Diskrete Mathematik für Informatik (SS 2020)held/teaching/diskrete_mathematik/dm... ·...

Diskrete Mathematik für Informatik(SS 2020)

Martin Held

FB ComputerwissenschaftenUniversität Salzburg

A-5020 Salzburg, [email protected]

25. August 2020

Computational Geometry and Applications Lab

UNIVERSITAT SALZBURG

mailto:[email protected]

Personalia

LVA-Leiter (VO+PS): Martin Held.

Email-Adresse: [email protected].

Basis-URL: https://www.cosy.sbg.ac.at/˜held.

Büro: Universität Salzburg, Computerwissenschaften, Zi. 1.20,Jakob-Haringer Str. 2, 5020 Salzburg-Itzling.

Telefonnummer (Büro): (0662) 8044-6304.

Telefonnummer (Sekr.): (0662) 8044-6328.

c© M. Held (Univ. Salzburg) Diskrete Mathematik (SS 2020) 2/333


https://www.cosy.sbg.ac.at/~held

Personalia

LVA-Leiter (PS): Günther Eder.







Personalia

LVA-Leiter (PS): Peter Palfrader.







Formalia

LVA-URL (VO+PS): Basis-URL/teaching/diskrete_mathematik/dm.html.

Allg. Information: Basis-URL/for_students.html.

Abhaltezeit der VO: Donnerstag 800–1110, mit etwa 20 Minuten Pause.

Abhalteort der VO: T01, Computerwissenschaften, Jakob-Haringer Str. 2.

Abhaltezeit des PS: Freitag 800–1000.

Abhalteort des PS: T01+T02+T03, Computerwissenschaften, Jakob-HaringerStr. 2.

Tutorium: Andreas Schranzhofer:Donnerstag 1600–1800 im T01,Computerwissenschaften, Jakob-Haringer Str. 2.

Achtung — das Proseminar ist prüfungsimmanent!


https://www.cosy.sbg.ac.at/~held/teaching/diskrete_mathematik/dm.html

https://www.cosy.sbg.ac.at/~held/for_students.html

Electronic Slides and Online Material

In addition to these slides, you are encouraged to consult the WWW home-page ofthis lecture:

https://www.cosy.sbg.ac.at/˜held/teaching/diskrete_mathematik/dm.html.

In particular, this WWW page contains up-to-date information on the course, plus linksto online notes, slides and (possibly) sample code.


https://www.cosy.sbg.ac.at/~held/teaching/diskrete_mathematik/dm.html

A Few Words of Warning

I hope that these slides will serve as a practice-minded introduction to various aspectsof discrete mathematics which are of importance for computer science. I would like towarn you explicitly not to regard these slides as the sole source of information on thetopics of my course. It may and will happen that I’ll use the lecture for talking aboutsubtle details that need not be covered in these slides! In particular, the slides won’tcontain all sample calculations, proofs of theorems, demonstrations of algorithms, orsolutions to problems posed during my lecture. That is, by making these slidesavailable to you I do not intend to encourage you to attend the lecture on an irregularbasis.


Acknowledgments

These slides are a revised and extended version of a draft prepared by KamranSafdar. Included is material written by Christian Alt, Caroline Atzl, Michael Burian,Peter Gintner, Bernhard Guillon, Yvonne Höller, Stefan Huber, Sandra Huemer,Christian Lercher, Sebastian Stenger, Alexander Zrinyi. I also benefited fromcomments and suggestions made by Stefan Huber and Peter Palfrader.

This revision and extension was carried out by myself, and I am responsible for allerrors.

Salzburg, February 2020 Martin Held


Legal Fine Print and Disclaimer

To the best of our knowledge, these slides do not violate or infringe upon somebodyelse’s copyrights. If copyrighted material appears in these slides then it wasconsidered to be available in a non-profit manner and as an educational tool forteaching at an academic institution, within the limits of the “fair use” policy. Forcopyrighted material we strive to give references to the copyright holders (if known).Of course, any trademarks mentioned in these slides are properties of their respectiveowners.

Please note that these slides are copyrighted. The copyright holder(s) grant you theright to download and print it for your personal use. Any other use, including non-profitinstructional use and re-distribution in electronic or printed form of significant portionsof it, beyond the limits of “fair use”, requires the explicit permission of the copyrightholder(s). All rights reserved.

These slides are made available without warrant of any kind, either express orimplied, including but not limited to the implied warranties of merchantability andfitness for a particular purpose. In no event shall the copyright holder(s) and/or theirrespective employers be liable for any special, indirect or consequential damages orany damages whatsoever resulting from loss of use, data or profits, arising out of or inconnection with the use of information provided in these slides.


Recommended Textbooks I

S. Maurer, A. Ralston.Discrete Algorithmic MathematicsA.K. Peters, 3rd edition, Jan 2005; ISBN 978-1-56881-166-6

K.H. Rosen.Discrete Mathematics and Its ApplicationsMcGraw-Hill, 7th edition, June 2011; ISBN 9780073383095

B. Kolman, R.C. Busby, S.C. Ross.Discrete Mathematical StructuresPearson India, 6th edition, 2017; ISBN 978-0134696447.

K.A. Ross, C.R.B. Wright.Discrete MathematicsPearson Prentice Hall, 5th edition, Aug 2002; ISBN 9780130652478

C. Stein, R.L.S. Drysdale, K. Bogart.Discrete Mathematics for Computer ScienceAddison-Wesley, March 2010; ISBN 978-0132122719.


Recommended Textbooks II

J. O’Donnell, C. Hall, R. Page.Discrete Mathematics Using a ComputerSpringer, 2nd edition, 2006; ISBN 978-1-84628-241-6

N.L. Biggs.Discrete MathematicsOxford University Press, 2nd edition, Feb 2003, reprinted (with corrections) 2008;ISBN 978-0-19-850717-8

M. Smid.Discrete Structures for Computer Science: Counting, Recursion, and Probabilityhttp://cglab.ca/~michiel/DiscreteStructures/, 2019

E. Lehman, F.T. Leighton, A.R. Meyer.Mathematics for Computer Sciencehttps://courses.csail.mit.edu/6.042/spring18/, 2018

M.M. Fleck.Building Blocks for Theoretical Computer Sciencehttp://mfleck.cs.illinois.edu/building-blocks/, 2013


http://cglab.ca/~michiel/DiscreteStructures/

https://courses.csail.mit.edu/6.042/spring18/

http://mfleck.cs.illinois.edu/building-blocks/

Table of Content

Introduction

Formalism: Definitions and Theorem Proving

Numbers and Basics of Number Theory

Principles of Elementary Counting and Combinatorics

Complexity Analysis and Recurrence Relations

Graph Theory

Cryptography


IntroductionWhat is Discrete Mathematics?Motivation


What is Discrete Mathematics?

I No universally accepted definition of the scope of DM exists . . .I Typically, objects studied in DM can only assume discrete, separate values rather

than values out of a continuum; sets of such objects are countable.I The term “discrete” is often used in contrast to “continuous”, where the values of

objects may vary smoothly.I Calculus does not belong to DM, but to “continuous mathematics”.


Typical Topics of Discrete Mathematics

I Depending on what is covered in other courses a variety of topics tends to bestudied within a course on DM:I Logic and Boolean algebra,I Mathematical language,I Algebraic structures,I Set theory,I Functions and relations;I Formal languages,I Automata theory;I Number theory,I Proofs and mathematical reasoning,I Counting and elementary combinatorics,I Graph theory,I Complexity theory,I Encoding and cryptography;I Computability theory,I Elementary probability theory.


Applications of Discrete Mathematics

I DM forms the mathematical language of computer science.I Thus, its importance has increased significantly in recent years.I DM is at the very heart of several other disciplines of computer science.I Applications of DM include — but are not limited to —

I Algorithms and data structures,I Automated programming,I Automated theorem proving,I Combinatorial geometry,I Computational geometry,I Cryptography and cryptanalysis,I Discrete simulation,I Game theory,I Operations research and combinatorial optimization,I Theory of computing,I Queuing theory.

I We start with a set of sample problems; solutions for all problems will be workedout or, at least, sketched during this course.


Sample Problem: Summation Formula

I Suppose that an algorithm needs 1 + 2 + 3 + · · ·+ (n − 1) + n manycomputational steps (of unit cost) to handle an input of size n.

I Question: Can we express this sum by means of a closed formula?I Basic math:

1 = 1 = 1·2/2

1 + 2 = 3 = 2·3/2

1 + 2 + 3 = 6 = 3·4/2

1 + 2 + 3 + 4 = 10 = 4·5/2

1 + 2 + 3 + 4 + 5 = 15 = 5·6/2

1 + 2 + 3 + 4 + 5 + 6 = 21 = 6·7/2

1 + 2 + 3 + 4 + 5 + 6 + 7 = 28 = 7·8/2

I An inspection of the numbers on the right-hand side might let us suspect that

1 + 2 + 3 + · · ·+ (n − 1) + n =n(n + 1)

2.

I But is this indeed correct? And, by the way, what do the dots in this equationreally mean??


Sample Problem: Summation Formula

I An answer can be established by means of number theory (natural numbers,induction). And we get indeed

1 + 2 + 3 + · · ·+ (n − 1) + n =n(n + 1)

2

for all “natural numbers” n.

I Caution: Even after calculating this sum for all values of n between 1 and 500one can not legitimately claim to know the sum for, say, n = 1000.

I Note: It would constitute a horrendous waste of CPU time to let a computercompute 1 + 2 + 3 + · · ·+ (n − 1) + n by successively adding numbers if wecould simply obtain the result by evaluating n(n+1)

2 .


Sample Problem: Chessboard Tilings

I Consider an 8× 8 chessboard with the upper-left and lower-right cells removed,and assume that we are given red/yellow and green/blue domino blocks whosesizes match the size of two adjacent squares of the chessboard.

I Question: Can this chessboard be covered completely by 31 domino blocks ofarbitrary color combinations?

?

?

?

?

?

I We consult counting principles and obtain the answer: No!I Caution: Simply trying out all possible placements of domino blocks hardly is an

option for an 8× 8 chessboard — and definitely no option for an n × n board!


Sample Problem: Route Calculation

I Question: What is the shortest route for driving from Salzburg to Graz?I Answer provided by computing a shortest path in a weighted graph: Salzburg→

Bad Ischl→ Bad Goisern→ Stainach/Irdning→ Liezen→ Leoben→Deutschfeistritz→ Graz.

Innsbruck

Worgl

BrixenLienz

Matrei

Mittersill

Kitzbuhel

Spittal/Drau

St. Michael/Lg.

Eben

Golling Goisern

Stainach/Ird.

Liezen

VillachKlagenfurt

GrazD.-feistritz

Hartberg

Bruck/Mur

Leoben

Wr. Neustadt

WienLinz

Steyr

Ischl

Sattledt

Salzburg

Munchen St. Polten

Rosenheim

60

47

81

2856

99

36

76 83

12496336

5311

83

10728

37

29

3261

7237

38

139

75

77

54

6612447

47

43

143

80

4924

34

15

94

I Note: Simply trying all possible routes gets tedious! (How would you evenguarantee that all possible routes have indeed been checked?)


Sample Problem: Channel Assignment

I Suppose that frequencies out of a set of m frequencies are to be assigned to nbroadcast stations within Austria. We are told that the area serviced by a stationlies within a disk with radius 50 kilometers. Obviously, no two different stationswhose broadcast areas overlap may use the same frequency.

I Question: Do we have enough frequencies? What is the minimum number offrequencies needed?

50km

I The solution can be obtained by using techniques of computational geometrycombined with graph coloring.


Sample Problem: Memory Required for Storing a Polyhedron

I Suppose that a polyhedral model has n vertices. How many edges and faces canit have at most? What is the storage complexity relative to n?

I Answer provided by graph theory: A polyhedron with n vertices has at most3n − 6 edges and 2n − 4 faces.


Sample Problem: Complexity of an Algorithm

I Suppose that an algorithm is given n numbers as input and that it solves aproblem by proceeding as follows: During one round of computation, it performsn computational steps. We know that during each round it discards at least 25%of the numbers. The algorithm executes one round after the other until only onenumber is left.

100755642after round 3:

after round 2:after round 1:input:

I Question: How many rounds does the algorithm run in the worst case (dependingon the input size n)? How many computational steps are carried out in the worstcase?

I Answer provided by the theory of recurrence relations: The number ofcomputational steps is linear in n, and the number of rounds is logarithmic in n.

I In asymptotic notation: O(n) and O(log n).


Sample Problem: Optimality of an Algorithm

I Tower-of-Hanoi Problem (TOH): Given three poles (labeled I,II,III) and a stack ofn disks arranged on Pole I from largest at the bottom to smallest at the top, weare to move all disks to Pole II such that only one disk is moved at a time andsuch that no larger disk ever is placed on a smaller disk.

I Attributed to Édouard Lucas [1883]. Supposedly based on an Indian legendabout Brahmin priests moving 64 disks in the Great Temple of Benares; oncethey are finished, life on Earth will end.

I Goal: Find an algorithm that uses the minimum number of moves.

I II IIII II III

I One can prove: A (straightforward) recursive algorithm needs 2n − 1 moves.I One can also prove: Every(!) algorithm that solves ToH needs at least 2n − 1

moves.I Thus, the solution achieved by the recursive algorithm is optimal as far as the

number of moves is concerned.I Note, though, that there exists a simple iterative solution due to Buneman&Levy

[1980] which avoids an exponential-sized stack!


Sample Problem: The Power of Exponential Growth

I According to legend, the power of exponential growth was already known by theBrahmin Sissa ibn Dahir (ca. 300-400 AD): As a reward for the invention of thegame of chess (or its Indian predecessor Chaturanga) he asked his king to placeone grain of rice in the first square of a chessboard, two in the second, four in thethird, and so on, doubling the amount of rice up to the 64-th square.

I So, how many grains of rice did Sissa ask for?I Let R(64) denote the number of rice grains for 64 squares. We get

R(64) = 1 + 2 + 4 + . . .+ 263

and, in general, using the capital-sigma notation and geometric series,

R(n) = 1 + 2 + 4 + . . .+ 2n−1 =n∑

i=1

2i−1 =n−1∑i=0

2i =2n − 12− 1

= 2n − 1.

I Hence, Sissa asked for

264 − 1 = 18 446 744 073 709 551 615

grains of rice. This is about 1000 times the current global yearly production!I [Sagan 1997]: “Exponentials can’t go on forever, because they will gobble up

everything”.I The “second half of the chessboard” is a phrase, coined by Kurzweil in 1999, to

refer to the point where exponential growth begins to have a significant impact.c© M. Held (Univ. Salzburg) Diskrete Mathematik (SS 2020) 27/333

Sample Problem: Key Distribution

I Suppose that two persons named Alice and Bob want to exchange a secretinformation, e.g., a key that can be used for decrypting their encrypted messages.

I Likely, they will not consider it to be safe to exchange the key as plain text via,say, email.

I What is a secure mechanism for them to exchange a key??I Meet in person at a secret place and share the key?!I Share in parts?!

I Answer provided by cryptography: The Diffie-Hellman Algorithm provides asimple way to exchange a key via public communication channels.

I By the way, how could Alice and Bob encrypt or decrypt messages once theyhave exchanged their key?

I Answer: This is yet another application of number theory!


Sample Problem: Hairy Siblings?

I Apparently, every human has a certain number of hairs on her/his body.I Question: Is it correct that there live at least two Austrians who have precisely the

same number of hairs on their bodies?I A combinatorial argument (pigeonhole principle) provides an affirmative answer:

Yes, this is correct!I Note: This is not a trick question — mild assumptions on the maximum number

of hairs per square centimeter on a human’s body will allow us to come up with arigorous mathematical argument.


Formalism: Definitions and Theorem ProvingDefinitionsSyntactical Proof TechniquesTypes of ProofsPigeonhole PrincipleNeed for Rigorous Analysis


How to Deal with Formal Statements . . .

I Experience tells me that students find it difficultI to parse and understand formal statements,I to formulate meaningful definitions,I to write clean and mathematically correct proofs.

I Hence, prior to diving into other areas of Discrete Mathematics, we start withtaking a practical look at the formal nuts and bolts of mathematical reasoning.

I In the following slides on definitions and theorem proving we pre-suppose an“intuitive” understanding of natural numbers, integers, reals, etc.; e.g., as taughtin school.

I We will later on put these number systems on slightly more formal grounds.


Definitions

I We distinguish between explicit and recursive definitions.I An explicit definition relates an entity that is to be specified (“definiendum”) to an

already known entity (“definiens”).I Explicit definition of an n-ary function f :

f (x1, x2, . . . , xn) := t ,

where the term t (normally) contains x1, x2, . . . , xn as free variables.I E.g., f (x , y) :=

√x2 + y2.

I Explicit definition of an n-ary predicate P:

P(x1, x2, . . . , xn) :⇔ A,

where the statement A (normally) contains x1, x2, . . . , xn as free variables.I E.g., P(x , y) :⇔ (x < y).

WarningThe definiendum does not occur in the definiens of an explicit definition of a function for predicate P! That is, the symbols f and P do not appear on the right-hand side.


Definitions: The Symbols “:=” and “:⇔”

I It is common to use the special symbols := and :⇔ for definitions, where thesymbol “:” appears on the side of the definiendum.

I Thus, one can also write =: or⇔: to indicate that the definiendum is on theright-hand side.

I Using =: and⇔: is very good practice sinceI it makes it immediately obvious to the reader that what follows constitutes a

definition rather than some lemma or claim,I it shows beyond doubt what is the definiens and what is the definiendum,

andI it forces the author to decide whether or not something is a consequence of

prior knowledge or some newly introduced entity.I However, if “:=” or “:⇔” are used once in a text then they have to be used for

absolutely all definitions in that text!!


Definitions: The Symbols “:=” and “:⇔”

I Poster seen in a tutoringinstitute at Salzburg:

I Can x1/2 be derived?I Can D be derived?

I Better formalism:I If x1, x2 are the roots of the

second-degree polynomial equationx2 + px + q = 0, with p, q ∈ R andunknown x ∈ R, then

x1/2 = −p2±√

p2

4− q.

I With D := p2 − 4q we get

D

>=<

0

2 distinct real roots,1 real root,0 no real roots.


Recursive Definitions

I Aka: Inductive definition.I How can we state

x is ancestor of y if x is parent of y, or if x is parent of parent of y, or if x isparent of parent of parent of y, or if . . .

in a form that does not need to resort to an ellipsis “. . .” ?I Recursive definitions (typically) consist of two parts:

I a basis in which the definiendum does not occur in the definiens, andI an inductive step in which the definiendum does occur.

I E.g.,x is an ancestor of y if x is parent of y or x is ancestor of parent of y.

WarningTo avoid infinite circles, the definiendum must not occur in the basis!


Recursive Definitions: Sum and Product

I Consider k real numbers a1, a2, . . . , ak ∈ R, together with some m, n ∈ N suchthat 1 ≤ m, n ≤ k .

n∑i=m

ai :=

0 if n < m

am if n = m(∑n−1

i=m ai ) + an if n > m

n∏i=m

ai :=

1 if n < m

am if n = m(∏n−1

i=m ai ) · an if n > m

I Consider a real number a ∈ R and a natural number n ∈ N0.

an :=

1 if n = 0 and a 6= 0a if n = 1

an−1 · a if n > 1


Recursive Definitions: Factorial and Fibonacci

Definition 1 (Factorial, Dt.: Fakultät, Faktorielle)For n ∈ N0,

n! :=

1 if n ≤ 1,n · (n − 1)! if n > 1.

n 0 1 2 3 4 5 6 7 8 9 10n! 1 1 2 6 24 120 720 5 040 40 320 362 880 3 628 800

Definition 2 (Fibonacci numbers)For n ∈ N0,

Fn :=

0 if n = 0,1 if n = 1,Fn−1 + Fn−2 if n ≥ 2.

n 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Fn 0 1 1 2 3 5 8 13 21 34 55 89 144 233 377 610


Fibonacci Numbers

I The Fibonacci numbers are named after Leonardo da Pisa (1180?–1241?), aka“figlio di Bonaccio”.

I The Fibonacci numbers have been studied extensively; they exhibit lots ofinteresting mathematical properties. For instance,

limn→∞

Fn+1

Fn= φ, where φ :=

1 +√

52

is known as golden ratio.

I The Fibonacci numbers are also found in nature: E.g., the numbers of CW/CCWspirals of sunflower heads are given by subsequent Fibonacci numbers.

[Image credit: Wikipedia.]


http://de.wikipedia.org

Caveats When Formulating Definitions

I Definitions like

P(x) :⇔ (x < 2y) or P(x , y , z) :⇔ (x < 2y)

can be seen as syntactically correct but they are semantically problematic!

Rule of thumbAll arguments of the definiendum have to appear as free variables in the definiens,and vice versa!

WarningAn entity introduced in a definition has to be free of internal inconsistencies and freeof contradictions with prior facts.

I E.g., assume that for mn ,

pq ∈ Q, with m, p, n, q ∈ N, we define

mn]pq

:=m + pn + q

.

I Then 11 ]

23 = 3

4 , but 22 ]

23 = 4

5 .I Since 1

1 = 22 , we conclude 4

5 = 34 , and, thus, 0 = 1. Yikes!


Terminology

Definition 3 (Proof, Dt.: Beweis)

To prove a statement means to derive it from axioms (or postulates) and otherpreviously established theorems by means of rules of logic.

I Note the difference between the English words “the proof” and “to prove”.I Common symbols to mark the end of a proof: 2, qued or qed (as an abbreviation

for the Latin words “quod erat demonstrandum”, i.e., for ”what was to be shown”).

Definition 4 (Theorem, Dt.: Satz, Theorem)

A statement is a theorem if it has been proved. If the statement is of the formH ⇒ C then we call H the hypothesis and C the conclusion.

I Of course, a theorem may involve quantifiers. E.g., ∀x H(x)⇒ C(x).I Depending on the importance of the result, terms like lemma (Dt.: Lemma,

Hilfssatz) or corollary (Dt.: Korollar) are also used instead of “theorem”.I A conjecture is a statement which has not yet been proved or disproved.I The status of a conjecture may remain unknown for decades or even centuries:

Fermat’s Last Theorem was stated by Pierre de Fermat in 1637 and proved byAndrew Wiles (with the help of Richard Taylor) in 1993–1995.


Syntactical Proof Techniques

I Syntactical proof techniques are proof techniques based on the analysis of thesyntactical structure of a statement.

I Syntactical proof techniques allow us to reason about statements and to simplifystatements with no or very little “understanding” of their mathematical meaning.

I In particular, syntactical proof techniques allow us to split complicated proofs intosimpler proofs, without any need for an ingenious idea for how to carry out aspecific proof.

I On the next slides we will study the standard proof situation H ⇒ C, andformulate rules which depend on the syntax of H and/or C.

I Recall the truth table for “⇒”:

H C H ⇒ C ¬H ∨ C0 0 1 10 1 1 11 0 0 01 1 1 1

H ⇒ C . . .. . . is true if either H is false (and C arbitrary)or if C is true for H being true.


Syntactical Proof Techniques for H ⇒ C

I If conclusion C is of the form (A ∧ B):I Prove A under the assumption H; andI Prove B under the assumption H.

I If conclusion C is of the form (A ∨ B):I Add ¬A to the assumption H and prove B. That is, assume both H and ¬A

to be true and use this to prove B.I Alternatively, add ¬B to the assumption H and prove A.

I If conclusion C is of the form (A⇒ B):I Add A to the assumption H and prove B.

I If conclusion C is of the form (A⇔ B):I Prove A⇒ B under the assumption H; andI Prove B ⇒ A under the assumption H.

WarningIn all the rules on this slide, A and B must not be part of a quantified formula.(Otherwise, get rid of the quantifier first!)



I If conclusion C is of the form (∀x A):I Proof technique: Let x0 be arbitrary but fixed (Dt.: “beliebig aber fix”). From

now on, x0 can be treated as a constant!I It remains to prove A[x0/x ] under the assumption H.I Often one does not trouble to explicitly label the particular arbitrary-but-fixed

choice of x as, say, x0 but only states that x is now regarded to be fixed.

WarningThe crucial point is that x0 has to be arbitrary, and the proof may not depend on theparticular choice of x0!I The symbol x0 may not occur anywhere in A, in the hypothesis H, or in some

other part of the conclusion.I We are not allowed to make any assumptions on x0 except for those that hold for

all x in the universe of discourse.



I If conclusion C is of the form (∃x A):I Constructive Proof (Dt.: konstruktiver Beweis):

I It “suffices” to find a suitable x0 such that A[x0/x ] if H.I Such an x0 is called the “solving term”.

I Existential Proof (Dt.: Existenzbeweis):I Prove that some suitable x0 exists.I No need to “construct” x0 explicitly.

I E.g., suppose that we want to prove the following claim: The polynomialp(x) := x3 − x2 + x − 1 has a real root over R.

Proof (constructive) : Factoring p(x) yields p(x) = (x − 1)(x2 + 1). Thus, welearn that 1 is a real root.

Proof (existential) : We have p(2) = 5 > 0 and p(0) = −1 < 0. Since p iscontinuous on the closed interval [0, 2], the Intermediate Value Theorem (Dt.:Zwischenwertsatz) tells us that there exists a real number x strictly between 0and 2 such that p(x) = 0.



I If conclusion C is of the form (∃!x A):I Prove that such an x exists.I Prove its uniqueness.

I If hypothesis H is of the form (∃x A):I Let x0 such that A[x0/x ].I Add A[x0/x ] to knowledge.I Again: x0 must not occur anywhere else in H or C!


Natural-Language Synonyms of Formal Terms

I On many occasions a conjecture will not be stated in formal terms but by using anatural language.

I Then one has to decode the natural-language formulation and translate it intoformal terms!

I Natural-language synonyms for A⇒ B:I B if A,I A only if B,I If A then B,I A is sufficient for B,I B is necessary for A.

I Natural-language synonyms for A⇔ B:I A if and only if B, A genau dann wenn B,I A is necessary and sufficient for B, A ist notwendig und hinreichend für B.


Equivalence Transformations

I First attempt to prove (∀n ∈ N 2n+1n+1 ≥

32 ):

2n + 1n + 1

≥ 32

2(2n + 1) ≥ 3(n + 1)4n + 2 ≥ 3n + 3

n ≥ 1

I Second refined attempt to prove (∀n ∈ N 2n+1n+1 ≥

32 ):

2n + 1n + 1

≥ 32

| · 2(n + 1)

=⇒ 2(2n + 1) ≥ 3(n + 1)=⇒ 4n + 2 ≥ 3n + 3 | − (3n + 2)=⇒ n ≥ 1

I Correct proof of (∀n ∈ N 2n+1n+1 ≥

32 ): Let n ∈ N be arbitrary but fixed. Then:

2n + 1n + 1

≥ 32

| · 2(n + 1)

⇐⇒ 2(2n + 1) ≥ 3(n + 1)⇐⇒ 4n + 2 ≥ 3n + 3 | − (3n + 2)⇐⇒ n ≥ 1


Equivalence Transformations

I Another way to prove (∀n ∈ N 2n+1n+1 ≥

32 ): We start with the term 2n+1

n+1 andconstruct a series of inequalities that ends in 3

2 .We get for all n ∈ N

2n + 1n + 1

=2(n + 1)

n + 1− 1

n + 1= 2− 1

n + 1≥ 2− 1

2=

32.

I Similarly we can prove(∀n ∈ N \ 1, 2 2n2 ≥ (n + 1)2):

2n2 = n2 + n2 = n2 + n · nn≥3≥ n2 + 3n ≥ n2 + 2n + 1 = (n + 1)2


Equivalence Transformations: Caveats

I Let a, b ∈ N be equal natural numbers. We “prove” that 1 = 2:

a = b | · a⇐⇒ a2 = ab | − b2

⇐⇒ a2 − b2 = ab − b2

⇐⇒ (a− b) · (a + b) = b · (a− b) | ÷ (a− b)⇐⇒ (a + b) = b | a := b⇐⇒ (b + b) = b⇐⇒ 2b = b | ÷ b⇐⇒ 2 = 1

I And here comes a “proof” of 4 = 5: Let x := 4 and y := 5. Thenx + y = 9 | · (x − y)

⇐⇒ x2 − y2 = 9x − 9y |+ 814 − 9x + y2

⇐⇒ x2 − 9x + 814 = y2 − 9y + 81

4⇐⇒ (x − 9

2 )2 = (y − 92 )2 | √

⇐⇒ x − 92 = y − 9

2 |+ 92

⇐⇒ x = y⇐⇒ 4 = 5


Equivalence Transformations: Caveats

I In the same way we can “prove” that 0 = 2. We start with the well-known identitycos2 x = 1− sin2 x , which holds for all x ∈ R:

cos2 x = 1− sin2 x | √

⇐⇒ cos x =√

1− sin2 x |+ 1⇐⇒ 1 + cos x = 1 +

√1− sin2 x | x := π

⇐⇒ 1− 1 = 1 +√

1− 0⇐⇒ 0 = 2

I Playing with square roots also allows to “prove” that 1 = −1:

1 =√

1 =√

(−1)(−1) =√−1√−1 = i · i = i2 = −1.

I We attempt to solve the equation√

x2 − 1 =√

x − 1 over R:√

x2 − 1 =√

x − 1 | 2

⇐⇒ x2 − 1 = x − 1 |+ 1− x⇐⇒ x2 − x = 0⇐⇒ x(x − 1) = 0

Hence, we seem to get 0, 1 as set of solutions.However, the equation is not even defined for x < 1.


Equivalence Transformations: Summary

Warnings

I Squaring is not an equivalence transformation!I If squaring is applied for solving an equation then all candidate solutions found

need to be tested with the original equation.I Taking a square root is only permissible if both signs are considered. That is,√

x2 yields ±x or |x |.I A division by x is only permissible if x 6= 0 can be assured.I Multiplication by a negative number is not an equivalence transformation for

inequalities.

Advice

I In general, a relation a b may only be replaced by a new relation a′ b′ if onecan argue that (a b)⇔ (a′ b′).

I It is advisable to prove a b, where ∈ =, <,>,≤,≥ , by constructing a chaina0 a1 a2 . . . an, with a0 = a and an = b, for some n ∈ N.


Types of Proofs

I “W.l.o.g.” for avoiding similar cases.I Direct enumeration.I Case analysis.I Direct proof.I Proof by contrapositive.I Proof by contradiction.I Indirect proof.I Disproving conjectures.


W.l.o.g.

I “W.l.o.g., A” means “Without loss of generality, we assume A”.I Dt.: O.B.d.A. (“Ohne Beschränkung der Allgemeinheit”).I This means that we could also carry on without the particular assumption A, and

would eitherI have to consider cases that are handled very similarly, orI could easily convert the general case to this special case.

I That is, a “w.l.o.g.” assumption allows us to save space/paper by avoiding toreplicate portions of a proof that differ only in trivial aspects.

WarningDo not use “w.l.o.g.” unless you could indeed explain explicitly and in full detail how tocarry on without that assumption!


Types of Proofs: Direct Enumeration

I Direct EnumerationI E.g.: The conjecture

2p + 1 is prime for all p ∈ 2, 3, 5

can be proved by considering all finitely many possible values for p.I Note: Direct enumeration only works if the set given is finite!


Types of Proofs: Case Analysis

I Aka Proof by Exhaustion. Dt.: Fallunterscheidung.I In order to prove H ⇒ C, it suffices to prove

A1 ∨ A2 ∨ . . . ∨ Ak

for some statements A1,A2, . . . ,Ak , and to prove

(H ∧ A1) ⇒ C,

(H ∧ A2) ⇒ C,...

(H ∧ Ak ) ⇒ C.

WarningIt is essential to guarantee that A1 ∨ A2 ∨ . . . ∨ Ak holds, i.e., that no case is missing!


Types of Proofs: Sample Case Analysis

I Suppose that we want to prove the following claim: For all n ∈ N0 the number 7divides n7 − n without remainder. E.g., for n = 3, we get 37 − 3 = 2184 = 7 · 312.

Proof : Factoring n7 − n yields

n7− n = n(n6− 1) = n(n3− 1)(n3 + 1) = n(n− 1)(n2 + n + 1)(n + 1)(n2− n + 1).

Let n = 7q + r with q, r ∈ N0 and 0 ≤ r ≤ 6. We consider seven cases, depending onwhether r = 0, 1, 2, 3, 4, 5 or 6.Case n = 7q: Then the factor n of n7 − n is divisible by 7.Case n = 7q + 1: Then the factor n − 1 = 7q of n7 − n is divisible by 7.Case n = 7q + 2: Then n2 + n + 1 = (7q + 2)2 + (7q + 2) + 1 = 49q2 + 35q + 7 is

divisible by 7.Case n = 7q + 3: Then n2 − n + 1 = (7q + 3)2 − (7q + 3) + 1 = 49q2 + 35q + 7 is

divisible by 7.Case n = 7q + 4: Then n2 + n + 1 = (7q + 4)2 + (7q + 4) + 1 = 49q2 + 63q + 21 is

divisible by 7.Case n = 7q + 5: Then n2 − n + 1 = (7q + 5)2 − (7q + 5) + 1 = 49q2 + 63q + 21 is

divisible by 7.Case n = 7q + 6: Then n + 1 = 7q + 7 is divisible by 7.


Types of Proofs: Direct Proof

I Dt.: direkter Beweis.I We want to prove H ⇒ C:

I We build a chain of reasoning that starts at H and ends in C.I This approach is the classical example of deductive reasoning, where a

logically valid sequence of steps establishes the truth of C under theassumption of H.

I Suppose we want to prove (∀x , y ∈ R+ (x < y)⇒ (x2 < y2)).

Proof : (Direct Proof)Let x0, y0 ∈ R+ be arbitrary but fixed, with x0 < y0.We have x0 < y0, and therefore x2

0 = x0 · x0 < y0 · x0. Since x0 < y0 we knowy0 · x0 < y2

0 , and obtain x20 < y0 · x0 < y2

0 , which finally establishes x20 < y2

0 :

x20 = x0 · x0 < y0 · x0 < y0 · y0 = y2

0


Types of Proofs: Proof by Contrapositive

I Dt.: Umkehrschluss, Kontraposition.I We want to prove H ⇒ C:

I In order to prove H ⇒ C we build a (direct) proof for (¬C ⇒ ¬H).I Again, suppose we want to prove (∀x , y ∈ R+ (x < y)⇒ (x2 < y2)).

Proof : Let x0, y0 ∈ R+ be arbitrary but fixed. We prove (x20 ≥ y2

0 )⇒ (x0 ≥ y0)similar to the direct proof before. Since (x2

0 ≥ y20 )⇔ (x2

0 − y20 ≥ 0), we get

0 ≤ x20 − y2

0 = (x0 − y0)(x0 + y0),

which implies y0 ≤ x0 since we may divide by the positive number x0 + y0.I Suppose we want to prove H ⇒ (∃x A).

Proof : Prove (∀x (¬A))⇒ ¬H.

WarningMake sure that the statements are negated correctly!


Types of Proofs: Proof by Contradiction

I Dt.: Widerspruchsbeweis.I We want to prove H ⇒ C:

I Formally, (H ⇒ C) ≡ ((H ∧ ¬C)⇒ ¬H).I Thus, assume (H ∧ ¬C) and prove ¬H.

I Warning: As when proving the contrapositive it is essential to check twice that thestatements are indeed negated correctly!


Types of Proofs: Indirect Proof

I Aka Reductio ad absurdum.I Dt.: indirekter Beweis.I We want to prove H ⇒ C.

I Consider a statement R that is known to be true, like 0 6= 1.I Now assume (H ∧ ¬C) and deduce ¬R, i.e., 0 = 1.I This is absurd, and we conclude that ¬C is false.I Formally, (H ∧ ¬C ∧ R)⇒ ¬R.I This is of the form (A⇒ B), and we have (A⇒ B) ≡ T , where B ≡ F . Thus,

A ≡ F .

NoteSince an indirect proof is similar to a proof by contradiction, many textbooks treat it asone proof technique, or use the terms “reductio ad absurdum”, “indirect proof”, and“proof by contradiction” as synonyms.

I In a nutshell: If we manage to prove ¬C ⇒ F to be true, then we have ¬C ≡ Fand, thus, C ≡ T .


Types of Proofs: Sample Indirect Proof

I Suppose that we want to prove that the polynomial equation x3 + x + 1 = 0 hasno rational solution.

Proof : Assume to the contrary that there exists a rational number pq which is a root of

that polynomial. W.l.o.g., we may assume pq to be irreducible. (A rational number p

q isirreducible if there exists no integer other than ±1 that divides both p and q.) We get

0 =p3

q3 +pq

+ 1 = p3 + pq2 + q3.

As statement R we take “0 is even”.We do a case analysis, depending on whether p, q are even or odd:Case p, q odd: Then p3 + pq2 + q3 is odd, but 0 is even, yielding a contradiction to R.Case p odd, q even: Then again p3 + pq2 + q3 is odd; contradiction.Case p even, q odd: Then again p3 + pq2 + q3 is odd; contradiction.Case p, q even: This is not possible since we assumed (rightfully) that p

q isirreducible.


Disproving Conjectures

I Sometimes conjectures are false . . .I If the conjecture is of the form (∀x A):

I Then we can disprove this conjecture by showing (∃x ¬A).I The latter is proved if we can come up with a counterexample (Dt.:

Gegenbeispiel) to the original claim.I E.g., the claim ∀p ∈ P (2p + 1) ∈ P is shown to be false by testing p := 7.

(Note, though, that it is true for p = 2, 3, 5, 11, 23, . . .)I Similarly, numbers of the form 22n

+ 1, for n ∈ N, were once assumed to beprimes. Indeed, this is correct for n = 1, 2, 3, 4 but n := 5 yields acounterexample:

225+ 1 = 4 294 967 297 = 641 · 6 700 417.

I If, however, the conjecture is of the form (∃x A):I Then a counterexample does not suffice!I Rather, to disprove this conjecture, we’d have to prove formally (∀x ¬A).


Caveat: Ex Falso Quodlibet!

I Consider the following lemma: Let x ∈ R. If xx2+1 > 2 then x < 1

2 .

I Formally: ∀x ∈ R(

xx2+1 > 2

)=⇒

(x < 1

2

).

Proof : Let x ∈ R arbitrary but fixed and suppose that xx2+1 > 2. This implies x > 0

and we get

1x

=xx2 >

xx2 + 1

> 2, thus1x> 2 and, therefore, x <

12.

I Note, though, that this lemma is of little use for mathematics: The hypothesis isnever true! We have

xx2 + 1

> 2 ⇐⇒ 0 > 2x2 − x + 2 ⇐⇒ 2x2 − x + 2 < 0.

However, by a simple case analysis,

if x ≤ 1 then 2x2 − x + 2 = 2x2 + 1 + (1− x) ≥ 2x2 + 1 > 0,

if x > 1 then 2x2 − x + 2 = x2 + 2 + x(x − 1) ≥ x2 + 2 > 0.


The Pigeonhole Principle

I Schubfachschluss: In 1834, Johann Dirichlet noted that if there are five objects infour drawers then there is a drawer with two or more objects.

I Pigeonhole Principle: If n letters are posted to m pigeonholes, thenI at least one pigeonhole receives more than one letter if n > m.I at least one pigeonhole remains empty if n < m.I each pigeonhole might receive exactly one letter if n = m.

Theorem 5 (Pigeonhole Principle, Dt.: Schubfachschluss)

Consider two finite sets A and B. There is a bijection between A and B if and only ifA and B have the same number of elements. If A has more elements then B thenevery mapping from A to B will cause at least one element of B to be the target oftwo or more elements of A.

Corollary 6

For n ∈ N let An := 1, 2, 3, . . . , n − 1, n ⊂ N, and let Bn denote an arbitrary set ofn elements. Then, for all n ∈ N and all sets Bn, no mapping from An+1 to Bn isinjective.


The Pigeonhole Principle: Sample Application

Lemma 7

Consider a rectangular grid of points which consists of four rows and 100 columns.Each point is colored with a color which is picked randomly among red, green andblue. Prove that there always exist four points of the same color that form thecorners of a rectangle (with sides parallel to the grid), no matter how the coloring isdone.

Proof : A column pattern is the top-to-bottom sequence of colors assigned to thepoints of a column of the grid. There are exactly 34 = 81 different column patterns.Since there are more than 81 columns, we are guaranteed to have at least twocolumns with the same column pattern. Consider two such columns. Since there arefour rows but only three colors, we conclude that two of the rows have the same color,thus giving us the four corners of the rectangle sought.

I Note: Just 19 columns suffice to guarantee the existance of such a rectangle.


The Pigeonhole Principle: A Hairy Application

I Suppose there exists an upper limit for the number of hairs on the human skinper square centimeter, say 140 hairs.

I Taking 5 square meters as a reasonable upper limit for the skin area of a human,we conclude that a human has at most 7 000 000 hairs.

I Since Austria has (significantly) more than 7 000 001 inhabitants, we concludethat, by the pigeonhole principle, at least two inhabitants of Austria have exactlythe same number of hairs.

#hairs hairy ID numbers0 11 2...

... It is impossible to map every inhabitant7 000 000 7 000 001 to his/her own "hair bin"!

7 000 0027 000 003


Real-World Application: Chessboard Tilings Revisited

I Question: Can our modified chessboard be covered completely by 31 dominoblocks of arbitrary color combinations?

?

?

?

?

?

I We observe that every permissible domino placement covers exactly one blacksquare and one white square of the chessboard.

I Thus, all domino placements would establish a one-to-one mapping betweenblack and white squares. However, there are 32 black squares and only 30 whitesquares! We conclude that our chessboard cannot be covered completely bydomino blocks.


Real-World Application: Analysis of Lossless Data Compression

I Could one design an algorithm for lossless data compression that is guaranteednot to increase the file size of some input file while achieving a genuinecompression for at least one other file? No!

I Assume that every file is represented as a string of bits of some arbitrary length.Suppose further that there exists a compression algorithm that transforms everyfile into a distinct file which is no longer than the original file, and that at least onefile will be compressed into something that is shorter than itself.

I Let m be the least number such that there is a file f with length m bits that getscompressed to something shorter. Let n be the number of bits of the compressedversion of f . Hence, n < m.

I Since n < m, every file of length n keeps its size during compression. There are2n many such files. Together with f we would have 2n + 1 files which allcompress into one of the 2n files of length n.

I By the pigeonhole principle there must exist some file f ′ of length n which is theoutput of the compression algorithm for two different inputs. That file f ′ cannot bedecompressed reliably, which contradicts the assumption that the algorithm islossless.

I Hence, every compression algorithm will increase the size of at least some file, orkeep the sizes of all files unchanged.


Need for Rigorous Analysis

I Suppose that we are to pick a bunch of integers between 1 and n such that notwo of them differ by exactly 3 or exactly 5. Let’s call these numbers “compatible”.

I How many compatible numbers can you pick for n := 20?I Intuition: Start at 1 and scan the integers from 1 to 20, successively picking those

integers which are compatible with all integers picked previously:

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

I We get 9 compatible integers. Our selection scheme makes it plausible that thisis indeed the maximum number of compatible integers within 1, 2, 3, . . . , 19, 20.Right?

I Well, what about the following 10 integers?

1 3 5 7 9 11 13 15 17 19

I Oops! Why should we believe that we can’t find 11 or more compatible integerswithin 1, 2, 3, . . . , 19, 20?

I The answer is provided by the pigeonhole principle: Every subset of compatibleintegers of 1, 2, 3, . . . , 19, 20 can contain at most one of each of the following10 pairs:

1 2 3 4 5 11 12 13 14 156 7 8 9 10 16 17 18 19 20


Need for Rigorous Analysis

Lesson Learned

1. An intuitively appealing argument or approach is no substitute for a formal proof:Intuition might be wrong!

2. Consent of the majority is no substitute for a formal proof either.

3. The so-called greedy approach need not always lead to the best solution for anoptimization problem.

Proofs Needed!Even though proofs and a rigorous formal analysis might seem boring (difficult,mind-boggling, mind-numbing, unnecessary, . . .) there is just no way around them ifwe want to be sure that our findings are correct!

I So, be prepared for at least some boring (difficult, mind-boggling, mind-numbing,unnecessary, . . .) proofs! §


Numbers and Basics of Number TheoryNatural NumbersIntegersRational NumbersReal NumbersWell-founded Induction


How Shall We Define Natural Numbers or Real Numbers?

I Three options:1. Ignore all formal details and presuppose an “intuitive” understanding of

reals, integers, . . .2. Introduce the natural numbers, N, and then construct a hierarchy of number

systems: N ⊂ Z ⊂ Q ⊂ R.3. Set up the reals, R, axiomatically and then define proper subsets for N,Z,Q.

I What is the best approach for a course on (applied) discrete mathematics? Muchscholarly debate — no consensus!

I We will start with introducing the natural numbers. However, since the gorydetails result in a lengthy discussion which provides little additional insight in N —and this is no course on number theory — we base our introduction of N on asimplified treatment of the so-called Peano axioms; see a book on number theoryfor a more formalized introduction of N.

I Note: Make sure to recall algebraic structures if you are not (or no longer) familiarwith them. (See, e.g., the course “Formale Systeme”).


Natural Numbers: N

I Intuitively, the natural numbers N are given by 1, 2, 3, 4, 5, . . . or by0, 1, 2, 3, 4, 5, . . ..

I Unfortunately, there is no general agreement on whether or not to include 0 . . .I Paulo Ribenboim (1996): “Let P be a set of natural numbers; whenever

convenient, it may be assumed that 0 is an element of P.”

ConventionIn this course we adopt the following convention:

N := 1, 2, 3, 4, 5, . . . and N0 := 0, 1, 2, 3, 4, 5, . . ..

I Caution: Read a text carefully to learn what an author means by "naturalnumber". In particular, watch for clues such as terms like "positive naturalnumbers" (which indicates that zero is included) or statements like "n is a naturalnumber, so it must be greater than zero" (which indicates that zero is notincluded).

I If one treats 0 as an element of N then 1, 2, 3, 4, 5, . . . is often denoted by N∗.


Orderings

Definition 8 (Partial order, Dt.: Halbordnung)

A partial order on a set S is a binary relation , i.e., a subset of S × S, such thatthe following three properties hold for all a, b, c ∈ S:

1. Reflexivity: a a.

2. Anti-symmetry: (a b ∧ b a) ⇒ a = b.

3. Transitivity: (a b ∧ b c) ⇒ a c.

If is a partial order on S then (S,) is called a partially ordered set, aka a poset.

Definition 9 (Strict partial order, Dt.: strikte Halbordnung)

A binary relation ≺ on a set S forms a strict partial order on S if the following twoproperties hold for all a, b, c ∈ S:

1. Irreflexivity: ¬(a ≺ a).

2. Transitivity: (a ≺ b ∧ b ≺ c) ⇒ a ≺ c.

I A strict partial order is always asymmetric: If a ≺ b then ¬(b ≺ a).(a ≺ b ∧ b ≺ a)

trans.⇒ a ≺ a, in contradiction to the irreflexivity: ¬(a ≺ a).


Orderings

Theorem 10There is a one-to-one correspondence between non-strict and strict partial orders.Let S be a set and a, b ∈ S.

1. If is a non-strict partial order on S then the corresponding strict partial order"≺" on S is the reflexive reduction given by

a ≺ b :⇔ a b and a 6= b.

2. If, on the other hand, ≺ is a strict partial order on S then the correspondingnon-strict partial order "" on S is the reflexive closure given by

a b :⇔ a ≺ b or a = b.

I As a notational convention, we omit the indication of an equality sign if we refer toa strict order, e.g., we write ≺ rather than or ⊂ rather than ⊆.


Orderings

I E.g., (Z,) with (the non-strict order) as defined below forms a poset:

if a and b are even: a b :⇔ a ≥ b

if a and b are odd: a b :⇔ a ≤ b

I The subset relation, ⊂, on the powerset P(X ) of a set X is a strict partial order.

Definition 11 (Dual order, Dt.: duale Ordnung)

Let (S,) resp. (S,≺) be a (strict) poset. The dual order (or reverse order ) on S, resp. , is defined as follows for a, b ∈ S:

a b :⇔ b a a b :⇔ b ≺ a.


Extreme Elements

Definition 12 (Minimal element, Dt.: minimales Element)

Let (S,) be a poset and T ⊆ S. An element a ∈ T is a minimal element of T if nob ∈ T \ a exists such that b a.

Definition 13 (Least element, Dt.: kleinstes Element, Minimum)

Let (S,) be a poset and T ⊆ S. An element a ∈ T is a least element (orminimum) of T if ∀b ∈ T \ a a b.

Definition 14 (Maximal element, Dt.: maximales Element)

Let (S,) be a poset and T ⊆ S. An element a ∈ T is a maximal element of T if nob ∈ T \ a exists such that a b.

Definition 15 (Greatest element, Dt.: Maximum)

Let (S,) be a poset and T ⊆ S. An element a ∈ T is a greatest element (ormaximum) of T if ∀b ∈ T \ a b a.

I Note: If a minimum or maximum exists then the anti-symmetry ensures that it isunique. Minimal or maximal elements need not be unique, though.


Orderings

Definition 16 (Total order, Dt.: totale Ordnung)

A binary relation on a set S forms a total order (or linear order ) on S if thefollowing three statements hold for all a, b, c ∈ S:

1. Totality: a b ∨ b a.

2. Anti-symmetry: (a b ∧ b a) ⇒ a = b.

3. Transitivity: (a b ∧ b c) ⇒ a c.

If is a total order on S then (S,) is called a totally ordered set.

I Note that (1) in Def. 16 implies reflexivity: a a for all a ∈ S.I That is, a total order on S is a (non-strict) partial order such that every pair of

elements of S is comparable.

Definition 17 (Well-order, Dt.: Wohlordnung)

A total order on a set S forms a well-order if every non-empty subset of S has aleast element.


Natural Numbers and Peano’s Axioms

I The following definition of N is based on a simplified version of Peano’s Axioms,as proposed by Giuseppe Peano (1858–1932) in 1889.

Definition 18 (Natural numbers, Dt.: natürliche Zahlen)

The set of all natural numbers, N, together with an order relation ≤, is a totallyordered set defined as follows:

N1 1 ∈ N.

N2 ∀n ∈ N n + 1 ∈ N ∧ n < n + 1.

N3 ∀n ∈ N, n 6= 1 ∃m ∈ N n = m + 1.

N4 Every non-empty subset of N has a least element.

The number n + 1 is called the successor of n, and sometimes denoted by succ(n).

I N1 together with N2 establish the infinite sequence 1 < 2 < 3 < . . .

I N3 guarantees that every n ∈ N (except 1) is the successor of some number in N.I The so-called well-ordering principle, N4, weeds out numbers like 1

2 or π.I One can show that the standard algebraic rules are compatible with the

conditions imposed on N, and that algebra and order interact smoothly within N.I One can also show that (up to a renaming of elements) there is only one set that

fulfills all conditions of Def. 18. Hence, N is uniquely defined.


The Principle of Mathematical Induction

I Franciscus Maurolicus (1494–1575), an abbot of Messina, seems to have beenfirst to use induction for proving a theorem. (He proved

∑ni=1(2i − 1) = n2.)

I Blaise Pascal (1623–1662) used induction to prove a relation among binomialcoefficients.

I Augustus de Morgan (1806–1871) was the first to use the term “induction”.I Today’s view of induction is based on the work of Giuseppe Peano (1858–1932).

Theorem 19

Consider a set K ⊆ N and assume that it is inductive, i.e., that

1. 1 ∈ K ,

2. ∀k ∈ K (k + 1) ∈ K .

Then K = N.

Proof : Suppose that K 6= N, i.e., K ⊂ N. Hence, K ′ := N \ K is not empty. By thewell-ordering principle, (N4), K ′ has a least element, n. Since n 6= 1 there exists anumber k ∈ N such that k + 1 = n. As n is the least element of K ′, we have k ∈ K .Applying modus ponens yields k + 1 = n ∈ K , i.e., a contradiction.


The Principle of Mathematical Induction

Theorem 20 (Weak Principle of Induction (W.P.I.))

Consider a predicate P over N.If

P(1)

and if

∀k ∈ N (P(k)⇒ P(k + 1))

then

∀n ∈ N P(n).

Proof : Define K := n ∈ N : P(n). We have1. 1 ∈ K , and2. ∀k ∈ K (k + 1) ∈ K .

Thus, Thm. 19 is applicable and we conclude K = N. That is, the predicate P holdsfor all natural numbers.


Three Main Steps of a Proof by Induction

I Suppose that we want to prove

∀n ∈ N P(n),

for some predicate P over N.

I We proceed as follows:1. Induction basis (“IB”): A basis step is done, i.e., P(1) is proved to be true.2. Induction hypothesis (“IH”): We assume P(k) to be true for an arbitrary but

fixed k ∈ N.3. Inductive step (“IS”): We prove P(k + 1) based on the knowledge that P(k)

is true.


Gauß’ Problem Revisited: Sample Inductive Proof

I We claim that∑n

i=1 i = n·(n+1)2 holds for all n ∈ N.

Proof : We use induction to prove our claim as follows:I We define a suitable predicate P:

∀n ∈ N

(P(n) :⇔

n∑i=1

i =n · (n + 1)

2

).

I Induction basis (IB): We establish the truth of P(1):

1∑i=1

i = 1 =1 · 2

2.

I Induction hypothesis (IH): Assume P(k) true for an arbitrary but fixed k ∈ N. Thatis, we assume

k∑i=1

i =k · (k + 1)

2.


Gauß’ Problem Revisited: Sample Inductive Proof

Proof (cont’d) :I Inductive step (IS): We have to prove P(k + 1) based on the induction

hypothesis. That is, we have to prove

k+1∑i=1

i =(k + 1) · (k + 2)

2.

We get

k+1∑i=1

i =k∑

i=1

i + (k + 1)

I.H.=

k · (k + 1)

2+ (k + 1)

=k · (k + 1) + 2(k + 1)

2

=(k + 1) · (k + 2)

2.


Variations of the Induction Principle

Theorem 21 (Strong Principle of Induction (S.P.I.))

Consider a predicate P over N.If

P(1)

and if

∀k ∈ N [(P(1) ∧ P(2) ∧ . . . ∧ P(k)) ⇒ P(k + 1)]

then

∀n ∈ N P(n).

I Obviously, all theorems that can be proved by W.P.I. can also be proved by S.P.I.,and one can actually show the equivalence of W.P.I. and S.P.I.


Variations of the Induction Principle

Theorem 22 (S.P.I. with Larger Base)

Consider a predicate P over N, and let m ∈ N.If

P(m)

and if

∀(k ∈ N, k ≥ m) [(P(m) ∧ P(m + 1) ∧ . . . ∧ P(k)) ⇒ P(k + 1)]

then

∀(n ∈ N, n ≥ m) P(n).

Proof : We define a new predicate P′ over N with

P′(n) :⇐⇒ P(m − 1 + n) for all n ∈ N,

and apply the standard S.P.I.

I We could also carry out induction for smaller base values. That is, inductionworks for claims over N0. (And even for negative base values!)


Mathematical Induction: Caveats

I We may not assume anything in the inductive step n→ n + 1 besides that P(n)holds and, of course, the standard properties of N.

I The inductive step alone does not suffice! By carrying out only the inductive stepone can “prove” that ∀n ∈ N n = n + 5. Let k ∈ N be arbitrary but fixed andassume as I.H. that k = k + 5:

k + 1 I.H.= (k + 5) + 1 = (k + 1) + 5.

Thus, proving the base is mandatory!I Several base cases alone do not suffice! E.g., consider

∀n ∈ N (n2 − n + 41 is a prime).

For n up to 20, the number n2 − n + 41 actually is a prime, but this is not true forn = 41. Thus, proving the inductive step is mandatory!



I George Pólya (1887–1985): "All cats have the same color", or∀n ∈ N (for all sets S of n cats (all cats of S have the same color)).

I We use induction to prove this claim.I IB k = 1: obviously true.I IH: For all sets S of k cats, all cats of S have the same color, for k arbitrary

but fixed.I IS from k to k + 1: Consider a set S of k + 1 cats, and let A and B be two

subsets of S such that

|A| = |B| = k and A ∪ B = S.

Using the induction hypothesis and the transitivity of the equivalence, weconclude that all cats of the set S have the same color!

As nature shows, this “proof” is seriously flawed . . .



I We claim that 2 · n = 0 for all n ∈ N0.I We use induction to prove this claim:

I IB k = 0: obviously true.I IH: Suppose that the claim holds for all k ∈ N0 with k ≤ n, for some arbitrary

but fixed n ∈ N0.I IS from n to n + 1: We write n + 1 as n + 1 = k1 + k2, where k1, k2 ∈ N0 with

k1, k2 ≤ n. Then

2 · (n + 1) = 2 · (k1 + k2) = 2 · k1 + 2 · k2I.H.= 0 + 0 = 0,

thus finishing the inductive “proof” . . .


Real-World Application: Fair Resource Distribution

I Suppose that some limited and non-uniform resource has to be distributed fairlyamong n receivers, for some n ∈ N with n > 1.

I E.g., a cake (with fruits, whipped cream, chocolate crumbs, icing, etc.) mighthave to be distributed fairly among n kids. Aka: “Cake Cutting Problem”.

I To make the situation worse, each kid might value different portions of the cakedifferently. (Bob likes fruits, Alice hates them; Alice likes whipped cream, but Bobhates it.)

I The distribution should involve all kids such that each kid has to agree that itreceived a fair share of the cake by his/her preferences.

Definition 23 (Fair distribution protocol)

A protocol for the distribution of a resource among n receivers is considered fair ifeach receiver gets at least 1/n-th of the resource (by his/her preferences), no matterwhat the preferences of the other receivers are and what the other receivers get.

I How can we come up with a fair distribution protocol? Is there a generalalgorithm for fair cake cutting in the presence of n kids??



If n = 2: Cut-and-choose distribution protocol.1. Alice cuts the cake into two equal pieces (equal by her preferences).2. Bob chooses whichever piece seems larger (by his preferences).3. Alice takes the remaining piece.

If n > 2: Recursive application of the cut-and-choose distribution protocol.1. The first n − 1 kids cut the cake into n − 1 pieces by applying the

cut-and-choose distribution protocol recursively to n − 2, n − 3 etc. kids, thuseach obtaining (hopefully) at least a fair 1/n−1 portion of the cake.

2. The n-th kid asks all other n − 1 kids to cut his/her portion of the cake into npieces such that the cutting is fair according to his/her preferences. (That is,according to each kid’s preferences, each of the n pieces of his/her portion isequally desirable, for all of the first n − 1 kids.)

3. The n-th kid walks around and collects one piece — the most desirable pieceaccording to his/her preferences! — from all the other n − 1 kids.

Theorem 24

The recursive cut-and-choose distribution protocol is fair.



Proof of Thm. 24 by induction : Assume that the total cake is worth 1 for each kid.I.B.: n = 2 Alice cut the cake into two pieces that are equally desirable (according to

her preferences) and, thus, both worth 1/2. Hence, she will get one half of thecake (by her preferences), no matter how Bob behaves.Bob sees two pieces, one worth w1 and the other one worth 1− w1 (by hispreferences). Trivially, either w1 ≥ 1/2 or 1− w1 ≥ 1/2.Hence, Bob can choose at least one half of the cake (according to hispreferences), and both kids have no reason to complain about an unfair cutting.

I.H.: Assume that the recursive cut-and-choose cake cutting has been considered fairby the first k − 1 kids, for k ≥ 3 arbitrary but fixed. Hence, each of the fist k − 1kids got a portion that is a least worth (according to the kid’s preferences) 1

k−1 .I.S.: After the cuts for the k -th kid were made, each kid has k pieces each worth

1(k−1)·k . After the k -th kid took one piece from each of them, each of the firstk − 1 kids is left with k − 1 pieces each worth 1

(k−1)·k , i.e., with a total worth of 1k .

Suppose that the k -th kid values the portion of the i-th kid with wi , fori ∈ 1, 2, . . . , k − 1. Of course, w1 + w2 + . . .+ wk−1 = 1. Since the k -th kidgets at least wi/k from the i-th kid, the k -th kid gets in total at least

w1

k+

w2

k+ . . .+

wk−1

k=

1k

(w1 + w2 + . . .+ wk−1) =1k.


Cardinality of N

I Intuitively, the cardinality of a set A specifies the number of elements of A.I It is common to write |A| to denote the cardinality of A. E.g., |1, 2, 4, 8, 16| = 5.I The notion of cardinality becomes tricky for “infinite” sets, where the cardinality is

not simply given by the number of elements.

Definition 25 (Comparing cardinality )

The sets A,B have the same cardinality, denoted by |A| = |B|, if there exists abijection from A to B.The set A is of strictly smaller cardinality than B, denoted by |A| < |B|, if thereexists an injective function but no bijective function from A to B.

Definition 26 (Finite, countably infinite, uncountable, Dt: endlich, abzählbar unendlich, überabzählbar unendlich)

The set A is a finite set if |A| < |N|.The set A is a countably infinite set if |A| = |N| =: ℵ0.The set A is an uncountable set if |A| > |N|.

Theorem 27

A subset of a countably infinite set is a finite or a countably infinite set itself.


Integers: Z

I Intuitive way to define the integers: Z := N0 ∪ −n : n ∈ N.I Thus, Z = 0,±1,±2,±3,±4,±5, . . ..I Z+ := N and Z+

0 := N0.I The blackboard-bold letter Z stands for the German word “Zahlen”.

I But what are the properties of the elements −n??I And how could we define a + b and a · b for a, b ∈ Z??

I In order to put Z on a more solid basis, we “extend” N to obtain Z.


Construction of Z Based on N

I Let ∼=Z be a relation over N0 such that

(a, b) ∼=Z (c, d) :⇔ a + d = c + b.

I Easy to show: ∼=Z is an equivalence relation over N0, with the equivalenceclasses shown below.

(1, 1) (1, 2) (1, 3) ...

(2, 1) (2, 2) (2, 3) ...

(3, 1) (3, 2) (3, 3) ...

(4, 1) (4, 2) (4, 3) (4, 4) ...

(3, 4)

(2, 4)

(1, 4)

: : : : :...

0

-1

-2

-3

:

123...



I We interprete [(a, b)]∼=Z as a− b.I For n ∈ N, the equivalence classes [(n, 0)]∼=Z form the natural numbers, while

[(0, n)]∼=Z form the negative integers.I Zero is given by [(0, 0)]∼=Z .

(1, 1) (1, 2) (1, 3) ...

(2, 1) (2, 2) (2, 3) ...

(3, 1) (3, 2) (3, 3) ...

(4, 1) (4, 2) (4, 3) (4, 4) ...

(3, 4)

(2, 4)

(1, 4)

: : : : :...

0

-1

-2

-3

:

123...



I It remains to define addition, multiplication and order. For a, b, c, d ∈ N0 wedefine an addition +Z , a multiplication ·Z and an order ≤Z as follows:

[(a, b)]∼=Z +Z [(c, d)]∼=Z := [(a + c, b + d)]∼=Z

[(a, b)]∼=Z ·Z [(c, d)]∼=Z := [(a · c + b · d , a · d + b · c)]∼=Z

[(a, b)]∼=Z ≤Z [(c, d)]∼=Z :⇔ a + d ≤ b + c

I It is easy to show thatI addition, multiplication and order are well-defined,I the standard rules of arithmetic hold, with [(0, 0)]∼=Z as zero element (“zero”),I ≤Z defines a total order on N0 × N0.

Definition 28 (Positive/negative)

An integer is positive if it is greater than zero and negative if it is less than zero;zero is neither positive nor negative.

Theorem 29

Z is a countably infinite set.


Divisibility Rules

Lemma 32A number a ∈ N is divisible by

2. if its last digit is even;

3. if the sum of its digits is divisible by three;

4. if its last two digits form a number that is divisible by four;

5. if its last digit is 0 or 5;

6. if it is divisible by two and three;

8. if the hundreds digit is even and the number formed by the last two digits isdivisible by eight, or if the hundreds digit is odd and the number formed by thelast two digits plus four is divisible by eight;

9. if the sum of its digits is divisible by nine;

10. if its last digit is 0;

11. if the alternating sum of its digits is divisible by eleven;

12. if it is divisible by three and four.

I There also exist divisibility rules for 7 but all of them are a bit ackward . . .


Divisibility Rules

Proof of Lem. 32 : We prove only the divisibility by three. Let n ∈ N anda0, a1, . . . , an ∈ 0, 1, . . . , 9 such that

a =n∑

i=0

ai · 10i .

We get

a =n∑

i=0

ai · 10i =n∑

i=0

ai · (10i − 1) +n∑

i=0

ai .

Since

3 |

(n∑

i=0

ai · (10i − 1)

),

the number a is divisible by three if and only if

3 |

(n∑

i=0

ai

).


Prime Numbers

Definition 33 (Prime, Dt.: Primzahl)

A natural number p ∈ N is a prime number, or is prime, if p ≥ 2 and if p is divisibleonly by 1 and p itself. All other numbers p ≥ 2 are called composite.

I The number 1 is no prime number!I The only even prime number is 2.I All primes greater than 2 are odd numbers.I The set of all prime numbers is frequently (but not always) denoted by P.

Definition 34 (Prime factor, Dt.: Primfaktor )

A natural number p ∈ N is a prime factor of n ∈ N if p is prime and p | n. If p is aprime factor of n then its multiplicity (Dt.: Vielfachheit) is the largest exponent k forwhich pk | n.


Prime Numbers

Lemma 35Let k ∈ N and a1, a2, . . . , ak ∈ Z and p ∈ P. Then

p | a1 · a2 · . . . · ak ⇔ (∃(1 ≤ j ≤ k) p | aj ).

Corollary 36

If two products of primes are identical then the primes are identical up to the orderin which they appear in the products.


Fundamental Theorem of Arithmetic

Theorem 37

Every natural number n > 1 is representable uniquely in the form

n = pm11 · p

m22 · . . . · p

mkk ,

where p1 < . . . < pk are primes and mj ∈ N are multiplicities for every j = 1, . . . , k .

Corollary 38

There are infinitely many prime numbers.

Proof attributed to Euclid of Alexandria : Suppose that p1 < p2 < . . . < pk are all theprimes. Let

n := p1 · p2 · . . . · pk + 1.

We have n ∈ N and n > 1. By Thm. 37, pj | n for some j = 1, . . . , k . This impliespj | (n − p1 · p2 · . . . · pk ), but n − p1 · p2 · . . . · pk = 1, and we get a contradiction.


More on Prime Numbers

Definition 39 (Mersenne prime)

A Mersenne number is of the form 2n − 1 for n ∈ N. A Mersenne prime is aMersenne number which is prime.

I Mersenne primes are used by the Mersenne twister, a pseudo-random numbergenerator developed in 1997 by Makoto Matsumoto and Takuji Nishimura.

I Several unsolved problems related to Mersenne numbers:I We have 211 − 1 = 2047 = 23 · 89. Thus, not all Mersenne numbers are

primes!I Are there infinitely many Mersenne primes? As of 07-December-2018, only

51 Mersenne primes are known, with 282 589 933 − 1 being the largest knownprime. (It was discovered by the “Great Internet Mersenne Prime Search”,www.mersenne.org.)

I What is a sufficient condition on n for 2n − 1 to be prime?

Lemma 40

If 2n − 1 is prime for some n ∈ N then n is prime.


www.mersenne.org

Chances to Become Famous: Conjectures About Primes

Conjecture 41 (Goldbach 1742, “weak version” or “ternary conjecture” )

Every odd natural number greater than 5 can be written as the sum of three primes.

Conjecture 42 (Goldbach-Euler 1742, “strong version” )

Every even natural number greater than 3 can be written as the sum of two primes.

I Christian Goldbach (1690–1764), Leonhard Euler (1707–1783).I The strong version of this conjecture implies the weak version: If n ∈ N, with

n ≥ 7, is odd then n′ := n − 3 is even with n′ > 3. Hence, if n′ can be written asthe sum of two primes, then n can be written as the sum of three primes.

I In 1937, Vinogradov proved the weak version for "sufficiently large numbers", andlater on his student Borozdin proved 3315

to be sufficiently large.I By means of distributed computer search, as of Dec 2012, Tomás Oliveira e Silva

verified the strong version of Goldbach’s conjecture up to 4 · 1018.I Also by distributed computing, in 2013 Harald Helfgott and David Platt verified

the weak Goldbach conjecture up to (roughly) 8 · 1030.I In 2013, Harald Helfgott released papers that, if accepted as correct, yield a

formal proof of the weak conjecture for all natural numbers greater than ≈ 1030.


Chances to Become Famous: Conjectures About Primes

Conjecture 43 (Polignac, 1849)

For every natural number k there exist infinitely many numbers p such that p andp + 2k are primes.

I For k := 1, the conjecture by Alphonse de Polignac (1817–1890) is known as thetwin prime conjecture. As of 19-Sep-2016, the largest known twin primes are2 996 863 034 8951 .290 000 ± 1; these numbers have 388 342 digits.

I In April 2013, Yitang Zhang proved that their exist infinitely many prime numberspn+1 and pn such that pn+1 − pn < 7 · 107.

I In November 2013, James Maynard reduced this bound to 600.I This bound seems to have been further reduced to 246 by the Polymath project

led by Terence Tao.


A Chance Missed to Become Famous: Fermat’s Last Theorem

I A Diophantine equation is an equation for which only integer solutions are sought.I E.g., (3, 4, 5) is an integer solution triple for a2 + b2 = c2.

Theorem 44 (Wiles&Taylor, 1995)

For every natural number n > 2, the Diophantine equation an + bn = cn has nosolution over N.

I Dt.: Großer Satz von Fermat.I Stated in 1637 by Pierre de Fermat (1607(?)–1665) without proof, but with a

famous side remark: "Cuius rei demonstrationem mirabilem sane detexi. Hancmarginis exiguitas non caperet."

I Proved for n = 4 by Fermat himself.I Finally proved by Andrew Wiles in 1993; a gap in the proof was fixed by Wiles

and his former student Richard Taylor; the full proof was published in 1995.


Quotient and Remainder

Lemma 45Let a ∈ N and b ∈ Z. Then there exist a unique quotient q ∈ Z and a uniqueremainder r ∈ N0 such that

b = aq + r and 0 ≤ r < a.

I We will use the operators div and mod for computing the divisor and remainder.That is, q and r of Lemma 45 are given by q := b div a and r := b mod a.

I The modulo of powers of 2 can be expressed as a bitwise AND operation:b mod 2n == b & (2n − 1).

I In many programming languages the remainder r can be obtained by means ofthe modulo operator. See, e.g., the operator % in C, C++, C#, Java, and Perl.

WarningIf one or both of a and b are allowed to be negative integers then the sign of theremainder may differ among different implementations!


Real-World Application: Base Conversion

I We know that 25 = (11001)2, i.e., (11001)2 is the base-two representation of25 = (25)10. (After all, 25 = 1 · 24 + 1 · 23 + 0 · 22 + 0 · 21 + 1 · 20.)

I How can we represent an integer relative to an arbitrary base b ∈ N \ 1?I Lemma 45 tells us that there exist unique q0, r0 ∈ N0 such that

n = bq0 + r0 with 0 ≤ r0 < b.

I The number r0 becomes the rightmost digit of the base-b representation of n, andwe seek q1, r1 such that

q0 = bq1 + r1 with 0 ≤ r1 < b,

and so on until some qi = 0.I E.g.,

25 = 12 · 2 + 112 = 6 · 2 + 06 = 3 · 2 + 03 = 1 · 2 + 11 = 0 · 2 + 1

and therefore 25 = (11001)2.


Congruences

I Introduced by Carl Friedrich Gauss (1777–1855) in 1801.

Definition 46 (Congruence, Dt.: Kongruenz)Let a, b ∈ Z and m ∈ N. We say that a is congruent to b modulo m, and write

a ≡m b,

if a− b is divisible by m. The term a ≡m b is called a congruence.

I Hence: a ≡m b :⇔ m | (a− b).I If a ≡m b then a and b have the same remainder after dividing by m.

That is, a mod m = b mod m.I Note: Some authors prefer to write a ≡ b (m) or a ≡ b mod m for a ≡m b.

Definition 47 (Even/odd, Dt.: gerade/ungerade)

A number n ∈ Z is said to be odd if and only if n ≡2 1; otherwise, n is even.


Congruences

38 ≡12 2 even + even ≡2 even

−3 ≡5 2 even + odd ≡2 odd

0 ≡3 3 odd + odd ≡2 even

8 ≡3 2 even · even ≡2 even

7 ≡3 1 even · odd ≡2 even

7 ≡3 −8 odd · odd ≡2 odd

Lemma 48

For m ∈ N, the relation ≡m is an equivalence relation on Z, i.e., for all a, b, c ∈ Z,

reflexivity a ≡m a,

symmetry if a ≡m b then b ≡m a, and

transitivity if a ≡m b and b ≡m c then a ≡m c

hold.


Residues and Zm

Lemma 49For m ∈ N, the relation ≡m is a congruence relation on Z, i.e., it respects addition,subtraction, and multiplication: Let a, b, c, d ∈ Z and m ∈ N, and suppose that

a ≡m b and c ≡m d .

Then

a + c ≡m b + d and a− c ≡m b − d and a · c ≡m b · d .

Definition 50 (Residue, Dt.: Residuum, Restklasse)

Let m ∈ N with m ≥ 2. The equivalence classes of Z modulo m are called residues(or remainders) modulo m. For a ∈ Z, its equivalence class modulo m is denoted by[a]m. The set of residues modulo m is denoted by Zm or Z/mZ.

Lemma 51

Let m ∈ N with m ≥ 2. Then Zm = [a]m : a ∈ N0 ∧ a < m.


Residues and Zm: Modulo Arithmetic

Definition 52 (Arithmetic on Zm)

Let m ∈ N with m ≥ 2, and [a]m, [b]m ∈ Zm. On Zm we define an addition +m and amultiplication ·m as follows.

[a]m +m [b]m := [a + b]m

[a]m ·m [b]m := [a · b]m

Lemma 53Let m ∈ N with m ≥ 2. Then addition +m and multiplication ·m on Zm arewell-defined. Furthermore, (Zm,+m, ·m) forms a commutative ring.

I Note: Often the notation [a]m is simplified by omitting the modulus m, i.e., bywriting [a], or even by simply writing a if it is clear that a ∈ Zm.

I It is also common to write

a mod m

instead of

[a]m.


Sample Application of Residues

Lemma 54

For any n ∈ N and any set of n distinct natural numbers a1, a2, . . . , an ⊂ N, thesum of a (suitable) subset of these numbers is divisible by n.

Proof : Let n be arbitrary but fixed and consider n arbitrary (but fixed and distinct)natural numbers a1, a2, . . . , an. We define n numbers b1, b2, . . . , bn ∈ N0 as follows:

bk := (k∑

j=1

aj ) mod n for k = 1, 2, . . . , n.

If one of the numbers bk is zero then we are done. Otherwise, we have

b1, b2, . . . , bn ⊆ 1, 2, . . . , n − 1.

By the pigeonhole principle, there exist i 6= j with 1 ≤ i, j ≤ n such that bi = bj .W.l.o.g, i < j . However, then

(ai+1 + ai+2 + · · ·+ aj ) mod n = 0, i.e., (ai+1 + ai+2 + · · ·+ aj ) ≡n 0,

thus settling our claim.


Real-World Application: Pseudo-Random Numbers

I Since computers cannot flip a coin to obtain a random result, one resorts toalgorithms that generate “random” numbers: pseudo-random number generators.

I Linear congruential generators (LCG, [Lehmer 1954]) have been well studied,are easy to implement and used frequently.

I They generate a sequence of non-negative integers less than some specifiedmodulus m ∈ N according to the following recursive definition:

xn+1 := (a · xn + c) mod m,

where

m ∈ N with m > 1 . . . . . . . . . modulus,a ∈ N with a < m . . . . . . . . . multiplier,c ∈ N0 with c < m . . . . . . . . . increment,x0 ∈ N0 with x0 < m . . . . . . . . . seed.

I E.g., m := 15, a := 1, c := 4 and x0 := 2 yields the following sequence ofnumbers:

2 6 10 14 3 7 11 0 4 8 12 1 5 9 13 2 6 . . .

I GCC/glibc: m := 231 − 1, a := 1103515245, c := 12345. More advancedpseudo-random number generators exist, e.g., Mersenne twister.


Greatest Common Divisor

Lemma 55Let a, b ∈ N. Then there exists a unique n ∈ N such that

1. n | a and n | b, and

2. for all m ∈ N, if m | a and m | b then m ≤ n.

Definition 56 (Greatest common divisor, Dt.: größter gemeinsamer Teiler (ggT))Let a, b ∈ N. The unique number n ∈ N that exists according to Lem. 55 is calledgreatest common divisor of a and b, and is denoted by gcd(a, b). Conventionally,gcd(a, 0) = gcd(0, a) := a, since 0 is divisible by all natural numbers.

Definition 57 (Relatively prime, Dt.: teilerfremd, relativ prim)

The numbers a, b ∈ N are relatively prime, or coprime, if gcd(a, b) = 1.

Definition 58 (Pairwise relatively prime)

A set S of natural numbers is called pairwise relatively prime (or pairwise coprimeor mutually coprime) if all pairs of numbers a and b in S, with a 6= b, are relativelyprime.


Greatest Common Divisor

Lemma 59 (Bézout’s Identity )

Let a, b ∈ N. Then there exist x , y ∈ Z such that gcd(a, b) = ax + by . Conversely,the smallest positive number ax + by , for all x , y ∈ Z, equals gcd(a, b).

I Lemma 59 was first stated by Étienne Bézout (1730–1783), and numbersx , y ∈ Z with gcd(a, b) = ax + by are called Bézout numbers.

I Note: Bézout numbers are not unique! For instance, gcd(10, 15) = 5, and10x + 15y = 5 has the solutions x = −1 and y = 1, and x = 2 and y = −1.

I For a, b, d ∈ Z given, the identity d = ax + by over Z× Z is called a linearDiophantine equation in x and y .

Corollary 60

The numbers a, b ∈ N are relatively prime if and only if the linear Diophantineequation ax + by = 1 has a solution, i.e., if and only if there exist x , y ∈ Z such thatax + by = 1.


Euclidean Algorithm for GCD Computation

Theorem 61 (Euclidean Algorithm)

The following algorithm computes gcd(a, b) for a, b ∈ N0 with a > b.

function gcd(a, b)precondition: a, b ∈ N0 with a > b.postcondition: t = gcd(a, b)

while b > 0 dot ← bb ← a mod ba← t

end whilet ← a


Euclidean Algorithm for GCD Computation: Sample Run

function gcd(a, b)precondition: a, b ∈ N0 with a > b.postcondition: t = gcd(a, b)

while b > 0 dot ← bb ← a mod ba← t

end whilet ← a

I We want to compute the gcd of 78 and 99. Hence, b := 78 anda := 99 = 1 · 78 + 21. We get after different passes through the loop:

after 1st pass: t = 78, b = 21, a = 78 = 3 · 21 + 15after 2nd pass: t = 21, b = 15, a = 21 = 1 · 15 + 6after 3rd pass: t = 15, b = 6, a = 15 = 2 · 6 + 3after 4th pass: t = 6, b = 3, a = 6 = 2 · 3 + 0after 5th pass: t = 3, b = 0, a = 3

I Hence, t = 3 = gcd(78, 99).


Does (Zm,+m, ·m) Form a Field?

Theorem 62

Let m ∈ N with m ≥ 2. An element [a]m ∈ Zm has a multiplicative inverse if and onlyif a is relatively prime to m.

Corollary 63

Let m ∈ N with m ≥ 2. The ring (Zm,+m, ·m) forms a (finite) field if and only if m isprime.

Lemma 64

Let m ∈ N with m ≥ 2 and [a]m ∈ Zm such that m and a are relatively prime. Letx , y ∈ Z such that ax + my = 1. Then [a]m ·m [x ]m = [1]m, i.e., [x ]m is themultiplicative inverse element for [a]m.


Extended Euclidean Algorithm for GCD Computation

Theorem 65 (Extended Euclidean Algorithm)

The following algorithm computes x , y ∈ Z and d ∈ N such thatgcd(a, b) = d = ax + by for a, b ∈ N with a > b.

function gcd_extended(a, b)precondition: a, b ∈ N0 with a > b.postcondition: (d , x , y) ∈ N× Z× Z such that gcd(a, b) = d = ax + by

if (a mod b) = 0 thenreturn (b, 0, 1)

else(d , x , y)← gcd_extended(b, a mod b)return (d , y , x − y · (a div b))

end if


Extended Euclidean Algorithm for GCD Computation: Sample Run

function gcd_extended(a, b)postcondition: (d , x , y) ∈ N× Z× Z such that gcd(a, b) = d = ax + by

if (a mod b) = 0 thenreturn (b, 0, 1)

else(d , x , y)← gcd_extended(b, a mod b)return (d , y , x − y · (a div b))

end if

I We want to compute x , y ∈ Z and d ∈ N such that gcd(99, 78) = d = 99x + 78y .

a b a div b a mod b d x y99 78 1 21 3 −11 1478 21 3 15 3 3 −1121 15 1 6 3 −2 315 6 2 3 3 1 −2

6 3 – 0 3 0 1

I Hence, gcd(99, 78) = −11 · 99 + 14 · 78 = −1089 + 1092 = 3.


Chinese Remainder Theorem

I Old Chinese folk tale: A Chinese Emperor used to count his army after a battleby ordering them to form groups of different sizes:

1. The soldiers should form groups of 3 and report back the number of soldiersthat could not join a group consisting of 3 soldiers.

2. Then the soldiers should form groups of 5 and report back the number ofsoldiers that could not join a group consisting of 5 soldiers.



5. · · ·I Based on this information he was able to figure out the number n of soldiers in

his army.I Indeed, a mathematical solution was provided by the Chinese mathematician Sun

Tzu sometime in the third to fifth century, and republished by Qin Jiushao in 1247!



n mod 3 = 1 n mod 5 = 2 n mod 7 = 2



Theorem 66 (Chinese Remainder Theorem, Dt.: Chinesischer Restsatz)

If, for some k ∈ N, the numbers m1,m2, · · · ,mk ∈ N are pairwise relatively prime,then the following system of simultaneous congruences has an integer solution bfor a1, a2, . . . , ak ∈ Z given:

b ≡m1 a1

b ≡m2 a2...

b ≡mk ak

(∗)

Furthermore, all solutions of (∗) are congruent modulo m :=∏k

i=1 mi . That is, thesolution is unique if constrained to 1, 2, . . . ,m.


Constructive Proof of Chinese Remainder Theorem 66

Proof : We show the existence of an integer solution. Consider i ∈ N with 1 ≤ i ≤ k .Since m1,m2, · · · ,mk are pairwise relatively prime, gcd( m

mi,mi ) = 1. Using the

extended Euclidean algorithm (Thm. 65), we can find integers xi and yi such that

xi ·mi + yi ·mmi

= 1. (?)

Let bi := yi · mmi

. Equation (?) guarantees that the remainder of bi when divided by mi

is 1. On the other hand, for i 6= j every mj divides bi evenly. Thus,

bi ≡mi 1 and bi ≡mj 0 for all j with i 6= j, 1 ≤ j ≤ k .

Since congruences respect multiplication, we get

ai · bi ≡mi ai and ai · bi ≡mj 0 for all j with i 6= j, 1 ≤ j ≤ k .

Thus, one solution of the simultaneous congruences is given by

b :=k∑

i=1

aibi .


Helping the Emperor

I The Emperor collected the following information:I When the soldiers formed groups of 3, one soldier was left out.I When the soldiers formed groups of 5, two soldiers were left out.I When the soldiers formed groups of 7, again two soldiers were left out.

I That is, since a1 = 1, a2 = 2, a3 = 2 and m1 = 3,m2 = 5,m3 = 7 andm = 3 · 5 · 7 = 105:

n ≡3 1 n ≡5 2 n ≡7 2

I Hence, we are to find x1, x2, x3, y1, y2, y3 ∈ Z such that

3x1 + 35y1 = 1 5x2 + 21y2 = 1 7x3 + 15y3 = 1.

I We have x1 := 12, y1 := −1, x2 := −4, y2 := 1, x3 := −2, y3 := 1 and, thus,

n = (35 · (−1) · 1 + 21 · 1 · 2 + 15 · 1 · 2) mod 105 = 37 mod 105 = 37.


Real-World Application: Secret Sharing

I Secret sharing refers to the distribution of information related to a secret (e.g., anumber) among a group of receivers such that the secret can only bereconstructed if all or, at least, a large percentage of the receivers cooperate.

I Ideally, the information received by one individual receiver shall be of no (or verylittle) help for the receiver to obtain the secret without the help of the others.

I A secret sharing scheme is called a (t , n) threshold scheme, or t-out-of-nscheme, if at least t of the n receivers have to cooperate. (Of course, t ≤ n.)

I Typically, t is large relative to n but not identical to n.I Several different variants of schemes for secret sharing are used in practice.I At least two published schemes rely on the Chinese Remainder Theorem 66.I We sketch the very basic idea of a scheme based on the Chinese Remainder

Theorem 66. (In our simple scheme we have t = n.)



I Suppose that the number 1234 is the secret to be shared by five receivers.I We choose

m1 := 2, m2 := 3, m3 := 5, m4 := 7, m5 := 11.

I Note that

m :=5∏

i=1

mi = 2 · 3 · 5 · 7 · 11 = 2310 > 1234.

I Now consider ai := 1234 mod mi , for 1 ≤ i ≤ 5. This gives us the numbers

a1 = 0, a2 = 1, a3 = 4, a4 = 2, a5 = 2.

I The numbers mi and ai are passed to the i-th receiver.I Note that each individual receiver has gained little information about the original

secret.I Rather, in our simple approach, all five receivers need to cooperate in order to

recover the secret: They have to solve the following set of five congruences:

b ≡2 0 b ≡3 1 b ≡5 4 b ≡7 2 b ≡11 2



I The five receivers have to solve the following set of five congruences:

b ≡2 0 b ≡3 1 b ≡5 4 b ≡7 2 b ≡11 2

I Since a1 = 0, we need to solve only four congruences and get the following fourDiophantine equations.

3x2+770y2 = 1 5x3+462y3 = 1 7x4+330y4 = 1 11x5+210y5 = 1

I Solving these equations yields the following solutions:

x2 = 257, y2 = −1 x3 = 185, y3 = −2 x4 = −47, y4 = 1 x5 = −19, y5 = 1

I Hence, the secret sought is recovered as

b = (−1) · 770 · 1− 2 · 462 · 4 + 1 · 330 · 2 + 1 · 210 · 2 = −3386 ≡2310 1234.


Real-World Application: Arithmetic with Large Integers

I Standard integer arithmetic cannot handle arbitrarily large integers.I One way to carry out complex integer arithmetic with large integers is to apply

modulo arithmetic and the Chinese Remainder Theorem 66:1. Select k modules m1,m2, . . . ,mk ∈ N \ 1 which are relatively prime, for

some k ∈ N.2. Let m := m1 ·m2 · . . . ·mk .3. Represent an integer n < m by its k remainders n1, n2, . . . , nk upon division

by m1,m2, . . . ,mk .4. Perform the arithmetic operations of your algorithm on these remainders,

with the calculations involving ni being carried out modulo mi .5. Recover the actual result by applying the Chinese Remainder Theorem 66.

I This approach works as long as all intermediate results are less than m.I Advantages:

I One can use (mostly) standard arithmetic to handle integers larger thanthose normally handled.

I One can run the computations for the different remainders in parallel, thusspeeding up the computation.

I Standard choices for the modules are numbers of the form 2i − 1:I One can prove gcd(2i − 1, 2j − 1) = 2gcd(i,j) − 1, which makes it easy to

ensure that the modules are relatively prime.


Real-World Application: Arithmetic with Large Integers

I Suppose that we want to limit our arithmetic operations to numbers less than 12.I We choose the five modules

m1 := 2, m2 := 3, m3 := 5, m4 := 7, m5 := 11.

and remember that m := m1 ·m2 ·m3 ·m4 ·m5 = 2310.I Hence, we can deal with numbers less than 2310.I Recall that n := 1234 can be represented by the five remainders (0, 1, 4, 2, 2).I Similarly, 1000 can be represented by the five remainders (0, 1, 0, 6, 10).I We get

(0, 1, 4, 2, 2) + (0, 1, 0, 6, 10) = (0, 2 mod 3, 4 mod 5, 8 mod 7, 12 mod 11)= (0, 2, 4, 1, 1).

I Thus, b := 1234 + 1000 is uniquely determined as the solution of the followingset of five congruences:

b ≡2 0 b ≡3 2 b ≡5 4 b ≡7 1 b ≡11 1


Rational Numbers: Q

Definition 67 (Rational equivalence)

On Z× N we define the binary relation ∼=Q as follows:

(p1, q1) ∼=Q (p2, q2) :⇔ p1q2 = p2q1.

Lemma 68The relation ∼=Q is an equivalence relation on Z× N.

Definition 69 (Rational numbers)

The rational numbers Q are defined as

Q := [(p, q)]∼=Q : p ∈ Z, q ∈ N.

The canonical representative of [(p, q)]∼=Q is denoted by p′

q′ , wherep′ := p div gcd(p, q) and q′ := q div gcd(p, q).


Rational Numbers: Q

I We have N ⊂ Q, since for every n ∈ N the fraction n1 belongs to Q.

I It is easy to define an addition +, multiplication · and order ≤ on Q that turns(Q,+, ·) into a totally ordered field. E.g.,

[(p1, q1)]∼=Q +∼=Q [(p2, q2)]∼=Q := [(p1q2 + p2q1, q1q2)]∼=Q

I Of course, it is standard to simplify the notation and write

pq

instead of [(p, q)]∼=Q.

But keep in mind that fractions are equivalence classes. Thus,

13∼=Q

26∼=Q

39∼=Q

20006000

.

I In the sequel we resort to standard knowledge and deal with rational numbers aswe learned in school. (However, this could be formalized, based on Def. 69!)


Q Is Dense in R But Still Countably Infinite!

Lemma 70There exists a rational number between any two distinct rational numbers.

Theorem 71

Q is everywhere dense in R. That is, for every x ∈ R, every arbitrarily smallneighborhood of x contains a rational number.

Theorem 72

The equation x2 = 2 has no solution over Q.

Proof : Suppose that there exist x ∈ Q such that x2 = 2. Hence, there exist p ∈ Z andq ∈ N such that

gcd(p, q) = 1 and 2 =

(pq

)2

=p2

q2 .

This equation is equivalent to 2q2 = p2, implying that p2 ≡2 0, and, thus, also p ≡2 0.This in turn implies q2 ≡2 0, and, therefore, also q ≡2 0. We have a contradiction togcd(p, q) = 1.


Q Is Dense in R But Still Countably Infinite!

Theorem 73

Q is a countably infinite set.

Proof by Cantor : Construct a bijection between N and Z× N (as a “superset” of Q) .1 2 3 4 · · ·12

22

32

42 · · ·

13

23

33

43

· · ·14

24

34

44

· · ·

......

......

. . .

This gives the sequence 1, 2, 12 ,

13 ,

22 , 3, . . .. Now start with zero and include every

number’s negative number, thus obtaining a systematic order of Z× N:

0 1 − 1 2 − 2 12 − 1

213 − 1

322 − 2

2 3 − 3 . . .

Numbering this sequence yields a bijection from N onto Z× N.


Real Numbers: R

I Intuitively, the reals comprise both rational and irrational numbers like√

2 or π.I A formal introduction of the reals based on N and Q — e.g., based on Dedekind

cuts or based on equivalence classes of Cauchy sequences — is beyond thescope of this lecture.

I Convenient notations for intervals of real numbers:∀a, b ∈ R [a, b] := x ∈ R : a ≤ x ≤ b;∀a, b ∈ R ]a, b[ := x ∈ R : a < x < b;∀a, b ∈ R [a, b[ := x ∈ R : a ≤ x < b;∀a, b ∈ R ]a, b] := x ∈ R : a < x ≤ b.

I Note: Some authors prefer to denote the open interval ]a, b[ by (a, b).I Floor and ceiling function (Dt.: Ab- und Aufrundungsfunktion): For x ∈ R,

bxc := maxk ∈ Z : k ≤ x,

dxe := mink ∈ Z : k ≥ x.I Gauß introduced the square-bracket notation [x ] (“Gaussklammer”) in 1808. The

names “floor” and “ceiling” and the corresponding notations were introduced byIverson in 1962 in his book on APL.

I We have [x ] = bxc for all x ∈ R.


Decimal Notation

Definition 74 (Decimal representation, Dt.: Dezimalzahl)

A real number x ∈ R+0 is in decimal representation (or a decimal number ) if it is

represented as a sum of (negative) powers of ten:

x = x0+∞∑i=1

xi

10i , with an integer part x0 ∈ N0 and with 0 ≤ xi ≤ 9 for all i ∈ N.

The decimal representation is finite if, for some n0 ∈ N0, we have xi = 0 for alli ≥ n0.

I By definition,

x = x0 + limn→∞

n∑i=0

xi

10i .

I It is straightforward to extend Def. 74 to negative reals.


Decimal Notation

Definition 75 (Decimal separator )The decimal separator is a symbol which is used to mark the boundary between theinteger part and the fractional part of a number in decimal representation.

WarningA least two symbols are in wide-spread use for the decimal separator!

I Most of Europe, most of South America and French Canada use the comma,while the UK, USA, Australia, English Canada and several Asian countries use adot (“period”, “full stop”). The dot also prevails in English-language publications.

I Dots or commas are frequently used to group three digits into groups within theinteger part. However, this practice is discouraged by ISO!

I Note: The decimal representation is not unique: we have 1.0 = 0.9999 . . ., wherethe ellipsis “. . .” represents an infinite sequence of the digit 9.

I In fact, every non-zero, finitely represented decimal number has an alternaterepresentation with trailing 9s, such as 123.4567 and 123.4566999 . . .

I Of course, the finite representation is (almost) always the preferredrepresentation.


Decimal Notation

Definition 76 (Recurring decimal, Dt.: periodische Dezimalzahl)

A decimal representation of a real number is a recurring decimal (or repeatingdecimal) if it becomes periodic at some point: a finite subsequence of the digitsafter the decimal separator is repeated indefinitely.

I Recurring decimals, e.g.,

13

= 0.333 · · ·

or

17

= 0.142857142857142857 · · ·

are written as 0.3 or 0.3, and 0.142857. (The horizontal line is known asvinculum.)

Lemma 77

A real number has a finite or recurring decimal representation if and only if it is arational number.


Decimal Notation: Conversion to a Fraction

I We proceed as follows to convert 0.4321 to a rational number.I Let x := 0.0021. Then 100x = 0.21.I This gives

99x = 100x − x = 0.21− 0.0021 = 0.21 =21

100.

I We get

x =21

9900=

73300

.

I Hence,

0.4321 = 0.43 + x =43

100+

73300

=14263300

=713

1650.


The Reals are Not Countable

Theorem 78

The real numbers are uncountable, i.e., there exists no bijection from N onto R.

Proof by Cantor (1891) : Suppose to the contrary that there exists a bijectiona : N→ R. We show that we can construct a number r which is not in the lista1, a2, a3, . . .: For k ∈ N let dk be the k -th digit after the decimal separator in ak if ak

has at least k digits after the decimal separator, and dk := 0 otherwise.

a1 = —.d1 – – – – . . .a2 = —.– d2 – – – . . .a3 = —.– – d3 – – . . .

...

If dk = 1 then rk := 2 else rk := 1. Now regard rk as the k -th digit of a number r ∈ R:we have r = 0.r1r2r3r4 . . .. Since at least the k -th digit of r differs from the k -th digit ofak , we conclude that r 6= an for all n ∈ N.

I Arguments of this form are called diagonal arguments.I Cantor proved |R| = 2ℵ0 > ℵ0 = |N|.I Continuum hypothesis: There is no set with cardinality strictly between that of the

integers and the reals. (This is a hypothesis, but not a theorem!)


Well-Ordering the Reals

I By definition, (N,≤) is well-ordered. And we have already hinted atwell-orderings for Z and Q.

I Question: Can the reals be well-ordered?I Answer: We don’t know it for sure!I It has been proved that it is impossible to write down an explicit well-ordering for

the reals.

Well-Order “Theorem”Every set can be well-ordered.

I In 1883, Georg Cantor stated that the Well-Order Theorem is a "fundamental lawof thought". This statement started a mathematical flame war!

I In any case, this “theorem” can only be taken as an axiom, since it has beenproved that it does not follow from any of the other commonly accepted axioms ofset theory.

I In first-order logic, the Well-Order Theorem is equivalent to the Axiom of Choice(Dt.: Auswahlaxiom) and to Zorn’s Lemma, in the sense that either one of themtogether with the Zermelo-Fraenkel Axioms allows to deduce the other ones.


Well-founded Order

Definition 79 (Well-founded order, Dt.: wohlfundierte Ordnung)

A strict partial order ≺ on M is called well-founded if every X ⊆ M, with X 6= /0, hasat least one minimal element relative to ≺. A poset (M,≺) is called a well-foundedposet if ≺ is well-founded.

I Of course, (N, <) is well-founded.

Lemma 80

The poset (M,≺) is well-founded if and only if no infinite strictly decreasingsequence in M exists, i.e., if an a : N→ M with ai+1 ≺ ai for all i ∈ N does not exist.

I Some authors call a well-founded order also a Noetherian order, named afterEmmy Noether (1882-1935).

I Not to be confused with a well-order (Dt.: Wohlordnung).


Lexicographical Order

Definition 81

Let (M1,≺1) and (M2,≺2) be two posets. The lexicographical ordering (≺1,≺2)lex

on M1 ×M2 is defined as

(a1, b1) (≺1,≺2)lex (a2, b2) :⇔ ((a1 ≺1 a2) ∨ ((a1 = a2) ∧ (b1 ≺2 b2))),

where (a1, b1), (a2, b2) ∈ M1 ×M2.

Lemma 82

Let (M1,≺1) and (M2,≺2) be two posets. Then M1 ×M2 together with thelexicographical order (≺1,≺2)lex is a poset.

I Similarly for a non-strict partial order .

Lemma 83

The posets (M1,≺1) and (M2,≺2) are well-founded if and only if(M1 ×M2, (≺1,≺2)lex ) is well-founded.


Induction Revisited

I Consider a predicate P over N and recall the Strong Induction Principle (Thm 21):If P(1) and if

∀k ∈ N [(∀(m ∈ N,m ≤ k) P(m)) ⇒ P(k + 1)]

then

∀n ∈ N P(n).


Induction Revisited

I And yet another version with “implicit” base:If

∀k ∈ N [(∀(m ∈ N,m < k) P(m)) ⇒ P(k)]

then

∀n ∈ N P(n).

I Note: The base case was not lost! Rather, it is included since we have to proveP(1) using the “helpful knowledge” that P(m) holds for all m ∈ N with m < 1.


Well-founded Induction

Theorem 84 (Principle of Well-founded Induction, Dt.: wohlfundierte Induktion)

Let (M,≺) be well-founded and P be a predicate on M.If

∀k ∈ M [(∀(m ∈ M,m ≺ k) P(m)) ⇒ P(k)]

then

∀m ∈ M P(m).

I That is, as inductive step we have to prove that the predicate holds for k if it holdsfor all predecessors m of k relative to ≺.

Proof : Let X := m ∈ M : P(m) is false, and suppose X 6= /0. Since (M,≺) iswell-founded, X has a minimal element n. Thus, ∀m ∈ M with m ≺ n the predicateP(m) holds. The inductive step

∀(m ∈ M,m ≺ n) P(m) ⇒ P(n)

yields that P(n) holds, in contradiction to n ∈ X .


Sample Well-founded Induction

I We give a proof of the existance claim made by the Fundamental Theorem ofArithmetic (Thm. 37): Every natural number n > 1 is either a prime number orhas a prime factorization.

Proof : We begin with observing that the relation “is genuine divisor of” (Def. 30) overN \ 1 is well-founded. The minimal elements relative to this relation are the primes.

We consider an arbitrary but fixed k ∈ N \ 1 and assume as inductive hypothesisthat the claim holds for all m ∈ N \ 1 that are smaller than k relative to this order.

Of course, if k is prime then the claim given by the theorem holds.So suppose that k is not prime. By definition of primality, this means that there existm1,m2 ∈ N \ 1 such that k = m1 ·m2.

Now we have

m1 is genuine divisor of k and m2 is genuine divisor of k .

Hence, both m1 and m2 are predecessors of k . By the inductive hypothesis, we knowthat m1 is either prime or has a prime factorization; same for m2. Thus, also k has aprime factorization, which establishes the inductive step.


Real-World Application: Functional Completeness of NAND

I A NAND gate is a logic gate which produces an output that is false only if all itsinputs are true. That is, (p | q) ≡ ¬(p ∧ q) for two Boolean variables p, q.

Theorem 85 (Functional completeness of NAND)

Every formula of propositional logic (that contains at least one junctor) can beimplemented by using only a combination of NAND gates.

I Thus, any digital circuit can be realized by using only one type of gate: NANDs.(This is also true for the NOR inverter.)

Lemma 86

Let p, q denote two Boolean variables. The following logical equivalences hold:

¬p ≡ (p | p) (p ∧ q) ≡ ((p | q) | (p | q)) (p ∨ q) ≡ ((p | p) | (q | q))

(p ⇒ q) ≡ (¬p ∨ q) (p ⇔ q) ≡ ((p ⇒ q) ∧ (q ⇒ p))

> ≡ (p | (p | p)) ⊥ ≡ (> | >)



Proof of Thm. 85 : Recall that propositional formulas (over some fixed set of npropositional variables p1, p2, . . . , pn) follow a rigid construction scheme: Apropositional formula over the n propositional variables p1, p2, . . . , pn is constructedinductively from a set ofI junctors: ¬,∧,∨,⇒,⇔;I parentheses: (, );I constants (truth values): ⊥,> (or F ,T );

based on the following rules:I A propositional variable is a propositional formula.I The constants ⊥ and > are propositional formulas.I If φ1 and φ2 are propositional formulas then so are the following:

(¬φ1), (φ1 ∧ φ2), (φ1 ∨ φ2), (φ1 ⇒ φ2), (φ1 ⇔ φ2).



Proof of Thm. 85 continued : On the set of of propositional formulas overp1, p2, . . . , pn we define the following order ≺PL:

φ1 ≺PL φ0 :⇐⇒

φ0 =syn (¬φ1) orφ0 =syn (φ1 ∧ φ2) for a suitable φ2, orφ0 =syn (φ2 ∧ φ1) for a suitable φ2, orφ0 =syn (φ1 ∨ φ2) for a suitable φ2, orφ0 =syn (φ2 ∨ φ1) for a suitable φ2, orφ0 =syn (φ1 ⇒ φ2) for a suitable φ2, orφ0 =syn (φ2 ⇒ φ1) for a suitable φ2, orφ0 =syn (φ1 ⇔ φ2) for a suitable φ2, orφ0 =syn (φ2 ⇔ φ1) for a suitable φ2,

where =syn denotes syntactical equivalence, i.e., equivalence among strings ofcharacters.Easy to see: ≺PL is a partial order. If φ1 ≺PL φ0 then the number of junctors of φ1 isone smaller than the number of junctors of φ0. Hence, the order ≺PL is well-founded.



Proof of Thm. 85 continued : We use a well-founded induction:1. The minimal elements of ≺PL are given by the variables p1, p2, . . . , pn and the

constants ⊥ and >. Lem. 86 tells us that ⊥ and > can be expressed usingNANDs.

2. Consider an arbitrary but fixed propositional formula φ0 that contains at least onejunctor, and assume as inductive hypothesis that all formulas smaller than φ0

relative to ≺PL can be expressed using only NANDs (or are simply variables).

By the construction scheme of propositional formulas, the formula φ0 is of theform φ0 =syn (¬φ1) or φ0 =syn (φ1 #φ2), for suitable φ1, φ2 and where # is one ofthe junctors ∧,∨,⇔,⇒.

Since φ1 (and φ2) are smaller than φ0, the inductive hypothesis applies and φ1

(and φ2) can be expressed using only NANDs (or are simply variables).

By using the scheme outlined in Lem. 86, also φ0 can be expressed using onlyNANDs.


Principles of Elementary Counting and CombinatoricsSum and Product RuleInclusion-Exclusion PrincipleBinomial CoefficientPermutationsOrdered Selection (Variation)Unordered Selection (Combination)


Sum and Product Rule

Theorem 87 (Sum rule, Dt.: Additionsprinzip)

Let A,B be two finite sets with A ∩ B = /0. Then

|A ∪ B| = |A|+ |B|.

Corollary 88

For n ∈ N, let A1,A2, . . . ,An be n finite sets that are pairwise disjoint. Then

|A1 ∪ A2 ∪ . . . ∪ An| =n∑

i=1

|Ai |.

Theorem 89 (Product rule, Dt.: Multiplikationsprinzip)

Let A,B be two finite sets. Then

|A× B| = |A| · |B|.


Sum and Product Rule

Proof of Theorem 89 :I We observe that

A× B =⋃b∈B

(A× b), with (A× b1) ∩ (A× b2) = /0 if b1 6= b2.

I Furthermore, there exists a bijective mapping between A and A× b. Thus,|A| = |A× b|, and the theorem is a consequence of Corollary 88.

Corollary 90

For n ∈ N, let A1,A2, . . . ,An be n finite sets. Then

|A1 × A2 × . . .× An| =n∏

i=1

|Ai |.

Corollary 91

For a propositional formula that contains n variables, 2n evaluations are necessaryin order to test all possible combinations of truth assignments to its variables.


Characteristic Function and Cardinality of Power Set

Definition 92 (Characteristic function, Dt.: Indikatorfunktion)

Let A be a finite set, and B ⊆ A. The characteristic function indicates membershipof an element of A in B:

1B : A→ 0, 1, 1B(a) :=

1 if a ∈ B,0 if a 6∈ B.

Lemma 93

A finite set A has 2|A| many different subsets. That is, |P(A)| = 2|A|.

Proof : We observe that every subset of A, including /0 and A itself, has a one-to-onecorrespondance to a characteristic function. Thus, every subset of A corresponds to asequence of n 0’s and 1’s, where n := |A|. We conclude that the power set P(A) has2n members.

Lemma 94

Let A be a finite set, and B ⊆ A. Then |B| =∑

a∈A 1B(a).


Real-World Application: Counting Strings

I How many 3-element strings s can be formed over the standard alphabet — 26lower-case letters — such that every string contains at least one x?

I Obviously such an s is in exactly one of the following sets:

A1 := s : first x in first place of s,

A2 := s : first x in second place of s,

A3 := s : first x in third place of s.

I Applying the Product Rule 89 yields

|A1| = |x × a, b, . . . , z × a, b, . . . , z| = 26 · 26,

|A2| = |(a, b, . . . , z \ x)× x × a, b, . . . , z| = 25 · 26,

|A3| = |(a, b, . . . , z \ x)× (a, b, . . . , z \ x)× x| = 25 · 25.

I Since A1,A2,A3 are pairwise disjoint, the Sum Rule 87 implies

|A1 ∪ A2 ∪ A3| = 1951.


Real-World Application: Counting Passwords

I Suppose that passwords are limited to strings of six to eight characters, whereeach character is one of the 26 uppercase letters or a digit. Every password hasto contain at least one digit.

I How many different passwords do exist under these restrictions?I Let N be the total number of passwords, and let N6,N7,N8 denote the number of

passwords with six (seven, eight, resp.) characters.I By the Product Rule 89, the total number of six-character strings (over the 26

letters and the 10 digits) is 366, with 266 of them containing no digit at all. Hence,

N6 = 366 − 266 = 1 867 866 560.

I Similarly,

N7 = 367 − 267 = 70 332 353 920

and

N8 = 368 − 268 = 2 612 282 842 880.

I Hence, by the Sum Rule 87,

N = N6 + N7 + N8 = 2 684 483 063 360.


Inclusion-Exclusion Principle

Theorem 95 (Inclusion-exclusion principle, Dt.: Siebprinzip, Poincaré-Formel)

Let A1,A2, . . . ,An be finite sets. Then

|n⋃

i=1

Ai | =∑I 6=/0

I⊆1,...,n

(−1)|I|+1 |⋂i∈I

Ai |.

I For |I| = 1:

∑1≤i≤n

(−1)1+1 |Ai | =n∑

i=1

|Ai |.

I For |I| = 2:∑1≤i<j≤n

(−1)2+1 |Ai ∩Aj | = −∑

1≤i<j≤n

|Ai ∩Aj |.

I In particular:

|A1 ∪ A2| = |A1|+ |A2| − |A1 ∩ A2|

|A1∪A2∪A3| = |A1|+|A2|+|A3|−|A1∩A2|−|A1∩A3|−|A2∩A3|+|A1∩A2∩A3|.


Real-World Application: Counting Bit Strings

I How many bit strings of length eight either start with 1 as first bit or end in 00 asthe two last bits? (This is a non-exclusive or!)

I Let A1 be the set of 8-bit strings that start with 1. Similarly, let A2 be the set of8-bit strings that end in 00.

I Then the number sought equals |A1 ∪ A2|.I By the Product Rule 89,

|A1| = 27 = 128 and |A2| = 26 = 64 and |A1 ∩A2| = 25 = 32.

I Hence, by the Inclusion-Exclusion Principle (Thm. 95),

|A1 ∪ A2| = |A1|+ |A2| − |A1 ∩ A2| = 128 + 64− 32 = 160.


Binomial Coefficients

Definition 96 (Binomial coefficient, Dt.: Binomialkoeffizient)

Let n ∈ N0 and k ∈ Z. Then the binomial coefficient(n

k

)of n and k is defined as

follows:

(nk

):=

0 if k < 0,

n!k!·(n−k)! if 0 ≤ k ≤ n,

0 if k > n.

I Recall k ! := 1 for k = 0.I The binomial coefficient

(nk

)is pronounced as “n choose k ”; Dt.: “n über k ”.

Lemma 97Let n ∈ N0 and k ∈ Z.(

n0

)=

(nn

)= 1

(n1

)=

(n

n − 1

)= n

(nk

)=

(n

n − k

)


Binomial Coefficients

I The following table contains the non-zero values of(n

k

)for 0 ≤ n, k ≤ 6.

kn 0 1 2 3 4 5 60 11 1 12 1 2 13 1 3 3 14 1 4 6 4 15 1 5 10 10 5 16 1 6 15 20 15 6 1

I Trivial to observe:I Each row begins and ends with 1.I Initially each row contains increasing numbers till its middle but then the

numbers start to decrease.I Each row’s first half is exactly the mirror image of its second half.


Binomial Coefficients: Pascal’s Triangle

I A simple rearrangement of the previous table yields what is known as Pascal’sTriangle in the Western world (Blaise Pascal, 1623–1662). But it was alreadystudied in India in the 10th century, and discussed by Omar Khayyam(1048–1131)!

11 1

1 2 11 3 3 1

1 4 6 4 11 5 10 10 5 1

1 6 15 20 15 6 1

I All entries in this triangle, except for the left-most and right-most entries per row,are the sum of the two entries above them in the previous row.

Theorem 98 (Khayyam, Yang Hui, Tartaglia, Pascal)For n ∈ N and k ∈ Z,(

nk

)=

(n − 1k − 1

)+

(n − 1

k

).


Binomial Theorem

I We know: (a + b)2 = a2 + 2ab + b2 and (a + b)3 = a3 + 3a2b + 3ab2 + b3.

Theorem 99 (Binomial Theorem, Dt.: Binomischer Lehrsatz)For all n ∈ N0 and a, b ∈ R,

(a + b)n =

(n0

)an +

(n1

)an−1b + · · ·+

(nn

)bn

or, equivalently,

(a + b)n =n∑

i=0

(ni

)an−ibi .

Corollary 100For all n ∈ N and all x ∈ R:

n∑i=0

(ni

)x i = (1 + x)n

n∑i=0

(ni

)= 2n

n∑i=0

(−1)i

(ni

)= 0


Permutations

Definition 101 (Permutation)

Let A be a finite set. A permutation of A is a bijective function from A to A. The setof all permutations of In := 1, 2, . . . , n, for n ∈ N, is denoted by Sn.

I A permutation on a finite set A of cardinality n can be regarded as an (ordered)sequence of length n in which every element of A appears exactly once.

I Standard notation for a permutation π of Sn:(1 2 3 . . . n

π(1) π(2) π(3) . . . π(n)

) I E.g., for n := 4:(1 2 3 42 3 1 4

)Definition 102 (Product of permutations)

Let A be a finite set together with two permutations α, β. Then the product (orcomposition) α β is the function

α β : A→ A with (α β)(a) := α(β(a)) for all a ∈ A.

WarningSome authors take (α β)(a) as (β(α(a)))!


Permutations

I Of course, the product of two permutations is itself a bijective function.I Note: It is common to drop in α β and simply write αβ.I The product of two permutations is not commutative.

α :=

(1 2 3 41 3 4 2

)β :=

(1 2 3 42 1 3 4

)

αβ =

(1 2 3 43 1 4 2

)βα =

(1 2 3 42 3 4 1

)Lemma 103

For all n ∈ N, the set of all permutations, Sn, over In together with as operationforms a group, the so-called symmetric group.

Lemma 104

For all n ∈ N, the set of all permutations, Sn, over In has exactly n! members.


Permutations

Definition 105 (Cycle, Dt.: Zyklus)

Let A be a finite set of cardinality n. A permutation π of A is a cycle of length k ≤ nif there exists a set B ⊆ A with |B| = k such that, with B := b1, b2, . . . , bk,

π(b1) = b2, π(b2) = b3, . . . , π(bk−1) = bk , π(bk ) = b1,

and π(a) = a for all a ∈ A \ B. In this case this k -cycle is written as

(b1 b2 . . . bk ) or as b1 7→ b2 7→ . . . 7→ bk 7→ b1.

Definition 106 (Transposition)A transposition is a 2-cycle.

Lemma 107Every permutation can be written as

(1) a product of cycles,(2) a product of transpositions.


Permutations

Lemma 108

If two different products of transpositions correspond to the same permutation thenboth products consist of either an even or an odd number of transpositions.

Definition 109 (Signature, Dt.: Signum)

The signature of a permutation is +1, and the permutation is even, if it consists ofan even number of transpositions. Otherwise, the signature is −1 and thepermutation is odd.

Definition 110 (Derangement, Dt.: Permutation ohne Fixpunkt)

A permutation π of A is a derangement if π(a) 6= a for all a ∈ A.


Ordered Selection

Definition 111 (Ordered selection without repetition, Dt.: Variation ohne Zurücklegen, Variation ohne Wiederholung)

Let n ∈ N and k ∈ N0, with k ≤ n, and A be a finite set of cardinality n. An orderedselection without repetition of k elements from A is a k -tuple

(a1, a2, . . . , ak ) with ai ∈ A for i = 1, 2, . . . , k and ai 6= aj for 1 ≤ i < j ≤ k .

Lemma 112

Let n ∈ N and k ∈ N0, with k ≤ n, and A be a finite set of cardinality n. There exist

V nk :=

n!

(n − k)!

many different ordered selections without repetition of k elements from A.

I Convention: V nk := 0 for k > n.

I V nk is the number of injective functions from Ik to A.

I Sometimes, V (n, k) is written instead of V nk . Also, English-language textbooks

often speak of a k -permutation rather than of an ordered selection withoutrepetition of k elements.


Ordered Selection

Definition 113 (Ordered selection with repetition, Dt.: Variation mit Zurücklegen)

Let n ∈ N and k ∈ N0, and A be a finite set of cardinality n. An ordered selectionwith repetition of k elements from A is a k -tuple

(a1, a2, . . . , ak ) with ai ∈ A for i = 1, 2, . . . , k .

Lemma 114

Let n ∈ N and k ∈ N0, and A be a finite set of cardinality n. There exist

r V nk := nk

many different ordered selections with repetition of k elements from A.

I Dt.: Auch Variation mit Wiederholung.I Note: r V n

k = |Ak |.I Sometimes, Vr (n, k) is written instead of r V n

k .


Unordered Selection

Definition 115 (Unordered selection without repetition, Dt.: Kombination ohne Zurücklegen, Kombination ohne Wiederholung)

Let n ∈ N and k ∈ N0, with k ≤ n, and A be a finite set of cardinality n. Anunordered selection without repetition of k elements from A is a set B such that

B ⊆ A with |B| = k .

Lemma 116

Let n ∈ N and k ∈ N0, with k ≤ n, and A be a finite set of cardinality n. There exist

Cnk :=

(nk

)

many different unordered selections without repetition of k elements from A.

I Convention: Cnk := 0 for k > n.

I Lemma 116 yields an alternate proof of |P(A)| = 2n. It also implies that thereexist

(nk

)different binary sequences where exactly k elements are 1.

I Sometimes, C(n, k) is written instead of Cnk .


Unordered Selection

Definition 117 (Unordered selection w. repetition, Dt.: Kombination m. Zurücklegen)

Let n ∈ N and k ∈ N0, and A be a finite set of cardinality n. An unordered selectionwith repetition of k elements from A is a k -element multiset, i.e., a set B togetherwith a multiplicity function, mult , such that

B ⊆ A and mult : B → N with∑b∈B

mult(b) = k .

Lemma 118

Let n ∈ N and k ∈ N0, and A be a finite set of cardinality n. There exist

r Cnk :=

(n + k − 1

k

)

many different unordered selections with repetition of k elements from A.

I Dt.: Auch Kombination mit Wiederholung.I Sometimes, Cr (n, k) is written instead of r Cn

k .


Proofs of Lemmas 112–118

Proof of Lemma 112 : We have n options for a1, leaving n − 1 options for a2, etc.Thus, we have n · (n − 1) · . . . · (n − k + 1) = n!

(n−k)! options.

Proof of Lemma 114 : We have n options for every selection. Thus, we have nk

options in total.

Proof of Lemma 116 : We know that V nk = n!

(n−k)! . There are k ! many different orderedselections that correspond to the same unordered selection. Thus,Cn

k = V nk /k ! = n!

(n−k)!k! =(n

k

).

Proof of Lemma 118 : Let a1, . . . , an be the n elements of A, and k ∈ N0. We encodesuch an unordered selection with repetition of k elements from A as a sequence oflength n + k − 1 of k crosses × which are separated by n − 1 vertical bars |, where icrosses between the j-th vertical bar and the (j + 1)-st vertical bar, for 1 ≤ j ≤ n − 2,indicate that element aj+1 was chosen with multiplicity i . Similarly for the multiplicitiesof a1 and an for crosses before the first and after the last vertical bar.We note that we have exactly

Cn+k−1k =

(n + k − 1

k

)

ways to choose the positions of the k crosses within this sequence.


Sample Unordered Selection With Repetition

I Suppose you are buying a 3-scoop icecream cup and that the flavor options arechocolate, vanilla, strawberry, and blackberry.

I You can also choose all the three scoops to be of the same flavor, say, vanilla.I Hence, we have A = chocolate, vanilla, strawberry, blackberry and n = 4 and

k = 3.I Using

r Cnk =

(n + k − 1

k

)=

(n + k − 1)!

k !(n − 1)!,

for our current example, we have 20 options to choose our icecream:

r C43 =

(4 + 3− 1

3

)=

6!

3!(4− 1)!= 20.


Sample Unordered Selection With Repetition

I What is the number of non-negative integer solutions of the Diophantine equationx1 + x2 + x3 + x4 = 10?

I Think of the variables xi as four categories, ×1,×2,×3,×4, which are separatedby three vertical bars. Let the number of crosses in each category indicate thevalue of xi in the solution. Thus

×1 ×1 || ×3 ×3 ×3 ×3| ×4 ×4 ×4 ×4

represents the solution x1 = 2, x2 = 0, x3 = 4, x4 = 4.I We have A = ×1,×2,×3,×4 and n = 4. There must be 10 crosses in all, so

k = 10. In all, there are r C410 = 286 different solutions.

I What if we want to count the number of solutions where each xi ≥ 1?I One way to do this is to initially put one cross in each category. There are now

k = 10− 4 = 6 crosses remaining, and still 4 categories. So there are r C46 = 84

such solutions.


Real-World Application: Elementary Probability

I What is the probability to win in the Austrian “6-aus-45” lottery after choosing oneset of six numbers?

I As usual, we define the probability of an event among equally-likely outcomes asthe number of favorable outcomes divided by the total number of possibleoutcomes.

I Assuming that the lottery is fair and, thus, that all combinations are equally likelyto win, we get

1C45

6=

1(456

) =1

8145060≈ 1.22774 · 10−7

as probability for having all six numbers right.



I A standard deck of cards contains 52 cards grouped into four suits — diamonds(Dt.: Karo), clubs (Dt.: Kreuz), hearts (Dt.: Herz), and spades (Dt.: Pik) — with13 cards in each suit (ace, 2, 3, 4, 5, 6, 7, 8, 9, 10, jack, queen, king).

I What is the probability that all hearts appear in consecutive order after a decentshuffling of the deck?

I There are 52! different permutations of the 52 cards.I There are 40! different permutations of the 39 cards plus one block of 13 hearts,

and 13! many permutations of the 13 hearts.I Hence, the probability that all hearts are consecutive is given by

40! · 13!

52!≈ 6.29908 · 10−11.



I All 52 cards are distributed to four players. (Again we assume that the deck wasshuffled decently.) What is the probability that one of the four players gets fouraces?

I For each player we have C5213 different outcomes, among which C48

9 are favorable.I Hence, the probability that one of the four players gets four aces is given by

C41 · C48

9

C5213

=

(41

)·(48

9

)(5213

) =44

4165≈ 0.010564


Complexity Analysis and Recurrence RelationsComplexity of an AlgorithmAsymptotic AnalysisRecurrence RelationsMaster Theorem


Complexity of an Algorithm

I Typical kinds of complexities studied:I time complexity, i.e., a mathematical estimation of the running time

independent of a particular implementation or platform;I space complexity, i.e., a mathematical estimation of the number of memory

units consumer by the algorithm;I complexity of the output generated.

I We start with four (informal!) definitions that pertain to the complexity analysis ofalgorithms.

Definition 119 (Elementary Operation, Dt.: Elementaroperation)An elementary operation is an operation whose running time is assumed not todepend on the specific values of its operands, such as the time for the comparisonof two floating-point numbers.

Definition 120 (Input Size, Dt.: Eingabegröße)

The size of the input of an algorithm is a quantity that measures the number ofinput items relevant for elementary operations of the algorithm, such as the numberof memory units needed to represent the input data, or the number of records to besorted (if constant memory and comparison time per record may be assumed).


Complexity of an Algorithm

Definition 121 (Worst-Case Complexity, Dt.: Komplexität im schlimmsten Fall)

A worst-case complexity of an algorithm is a function f : N→ R+ that gives anupper bound on the number of elementary operations of an algorithm with respectto the size of its input, for all inputs of the same size.

Definition 122 (Average-Case Complexity, Dt. Komplexität im durchschnittl. Fall)

An average-case complexity of an algorithm is a function g : N→ R+ that modelsthe average number of elementary operations of an algorithm with respect to thesize of its input.

I It is common to denote the input size by the variable n.I The worst-case complexity is a pessimistic estimator; the average-case

complexity often is more relevant if the worst case is encountered rarely.I In any case, we seek sharp bounds.


Growth Rate of Functions

I Small input sizes can usually be computed instantaneously. Therefore we aremost interested in how an algorithm performs as the input size n gets large:asymptotic complexity analysis.

I We are interested in the growth of the complexity as a function of n.I Determine the dominating term in the complexity function — it gives the order of

magnitude of the asymptotic behavior.

1, log n, log2n,√

n, n, n log n, n log2n, n76 , n2, n3, . . . , 2n, 3n, 2(2n), . . .

I Note: In this course, log n will always denote the logarithm of n to the base 2, i.e.,log n := log2 n. Recall that logα n = 1

log2 αlog2 n.


How Can We Compare the Growth Rate of Functions?

I Let’s consider f , g : N→ R+ with f (n) := n and g(n) := 9n + 20.I We get for all n ∈ N with n ≥ 20

g(n) = 9n + 20 ≤ 9n + n = 10n = 10f (n), that is g(n) ≤ 10f (n).

I Also for all n ∈ N

f (n) ≤ g(n).

c1 · f(n) ≤ g(n) ≤ c2 · f(n)

g grows at least as fast as c1 · ff is an asymptotic lower bound on g

we’ll say that g ∈ Ω(f)

g grows at most as fast as c2 · ff is an asymptotic upper bound on g

we’ll say that g ∈ O(f)

︸︷︷︸

g has same growth rate as f

we’ll say that g ∈ Θ(f)

for all n ≥ n0

where n0 := 20,c1 := 1, c2 := 10.

Thus, we have


Asymptotic Notation: Big-O

c1 · f(n) ≤ g(n) ≤ c2 · f(n)

g grows at most as fast as c2 · ff is an asymptotic upper bound on g

we’ll say that g ∈ O(f)

for all n ≥ n0 andfixed c1, c2 ∈ R+.

Definition 123 (Big-O, Dt.: Groß-O)

Let f : N→ R+. Then the set O(f ) is defined as

O(f ) :=

g : N→ R+ | ∃c2 ∈ R+ ∃n0 ∈ N ∀n ≥ n0 g(n) ≤ c2 · f (n).

I Note: O(f ) is a set of functions!I Definitions of the form

O(f (n)) := g : N→ R+ | ∃c2 ∈ R+ ∃n0 ∈ N ∀n ≥ n0 g(n) ≤ c2 · f (n)are a (wide-spread) formal nonsense.

I Some authors prefer to use the symbol O instead of O.


Graphical Illustration of O(f )

Definition 123 (Big-O, Dt.: Groß-O)

Let f : N→ R+. Then the set O(f ) is defined as

O(f ) :=

g : N→ R+ | ∃c2 ∈ R+ ∃n0 ∈ N ∀n ≥ n0 g(n) ≤ c2 · f (n).

n

c2 · f

g

︸︷︷︸

n0

g(n) ≤ c2 · f (n) for all n ≥ n0

I Equivalent definition used by some authors:

O(f ) :=

g : N→ R+ | ∃c2 ∈ R+ ∃n0 ∈ N ∀n ≥ n0

g(n)

f (n)≤ c2

.


Why Don’t We Care About Constants?

I Note that this notation hides all lower-order terms and multiplicative constants.Why don’t we care?

I Since it doesn’t matter for large values of n!I Consider the following two nested for-loops:

for i = 1 to n dofor j = i to n do

Compute(i, j)end for

end for

I How often is Compute() being called?Let g : N→ R+ be the function thatmodels the number of calls independence on n.

I We get

g(n) = n + (n − 1) + . . .+ 2 + 1

=n(n + 1)

2=

12

n2 +12

n.

I Consider f : N→ R+ with f (n) := n2.I Let’s compare the growth rates of f

and g when we double n:

n g(n) f (n)5 15 25

10 55 10020 210 40040 820 160080 3240 6400

I Doubling n causes both f (n) and g(n)to (roughly) quadruple!


Why Don’t We Care About Constants?

I We plot the growth ratio g(n)f (n) for f , g : N→ R+ with f (n) := n2 and

g(n) := 12 n2 + 1

2 n.

0.5

0.51

0.52

0.53

0.54

0.55

0 200 400 600 800 1000

g(x)/f(x)

0.5

0.5002

0.5004

0.5006

0.5008

0.501

1000 3000 5000 7000 9000

g(x)/f(x)

I The plots suggest g(n)f (n) ≤ 1 for all n ≥ 200, that is, g(n) ≤ f (n), which would imply

g ∈ O(f ).I More precisely, they suggest g(n)

f (n) ≤12 + ε for any positive ε and all sufficiently

large values of n.I The plots also suggest g(n)

f (n) ≥12 , which would imply g ∈ Ω(f ).

I Hence g(n) ≈ 12 f (n), which would imply g ∈ Θ(f ).


Asymptotic Notation: Big-Omega

c1 · f(n) ≤ g(n) ≤ c2 · f(n)

g grows at least as fast as c1 · ff is an asymptotic lower bound on g

we’ll say that g ∈ Ω(f)


Definition 124 (Big-Omega, Dt.: Groß-Omega)

Let f : N→ R+. Then the set Ω(f ) is defined as

Ω(f ) :=

g : N→ R+ | ∃c1 ∈ R+ ∃n0 ∈ N ∀n ≥ n0 c1 · f (n) ≤ g(n).

I Equivalently,

Ω(f ) :=

g : N→ R+ | ∃c1 ∈ R+ ∃n0 ∈ N ∀n ≥ n0 c1 ≤

g(n)

f (n)

.


Graphical Illustration of Ω(f )

Definition 124 (Big-Omega, Dt.: Groß-Omega)

Let f : N→ R+. Then the set Ω(f ) is defined as

Ω(f ) :=

g : N→ R+ | ∃c1 ∈ R+ ∃n0 ∈ N ∀n ≥ n0 c1 · f (n) ≤ g(n).

n

g

︸︷︷︸

n0

c1 · f (n) ≤ g(n) for all n ≥ n0

c1 · f


Asymptotic Notation: Big-Theta

c1 · f(n) ≤ g(n) ≤ c2 · f(n)︸︷︷︸g has same growth rate as f

we’ll say that g ∈ Θ(f)


Definition 125 (Big-Theta, Dt.: Groß-Theta)

Let f : N→ R+. Then the set Θ(f ) is defined as

Θ(f ) :=

g : N→ R+ | ∃c1, c2 ∈ R+ ∃n0 ∈ N ∀n ≥ n0

c1 · f (n) ≤ g(n) ≤ c2 · f (n) .


Graphical Illustration of Θ(f )

Definition 125 (Big-Theta, Dt.: Groß-Theta)

Let f : N→ R+. Then the set Θ(f ) is defined as

Θ(f ) :=

g : N→ R+ | ∃c1, c2 ∈ R+ ∃n0 ∈ N ∀n ≥ n0

c1 · f (n) ≤ g(n) ≤ c2 · f (n) .

n

c2 · f

g

︸︷︷︸

n0

c1 · f (n) ≤ g(n) ≤ c2 · f (n) for all n ≥ n0

c1 · f

which is equivalent to c1 ≤ g(n)f (n) ≤ c2 for all n ≥ n0


Sample Proof of g ∈ Θ(f )

I We prove g ∈ Θ(f ) for f (n) := n2 and g(n) := 12 n2 + 1

2 n.Proof :I We get, for all n ∈ N,

g(n) =12

n2 +12

n ≤ 12

n2 +12

n2 = n2 = f (n), that is g(n) ≤ f (n).

I Thus, g ∈ O(f ) with c2 := 1 and n0 := 1.I Now we prove g ∈ Ω(f ) and get, again for all n ∈ N,

g(n) =12

n2 +12

n ≥ 12

n2 =12

f (n), that is12

f (n) ≤ g(n).

I Thus, g ∈ Ω(f ) with c1 := 12 and n0 := 1. Lemma 132 implies g ∈ Θ(f ).

I There is no need to try to obtain the smallest-possible values for n0 and c1, c2!

I Can we also prove h ∈ O(f ) for h : N→ R+ with h(n) := n3? We get, for all n ∈ N,

h(n)

f (n)=

n3

n2 = n.

I Thus, h(n)f (n) grows unboundedly as n grows, while it ought to be bounded by some

constant c2 for all n ≥ n0, for some fixed n0 ∈ N. We conclude that h 6∈ O(f ).


Asymptotic Notation: Small-Oh

Definition 126 (Small-Oh, Dt.: Klein-O)

Let f : N→ R+. Then the set o(f ) is defined as

o (f ) :=

g : N→ R+ | ∀c ∈ R+ ∃n0 ∈ N ∀n ≥ n0 g(n) ≤ c · f (n).

Mind the difference

O(f ) :=

g : N→ R+ | ∃c ∈ R+ ∃n0 ∈ N ∀n ≥ n0 g(n) ≤ c · f (n)

o (f ) :=

g : N→ R+ | ∀c ∈ R+ ∃n0 ∈ N ∀n ≥ n0 g(n) ≤ c · f (n)

I Similarly, ω(f ) can be defined relative to Ω(f ).I It is trivial to extend Definitions 123–126 such that N0 rather than N is taken as

the domain (Dt.: Definitionsmenge).I We can also replace the codomain (Dt.: Zielbereich) R+ by R+

0 (or even R)provided that all functions are eventually positive.

I The same comments apply to the subsequent slides.


Asymptotic Notation: Limit of a Sequence

Definition 127 (Sequence, Dt.: Folge)

A (real) sequence is a function from N (or N0) to R. For x : N→ R it is written as(xn)n∈N or 〈xn〉n∈N, or simply (xn) or 〈xn〉.

Definition 128 (Limit, Dt. Grenzwert)

The value x ∈ R is the limit of the (real) sequence (xn), denoted by limn→∞ xn = x ,if

∀ε ∈ R+ ∃n0 ∈ N ∀n ≥ n0 |xn − x | < ε.

Theorem 129 (Squeeze theorem, Dt.: Einschnürungssatz)

Consider three real sequences (xn), (yn), (zn) and suppose that xn ≤ yn ≤ zn for alln ≥ n0 for some n0 ∈ N. If the limits of (xn) and (zn) exist such that

limn→∞

xn = limn→∞

zn,

then the limit of (yn) exists with

limn→∞

xn = limn→∞

yn = limn→∞

zn.


Asymptotic Notation: Limit of a Sequence

I The following theorem (by Guillaume de l’Hôpital, 1661–1704) allows to handlelimits that involve indeterminate terms of the form

00

or∞∞ .

Theorem 130 (L’Hôpital’s rule)

Consider two real functions f and g, and a real value c.If

1. limx→c f (x) = 0 = limx→c g(x) or limx→c f (x) = ±∞ = limx→c g(x),

2. f and g are differentiable in an open interval I with c ∈ I, except possibly at citself,

3. g′(x) 6= 0 for all x ∈ I \ c, and if

4. limx→cf ′(x)g′(x) exists,

then

limx→c

f (x)

g(x)= lim

x→c

f ′(x)

g′(x).


Asymptotic Notation: Basic Facts

Lemma 131

Let f1, f2, g1, g2 : N→ R+, and c ∈ R+. Then the following relations hold:

1. g1 ∈ O(f1) ∧ g2 ∈ O(f2) ⇒ g1 + g2 ∈ O(f1 + f2)

2. g1 ∈ O(f1) ∧ g2 ∈ O(f2) ⇒ g1 · g2 ∈ O(f1 · f2)

3. f2 ·O(f1) ⊆ O(f1 · f2)

4. O(c · f1) = O(f1)

5. g1 ∈ O(f1) ⇒ c · g1 ∈ O(f1)

6. Θ(f1) = O(f1) ∩ Ω(f1)

7. g1 ∈ Θ(f1) ⇔ f1 ∈ Θ(g1)

8. g1 ∈ Θ(f1) ∧ g2 ∈ Θ(f1) ⇒ g1 ∈ Θ(g2)


Asymptotic Notation: Basic Facts

Lemma 132

Let f , g : N→ R+ and c ∈ R+. Then:

limn→∞

g(n)

f (n)= c ⇒ g ∈ Θ(f ),

and

limn→∞

g(n)

f (n)= 0 ⇔ g ∈ o(f ).

I For example, let f , g, h : N→ R+ with f (n) := n2 − 7n, g(n) := 3n2 + 5n√

n andh(n) := n2.We have g ∈ Θ(f ) since f ∈ Θ(h) and g ∈ Θ(h):

limn→∞

f (n)

h(n)= lim

n→∞

n2 − 7nn2 = lim

n→∞

(1− 7

n

)= 1

limn→∞

g(n)

h(n)= lim

n→∞

3n2 + 5n√

nn2 = lim

n→∞

(3 +

5√n

)= 3


Asymptotic Notation: Wide-spread Notational Abuse

I It is common to write

g(n) = O(n2) or g ∈ O(n2)

as an informal short-hand notation for

g ∈ O(f ) with f : N→ R+, n 7→ n2.

I Similarly,

g(n) = h(n) + O(n3)

means

|g − h| ∈ O(f ) with f : N→ R+, n 7→ n3.

I Furthermore,

g(n) = nO(1)

indicates that

g ∈ O(f ) with f : N→ R+, n 7→ nc

for some constant c ∈ R+.


Asymptotic Notation: Wide-spread Notational Abuse — Caveats!

Warning

1. In the equation-based notation the equality sign does not assert the equality oftwo functions or sets!

2. The property expressed by this equality sign is not symmetric! That is,

O(n2) = O(n3) but O(n3) 6= O(n2).

3. Stipulating

g(m) = O(mn)

is not the same as stipulating

g(n) = O(mn).

I It is convenient to be a bit sloppy and write, e.g., n2 = O(n3), rather than to resortto the λ-quantifier and write λn.n2 ∈ O(λn.n3). But keep in mind that anis-element-of or subset relation is meant even if an equality sign is used!

I Unfortunately, several textbooks are fuzzy about this important distinction . . .


Conditional Asymptotic Notation

Definition 133 (Conditional Asymptotic Notation)

Consider a function f : N→ R+ and a predicate P : N→ F ,T.

O(f | P) :=

g : N→ R+ | ∃c ∈ R+ ∃n0 ∈ N ∀n ≥ n0 :

P(n) ⇒ g(n) ≤ c · f (n) .

Ω(f | P) :=

g : N→ R+ | ∃c ∈ R+ ∃n0 ∈ N ∀n ≥ n0 :

P(n) ⇒ g(n) ≥ c · f (n) .

Θ(f | P) :=

g : N→ R+ | ∃c1, c2 ∈ R+ ∃n0 ∈ N ∀n ≥ n0 :

P(n) ⇒ c1 · f (n) ≤ g(n) ≤ c2 · f (n) .

o (f | P) :=

g : N→ R+ | ∀c ∈ R+ ∃n0 ∈ N ∀n ≥ n0 :

P(n) ⇒ g(n) < c · f (n) .

I E.g., let P(n) :⇔ n ≡2 0, or P(n) :⇔ (∃k ∈ N0 n = 2k ).


Smoothness

Definition 134 (Eventually non-decreasing, Dt.: schlußendlich nicht abnehmend)

A function f : N→ R+ is eventually non-decreasing exactly if

∃n0 ∈ N ∀n ≥ n0 f (n) ≤ f (n + 1).

Definition 135 (b-smooth, Dt.: b-glatt)

A function f : N→ R+ is b-smooth for some integer b ≥ 2 exactly if f is eventuallynon-decreasing and if

∃c ∈ R+ ∃n0 ∈ N ∀n ≥ n0 f (b · n) ≤ c · f (n).

Definition 136 (smooth, Dt.: glatt)

A function f : N→ R+ is smooth if it is b-smooth for all integers b ≥ 2.

Lemma 137

If f : N→ R+ is b-smooth for some integer b ≥ 2 then it is smooth.


Smoothness Rule

Theorem 138 (Smoothness Rule)

Let f , g : N→ R+, and consider an integer b ≥ 2. If

1. f is a smooth function,

2. g ∈ O(f | “is power of b”), and if

3. g is an eventually non-decreasing function,

then g ∈ O(f ).

I Similarly for Ω(f ) and Θ(f ).I Again, it is trivial to extend the definitions and lemmas such that N0 rather than N

is taken as the base set. Similarly, we can replace R+ by R+0 provided that all

functions are eventually non-zero.I The same comments apply to the subsequent slides.


Smoothness Rule: Sample Application

I For a, b ∈ R+0 we define g : N→ R+

0 as

g(n) :=

a if n = 1,4g(⌈ n

2

⌉) + b · n otherwise.

I Note that⌈ n

2

⌉= 2k−1 if n = 2k .

I We would like to show that g ∈ Θ(n2):It suffices toI prove that f , with f (n) := n2, is smooth,I prove that g ∈ Θ(f | “is power of 2”),I prove that g is eventually non-decreasing.

I Standard application in computer science: Solving the recurrence relation

T (n) = T(⌈n

2

⌉)+ T

(⌊n2

⌋)+ b · n,

e.g., as derived when analyzing the complexity of merge sort.


Recurrence Relations

I Sample sequence t : N0 → R: (1,2,4,8,16,32,64,128,. . .)

Definition 139 (Recurrence relation, Dt.: Rekurrenzgleichung)

A recurrence relation is an equation that relates elements of a sequence t . It is oforder k , for some k ∈ N, if tn can be expressed in terms of n and tn−1, tn−2, . . . , tn−k ,i.e., if tn is of the form tn = f (tn−1, tn−2, . . . , tn−k , n) for f : Rk × N→ R.

I Recurrence relation (of order 1) for the sample sequence given above:

tn :=

1 if n = 0,2 · tn−1 if n > 0.

I Easy to see: tn = 2n for all n ∈ N0.

NoteWe will freely mix the notations tk and t(k) for denoting the k -th element of asequence (tn)n∈N or (tn)n∈N0 .


Recurrence Relations: The Tower-of-Hanoi Recurrence

I According to legend, life on Earth will end once the Brahmin priests managed tomove the last disk in their 64-disk Tower-of-Hanoi problem . . .

I Also according to legend, the priests apply a recursive algorithm, thereby moving(1) the top n − 1 disks (recursively) from pole I to the auxiliary pole III,(2) the largest (bottom-most) disk from pole I to pole II,(3) the top n − 1 disks (recursively) from pole III to pole II.

I If T (n) denotes the number of moves for the n-disk ToH problem, they need twotimes T (n − 1) moves for the recursive Steps (1) and (3), and one move forgetting the largest disk from pole I to II in Step (2).

I Of course, T (1) = 1.I Hence, we get the recurrence relation

T (n) = 2T (n − 1) + 1 with T (1) := 1

for the number T of moves for solving the Tower-of-Hanoi problem recursively.I A solution of this recurrence relation tells us when life on Earth might end . . .I So, is it already time for an apocalyptic mood?I We start with classifying recurrence relations.



Definition 140 (Homogeneous recurrence, Dt.: homogene Rekurrenz)

A recurrence relation of order k is homogeneous if it is satisfied by the zerosequence.

I E.g., tn := 3 · n2 · tn−1 · tn−2.

Definition 141 (Linear homogeneous recurrence)

A homogeneous recurrence relation of order k is linear if tn =∑k

i=1 ai (n) · tn−i ,where ai : N→ R for i = 1, 2, . . . , k .

I E.g., tn := n2 · tn−1 + 3 · tn−2.

Definition 142 (Linear homogeneous recurrence with constant coefficients)

A linear homogeneous recurrence relation of order k has constant coefficients iftn =

∑ki=1 ai · tn−i , where a1, a2, . . . , ak ∈ R.

I E.g., tn := 2 · tn−1 + 3 · tn−2.



CaveatEven seemingly simple recurrence relations need not be easy to understand andsolve.

I For instance, it is not known whether the so-called Collatz 3n + 1 recursion[Lothar Collatz 1937] will result in T (n) = 1 for all n ∈ N:

T (n) :=

1 if n = 1,T( 3n+1

2

)if n > 1 ∧ n ≡2 1,

T( n

2

)if n > 1 ∧ n ≡2 0.

I E.g., for n := 6 one gets the equalities

T (6) = T (3) = T (5) = T (8) = T (4) = T (2) = T (1)

and, thus, T (6) = 1.I Experiments have confirmed the Collatz conjecture up to 264 ≈ 1.8 · 1019 . . .


Heuristics for Solving Recurrences

I Constructive Induction:I First "guess" a solution.I Use "constructive" induction to verify that the solution guessed is correct.

I Cascading:I Restate the recurrence relation for tn, tn−1, tn−2, . . ..I Manipulate and rearrange the individual equations such that summing over

all equations yields a closed-form expression for tn.I Iteration:

I Expand the recurrence relation.I Derive a closed-form solution.

NoteAll heuristics require induction to prove that the result obtained is indeed correct!


Heuristics for Solving Recurrences: Constructive Induction

I Solve the recurrence relation tn = tn−1 + n, with t0 := 0.I Guess: tn ∈ O(n2).I Our guess can be verified by showing tn ≤ a · n2 for a suitable (but yet unknown)

a ∈ R+.I We get:

tn+1 = tn + (n + 1) ≤ a · n2 + (n + 1)a:=2= 2n2 + (n + 1) ≤ 2(n + 1)2.

I Now use standard induction to show that tn ≤ 2n2 is indeed correct for all n ∈ N0.


Heuristics for Solving Recurrences: Cascading

I Solve the recurrence relation tn = tn−1 + n, with t0 := 0.I Restating the recurrence yields the following set of equations:

tn = tn−1 + n

tn−1 = tn−2 + n − 1

tn−2 = tn−3 + n − 2

...

t2 = t1 + 2

t1 = t0 + 1

tn = t0 + 1 + 2 + · · ·+ (n − 2) + (n − 1) + n

I This indicates that

tn =n∑

i=0

i =n(n + 1)

2∈ Θ(n2),

which is proved by induction.


Heuristics for Solving Recurrences: Iteration

I Solve the recurrence relation tn = tn−1 + n, with t0 := 0.I Iterating the recurrence yields

tn = tn−1 + n

= (tn−2 + (n − 1)) + n

= (tn−3 + (n − 2)) + (n − 1) + n

= (tn−4 + (n − 3)) + (n − 2) + (n − 1) + n

...

= t0 + 1 + 2 + · · ·+ (n − 1) + n

= 0 + 1 + 2 + · · ·+ (n − 1) + n.

I Again, this indicates that

tn =n∑

i=0

i =n(n + 1)

2∈ Θ(n2),

which is proved by induction.


Real-World Problem: When Will Life on Earth End?

I We have the Tower-of-Hanoi recurrence relation

T (n) = 2T (n − 1) + 1 with T (1) := 1.

I Iteration yields the following identities:

T (n) = 2T (n − 1) + 1 = 21T (n − 1) + 20

= 2(21T (n − 2) + 20)+ 20 = 22T (n − 2) + 21 + 20

= 22(21T (n − 3) + 20)+ 21 + 20 = 23T (n − 3) + 22 + 21 + 20

...

= 2n−1T (n − (n − 1)) + 2n−2 + . . .+ 22 + 21 + 20

= 2n−1 + 2n−2 + . . .+ 22 + 21 + 20

= 2n − 1

I Hence, if the priests manage to move one disk per second then we would have toexpect the end of Earth 264 − 1 seconds after they started, i.e., roughly within5 · 1011 years . . .


Solving Linear Homogeneous Recurrence RelationsWith Constant Coefficients

Lemma 143

Consider the recurrence relation a0tn + a1tn−1 + · · ·+ ak tn−k = 0, with ai ∈ R. If(fn) and (gn) satisfy the recurrence relation then (αfn + βgn) satisfies therecurrence relation for all α, β ∈ R.

Proof : Suppose that

k∑i=0

ai fn−i = 0 andk∑

i=0

aign−i = 0

for all n ≥ k . Then:

k∑i=0

ai (αfn−i + βgn−i ) = α

k∑i=0

ai fn−i + β

k∑i=0

aign−i = 0.



I So, consider a0tn + a1tn−1 + · · ·+ ak tn−k = 0I Guess tn = xn for some unknown x ∈ R.I Then a0xn + a1xn−1 + · · ·+ ak xn−k = 0.I Further xn−k (a0xk + a1xk−1 + · · ·+ ak ) = 0.I If we ignore the trivial solution x = 0 then we get

a0xk + a1xk−1 + · · ·+ ak = 0

as the so-called characteristic equation of the recurrence relation

a0tn + a1tn−1 + · · ·+ ak tn−k = 0.

I Hence, any root r of this equation serves as a partial solution of the recurrencerelation, with tn := r n.



I Suppose that the characteristic equation has k distinct roots r1, . . . , rk such thatall roots are real numbers. I.e., the characteristic equation is given as

k∏i=1

(x − ri ) = 0.

I Then, the general solution of the recurrence relation is of the form

tn =k∑

i=1

ci r ni ,

for some constants c1, c2, . . . , ck ∈ R.I The constants ci are determined based on the initial condition(s).


Solving Linear Homogeneous Recurrence RelationsWith Constant Coefficients: Fibonacci Sequence

I Consider the Fibonacci sequence (over N0)

Fn :=

0 if n = 0,1 if n = 1,Fn−1 + Fn−2 if n ≥ 2.

I Hence, Fn − Fn−1 − Fn−2 = 0, and we get

x2 − x − 1 = 0

as the characteristic equation.I The characteristic equation has the roots

x1 =1 +√

52

and x2 =1−√

52

.

I Note: x1 is known as the golden ratio, φ = 1.618 . . ..I This yields

Fn = c1 ·(

1 +√

52

)n

+ c2 ·(

1−√

52

)n

.


Solving Linear Homogeneous Recurrence RelationsWith Constant Coefficients: Fibonacci Sequence

I This yields

Fn = c1 ·(

1 +√

52

)n

+ c2 ·(

1−√

52

)n

.

I The constants c1, c2 are determined by resorting to the initial conditions.

n = 0 : F0 = 0 = c1 + c2

n = 1 : F1 = 1 = c1 ·1 +√

52

+ c2 ·1−√

52

I By solving this linear system we obtain c1 = −c2 = 1√5.

I Hence,

Fn =1√5·(

1 +√

52

)n

− 1√5·(

1−√

52

)n

.



I Multiple roots: Suppose that the characteristic equation has s distinct rootsr1, . . . , rs of multiplicities m1, . . . ,ms such that all roots are real numbers. I.e., thecharacteristic equation is given as

s∏i=1

(x − ri )mi = 0.

I Then we have

tn =s∑

i=1

mi−1∑j=0

cijnj r ni ,

for constants cij ∈ R.I E.g., for (x − 1) · (x − 2)2 = 0 we get

tn = c10 · n0 · 1n + c20 · n0 · 2n + c21 · n1 · 2n = c10 + c20 · 2n + c21 · n · 2n.


Solving Linear Inhomogeneous Recurrence RelationsWith Constant Coefficients

I Assume we have an inhomogeneous recurrence relation of the following form:

a0 · tn + a1 · tn−1 + · · ·+ ak · tn−k = bn1 · p1(n) + bn

2 · p2(n) + · · ·+ bnm · pm(n).

where bi are constants and pi are polynomials in n of degree di ∈ N0.I Then the characteristic polynomial is

(a0 · xk + a1 · xk−1 + · · ·+ ak ) ·m∏

i=1

(x − bi )di+1 = 0.

I Now proceed as in the homogeneous case.


Solving Linear Inhomogeneous Recurrence RelationsWith Constant Coefficients

Theorem 144

Consider the linear inhomogeneous recurrence relation

a0tn + a1tn−1 + · · ·+ ak tn−k =m∑

i=1

bni · pi (n),

where bi are constants and pi are polynomials in n of degree di ∈ N0, for m ∈ N0,and suppose that its characteristic equation

(a0xk + a1xk−1 + · · ·+ ak ) ·m∏

i=1

(x − bi )di+1 = 0

has s distinct roots r1, . . . , rs of multiplicities m1, . . . ,ms such that all roots are realnumbers. Then the general solution of the recurrence relation is given by

tn =s∑

i=1

mi−1∑j=0

ci,j · nj · r ni ,

for constants cij ∈ R.


Solving Linear Inhomogeneous Recurrence RelationsWith Constant Coefficients: Sample Solution

I Consider

tn :=

0 if n = 0,2tn−1 + n + 2n otherwise.

I The standard form of this recurrence is

tn − 2tn−1 = n + 2n = 1n · n1 + 2n · n0.

I Hence, relative to Thm. 144, we get

k = 1 a0 = 1 a1 = −2 m = 2

b1 = 1 p1(n) = n d1 = 1 b2 = 2 p2(n) = 1 d2 = 0.

I This results in

0 = (x − 2) · (x − 1)2 · (x − 2)1 = (x − 1)2 · (x − 2)2

as the characteristic equation, and we get, with r1 := 1, r2 := 2,m1 = m2 := 2,

tn = c10 · n0 · 1n + c11 · n1 · 1n + c20 · n0 · 2n + c21 · n1 · 2n

= c10 + c11 · n + c20 · 2n + c21 · n · 2n.


Solving Linear Inhomogeneous Recurrence RelationsWith Constant Coefficients: Sample Solution

I This yields

tn = c10 + c11 · n + c20 · 2n + c21 · n · 2n.

I The constants c10, c11, c20, c21 are determined by resorting to the initial conditions:

n = 0 : 0 = c10 + c11 · 0 + c20 · 20 + c21 · 0 · 20 = c10 + c20

n = 1 : 3 = c10 + c11 + 2 · c20 + 2 · c21

n = 2 : 12 = c10 + 2 · c11 + 4 · c20 + 8 · c21

n = 3 : 35 = c10 + 3 · c11 + 8 · c20 + 24 · c21

I Solving this system of four linear equations for c10, c11, c20, c21 yields

c10 = −2, c11 = −1, c20 = 2, c21 = 1.

I We conclude that

tn = −2− n + 2 · 2n + n · 2n, i.e., tn = −2− n + 2n+1 + n · 2n.


Master Theorem

Theorem 145 (Master theorem, Dt.: Hauptsatz der Laufzeitfunktionen)

Consider constants c ∈ R+, k , n0 ∈ N and a, b ∈ N with b ≥ 2, and let T : N→ R+0

be an eventually non-decreasing function such that

T (n) = a · T(n

b

)+ c · nk

for all n ∈ N with n ≥ n0, where we interpret nb as either d n

b e or b nb c.

Then we have

T ∈

Θ(nk ) if a < bk ,

Θ(nk log n) if a = bk ,

Θ(nlogb a) if a > bk .

I E.g., we get T ∈ Θ(n log n) for T defined as follows:

T (n) = T(⌈n

2

⌉)+ T

(⌊n2

⌋)+ c · n.


Master Theorem (Asymptotic Version)

Theorem 146

Consider constants k , n0 ∈ N and a, b ∈ N with b ≥ 2, and a function f : N→ R+0

with f ∈ Θ(nk ). Let T : N→ R+0 be an eventually non-decreasing function such that

T (n) = a · T(n

b

)+ f (n)


b e or b nb c.

Then we have

T ∈

Θ(nk ) if a < bk ,

Θ(nk log n) if a = bk ,

Θ(nlogb a) if a > bk .


Master Theorem (Refined Asymptotic Version)

Theorem 147

Consider constants n0 ∈ N and a ∈ N, b ∈ R with b > 1, and a function f : N→ R+0 .

Let T : N→ R+0 be an eventually non-decreasing function such that

T (n) = a · T(n

b

)+ f (n)


b e or b nb c.

Then we have

T ∈

Θ(f ) if

f ∈ Ω

(n(logb a)+ε

)for some ε ∈ R+,

and if the following regularity condition holdsfor some 0 < s < 1 and all sufficiently large n:

a · f( n

b

)≤ s · f (n),

Θ(nlogb a log n

)if f ∈ Θ

(nlogb a) ,

Θ(nlogb a) if f ∈ O(

n(logb a)−ε)

for some ε ∈ R+.

I This is a simplified version of the Akra-Bazzi Theorem [Akra&Bazzi 1998].


Real-World Application: Analysis of Fast Integer Multiplication

I The standard multiplication of two integers a, b represented as binary numberswith 2n bits each requires Θ(n2) many additions and shifts of bits.

I Can we do any better and achieve o(n2) time? Yes!I [Anatoliı Karatsuba 1960–1963:] Let

(a2n−1a2n−2 · · · a1a0)2 and (b2n−1b2n−2 · · · b1b0)2

be the 2n-bit binary representations of a and b. Hence, a =∑2n−1

i=0 ai2i andb =

∑2n−1i=0 bi2i .

I We have

a ∼ 2nA1 + A0 and b ∼ 2nB1 + B0

with

A1 := (a2n−1a2n−2 · · · an+1, an)2, A0 := (an−1an−2 · · · a1a0)2,

B1 := (b2n−1b2n−2 · · · bn+1, bn)2, B0 := (bn−1bn−2 · · · b1b0)2.

I We get

a · b ∼ 22nA1 · B1 + 2n(A1 · B0 + A0 · B1) + A0 · B0.


Real-World Application: Analysis of Fast Integer Multiplication

I We get

a · b ∼ 22nA1 · B1 + 2n(A1 · B0 + A0 · B1) + A0 · B0,

which can be rewritten as

a · b ∼ (22n + 2n)A1 · B1 + 2n(A1 − A0) · (B0 − B1) + (2n + 1)A0 · B0.

I Thus, the multiplication of two 2n-bit binary numbers can be carried outrecursively by computing

1. three multiplications of n-bit binary numbers, plus2. a constant number of additions and shifts on n-bit binary numbers.

I Hence, if T (n) denotes the total number of bit operations used by this recursivealgorithm for n-bit binary numbers, then

T (n) = 3T(n

2

)+ f (n) with f ∈ Θ(n).

I The asymptotic version of the Master Theorem 146 allows us to conclude that

T ∈ Θ(

nlog2 3), i.e., that T ∈ Θ

(n1.58496...

)and, thus, T ∈ o

(n2).

I Even faster methods for integer multiplication exist, based on Fast FourierTransform: [Schönhage&Strassen 1971], [Fürer 2007].


Graph TheoryWhat is a (Directed) Graph?PathsTreesSpecial GraphsGraph Coloring


Basic Definitions: Undirected Graph

Definition 148 (Graph, Dt.: (schlichter endlicher ungerichteter) Graph)

A (simple finite undirected) graph G = (V ,E) with n vertices (aka nodes) and medges consists of a vertex set V = v1, v2, . . . , vn and an edge setE = e1, e2, . . . , em, where V ∩ E = /0 and each edge is an unordered pair ofdistinct vertices:

e ∈ E ⇒ e = u, v with u, v ∈ V and u 6= v .

a

b

c

d

e a

b

c

d

e

I It is common to mix the terms “node” (Dt.: Knoten) and “vertex” (Dt.: Ecke) freely.I An edge u, v is often denoted by uv .I If we allow edges of the form uu then we get a loop (Dt.: Schlinge, Schleife) and

the graph is no longer simple (Dt.: schlicht, einfach).I If we allow multiple edges between two vertices then we get a multigraph.c© M. Held (Univ. Salzburg) Diskrete Mathematik (SS 2020) 251/333

Basic Definitions: Graphical Representation

I Graphical representation of a graph:I Denote the vertices by markers of the same form (circles, dots, squares, . . .).I For every pair of vertex markers, draw a curve between them if the graph

contains an edge between the corresponding vertices.I The edges drawn may be curved and may intersect.I However, it is poor practice to let an edge pass or touch any other vertex in

addition to its two defining vertices.I Use arrows to denote directed edges.


Basic Definitions: Graphical Representation

I Which of the following drawings show simple graphs?

not a simple graph: loop!

not a graph

this is a graph!

multigraph


Basic Definitions: Directed Graph

Definition 149 (Directed graph, Dt.: gerichteter Graph)

A (finite simple) directed graph, or digraph, G = (V ,E) with n vertices (aka nodes)and m edges consists of a vertex set V = v1, v2, . . . , vn and an edge setE = e1, e2, . . . , em, where V ∩ E = /0 and each edge is an ordered pair of distinctvertices:

e ∈ E ⇒ e = (u, v) with u, v ∈ V and u 6= v .

a

b

c

d

e a

b

c

d

e

I For a digraph, uv indicates the edge (u, v), i.e., an edge where u is the tail and vis the head.

I In this lecture we will always specify a directed graph explicitly; that is, the term“graph” without the qualifier “directed” shall mean “undirected graph”.


Basic Definitions: How to Deal with V = /0

I There is no consensus on whether or not to allow V = /0 in the definition of agraph. (Of course, if V = /0 then E = /0.)

I And, indeed, there are pros and cons of allowing V = /0.I Furthermore, if V = /0 is allowed then there is little consensus on how to call such

a graph:I Common terms are order-zero graph, K0, and null graph.I Some authors also use the term empty graph to indicate V = /0 while other

authors prefer to reserve this term for a graph with E = /0 but V 6= /0.

ConventionWe will always assume that every (directed) graph has at least one node.


Basic Definitions — Warning!

No common terminologyThe terminology in graph theory lacks a rigorous standardization, both in the Germanand in the English literature.

I In several cases the meanings of different terms coincide for simple undirectedgraphs, which seems to serve as a justification for authors to freely mix andmatch terms.

I Thus, always make sure to check how some author defines standard terms ofgraph theory . . .


Undirected Graphs as Directed Graphs

I It is straightforward to represent an undirected graph as a directed graph.I Hence, undirected graphs can be seen as a special case of directed graphs, and

most algorithms that work for directed graphs are applicable to undirectedgraphs, too.

a b c

d e f

a b c

d e f


Directed Graphs and Relations

I There is an elementary one-to-one mapping from relations to digraphs!I E.g., the relation R on the set a, b, c, d , e, with

R := (a, b), (b, a), (d , c), (e, a), (e, b),

corresponds to the following directed graph:

a

b

c

d

e

I Hence, statements about relations can be translated to statements aboutdigraphs, and vice versa.

I Note, though, that the digraph corresponding to a relationI need not be simple but might contain loops,I need not have a finite vertex set.


Directed Graphs and Relations: Hasse DiagramI Consider the poset (S,R), where S := n ∈ N : 1 < n ≤ 12 and R denotes the

partial order of divisibility on S. (That is, for a, b ∈ S, we have a R b iff a | b.)

12 8

9 6 4 10

7 3 2 5 11

12 8

9 6 4 10

5 11237

Hasse diagram

(1) Redraw the digraph such that all oriented (non-loop) edges point upwards.

(2) Now remove all loops (that result from the reflexivity of the partial order).

(3) Next, remove all edges implied by transitivity.

(4) Finally, shrink all node markers to dots.

Definition 150 (Hasse diagram)

The graph obtained after carrying out Steps (1)–(4) is the Hasse diagram of theposet.


Real-World Application: Precedence Graph

I Typically, some but likely not all statements of a computer program could beexecuted in parallel. Care has to be taken not to execute a statement thatdepends on results of statements that were not yet executed.

I A precedence graph is a directed graph that models dependences. E.g., thedependence of statements of a computer program on other statements:I Each statement is represented by a vertex.I There is an edge from vertex u to vertex v if the statement that corresponds

to v has to be executed after the statement of u.I Precedence graphs are used in all sorts of scheduling tasks: E.g., job

scheduling, concurrency control and instruction scheduling, resolving linkerdependencies, data serialization, automated parallelization of sequential code.

(1) a := 1

(2) b := 2

(3) c := 3

(4) d := a+ 2

(5) e := 2a+ b

(6) f := d+ c

(7) g := c+ e

(8) h := d+ e+ f

(1)

(2)

(3)

(4)

(5)

(6)

(7)

(8)


Basic Definitions: Adjacency and Degree

Definition 151 (Adjacent, Dt.: benachbart)

Two vertices u, v ∈ V of a graph G = (V ,E) are adjacent if uv ∈ E ; the edge uv isincident to the vertices u and v .

Definition 152 (Degree, Dt.: Grad)

The degree (aka valence) of a vertex u of a graph G = (V ,E) is the number ofedges incident to u. It is denoted by deg(u).For directed graphs, it is common to distinguish between the in-degree, deg−(u),i.e., the number of edges vu for v ∈ V , and the out-degree, deg+(u), i.e., thenumber of edges uv for v ∈ V .The degree of a graph is the maximum of the degrees of its vertices.

Definition 153 (Subgraph, Dt.: Teilgraph)

A graph G′ = (V ′,E ′) is a subgraph of a (directed) graph G = (V ,E) if V ′ ⊆ V andE ′ ⊆ E such that all edges of E ′ are formed by vertices of V ′.


Basic Definitions: Adjacency Matrix

Definition 154 (Adjacency matrix, Dt.: Adjazenzmatrix)

The adjacency matrix of a (directed) graph G = (V ,E) is an n × n matrix M, wheren := |V | and

mij :=

1 if vivj ∈ E ,0 otherwise.

a

b

c

d

e a b c d eabcde

0 1 1 1 01 0 0 1 01 0 0 0 10 0 1 0 11 1 0 0 0

ij

I The adjacency matrix M is symmetric for undirected graphs, and all diagonalelements are zero for simple graphs.

I Note: Storing M (as an n × n array) requires Θ(n2) memory!I Adjacency lists (and their variants) help to preserve memory if |E | |V |2.


Basic Properties of Graphs

Lemma 155 (Euler’s Handshaking Lemma, Dt.: Handschlag-Lemma)

The sum over all degrees of vertices of a graph G = (V ,E) equals twice thenumber of its edges, i.e.,

∑ν∈V deg(ν) = 2|E |.

Sketch of Proof : Adding one edge increases the sum of the degrees by two.

Corollary 156

In every graph the number of vertices of odd degree is even.

I Simple application of Euler’s Handshaking Lemma:I Suppose that a party is attended by 15 guests. Is it possible that every guest

at the party knows all others except for precisely one guest?I No: Consider a graph with 15 nodes (guests) where two nodes are linked by

an edge if the corresponding guests do not know each other. Hence, wewould get 15 nodes of degree one, in contradiction to Cor. 156.


Walks

Definition 157 (Walk, Dt.: Wanderung, Kantenfolge)

A walk of length k , with k ∈ N0, on G = (V ,E) is an alternating sequence

v0e1v1e2v2 . . . ek vk

of k + 1 vertices v0, v1, . . . , vk ∈ V and k edges e1, . . . , ek ∈ E such that

∀(1 ≤ i ≤ k) ei = vi−1vi .

I Often, a walk of length k is written simply as

v0v1v2 . . . vk .

I Conventionally, v0 is called the start vertex (or initial vertex) of the walk, and vk iscalled its end vertex (or terminal vertex). Note that vi−1 6= vi for i ∈ 1, 2, . . . , k.

Definition 158 (Closed walk, Dt.: geschlossene Wanderung)

A walk is called closed if the start vertex and the end vertex are identical. A closedwalk of length k is called trivial if k ≤ 2.


Paths, Trails, Tours and Cycles

Definition 159 (Trail, Dt.: Weg)A trail in a graph G is a walk in which all edges are distinct.

Definition 160 (Path, Dt.: Pfad)A path in a graph G is a walk in which all vertices are distinct.

Definition 161 (Tour, Dt.: Tour )A tour in a graph G is a closed trail.

Definition 162 (Cycle, Dt.: Zyklus, Kreis)A cycle in a graph G is a non-trivial closed walk in which all but the start and theend vertices are distinct.

I Note: Distinct vertices implies distinct edges; i.e., every path is a trail and everycycle is a tour.

I Note that some authors prefer to use the terms “path”, “simple path”, “cycle” and“simple cycle” instead of “trail”, “path”, “tour” and “cycle” . . .


ConnectednessDefinition 163 (Connected component, Dt.: Zusammenhangskomponente)

A connected component of a graph G = (V ,E) is a maximal subgraphG′ = (V ′,E ′) of G such that for every unordered pair u, v, with u, v ∈ V ′ andu 6= v , there exists a path between u and v within G′.

cut vertex

Definition 164 (Cut vertex, Dt.: Artikulationspunkt, Schnittknoten)

A cut vertex of a graph G = (V ,E) is a vertex v ∈ V such that the removal of v andof all edges incident to v would increase the number of connected components.

Definition 165 (Connected, Dt.: zusammenhängend)

A graph is connected if it contains only one connected component.


Connectedness

Definition 166 (Weakly connected, Dt.: schwach zusammenhängend)

A directed graph is weakly connected if replacing all its directed edges byundirected edges results in a connected (undirected) graph.

Definition 167 (Strong component, Dt.: starke Zusammenhangskomponente)

A strong component (aka strongly connected component) of a directed graphG = (V ,E) is a maximal subgraph G′ = (V ′,E ′) of G such that for every orderedpair (u, v), with u, v ∈ V ′ and u 6= v , there exists a path from u to v within G′.

Definition 168 (Strongly connected, Dt.: stark zusammenhängend)

A directed graph G = (V ,E) is strongly connected if it consists of only one strongcomponent, i.e., if for every ordered pair (u, v), with u, v ∈ V and u 6= v , thereexists a path from u to v .


Seven Bridges of Königsberg

I Early 18th century: Does there exist a trail (or even a tour) through the city ofKönigsberg that crosses every of its seven bridges exactly once? (Of course,every bridge had to be crossed fully, and no other means to get across the riverPregel were allowed.)

A

BC

D

12

3

4

5

6 7

A

BC

D

1 2

3

4

5 6 7

[Image credit for background image: Wikipedia.]

I In 1736, Leonhard Euler (1707–1783) treated this problem as a graph problemand proved, using a parity argument, that such a trail or tour does not exist.

I His solution is generally regarded as the first theorem of graph theory.


http://en.wikipedia.org

Euler Tour and Hamilton Cycle

Definition 169 (Euler trail, Dt.: Eulerscher Weg)

An Euler trail is a trail that contains all edges of a graph exactly once.

Definition 170 (Euler tour, Dt.: Eulersche Tour )

An Euler tour is a tour that contains all edges of a graph exactly once. A graph is anEulerian graph if it has an Euler tour.

Definition 171 (Hamilton path, Dt.: Hamiltonscher Pfad)

A Hamilton path is a path that passes through all vertices of a graph exactly once.

Definition 172 (Hamilton cycle, Dt.: Hamiltonscher Kreis)

A Hamilton cycle is a cycle that passes through all vertices of a graph exactly once.


Euler Tour and Hamilton Cycle

walk trail path

tour cycle

Hamilton cycle

Hamilton path

Euler tour

Euler trailallvertices

allvertices

alledges

alledges

closed

distinct edges distinct vertices

distinct vertices

(except for initial, terminal)

(for undirected graphs)


Euler Tour

Theorem 173

Suppose that every node of a graph G has degree at least one. Then G has anEuler tour if and only if G is connected and every vertex of G has even degree.

Theorem 174

Suppose that every node of a graph G has degree at least one. Then G has anEuler trail (but no Euler tour) if and only if G is connected and exactly two vertices ofG have odd degrees.

Corollary 175

An Euler tour or trail in a graph G = (V ,E) can be determined in O(|E |) time, if itexists. Otherwise, again in O(|E |) time, we can determine that neither an Euler tournor an Euler trail exists in G.


Constructive Proof of Theorem 173

Sketch of Proof of Theorem 173 : Let G = (V ,E) be a graph such that every node ofa graph G has degree at least one.

Suppose that G has an Euler tour T . It is obvious that G is connected. Everyoccurrence of a vertex v ∈ V in T is preceded and followed by an edge. Thus, eachtime T passes through v , two of the edges incident to v are consumed. Since T doesneither start nor end in v , it is necessary that deg(v) is even.

Now suppose that every vertex of G has even degree, and, of course, that G isconnected. We give a constructive proof that G admits an Euler tour. Pick any vertexv to start with and trace out a trail T . Every edge that is being traversed is marked. Asabove, we observe that passing through a vertex that is neither the start nor the endvertex of T consumes two edges.We realize that, eventually, T will get us back to v . (We cannot be stuck in some othervertex w since w has even degree.) If at the time when we are back at v every vertexof T has no unmarked incident edge then we are done. Otherwise, we start a new trailT ′ at a vertex w of T which has an unmarked incident edge and follow it until we getback to w .This process continues until no unmarked edges remain. At the end the trails arespliced together appropriately.


Hamilton Cycle

Theorem 176It is NP-complete to determine whether a Hamilton cycle or Hamilton path exists ina general graph.

I Informally, Theorem 176 says that no (deterministic sequential) algorithm isknown which determines the existence of a Hamilton cycle or path in an n-vertexgraph in a time that is a polynomial function of n.

I Even worse, an efficient (polynomial-time) algorithm will never be found unlessP = NP holds, which seems rather unlikely.

Theorem 177 (Dirac, 1952)

If the degree of every vertex of an n-vertex graph G, with n ≥ 3, is at least d n2 e then

G has a Hamilton cycle.

Theorem 178 (Ore, 1960)

If the sum of the degrees of every pair of non-adjacent vertices of an n-vertex graphG, with n ≥ 3, is at least n then G has a Hamilton cycle.


Trees

Definition 179 (Acyclic, Dt.: zyklenfrei)

A graph is called acyclic if it contains no cycles.

Definition 180 (Tree, Dt.: Baum)A tree is an undirected graph that is acyclic and connected.

I For trees most authors prefer to speak about nodes rather than vertices.I Unless explicitly stated otherwise, we will only deal with trees that have at least

one node.

Definition 181 (Rooted tree, Dt.: Baum mit Wurzel, Wurzelbaum)A rooted tree is a directed graph with a node u such that

1. the graph contains u as node (“root”),

2. paths from u to all other nodes of the graph exist,

3. the in-degree of u is zero,

4. the in-degree of every other node of the graph is one.


Trees

Definition 182 (Child and parent, Dt.: Kind und Eltern)

For a rooted tree T = (V ,E) and nodes u, v ∈ V , the node v is a child of the nodeu, and u is the parent of v , if the edge uv belongs to E . Siblings are nodes whichshare the same parent.

Definition 183 (Descendant and ancestor, Dt.: Nachfahre und Vorfahre)

In a rooted tree T = (V ,E), with u, v ∈ V , a node v is a descendant of a node u,and u is an ancestor of v , if u 6= v and if the path from the root to v contains u.

Definition 184 (Leaf, Dt.: Blatt)

A leaf of a rooted tree is a node without children. For a tree (that is not rooted) aleaf is a node with degree 1. All non-leaf nodes of a (rooted) tree are called innernodes.


Real-World Application: File System as a Rooted Tree

/

bin home lib usr tmp var

bash cowi1 cowi2 bin include

man sshaugsten held

public-www

held.html

bin

cl.shpics

tmp

f.txt

img1.jpg img2.jpg img3.jpg

I The root of the tree is the rootdirectory /.

I Inner nodes are (non-empty)directories.

I Leaves are files (or emptydirectories).


Trees

Definition 185 (Subtree, Dt.: Teilbaum)

A tree T ′ = (V ′,E ′) is a (proper) subtree of a tree T = (V ,E) rooted at the node uif

1. T ′ is a subgraph of T ,

2. T ′ is rooted at a node v , where v is a child of u, and

3. T ′ contains all descendants of v within T , together with the appropriate edgesof E .

WarningSome authors do not require the node v to be a child of u but allow v to be anydescendant of u.

Definition 186 (Forest, Dt.: Wald)

A forest is a graph all of whose connected components are trees.


Trees: Elementary Properties

Theorem 187

Every pair of nodes in a tree is connected by exactly one path.

Theorem 188

In a rooted tree there exists exactly one path from the root to any node.

Lemma 189

Removing an edge from a (rooted) tree results in a graph with two connectedcomponents, each of which is a (rooted) tree.


Trees: Elementary Properties

Theorem 190

For every (rooted) tree T = (V ,E) we get |E | = |V | − 1.

Proof of Theorem 190 for rooted trees : We note that the subtree relation on rootedtrees is well-founded and, thus, we can employ the principle of well-founded induction.Obviously, the claim holds for the minimal elements relative to this relation, i.e., fortrees that contain no proper subtrees and, thus, have only a root and no edges.Now suppose that the equality claimed holds for all k subtrees (V1,E1), . . . , (Vk ,Ek )of a rooted tree T = (V ,E). (We do not need to assume explicitly that it holds also forall subtrees of subtrees of T .) We get

|E | = k +k∑

i=1

|Ei | = k +k∑

i=1

(|Vi | − 1) = k + (−k) +k∑

i=1

|Vi | =k∑

i=1

|Vi |

= |V | − 1,

thus establishing the claim also for T = (V ,E).

Corollary 191

If |V | > 1 holds for a (rooted) tree T = (V ,E), then T has at least one leaf.


Trees

Definition 192 (Depth, Dt.: Tiefe)

The depth of the root u of a rooted tree T = (V ,E) is 0, and the depth of a nodev 6= u of T is k if the depth of the parent of v is k − 1, for all v ∈ V .

WarningSome authors prefer to regard the root as a node at depth 1. Hence, make sure tocheck how depth is defined in a textbook prior to using the results stated!

0

1 1 1

2 2 2

3


Trees

Definition 193 (Level, Dt.: Niveau)

A level of a rooted tree T comprises all nodes of T which have the same depth.

Definition 194 (Height, Dt.: Höhe)

The height of a rooted tree T is the maximum depth of nodes of T .

0

1 1 1

2 2 2

3

Level 1

height


Trees

Definition 195 (Ordered tree, Dt.: geordneter Baum)

An ordered tree is a rooted tree T such that the children of every node of T arearranged in some specific order, e.g., by means of a numbering scheme.

Definition 196 (Binary tree, Dt.: Binärbaum)

A binary tree is an ordered tree T consisting of a root node u and at most twoproper subtrees, which are called left subtree, L, and right subtree, R. If L (R, resp.)is a proper subtree then it is in turn a binary tree rooted in the left (right, resp.) childof u.

Definition 197 (Binary search tree, Dt.: binärer Suchbaum)A binary search tree is a binary tree T which has distinct values associated with itsnodes such that (relative to some total order)I if it has a proper left subtree L then

1. all values of nodes in L are less than the root value,2. L is a binary search tree itself,

I if it has a proper right subtree R then3. all values of nodes in R are greater than the root value,4. R is a binary search tree itself.


Balanced Binary Trees

Definition 198 (k-balanced tree, Dt.: k-balanzierter Baum)

A rooted binary tree is height-balanced with balance factor k if it either has noproper subtrees or if

1. it has two proper subtrees and the heights of both subtrees differ by not morethan k , or if

2. it has one proper subtree of height at most k − 1,

and if

3. all proper subtrees are height-balanced with balance factor k .

I E.g., for k = 1: AVL tree.I Trees with balance factor 1 are simply called balanced or self-balancing.

Definition 199 (Perfectly balanced binary tree, Dt.: perfekt balanz. Binärbaum)

A binary tree T is perfectly balanced if leaf nodes occur at most in the levels h andh − 1, where h is the height of T .

I E.g., a (binary) heap is a perfectly balanced binary tree.


Height-Related Properties of Binary Trees

Lemma 200

For i ∈ N0, level i of a binary tree contains at most 2i nodes.

Proof by induction : The claim holds for i = 0. If we have at most 2k nodes on level kthen we have at most 2 · 2k = 2k+1 nodes on level k + 1.

Lemma 201

Let h be the height and n be the number of nodes of a binary tree. Thenh ≥ dlog2(n + 1)e − 1, i.e., h ∈ Ω(log n).

Proof : Lemma 200 implies that a binary tree with height h contains at most

h∑i=0

2i = 2h+1 − 1

nodes. Hence, n ≤ 2h+1 − 1 and, thus, h ≥ dlog2(n + 1)e − 1.

Theorem 202

If T is a balanced binary tree with n nodes and height h then h ∈ Θ(log n).


Spanning Trees

Definition 203 (Spanning tree, Dt.: spannender Baum)

A spanning tree of a connected graph G is a subgraph of G that

1. is a tree,

2. includes all vertices of G.

Theorem 204Every connected graph G contains a spanning tree.

a d

b c

e

a d

b c

ee.g.


Spanning Trees

Definition 205 (Weighted graph, Dt.: gewichteter Graph)

An (edge-)weighted graph is a graph in which every edge is assigned a(non-negative) real number, the so-called weight or cost.

Definition 206 (Minimum spanning tree, Dt.: minimal spannender Baum)

A minimum spanning tree (MST) of a weighted connected graph G is a spanningtree T of G such that the sum of the weights of the edges of T is minimum over allspanning trees of G.

a d

b c

e

3

3

1

42

1

5

a d

b c

e

1

2

1

3


Real-World Application: IP Multicast Distribution Tree

I IP Multicasting is a one-to-many communication within an IP-based network:One source sends data to several receivers.

I Multicasting is based on a receiver-driven generation of a multicast distributiontree, thus reducing the amount of data that has to be distributed concurrently.

I Roughly, a multicast distribution tree is a spanning tree of all multicast receivers(or receiver groups), the source and intermediate routers.

S

R R

R

R

R

R

R

R


Real-World Application: Collaboration Graph and the Erdös Number

I A collaboration graph for a set of n scientists is a graph with n vertices such thattwo vertices are connected by an edge if the corresponding scientists have atleast one joint publication.

I The Erdös number of a scientist is the “collaborative distance” of a scientist to theextremely prolific Hungarian mathematician Paul Erdös (1913–1996, more than500 co-authors and more than 1 500 publications): Erdös has 0, and a scientisthas Erdös number k + 1 if k is the lowest Erdös number of his/her co-authors.

I One’s Erdös number can be obtained by computing minimum-weight paths on acollaboration graph: Single-source shortest path or minimum-spanning tree (forappropriate weights).

Erdos

11

11 2

22

3

4567

∞

∞∞


Complete and Bipartite Graphs

Definition 207 (Complete graph, Dt.: vollständiger Graph)A complete graph on n vertices, commonly denoted by Kn, is an undirected graphwith n vertices in which every pair of vertices is adjacent.

a

b

c

d

eK5

a

b

c

d

f

e

a

b

c

d

f

eK3,3

Definition 208 (Bipartite graph, Dt.: bipartiter Graph)

An undirected graph G = (V ,E) is a bipartite graph if V can be partitioned into two(non-empty) subsets V1,V2 such that E ⊆ v1, v2 : v1 ∈ V1, v2 ∈ V2.

Definition 209 (Complete bipartite graph, Dt.: vollständig-bipartiter Graph)

An undirected graph G = (V ,E) is a complete bipartite graph if V can be partitionedinto two (non-empty) subsets V1,V2 such that E = v1, v2 : v1 ∈ V1, v2 ∈ V2. Ifn := |V1| and m := |V2| then G is denoted by Kn,m.



I The edges and corners of a cube can be interpreted as a bipartite graph.

E HA B

C D

E F

G H

A B

CD

F G

I If we add all diagonals that cross the cube then we get K4,4.

E HA B

C D

E F

G H

A B

CD

F G



Lemma 210

Let G = (V ,E) be a bipartite graph and let V1,V2 be the partition of V according toDef. 208. Then∑

v1∈V1

deg(v1) =∑

v2∈V2

deg(v2) = |E |.

Proof :I As each edge has exactly one vertex from V1, we can write∑

v1∈V1

deg(v1) = |E |.

I Similarly,∑v2∈V2

deg(v2) = |E |.


Real-World Application: Task Assignment and Matchings

I Suppose that we are given a set of tasks and a set of processors. We knowwhich processor can carry out which tasks.

I These relations can be represented as a bipartite graph.I How can we get the maximum number of tasks processed concurrently?

Definition 211 (Matching, Dt.: Paarung)

I A matching in a simple graph G = (V ,E) is a subset E ′ of E such that no twoedges of E ′ are incident upon the same vertex of V .

I A maximal matching is a matching that does not allow to add an additionaledge.

I A maximum matching is a matching with the largest-possible number of edges.I A perfect matching is a matching that leaves no vertex unmatched.

P1 P2 P3 P4 P5 P6

T1 T2 T3 T4 T5


Hypercube

Definition 212 (Hypercube)

For n ∈ N0, the hypercube Qn is defined recursively as follows:

1. Q0 is a single vertex;

2. Qn+1 is obtained by taking two disjoint copies of Qn and linking each vertex inone copy of Qn to the corresponding vertex in the other copy of Qn.

Q2

Q2

Q3 000 001

011010

Q2

Q0

Q1

101

111110

100

I We could also obtain Qn by labeling 2n vertices with distinct n-bit binary strings,and by connecting those vertices by edges whose strings differ in exactly one bit.

Lemma 213

Qn is a regular bipartite graph of degree n with 2n vertices and n · 2n−1 edges.


Real-World Application: Hamilton Cycles in Qn Yield Gray Codes

Definition 214 (Gray code)

A (cyclic) Gray code of an ordered sequence of 2n entities, for n ∈ N, is a sequenceof n-bit binary strings such that the encodings of two neighboring entities haveHamming distance one, i.e., differ by exactly one bit.

I Gray codes are widely used in position encoders and for error detection andcorrection in digital communication.

Lemma 215

For n ∈ N with n ≥ 2, the number of different n-bit Gray codes equals the number ofdifferent Hamilton cycles in Qn.

000 001

011010

101

111110

000 001

011010

101

111110

100100


Isomorphic Graphs

Definition 216 (Isomorphic, Dt.: isomorph)

Two (directed) graphs G1 = (V1,E1) and G2 = (V2,E2) are isomorphic, denoted byG1 ' G2, if there exists a one-to-one mapping f between V1 and V2 that preservesadjacency; i.e., uv ∈ E1 ⇔ f (u)f (v) ∈ E2 for all u, v ∈ V1. Such a suitable function fis called graph isomorphism.

a

b

c

d

e 1

2

3

4

5

G1 ' G2

G1 G2

Lemma 217The relation ' is an equivalence relation on graphs.


Isomorphic Graphs

I Don’t be fooled by drawings! Two graphs may be isomorphic even if theirdrawings look strikingly different.

A B C D

E F G H

1 2

34

5 6

78

I Necessary (but not sufficient) conditions for two graphs to be isomorphic: samenumbers of vertices and edges, same degrees.

I The complexity of the graph isomorphism problem for general n-vertex graphs isunknown. No polynomial-time algorithm is known, but the problem is also notknown to be NP-complete. In December 2015, Lazlo Babai announced adeterministic algorithm that runs in time 2O(logc n) time for some positive constantc, i.e., in quasi-polynomial time.

I Practically efficient algorithms for graph canonical labeling are known, though.


Real-World Application: Non-Isomorphic Trees Represent Molecules

I Molecules can be represented as graphs, where atoms are represented byvertices and bonds are represented by edges [Arthur Cayley 1857].

I Saturated hydrocarbons, CnH2n+2, are given by trees where each carbon atom isrepresented by a degree-four vertex and each hydrogen atom is a leaf.

I How many different isomers can exist for n := 4?I We have exactly two non-isomorphic trees of this type and, thus, two different

isomers of C4H10, namely butane and isobutane.

H C C C C H

H H H H

HHHH H C C

C

C H

H

H H

H

H

H

HH

Butane Isobutane


Planar Graphs

Definition 218 (Planar graph, Dt.: planarer oder plättbarer Graph)A planar graph is a graph which can be drawn in the plane without edge crossings.Such a drawing is called a (planar) embedding (Dt.: planare Einbettung).

I Note: A graph may be planar even if a non-planar embedding is seen!

A B C D

E F G H

A

B

C

D

E

F

G

H

Definition 219 (Planar subdivision, Dt.: planare Unterteilung)

A face of an embedding of a planar graph is a maximal connected region of theplane that is disjoint from all edges.An embedding of a planar graph induces a planar subdivision, i.e., it subdivides theplane into several connected faces. (One of those faces, the so-called outer face,may be unbounded.)


Subdivision of a Graph

Definition 220 (Subdivision, Dt.: Unterteilung)

An edge subdivision of the edge uv ∈ E by means of the vertex w 6∈ V transformsthe graph G = (V ,E) into the graph G′ = (V ′,E ′), where V ′ = V ∪ w andE ′ = (E \ uv) ∪ uw ,wv.A graph G′ = (V ′,E ′) is a subdivision graph of G = (V ,E) if G′ is obtained fromG = (V ,E) via a finite series of edge subdivisions.

a

b

c

d

e

a

b

c

d

ef h

g

Theorem 221 (Kuratowski (1930))

A graph is planar if and only if it does not contain a subgraph that is isomorphic to asubdivision graph of K5 or K3,3.

Corollary 222

If a graph contains K5 or K3,3 as a subgraph then it is not planar.


Planar Graphs

I The following graph contains a subdivision graph of K5 as a subgraph. Hence, itis not planar.

Theorem 223 (Wagner (1936), Fáry (1948), Stein (1951))Any planar graph can be drawn in the plane without edge crossings such that all itsedges are straight-line segments: planar straight-line graph (PSLG).

Theorem 224 (Hopcroft&Tarjan (1974))

Testing whether a given graph with n vertices is planar can be done in O(n) time.


Euler’s Formula for Planar Graphs

Theorem 225 (Euler, Dt.: Eulerscher Polyedersatz)

Consider an embedding of a connected planar graph G. We denoteI the number of its vertices by v ,I the number of its edges by e,I the number of its faces by f .

Thenv − e + f = 2.

Proof : Suppose that G is connected but no tree. Therefore G contains a cycle, andwe may remove an edge from G without destroying its connectivity. The removal ofone edge of a cycle decreases both e and f by one, implying that the value ofv − e + f does not change. By using induction we can prove that a series of suchedge removals (for breaking up cycles) does not change the value of v − e + f , whileallowing us to transform G into a tree.If, however, G is a tree then Thm. 190 tells us that 1 = v − e. Since f = 1, we get2 = v − e + f , thus establishing the claim.

I Euler’s Formula generalizes to v − e + f = 1 + c for a planar graph with cconnected components.



Corollary 226

Let v , e, f for a connected planar graph G as defined in Theorem 225. If v ≥ 3 then

e ≤ 3v − 6 and f ≤ 2v − 4 and f ≤ 23

e.

If every vertex of G has a degree of three or greater then we get

v ≤ 23

e and e ≤ 3f − 6 and v ≤ 2f − 4.

Furthermore, every planar graph contains one node with degree at most five.

Proof : We prove that 3f ≤ 2e, which is obvious for f = 1. We call an edge a “side” ofa face if the edge is in the boundary of the face. Let k denote the total number ofsides.If f > 1 then each face is bounded by at least three sides, so k ≥ 3f .But each edge has at most two different sides, so k ≤ 2e.We conclude 3f ≤ 2e.



Corollary 227K5 is not planar.

Proof : We get v = 5 and e =(5

2

)= 10. So, e ≤ 3v − 6 (Cor. 226) does not hold.

Definition 228 (Triangle-free, Dt.: dreiecksfrei)

A triangle-free graph is a graph which does not contain a cycle of length three, i.e.,in which no three vertices form a triangle of edges.

Corollary 229

A triangle-free planar graph has one node of degree at most three and e ≤ 2v − 4holds (if v ≥ 2).

Corollary 230K3,3 is not planar.

Proof : K3,3 is triangle-free and has six vertices and nine edges. If it were planar then,by Cor. 229, it could have at most 2 · 6− 4 = 8 edges. Thus, K3,3 is non-planar.


Real-World Application: Number of Edges of a Polyhedron

I Suppose that a polyhedral model has n vertices. How many edges and faces canit have at most? What is the storage complexity relative to n?

Theorem 231

The vertices and edges of a simple (bounded) polyhedron form a planar graph.

Corollary 232

A simple (bounded) polyhedron with n vertices has at most 3n − 6 edges and2n − 4 faces.


Graph Coloring

Definition 233 (Coloring, Dt.: Färbung)

An assignment of colors to all vertices of a graph G is called a (vertex) coloring ifadjacent vertices are assigned different colors.

Definition 234 (k-colorable, Dt.: k-färbbar )

A graph G is k-colorable if G admits a coloring that uses k colors.

Definition 235 (Chromatic number, Dt.: chromatische Zahl)

The chromatic number of a graph G, written as χ(G), is the least number of colorsrequired to color G.

I χ(Kn) = n.I Determining χ(G) is NP-hard even if G is a planar 4-regular graph. Thus, it is

rather unlikely that a polynomial-time algorithm will ever be found for determiningχ(G). However, fairly efficient heuristics exist.

Lemma 236

A graph is 2-colorable if and only if it is bipartite.


Graph Coloring

I It is straightforward that every planar graph can be colored by six colors and thatevery triangle-free planar graph can be colored by four colors.

I Still easy to see: Every planar graph can be colored by five colors.

Theorem 237 (Four Color Theorem, Haken and Appel (1976))

Every planar graph can be colored using no more than four colors.

I Wolfgang Haken and Kenneth Appel used a super-computer at the University ofIllinois to check 1936 “reducible” configurations. The proof is not accepted by allmathematicians as it has two parts, one of which can only be solved usingcomputers. (And the second part that is solveable by hand is also very tedious.)

I In 1996, Robertson et alii reduced the number of computer-checked cases to633.

I In 2005, Benjamin Werner and Georges Gonthier used a general-purpose proofassistant (“Coq”) to prove the theorem.

Corollary 238

If every entity of a topographic map is a connected area then four colors suffice tocolor the map such that no two entities that share a common border (other than acommon point) are colored with the same color.


Real-World Application: Channel Assignment

I We can solve the channel assignment problem by considering its so-calledunit-disk graph, whereI the vertices are given by the broadcast stations,I two vertices are connected by an edge if their service areas overlap.

I Obviously, the chromatic number of that graph equals the minimum number offrequencies needed.


Real-World Application: Index Registers

I Optimizing compilers try to store frequently used variables of the body of a loopin index registers of the CPU (rather than in regular memory).

I How many index registers are needed for a given loop?I We set up a graph whose vertices are given by the variables, and where two

vertices are connected by an edge if the corresponding variables ought to bekept in registers at the same time.

I Then the chromatic number of that graph gives the minimum number of registersneeded.

I Other applications of graph coloring:I Scheduling consumer-producer interactions to allow concurrency.I Sudoku puzzles.


CryptographyIntroductionSymmetric-Key CryptographyPublic-Key Cryptography


Introduction — What is Cryptography?

I Cryptography is the science of sending and receiving messages in secret code.I A sender A (“Alice”) sends an encoded message to a receiver B (“Bob”).I The goal is to keep the transmission of the message secure (from others to read

it) and to ensure successful communication of the information.I Cryptography has been used for at least two thousand years, see Caesar’s

cipher.I Likely, the most famous historical example of cryptography is the Enigma

machine; its security breach ultimately helped significantly with the defeat of thesubmarine force of the German Third Reich.

I Two main schemes are in use nowadays:Symmetric-Key Cryptography (SKC): the same secret key is used for both

encryption and decryption; aka secret-key cryptography.Public-Key Cryptography (PKC): different keys are used for encryption and

decryption, with some keys being known publicly; akaasymmetric-key cryptography.


Basic Terms

I Plaintext — original message.I Ciphertext — encoded/encrypted message.I Encryption — generating ciphertext from plaintext.I Decryption / Deciphering — generating plaintext from ciphertext.I Cryptanalysis — trying to break the encryption by applying various methods.I Adversary, Spy — the message thief.I Eavesdropper — a secret listener who listens to private conversations.I Authentication — the process of proving one’s identity.I Privacy — ensuring that the message is read only by the intended receiver.

(GnuPG: “Privacy is not a crime!”)


Eavesdropper Attacks

Alice Bob

Eve

ciphertext

ciphertextciphertext

plaintext = Decrypt( ciphertext, key )ciphertext = Encrypt( plaintext, key )

I Eve might attempt toI break the encryption,I replay the encrypted message (e.g., login) without breaking the encryption,I modify the message,I block the message,I fabricate a new message.


Classical Cryptography: Caesar’s Cipher

I According to Suetonius, Caesar (100BC–44BC) used an encryption scheme (forcommunication with his generals) that shifted the alphabet of the plaintext bysome fixed value.

X Y Z A B CW D E F G

B C D E F GA H I J K

V

Z

I With n := 4:Plaintext: alea iacta estCiphertext: epie megxe iwx

I Suppose that the (Roman) letters are mapped to the numbers 0, 1, . . . , 25.I Then Caesar’s encryption and decryption with shift n can be computed as follows:

ciphertext := Encryptn(plaintext) = (plaintext + n) mod 26

plaintext := Decryptn(ciphertext) = (ciphertext − n) mod 26


Classical Cryptography: Caesar’s Cipher

I Likely, Caesar’s cipher was reasonably secure at the time when it was used.I It is broken easily by means of frequency analysis and brute-force attacks — it

offers no security by today’s standards!I Nevertheless, Caesar’s cipher with n := 13, aka ROT13, has been (mis-)used for

serious applications even rather recently.I However, it is used within more complex systems, e.g., the Vigenère cipher.

I On a Unix machine, the tr utility can be used for carrying out Caesar’s cipher.E.g.,echo "alea iacta est" | tr ’A-Za-z’ ’E-ZA-De-za-d’yieldsepie megxe iwx.


Symmetric-Key Cryptography

I A single secret key is used for both encryption and decryption (aka “secret-keyalgorithms”).

plaintext ciphertext plaintextkey key

I Simple example: Suppose that Alice wants to encrypt a bit string A. Then Aliceand Bob could choose a secret key B and apply a bit-wise XOR (exclusive OR,⊕) — an output bit is 1 if exactly one of the two input bits is 1 — in order totransmit A⊕ B. Then Bob would compute (A⊕ B)⊕ B and, thus, retrieve A.

A B A⊕ B (A⊕ B)⊕ B0 0 0 00 1 1 01 0 1 11 1 0 1

I Of course, the key B must be known to both Alice and Bob, and, in fact, it mustnot be known to anybody else.


Key Distribution Problem

I The key B must be known to both Alice and Bob, and, in fact, it must not beknown to anybody else.

I That is, Alice and Bob need to share the secret key in order to be able to encryptand decrypt their messages!

I What is a secure mechanism for them to exchange a key??I Meet in person at a secret place and share the key?!I Share in parts?!

I The key distribution problem is a major roadblock on the road to securecommunication among folks who do not meet regularly.

I A second big disadvantage is the need for multiple keys in order to encryptmessages intended for different receivers.


Public-Key Cryptography

I A pair of keys is used to encrypt and decrypt the messages, with one key beingpublic.

plaintext ciphertext plaintextkey1 key2

I PKC make use of so-called one-way functions which are “easy” to compute forevery input but extremely “hard” to invert for an output given. For example,considerI multiplication versus factorization (“factorization problem”):

I If f (a, b) := a · b, thenf (a, b) = 533 for a := 13, b := 41;

I While you can calculate f (13, 41) in your head, it is less trivial to obtaina, b such that f (a, b) = 533;

I exponentiation versus logarithms (“discrete log problem”):I If f (a, b) := ab, then

f (a, b) = 243 for a := 3, b := 5;I Again, finding a and b such that loga 243 = b is considerably more

difficult.I Whitfield Diffie and his advisor Martin Hellman were the first to publish a PKC

scheme in 1976. (They were the recipients of the 2015 ACM Turing Award.)


Real-World Application: Diffie-Hellman Symmetric Key Exchange

I Alice and Bob share two public numbers: a (large) prime number p ∈ P and aso-called generator g ∈ 2, 3, . . . , p − 1 such that for every n ∈ 1, 2, . . . , p − 1there exists a k ∈ N with n = gk mod p. (Then g is a “primitive root” modulo p).

Alice Bob(1) selects s with 1 < s < p − 1 selects t with 1 < t < p − 1(2) sends S := gs mod p to Bob sends T := gt mod p to Alice(3) calculates T s mod p calculates St mod p

I We have

T s = (gt mod p)s ≡p (gt )s = gts = (gs)t ≡p St .

Hence, k := T s mod p = St mod p can be used as a common key by Alice andBob.

I In general, the public information is p, g,S and T , while s and t are secret.I To find s, Eve could attempt to solve the discrete log problem gs = S mod p.

Same for t . At present, nobody knows how to solve this problem efficiently. It isnot even known how to compute gst mod p efficiently if gs mod p and gt mod pare known.


Real-World Application: Diffie-Hellman Key Exchange Sample

Alice Bob(1) selects s with 1 < s < p − 1 selects t with 1 < t < p − 1(2) sends S := gs mod p to Bob sends T := gt mod p to Alice(3) calculates T s mod p calculates St mod p

I Alice and Bob make p := 13 and g := 2 public.I Alice chooses the private value s := 5, while Bob chooses t := 6.I We get S := gs mod p = 25 mod 13 = 32 mod 13 = 6, and

T := gt mod p = 26 mod 13 = 12, which can be exchanged publicly, i.e., via aninsecure communication channel.

I Finally, T s mod p = 125 mod 13 = (12140 · 13 + 12) mod 13 = 12, andSt mod p = 66 mod 13 = (3588 · 13 + 12) mod 13 = 12.

I Hence, Alice and Bob have managed to exchange 12 as a master key for theirfuture communication.

I Of course, in practice considerably larger values are chosen for p!!I The Diffie-Hellman key exchange is vulnerable to man-in-the-middle attacks!


Number Theory and Cryptography

Lemma 239

Let a, b ∈ N such that gcd(a, b) = 1. Then there exists x ∈ Z such that a · x ≡b 1.

Proof : Since gcd(a, b) = 1, Cor. 60 tells us that there exist x , y ∈ Z such thata · x + b · y = 1. Hence, a · x = 1− b · y ≡b 1.

Definition 240 (Euler’s Totient Function, Dt.: Eulersche ϕ-Funktion)

Euler’s totient function ϕ : N→ N is defined as

ϕ(n) := |Un|, with Un := 1 ≤ x ≤ n : gcd(x , n) = 1.

The set Un is called the group of units of n.

I Hence, ϕ(n) gives the number of integers among 1, 2, . . . , n which are coprime ton.

I We have ϕ(4) = 2, ϕ(5) = 4, ϕ(6) = 2, and ϕ(p) = p − 1 for every p ∈ P.


Number Theory and Cryptography

Lemma 241

Let p, q ∈ P. If p 6= q then ϕ(pq) = (p − 1)(q − 1).

Proof : There are q multiples of p and p multiples of q within 1, 2, . . . , pq, and theonly common multiple of both p and q is pq. Hence, by the Inclusion-ExclusionPrinciple (Thm. 95), ϕ(pq) = pq − p − q + 1 = (p − 1)(q − 1).

Lemma 242 (Fermat/Euler )

Let n ∈ N and x ∈ Un. Then xϕ(n) ≡n 1.

Corollary 243

Let n ∈ N and x ∈ Un. If n = pq, with p, q ∈ P and p 6= q, then x (p−1)(q−1) ≡n 1.


Real-World Application: RSA Cryptosystem

I The RSA system [Rivest, Shamir, Adleman 1977] makes use of Lemma 242 andof the fact that state-of-the-art factorization methods take far too long for productsof numbers with several hundred digits each.

I The basic idea is very simple:I Select two distinct prime numbers p and q (each of which, in practice, has at

least 150 digits) and compute n = p · q.I Lemma 241 tells us that ϕ(n) = (p − 1) · (q − 1).I Select an integer e ∈ N \ 1 such that gcd(e, ϕ(n)) = 1.I The numbers n and e are published (Bob’s public key ).I Compute a number d which is the inverse of e in Zϕ(n), i.e., such that

d · e ≡ϕ(n) 1. (Such a number exists due to Lem. 239.)I The number d is called Bob’s private key and is kept secret.


Real-World Application: RSA Cryptosystem

I Hence, we have n = p · q and, thus, ϕ(n) = (p − 1) · (q − 1).Furthermore, gcd(e, ϕ(n)) = 1 and d · e ≡ϕ(n) 1.

I Encoding the ciphertext:I Alice encodes a message x ∈ N, with x < n to keep it in Zn and with

gcd(x , n) = 1, by using Bob’s public key e and n:y := xe mod n with 0 < y < n.

I Decoding the ciphertext:I Bob computes z := yd mod n with 0 < z < n.

I Why does z = x mod n hold?I The condition d · e ≡ϕ(n) 1 ensures that there exists k ∈ Z such that

d · e = k · ϕ(n) + 1.

I It follows that

z ≡n yd ≡n xde = xkϕ(n)+1 = (xϕ(n))k xL. 242≡n (1)k x = x .


Real-World Application: RSA Cryptosystem Sample

I Suppose that p = 5 and q = 11. Hence n = 55 and ϕ(n) = 40. Suppose furtherthat three users chose the following keys:

e dAlice 23 7Bob 37 13Caesar 9 9

I We have 23 · 7 ≡40 1 and 37 · 13 ≡40 1 and 9 · 9 ≡40 1.I Let x := 2 (and use Mathematica to do the arithmetic).

Alice: 223 = 8388608 = 152520 · 55 + 8 ≡55 8 =: y87 = 2097152 = 38130 · 55 + 2 ≡55 2 =: z

Bob: 237 = 137438953472 = 2498890063 · 55 + 7 ≡55 7 =: y713 = 96889010407 = 1761618371 · 55 + 2 ≡55 2 =: z

Caesar: 29 = 512 = 9 · 55 + 17 ≡55 17 =: y179 = 118587876497 = 2156143209 · 55 + 2 ≡55 2 =: z


Real-World Application: RSA Cryptosystem Analysis

I Note that there are ϕ(n) many messages that can be sent for n given.I Since

ϕ(n)

n=

(p − 1)(q − 1)

pq= (1− 1

p)(1− 1

q)

is close to 1 for large p, q, the probability of selecting a message that is coprimeto n is the larger the larger p, q are.

I Although n is publicly known, it is important to keep ϕ(n) and, thus, also p, qsecret!

I An eavesdropper who only knows n, e, and y cannot do much with thisinformation. In particular, no efficient algorithm is known to factor n into p, q as asimple means to obtain ϕ(n).

I It is also important to ensure that xe > n, i.e., that y is obtained by exponentiationand then by a reduction modulo n.I If xe < n then one could simply recover x by taking the e-th root of y . (After

all, e is known publicly!)I Hence, it is wise to select e such that 2e > n.


The End!

I hope that you enjoyed this course, and I wish you all the best for your future studies.

Computational Geometry and Applications Lab

UNIVERSITAT SALZBURG


Diskrete Mathematik für Informatik (SS 2020)held/teaching/diskrete_mathematik/dm... ·...

Documents

Transcript of Diskrete Mathematik für Informatik (SS 2020)held/teaching/diskrete_mathematik/dm... ·...