Great Theoretical Ideas In Computer Science Steven Rudich Lecture 2 CS 15-251 Jan 16, 2003 Spring 2003 Carnegie Mellon University Unary, Binary, and Beyond Last time we talked about unary notation. nth Triangular Number n = 1 + 2 + 3 + . . . + n-1 + n = n(n+1)/2 nth Square Number n = 1 + 3 + … + 2n-1 = Sum of first n odd numbers nth Square Number n = n + n-1 = n2 ( n)2 = ( n-1)2 + ( n)2 = + +...+ n n Unary takes size n to represent the number n. Higher bases give a logarithmically more compact representation. Base X, X>1 A base X digit is any integer 0 · a < X Let S = an-1, an-2, …, a0 be a sequence of base X digits. S is said to be the base X representation of the number: an-1 Xn-1 + an-2 Xn-2 + . . . + a0 X0 Base 2 = Binary Base 10 = Decimal A base X digit is any integer 0 · a < X Let S = an-1, an-2, …, a0 be a sequence of base X digits. S is said to be the base X representation of the number: an-1 Xn-1 + an-2 Xn-2 + . . . + a0 X0 Sequence S = an-1, an-2, …, a0 represents the number: an-1 Xn-1 + an-2 Xn-2 + . . . + a0 X0 Base 2 101 represents 1 (2)2 + 0 (21) + 1 (20) Base 7 015 represents 0 (7)2 + 1 (71) + 5 (70) Sequence S = an-1, an-2, …, a0 represents the number: an-1 Xn-1 + an-2 Xn-2 + . . . + a0 X0 Base 2 101 represents 1 (2)2 + 0 (21) + 1 (20) = Base 7 015 represents 0 (7)2 + 1 (71) + 5 (70) = Biggest n “digit” number in base X would be: (X-1)Xn-1 + (X-1)Xn-2-1+…(X-1)X0 Base 2 111 represents 1 (2)2 + 1 (21) + 1 (20) Base 7 666 represents 6 (7)2 + 6 (71) + 6 (70) Biggest n “digit” number in base X would be: (X-1)Xn-1 + (X-1)Xn-2-1+…(X-1)X0 Base 2 111 represents 1 (2)2 + 1 (21) + 1 (20) 111 + 1 = 1000 represents 23 Thus, 111 represents 23 – 1 Base 7 666 represents 6 (7)2 + 6 (71) + 6 (70) 666 + 1 = 1000 represents 73 Thus, 666 represents 73 - 1 Biggest n “digit” number in base X would be: S= (X-1)Xn-1 + (X-1)Xn-2-1+…(X-1)X0 \Add 1 to get: Xn + 0 Xn-1 + … + 0 X0 Thus, S = Xn - 1 Base 2 111 represents 1 (2)2 + 1 (21) + 1 (20) 111 + 1 = 1000 represents 23 Thus, 111 represents 23 – 1 Base 7 666 represents 6 (7)2 + 6 (71) + 6 (70) 666 + 1 = 1000 represents 73 Thus, 666 represents 73 - 1 (X-1)Xn-1 + (X-1)Xn-2-1+…(X-1)X0 = Xn - 1 Factoring out (x-1). We obtain: (X-1) (Xn-1 + Xn-2 + ….. +X + 1) = Xn – 1 Dividing by X-1 (since X is not equal to 1): (Xn-1 + Xn-2 + ….. +X + 1) = (Xn – 1) / (X -1) Or. (1 + X + X2 + … + Xn-1) = (Xn – 1) / (X -1) 1 + X1 + X2 + X 3 + … + Xn-2 + Xn-1 = Xn – 1 X- 1 This is a super important sum called the Geometric Series (or Sum). Let’s prove this again, a different way. . . An Important Calculation (X-1) ( 1 + X1 + X2 + X 3 + … + Xn-2 + Xn-1 ) = X1 + X 2 + X 3 + … + Xn-1 + Xn - 1 - X1 - X2 - X 3 - … - Xn-2 – Xn-1 = - 1 = + Xn - 1 Xn The Geometric Series (X-1) ( 1 + X1 + X2 + X 1 + X1 + X2 + X 3 3 + … + Xn-1 ) = Xn - 1 + … + Xn-2 + Xn-1 = Xn – 1 X- 1 1 + X1 + X 2 + X 3 + … + Xn-1 + Xn = Xn+1 - 1 X- 1 The highest n digit number in base X. (X-1) ( 1 + X1 + X2 + X 3 + … + Xn-1 ) = Xn - 1 Base X. Let S be the sequence of n digits each of which is X-1. S is the largest number of length n that can be represented in base X. S = Xn - 1 Notice that we have not yet shown that each base-b number has a unique representation. In service of this, we will introduce a variation of base-b called “plus minus base-b”. Plus/Minus Base X An plus/minus base X digit is any integer –X < a < X Let S = an-1, an-2, …, a0 be a sequence of plus/minus base X digits. S is said to be the plus/minus base X representation of the number: an-1 Xn-1 + an-2 Xn-2 + . . . + a0 X0 Does each n-digit number in plus/minus base X represent a different integer? No. Consider plus/minus binary. 5 = 0 1 0 1 = 1 0 -1 -1 Humm.. So what is it good for? Not every number has a unique plus/minus binary representation, but 0 does. 0 has a unique n-digit plus/minus base X representation as all 0’s. Suppose an-1 Xn-1 + an-2 Xn-2 + . . . + a0 X0 = S =0, where there is some highest k such that ak 0. Wl.o.g. assume ak > 0. Because Xk > (X-1)(1 + X1 + X2 + X 3 + … + Xk-1) no sequence of digits from ak-1 to a0 can represent a number as small as –Xk Hence S 0. Contradiction. Each of the numbers from 0 to Xn-1 is uniquely represented by an n-digit number in base X. We already know that n–digits will represent something between 0 and Xn – 1. Suppose two distinct sequences represent the same number: an-1 Xn-1 + an-2 Xn-2 + . . . + a0 X0 = bn-1 Xn-1 + bn-2 Xn-2 + . . . + b0 X0 The difference of the two would be an plus/minus base X representation of 0, but it would have a non-zero digit. Contradiction. Each of the numbers from 0 to Xn-1 is uniquely represented by an n-digit number in base X. n digits represent up to Xn – 1 n-1 digits represents up to Xn-1 - 1 Let k be a number: Xn-1 ≤ k ≤ Xn - 1 So k can be represented with n digits. For all k: logxk = n – 1 So k uses logxk + 1 digits. Fundamental Theorem For Base X: Each of the numbers from 0 to Xn-1 is uniquely represented by an n-digit number in base X. k uses logxk + 1 digits in base X. n has length n in unary, but has length log2n + 1 in binary. Unary is exponentially longer than binary. Bases In Different Cultures Sumerian-Babylonian: 10, 60, 360 Egyptians: 3, 7, 10, 60 Africans: 5, 10 French: 10, 20 English: 10, 12,20 Egyptian Multiplication The Egyptians used decimal numbers but multiplied and divided in binary Egyptian Multiplication a times b by repeated doubling b has some n-bit representation: bn..b0 Starting with a, repeatedly double largest so far to obtain: a, 2a, 4a, …., 2na Sum together all 2ka where bk = 1 Egyptian Multiplication 15 times 5 by repeated doubling 5 has some 3-bit representation: 101 Starting with 15, repeatedly double largest so far to obtain: 15, 30, 60 Sum together all 2k(15) where bk = 1: 15 + 60 = 75 Why does that work? b = b020 + b121 + b222 + … + bn2n ab = b020a + b121a + b222a + … + bn2na If bk is 1 then 2ka is in the sum. Otherwise that term will be 0. Wait! How did the Egyptians do the part where they converted b to binary? They used repeated halving to do base conversion. Consider … Egyptian Base Conversion Output stream will print right to left. Input X. Repeat until X=0 { If X is even then Output O; Otherwise {X:=X-1; Output 1} } X:=X/2 Egyptian Base Conversion Output stream will print right to left. Input X. Repeat until X=0 { If X is even then Output O; Otherwise Output 1 } X:= X/2 Start the algorithm 010101 1 Repeat until X=0 { If X is even then Output O; Otherwise Output 1; X:= X/2 } Start the algorithm 01010 1 Repeat until X=0 { If X is even then Output O; Otherwise Output 1; X:= X/2 } Start the algorithm 01010 01 Repeat until X=0 { If X is even then Output O; Otherwise Output 1; X:= X/2 } Start the algorithm 0101 01 Repeat until X=0 { If X is even then Output O; Otherwise Output 1; X:= X/2 } Start the algorithm 0101 101 Repeat until X=0 { If X is even then Output O; Otherwise Output 1; X:= X/2 } Start the algorithm 010 101 Repeat until X=0 { If X is even then Output O; Otherwise Output 1; X:= X/2 } And Keep Going until 0 0 010101 Repeat until X=0 { If X is even then Output O; Otherwise Output 1; X:= X/2 } Sometimes the Egyptian combined the base conversion by halving and the multiplication by doubling into one algorithm Rhind Papyrus (1650 BC) 70*13 70 140 280 560 13 * 6 3 * 1 * 70 350 910 Rhind Papyrus (1650 BC) 70*13 70 140 280 560 13 * 6 3 * 1 * 70 350 910 Binary for 13 is 1101 = 23 + 22 + 20 70*13 = 70*23 + 70*22 + 70*20 Rhind Papyrus (1650 BC) 17 34 68 136 184 48 14 1 2 * 4 8 * Rhind Papyrus (1650 BC) 17 34 68 136 1 2 * 4 8 * 184 48 14 184 = 17*8 + 17*2 + 14 184/17 = 10 with remainder 14 This method is called “Egyptian Multiplication/Division” or “Russian Peasant Multiplication/Division”. Wow. Those Russian peasants were pretty smart. Standard Binary Multiplication = Egyptian Multiplication X ******** 101 ******** ******** *********** Egyptian Base 3 We have defined Base 3: Each digit can be 0, 1, or 2 Plus/Minus Base 3 uses -2, -1, 0, 1, 2 Here is a new one: Egyptian Base 3 uses -1, 0, 1 Example: 1 0 -1 -1 = 9 - 3 - 1 = 5 Unique Representation Theorem for Egyptian Base 3 No integer has 2 distinct, n-digit, Egyptian base-3 representations. We can represent all integers from -(3n-1)/2 to (3n-1)/2 Proof; If so, their difference would be a non-trivial plus/minus base 3 representation of 0. Contradiction. Highest number = 1111…1 = (3n-1)/2 Lowest number = -1-1-1-1…-1 = -(3n-1)/2 Unique Representation Theorem for Egyptian Base 3 No integer has 2 distinct, n-digit, Egyptian base-3 representations. We can represent all integers from -(3n-1)/2 to (3n-1)/2 3n, n-digit, base 3 representations of the numbers from 0 to 3n – 1 Subtract 111111…111 = (3n – 1)/2 from each to get an Egyptian base 3 representation of all the numbers from -(3n-1)/2 to (3n-1)/2. How could this be Egyptian? Historically, negative numbers first appear in the writings of the Hindu mathematician Brahmagupta (628 AD). One weight for each power of 3. Left = “negative”. Right = “positive” How to add 2 n-bit numbers. + * * * * * * * * * * * * * * * * * * * * * * How to add 2 n-bit numbers. + * * * * * * * * * * * * * * * * * * * * * * * * How to add 2 n-bit numbers. + * * * * * * * * * * * * * * * * * * * * * * * * * * How to add 2 n-bit numbers. + * * * * * * * * * * * * * * * * * * * * * * * * * * * * How to add 2 n-bit numbers. + * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * How to add 2 n-bit numbers. + * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * How to add 2 n-bit numbers. + * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * Let k be the maximum time that it takes you to do * * * * * * Time < kn is proportional to n t i m e # of bits in numbers The time grow linearly with input size. If n people agree to help you add two n bit numbers, it is not obvious that they can finish faster than if you had done it yourself. Is it possible to add two n bit numbers in less than linear parallel-time? Darn those carries. We will look at parallel binary addition in a later lecture. For now, let me show you a super parallel plus/minus binary addition. Plus/minus binary means base 2 allowing digits to be from {-1, 0, 1}. We can call each digit a “trit”. n people can add 2, n-trit, plus/minus binary numbers in constant time! An Addition Party to Add 110-1 to -111-1 - An Addition Party 1 1 -1 -1 0 1 1 -1 Invite n people to add two n-trit numbers Assign one person to each trit position An Addition Party 0 2 1 0 -2 1 -1 1 Each person should add the two input trits in their possession. Problem: 2 and -2 are not allowed in the final answer. Pass Left 1 0 0 1 1 0 -2 -1 -1 If you have a 1 or a 2 subtract 2 from yourself and pass a 1 to the left. (Nobody keeps more than 0) Add in anything that is given to you from the right. (Nobody has more than a 1) After passing left 1 1 0 -2 -1 1 There will never again be any 2s as everyone had at most 0 and received at most 1 more Passing left again 1 1 0 -1 -1 1 1 0 If you have a -1 or -2 add 2 to yourself and pass a -1 to the left (Nobody keeps less than 0) After passing left again 1 0 1 0 0 -1 1 No -2s anymore either. Everyone kept at least 0 and received At most -1. 1 1 -1 = -1 0 1 1 1 0 1 -1 0 0 1 1 Caution: Parties and Algorithms Do not Mix 1 0 1 Later we will see how this is used inside all arithmetic chips. References The Book Of Numbers, by J. Conway and R. Guy History of Mathematics, Histories of Problems, by The Inter-IREM Commission