Great Theoretical Ideas In Computer Science
Steven Rudich
Lecture 2
CS 15-251
Jan 16, 2003
Spring 2003
Carnegie Mellon University
Unary, Binary, and Beyond
Last time we talked
about unary
notation.
nth Triangular Number
n
= 1 + 2 + 3 + . . . + n-1 + n
= n(n+1)/2
nth Square Number
n = 1 + 3 + … + 2n-1
= Sum of first n odd numbers
nth Square Number
n = n + n-1
= n2
( n)2 = ( n-1)2 +
( n)2 =
+
+...+
n
n
Unary takes size n
to represent the
number n.
Higher bases give a
logarithmically
more compact
representation.
Base X, X>1
A base X digit is any integer 0 · a < X
Let S = an-1, an-2, …, a0 be a sequence of
base X digits. S is said to be the base
X representation of the number:
an-1 Xn-1 + an-2 Xn-2 + . . . + a0 X0
Base 2 = Binary Base 10 = Decimal
A base X digit is any integer 0 · a < X
Let S = an-1, an-2, …, a0 be a sequence of
base X digits. S is said to be the base
X representation of the number:
an-1 Xn-1 + an-2 Xn-2 + . . . + a0 X0
Sequence S = an-1, an-2, …, a0
represents the number:
an-1 Xn-1 + an-2 Xn-2 + . . . + a0 X0
Base 2
101 represents 1 (2)2 + 0 (21) + 1 (20)
Base 7
015 represents 0 (7)2 + 1 (71) + 5 (70)
Sequence S = an-1, an-2, …, a0
represents the number:
an-1 Xn-1 + an-2 Xn-2 + . . . + a0 X0
Base 2
101 represents 1 (2)2 + 0 (21) + 1 (20)
=
Base 7
015 represents 0 (7)2 + 1 (71) + 5 (70)
=
Biggest n “digit” number in base X
would be:
(X-1)Xn-1 + (X-1)Xn-2-1+…(X-1)X0
Base 2
111 represents 1 (2)2 + 1 (21) + 1 (20)
Base 7
666 represents 6 (7)2 + 6 (71) + 6 (70)
Biggest n “digit” number in base X
would be:
(X-1)Xn-1 + (X-1)Xn-2-1+…(X-1)X0
Base 2
111 represents 1 (2)2 + 1 (21) + 1 (20)
111 + 1 = 1000 represents 23
Thus, 111 represents 23 – 1
Base 7
666 represents 6 (7)2 + 6 (71) + 6 (70)
666 + 1 = 1000 represents 73
Thus, 666 represents 73 - 1
Biggest n “digit” number in base X
would be:
S= (X-1)Xn-1 + (X-1)Xn-2-1+…(X-1)X0
\Add
1 to get: Xn + 0 Xn-1 + … + 0 X0
Thus, S = Xn - 1
Base 2
111 represents 1 (2)2 + 1 (21) + 1 (20)
111 + 1 = 1000 represents 23
Thus, 111 represents 23 – 1
Base 7
666 represents 6 (7)2 + 6 (71) + 6 (70)
666 + 1 = 1000 represents 73
Thus, 666 represents 73 - 1
(X-1)Xn-1 + (X-1)Xn-2-1+…(X-1)X0
= Xn - 1
Factoring out (x-1). We obtain:
(X-1) (Xn-1 + Xn-2 + ….. +X + 1) = Xn – 1
Dividing by X-1 (since X is not equal to 1):
(Xn-1 + Xn-2 + ….. +X + 1) = (Xn – 1) / (X -1)
Or.
(1 + X + X2 + … + Xn-1) = (Xn – 1) / (X -1)
1 + X1 + X2 + X
3
+ … + Xn-2 + Xn-1 =
Xn – 1
X- 1
This is a super
important sum called
the Geometric
Series (or Sum).
Let’s prove this
again, a different
way. . .
An Important Calculation
(X-1) ( 1 + X1 + X2 + X
3
+ … + Xn-2 + Xn-1 )
=
X1 + X 2 + X 3 + …
+ Xn-1 + Xn
- 1 - X1 - X2 - X 3 - … - Xn-2 – Xn-1
=
- 1
=
+
Xn - 1
Xn
The Geometric Series
(X-1) ( 1 + X1 + X2 + X
1 + X1 + X2 + X
3
3
+ … + Xn-1 ) = Xn - 1
+ … + Xn-2 + Xn-1 =
Xn – 1
X- 1
1 + X1 + X 2 + X
3
+ … + Xn-1 + Xn =
Xn+1 - 1
X- 1
The highest n digit number in base X.
(X-1) ( 1 + X1 + X2 + X
3
+ … + Xn-1 ) = Xn - 1
Base X. Let S be the sequence of n
digits each of which is X-1. S is the
largest number of length n that can be
represented in base X.
S = Xn - 1
Notice that we have not
yet shown that each
base-b number has a
unique representation. In
service of this, we will
introduce a variation of
base-b called “plus
minus base-b”.
Plus/Minus Base X
An plus/minus base X digit is any
integer –X < a < X
Let S = an-1, an-2, …, a0 be a sequence of
plus/minus base X digits. S is said to be
the plus/minus base X representation
of the number:
an-1 Xn-1 + an-2 Xn-2 + . . . + a0 X0
Does each n-digit
number in plus/minus
base X represent a
different integer?
No.
Consider plus/minus
binary.
5 = 0 1 0 1 = 1 0 -1 -1
Humm..
So what is it good for?
Not every number has a
unique plus/minus binary
representation, but 0
does.
0 has a unique n-digit plus/minus
base X representation as all 0’s.
Suppose an-1 Xn-1 + an-2 Xn-2 + . . . + a0 X0 = S =0, where there
is some highest k such that ak  0. Wl.o.g. assume ak > 0.
Because Xk > (X-1)(1 + X1 + X2 + X 3 + … + Xk-1)
no sequence of digits from ak-1 to a0 can represent a number
as small as –Xk
Hence S  0. Contradiction.
Each of the numbers from 0 to Xn-1 is
uniquely represented by an n-digit
number in base X.
We already know that n–digits will represent something
between 0 and Xn – 1.
Suppose two distinct sequences represent the same
number:
an-1 Xn-1 + an-2 Xn-2 + . . . + a0 X0 =
bn-1 Xn-1 + bn-2 Xn-2 + . . . + b0 X0
The difference of the two would be an plus/minus base X
representation of 0, but it would have a non-zero digit.
Contradiction.
Each of the numbers from 0 to Xn-1 is
uniquely represented by an n-digit
number in base X.
n digits represent up to Xn – 1
n-1 digits represents up to Xn-1 - 1
Let k be a number: Xn-1 ≤ k ≤ Xn - 1
So k can be represented with n digits.
For all k:  logxk  = n – 1
So k uses  logxk  + 1 digits.
Fundamental Theorem For Base X:
Each of the numbers from 0 to Xn-1 is
uniquely represented by an n-digit
number in base X.
k uses  logxk  + 1 digits in base X.
n has length n in
unary, but has length
 log2n  + 1 in binary.
Unary is
exponentially longer
than binary.
Bases In Different Cultures
Sumerian-Babylonian: 10, 60, 360
Egyptians: 3, 7, 10, 60
Africans: 5, 10
French: 10, 20
English: 10, 12,20
Egyptian Multiplication
The Egyptians
used decimal
numbers but
multiplied and
divided in binary
Egyptian Multiplication a times b
by repeated doubling
b has some n-bit representation: bn..b0
Starting with a,
repeatedly double largest so far to
obtain: a, 2a, 4a, …., 2na
Sum together all 2ka where bk = 1
Egyptian Multiplication 15 times 5
by repeated doubling
5 has some 3-bit representation: 101
Starting with 15,
repeatedly double largest so far to
obtain: 15, 30, 60
Sum together all 2k(15) where bk = 1:
15 + 60 = 75
Why does that work?
b = b020 + b121 + b222 + … + bn2n
ab = b020a + b121a + b222a + … + bn2na
If bk is 1 then 2ka is in the sum.
Otherwise that term will be 0.
Wait! How did the
Egyptians do the part
where they converted b
to binary?
They used repeated
halving to do base
conversion. Consider …
Egyptian Base Conversion
Output stream will print right to left.
Input X.
Repeat until X=0
{
If X is even then Output O;
Otherwise {X:=X-1; Output 1}
}
X:=X/2
Egyptian Base Conversion
Output stream will print right to left.
Input X.
Repeat until X=0
{
If X is even then Output O;
Otherwise Output 1
}
X:= X/2
Start the algorithm
010101
1
Repeat until X=0
{
If X is even then Output O;
Otherwise Output 1;
X:= X/2
}
Start the algorithm
01010
1
Repeat until X=0
{
If X is even then Output O;
Otherwise Output 1;
X:= X/2
}
Start the algorithm
01010
01
Repeat until X=0
{
If X is even then Output O;
Otherwise Output 1;
X:= X/2
}
Start the algorithm
0101
01
Repeat until X=0
{
If X is even then Output O;
Otherwise Output 1;
X:= X/2
}
Start the algorithm
0101
101
Repeat until X=0
{
If X is even then Output O;
Otherwise Output 1;
X:= X/2
}
Start the algorithm
010
101
Repeat until X=0
{
If X is even then Output O;
Otherwise Output 1;
X:= X/2
}
And Keep Going until 0
0
010101
Repeat until X=0
{
If X is even then Output O;
Otherwise Output 1;
X:= X/2
}
Sometimes the Egyptian
combined the base
conversion by halving
and the multiplication by
doubling into one
algorithm
Rhind Papyrus (1650 BC)
70*13
70
140
280
560
13 *
6
3 *
1 *
70
350
910
Rhind Papyrus (1650 BC)
70*13
70
140
280
560
13 *
6
3 *
1 *
70
350
910
Binary for 13 is 1101 = 23 + 22 + 20
70*13 = 70*23 + 70*22 + 70*20
Rhind Papyrus (1650 BC)
17
34
68
136
184 48 14
1
2 *
4
8 *
Rhind Papyrus (1650 BC)
17
34
68
136
1
2 *
4
8 *
184 48 14
184 = 17*8 + 17*2 + 14
184/17 = 10 with remainder 14
This method is called
“Egyptian
Multiplication/Division” or
“Russian Peasant
Multiplication/Division”.
Wow. Those Russian
peasants were pretty
smart.
Standard Binary Multiplication
= Egyptian Multiplication
X
********
101
********
********
***********
Egyptian Base 3
We have defined
Base 3: Each digit can be 0, 1, or 2
Plus/Minus Base 3 uses -2, -1, 0, 1, 2
Here is a new one:
Egyptian Base 3 uses -1, 0, 1
Example: 1 0 -1 -1 = 9 - 3 - 1 = 5
Unique Representation Theorem for
Egyptian Base 3
No integer has 2 distinct, n-digit,
Egyptian base-3 representations. We
can represent all integers from
-(3n-1)/2 to (3n-1)/2
Proof; If so, their difference would be
a non-trivial plus/minus base 3
representation of 0. Contradiction.
Highest number = 1111…1 = (3n-1)/2
Lowest number = -1-1-1-1…-1 = -(3n-1)/2
Unique Representation Theorem for
Egyptian Base 3
No integer has 2 distinct, n-digit,
Egyptian base-3 representations. We
can represent all integers from
-(3n-1)/2 to (3n-1)/2
3n, n-digit, base 3 representations of the numbers
from 0 to 3n – 1
Subtract 111111…111 = (3n – 1)/2 from each to get an
Egyptian base 3 representation of all the numbers
from -(3n-1)/2 to (3n-1)/2.
How could this be
Egyptian? Historically,
negative numbers first
appear in the writings of
the Hindu mathematician
Brahmagupta (628 AD).
One weight for each power of 3.
Left = “negative”. Right = “positive”
How to add 2 n-bit numbers.
+
* * * * * * * * * * *
* * * * * * * * * * *
How to add 2 n-bit numbers.
+
*
* * * * * * * * * * *
* * * * * * * * * * *
*
How to add 2 n-bit numbers.
+
* *
* * * * * * * * * * *
* * * * * * * * * * *
* *
How to add 2 n-bit numbers.
+
* * *
* * * * * * * * * * *
* * * * * * * * * * *
* * *
How to add 2 n-bit numbers.
+
* * * *
* * * * * * * * * * *
* * * * * * * * * * *
* * * *
How to add 2 n-bit numbers.
+
* * * * * * * * * * *
* * * * * * * * * * *
* * * * * * * * * * *
* * * * * *
* * * * * *
How to add 2 n-bit numbers.
+
* * * * * * * * * * *
* * * * * * * * * * *
* * * * * * * * * * *
* * * * * *
Let k be the
maximum time that
it takes you to do
* * * * * *
Time < kn is
proportional to n
t
i
m
e
# of bits in numbers
The time grow linearly with
input size.
If n people agree to help you add two
n bit numbers, it is not obvious that
they can finish faster than if you had
done it yourself.
Is it possible to
add two n bit
numbers in less
than linear
parallel-time?
Darn those
carries.
We will look at
parallel binary
addition in a later
lecture. For now,
let me show you a
super parallel
plus/minus binary
addition.
Plus/minus binary
means base 2
allowing digits to be
from {-1, 0, 1}. We
can call each digit a
“trit”.
n people can add 2, n-trit, plus/minus
binary numbers in constant time!
An Addition Party
to Add 110-1 to -111-1
-
An Addition Party
1
1
-1
-1
0
1
1
-1
Invite n people to add two n-trit numbers
Assign one person to each trit position
An Addition Party
0
2
1
0
-2
1
-1
1
Each person should add the two input
trits in their possession. Problem: 2 and
-2 are not allowed in the final answer.
Pass Left
1
0
0
1
1
0
-2
-1
-1
If you have a 1 or a 2 subtract 2 from yourself and pass
a 1 to the left. (Nobody keeps more than 0)
Add in anything that is given to you from the right.
(Nobody has more than a 1)
After passing left
1
1
0
-2
-1
1
There will never again be any 2s
as everyone had at most 0
and received at most 1 more
Passing left again
1
1
0
-1
-1
1
1
0
If you have a -1 or -2 add 2 to yourself
and pass a -1 to the left
(Nobody keeps less than 0)
After passing left again
1
0
1
0
0
-1
1
No -2s anymore either.
Everyone kept at least 0 and received
At most -1.
1
1
-1
=
-1
0
1
1
1
0
1
-1
0
0
1
1
Caution:
Parties and Algorithms Do not Mix
1
0
1
Later we will see
how this is used
inside all arithmetic
chips.
References
The Book Of Numbers, by J. Conway
and R. Guy
History of Mathematics, Histories of
Problems, by The Inter-IREM
Commission