DD2458, Problem Solving and Programming Under Pressure
Lecture 9: Arithmetics II
Date: 2008-11-10
Scribe(s): Marcus Forsell Stahre and David Schlyter
Lecturer: Douglas Wikström
This lecture is a continuation of the previous one, and covers modular arithmetic
and topics from a branch of number theory known as elementary number theory.
Also, some abstract algebra will be discussed.
1
Greatest Common Divisor
Definition 1.1 If an integer d divides another integer n with no remainder, d is
said to be a divisor of n. That is, there exists an integer a such that a · d = n. The
notation for this is d | n.
Definition 1.2 A common divisor of two non-zero integers m and n is a positive
integer d, such that d | m and d | n.
Definition 1.3 The Greatest Common Divisor (GCD) of two positive integers m
and n is a common divisor d such that every other common divisor d0 | d. The
notation for this is GCD(m, n) = d.
That is, the GCD of two numbers is the greatest number that is a divisor of both
of them. To get an intuition of what GCD is, let’s have a look at this example.
Example Calculate GCD(9, 6).
Say we have 9 black and 6 white blocks. We want to put the blocks
in boxes, but every box has to be the same size, and can only hold
blocks of the same color. Also, all the boxes must be full and as large
as possible . Let’s for example say we choose a box of size 2:
As we can see, the last box of black bricks is not full. It’s easy to
see that it wouldn’t be, since 2 isn’t a divisor of 9. We know that 3 is
a divisor of both 9 and 6, so we try this size of boxes.
1
2
DD2458 – Popup HT 2008
There is no larger number that is a divisor of both 9 and 6, so there
can’t be any larger size of boxes that can be completely filled. We
therefore come to the conclusion that GCD(9, 6) must be 3.
From the third definition above, it’s not very hard to see that the Greatest Common
Divisor is equivalent to the greatest common factor (not to be confused with prime
factors!), since every other common divisor is smaller than GCD(m, n). The third
definition also gives us an inefficient but simpler way than the above to find the
Greatest Common Divisor of two numbers.
First, find the prime factorizations of m and n. Take all the common prime
factors and calculate the product of them. Let’s call this product d. Since both m
and n is divisible by this d, and since every common divisor is a divisor of d, we get
GCD(m, n) = d. Note that we know that every common divisor is a divisor of d
since every common divisor can be written as a product of common prime factors.
Example Calculate GCD(60, 42)
60 = 2 · 2 · 3 · 5
42 = 2 · 3 · 7
GCD(60, 42) = 2 · 3 = 6
Since finding the prime factorization of a number in general is hard, this is obviously
not a very good way to calculate GCD, especially when m and n grows larger. There
are much more efficient ways to calculate it, and we will talk about two of them in
a moment, but first let’s have a look at a few useful properties of GCD.
1.1
Properties of GCD
m and n are positive integers, and k is any integer.
GCD(m, n) = GCD(n, m)
(1)
GCD(m, n) = GCD(m + k · n, n)
GCD(m, n) = GCD(m mod n, n)
(2)
(3)
GCD(m, n) = 2 · GCD(m/2, n/2)
GCD(m, n) = GCD(m/2, n)
if m and n are even.
(4)
if m is even and n is odd.
(5)
It is not hard too see that (1) is true, but the others might need some explanation.
Property (2) is the basis for (3), so let’s start with (2).
If x is a common divisor of m and n, then x | m and x | n. Therefore
x | k · n, and thus x | (m + k · n).
Arithmetics II
3
If x is a common divisor of (m + k · n) and n, then x | m + k · n and
x | n. Therefore x | k · n, and thus x | (m + k · n) − k · n ⇔ x | m.
From this we conclude that the set of common divisors to m and n is equal to
the set of common divisors to m + k · n and n. This means that GCD(m, n) =
GCD(m + k · n, n)
We’ll now use this to show (3). We are going to use the fact that m mod n is defined
as an integer 0 ≤ r < n such that for some integer a, m = a · n + r. From this we
get r = m − a · n, and from (2) we know that GCD(m − a · n, n) = GCD(m, n).
Since m and n both being even is equivalent to m and n having a common divisor 2, we can move this factor outside of the GCD function, and still get the same
result (4). By similar reasoning, if one of m and n are even and not the other one,
2 is a non-common divisor, and the result is not affected if it’s removed (5).
1.2
Euclidean Algorithm
The Euclidean Algorithm makes use of property (3) from above. The idea is to
reduce m and n repeatedly until one of them is zero. The original version of the
algorithm actually used property (2) from above, and looks like this:
Algorithm 1: Euclidean algorithm for finding the GCD
Input: Two integers, m and n
Output: The GCD of m and n
EuclideanOld(m,n)
(1)
if m = 0
(2)
return n
(3)
while n 6= 0
(4)
if n > m
(5)
n←n−m
(6)
else
(7)
m←m−n
(8)
return m
It uses the fact that GCD(m, n) = GCD(m − n, n), and simply repeats the operation until n hits zero. A more efficient way of doing this is of course to use
(3).
4
DD2458 – Popup HT 2008
Algorithm 2: A better version of the Euclidean algorithm
Input: Two integers, m and n
Output: The GCD of m and n
EuclideanIterative(m,n)
(1)
while n 6= 0
(2)
t←n
(3)
n ← m mod n
(4)
m←t
(5)
return m
This is very much the same idea as the previous one. GCD(m, n) = GCD(n, m mod
n), repeat until n hits zero. Since modulo basically is repeated subtraction, this is
very much the same algorithm, but several subtractions are “done at once”. The
recursive version of the algorithm looks like this:
Algorithm 3: A recursive version of Algorithm 2
Input: Two integers, m and n
Output: The GCD of m and n
EuclideanRecursive(m,n)
(1)
if n = 0
(2)
return m
(3)
else
(4)
return EuclideanRecursive(n, m mod n)
1.3
Binary GCD Algorithm
When using a computer to calculate GCD, the Binary GCD Algorithm (also known
as Stein’s algorithm) might be preferable, since it makes good use of the binary
representation of numbers that a computer uses. The idea is to use property (4)
and (5) in order to step by step decrease the number of bits in the numbers.
Arithmetics II
5
Algorithm 4: Stein’s algorithm for binary GCD calculation
Input: Two integers, m and n
Output: The GCD of m and n
BinaryGCD(m,n)
(1)
if m = 0 or n = 0
(2)
return 0
(3)
s←0
(4)
while m and n are even
(5)
m ← m/2
(6)
n ← n/2
(7)
s←s+1
(8)
while n is even
(9)
n ← n/2
(10) while m 6= 0
(11)
while m is even
(12)
m ← m/2
(13)
if m < n
(14)
Swap(m,n)
(15)
m←m−n
(16)
m ← m/2
(17) return 2s · n
First, all the common factors 2 are removed. Since the numbers are stored in binary
on a computer, this can be done with shift-operations, which are very fast. If n
still has any factors 2, they can be discarded, as they’re not common factors.
The main-loop of the algorithm is then entered. At this point there are no
common factors 2 that are left, since they’ve all been removed. Therefore, m can
be divided by 2 until it is odd. If it then is smaller than n, they are swapped. This
is done so that the next operation never gives a negative result.
When row 15 is about to be executed, both m and n are guaranteed to be
odd numbers. And since the difference of two odd numbers always is even, row
15 always makes m into an even number. It can therefore be divided by two once
again. The loop with shifting right and making even continues until m is zero, and
then n is returned, but not before multiplying with the common factors 2 that were
removed on rows 4 to 7.
2
Bézout’s identity
Bézout’s identity states that for every pair of non-zero integers a and b, there exists
integers x and y such that ax + by = GCD(a, b). These can be found using the
Extended Euclidean Algorithm. Note though that there is an infinite number of
x and y that satisfies this equation. The Extended Euclidean Algorithm finds one
solution, and every other solution is on the following form, where k is an integer:
b
xk = x + k · GCD(a,b)
a
yk = y − k · GCD(a,b)
6
DD2458 – Popup HT 2008
Later on, there will be an example of when this identity and the Extended Euclidean
Algorithm can be useful.
2.1
Extended Euclidean Algorithm
Algorithm 5: Extended Euclidean algorithm for solving ax + by =
GCD(a,b)
Input: Two integers, m and n
Output: Two integers, x and y satisfying the equation
ExtendedEuclidean(m,n)
(1)
if n | m
(2)
return (0, 1)
(3)
else
(4)
(x0 , y 0 ) ← ExtendedEuclidean(n,
m mod n)
j k
m
0
0
(5)
return (y’, x − y · n )
If n is a divisor of m, then the solution to mx + ny = GCD(m, n) must be x =
0, y = 1, since we know then that GCD(m, n) is n. In the recursive step we
use GCD property (3) from above and instead try to solve the equation n · x0 +
0
(m mod n) · y 0 = GCD(m, n). By introducing a term n · b m
n c · y , we can use the
fact that n · b m
c
+
(m
mod
n)
=
m
(This
is
simply
integer
division
followed by
n
multiplication, and then adding back the missing remainder).
n · x0
+ (m mod n) · y 0
= GCD(m, n)
jmk
· y0 − n ·
+ (m mod n) · y 0 + n ·
n
jmk
jmk
· y 0 ) + ((m mod n) + n ·
) · y0
n · (x0 −
n
n
jmk
· y0 ) + m · y0
n · (x0 −
n
n · x0
jmk
n
· y0
= GCD(m, n)
= GCD(m, n)
= GCD(m, n)
This means that the solution to the equation m · x + n · y = GCD(m, n), by the
0
recursion, is x = y 0 , and y = (x0 − b m
n c · y ).
3
Abstract Algebra
Abstract algebra is a branch of mathematics that deals with algebraic structures
such as rings, groups, fields, and vector spaces. When dealing with number theory
these structures are very often useful, so this is a brief introduction to a few of
them.
3.1
Monoids
A monoid consists of a set M and a binary operator ◦, such that:
For every a, b ∈ M : a ◦ b ∈ M
For every a, b, c ∈ M : (a ◦ b) ◦ c = a ◦ (b ◦ c)
There is an e ∈ M such that for every a ∈ M : a ◦ e = e ◦ a = a
(Closure)
(Associativity)
(Identity)
Arithmetics II
7
Note that you could have a structure without the identity property, but it wouldn’t
be a monoid, and you’d end up with very strange results!
If the monoid also has the property that for every a, b ∈ M : a ◦ b = b ◦ a, the
monoid is said to be commutative, or abelian.
3.2
Groups
A group is a monoid (G, ∗) such that:
For every a ∈ G there is an b ∈ G such that: a · b = b · a = e
(Inverse)
A group does not need to be abelian, but it might be. It’s often natural to denote
the binary operation on a group as addition and multiplication, and in those cases
the group is usually referred to as an additive or multiplicative group, respectively.
In those cases it’s also often natural to have 0 or 1 as the identity element.
3.3
Application of groups
Exponentiation is in general the same thing as repeated multiplication, just as
multiplication is repeated addition. We can therefore in a multiplicative group do
exponentiation:
Example We have a group (Z, ∗). Find 23
Calculating 23 is the same as multiplying three times, so we get 2 · 2 · 2.
2·2·2=4·2=8
Therefore 23 = 8
When dealing with larger exponents, this way of doing it will be very slow. We can
instead use a simple method called Square-and-Multiply (or Double-and-Add when
dealing with an additive group). The following algorithm calculates g x .
Algorithm 6: Square and Multiply
Input: A base g and an exponent x
Output: The result of g x
SquareAndMultiply(g, x)
(1)
r←1
(2)
while x > 0
(3)
if x is odd
(4)
r ←r·g
(5)
g ←g·g
(6)
x ← Floor(x/2)
(7)
return r
Let g be a number, which in base 2 is represented by the binary digits an−1 an−2 ...a0 ,
where a0 is the least significant digit. We can then calculate g x as
Y
ak
g2
(6)
k:ak =1
8
DD2458 – Popup HT 2008
This algorithm uses the fact that g = a0 · 20 + a1 · 21 + ... + an−1 · 2n−1 . That is, if
we go through all the binary digits, and the k’th binary digit is a one, multiply the
k
k
result by g 2 . In the implementation this g 2 is kept between the digits, and only
one additional multiplication needs to be do for each bit in x. This means that the
running time of this algorithm is roughly O(n3 ) (since multiplication usually takes
n2 time).
It’s important to notice that the binary operator doesn’t need to be multiplication, it might just as well have been addition or anything else. This quite
simple algorithm is not only useful in simple groups, but can also be used to do
fast exponentiation with positive integers.
3.4
Ring
A commutative ring (R, +, ·) is a set R and two binary operators + and · such that:
(R, +) is an abelian group with identity 0
(R, ·) is an abelian monoid with identity 1
For every a, b, c ∈ R: a · (b + c) = a · b + a · c, (a + b) · c = a · c + b · c
3.5
Field
A field is a ring where multiplication is commutative, and where every element
except zero has a multiplicative inverse. Examples of fields are R and Zp (where p
is a prime). Fields are useful because they often behave just like regular numbers.
If (K, +, ·) is a field, then
(K − {0}, ·) is an abelian group
4
(Multiplicative Inverse)
Modular Arithmetics
Modular arithmetics, is very often useful when dealing with cyclic structures like
groups and rings, and they are also often useful in computer science when dealing
with for example cryptography. It basically deals with arithmetics with numbers
that “wrap around” at some value. In general, modular arithmetics can be considered to be arithmetics modulo some integer n.
Theorem 4.1 Given two integers n and m, there exists an integer q and an integer
r such that n = qm + r. The notation for this is n mod m = r.
n
When n and m are positive integers, we have that n = b m
c + (n mod m) (We used
this above when showing the correctness of the Extended Euclidean Algorithm).
Definition 4.2 Two integers n and n0 are congruent modulo m if and only if
n = n0 + k · m for some integer k, that is, n mod m = n0 mod m. The notation for
this is n ≡ n0 (mod m)).
Other common notations for this is n ≡ n0 mod m, n = n0 (mod m), n ≡m n0 , and
n ≡ n0 (m).
Arithmetics II
9
It’s also worth to note that the congruence relation is an equivalence relation.
This means that it has the following properties:
a≡a
if a ≡ b then b ≡ a
if a ≡ b and b ≡ c then a ≡ c
(Reflexive)
(Symmetric)
(Transitive)
The following theorems might also be useful to know:
Theorem 4.3 The set 0, 1, ..., n − 1 with modular addition and multiplication forms
a commutative ring. It can also be defined as the set of elements in the equivalence
classes under the congruence relation modulo n.
Common notations: Zn , Z/nZ, Z/(n), Z/n, Z/(n)Z
Theorem 4.4 If n is prime, Zn is a field.
5
Useful Theorems
Two integers are said to be relatively prime, or co-prime, if they don’t have any
common prime factors. That is:
Theorem 5.1 Two integers m and n are co-prime if their greatest common divisor
is 1.
5.1
The Chinese Remainder Theorem
The Chinese Remainder Theorem has its origins in a Chinese book called “The
Mathematical Classics by Sun Zi”, and is a statement about simultaneous congruences. It is said to have it origins from when the emperor was to count how many
soldiers he had. They ordered the soldiers to stand in lines of for example 5, and
then counted how many was left over. This was repeated a few times with different
line-widths, and at the end they could calculate how many soldiers there were.
Claim 5.2 Let n1 , n2 , ..., nk be pairwise relatively prime integers, and let a1 , a2 , ..., ak
be integers. Then the following system of equations has a solution:
x ≡ a1
x ≡ a2
x ≡ a3
...
x ≡ ak
(mod n1 )
(mod n2 )
(mod n3 )
(mod nk )
The constructive proof of this, which is also basis for the implementation of the
algorithm, is as follows:
(1) Let N = n1 · n2 · n3 · ... · nk
(2) Find an ri and si such that ri · ni + si ·
N
ni
=1
10
DD2458 – Popup HT 2008
(
1 mod ni
(3) Note that: si · nNi = 1 − ri · ni =
0 mod nj
k
X
N
· ai
(4) The solution is found by x =
si ·
n
i
i=1
if j 6= i
The key to the proof is in step 3. In step two we find the solution (ri , si ) to
the equation using the extended euclidean algorithm. Also take notice that ni and
N
ni are co-prime. In step 3 we have just rewritten the equation from step 2. It’s
easy to see that si · nNi ≡ 1 (mod ni ), because ri · ni ≡ 0 (mod ni ), and since N
has every factor nj except ni , it’s easy to see that si · nNi ≡ 0 (mod nj ). This fact
can be used to prove that (4) is a solution modulo N , but since the proof of this is
quite long and tedious, it will not be presented in this text.
5.2
Fermat’s little theorem
This theorem is also very useful when dealing with number theory, and is for example the basis of the Fermat primality test.
Theorem 5.3 Let p be any prime. ap ≡ a (mod p) for every integer a.
Theorem 5.4 Let p be any prime, and a an integer co-prime to p, then ap−1 ≡ 1
(mod p)
Both of the above statements are equivalent. There are several proofs for the
correctness of them, but one in particular is very short and easy to understand.
The basic idea is to make bracelets of length p using a different colors. Every
bracelet using only one kind of color is removed (this is exactly a bracelets), and
the proof continues by showing that the remaining number of bracelets is divisible
by p.
5.3
Euler’s totient function
Euler’s totient function, or Euler-phi is defined as follows:
Definition 5.5 φ(n) is the number of positive integers less than n that are relatively prime to n.
It’s not hard to see that φ(p) = p − 1 for every prime p. The formula for calculating
the value of the function in the general case is not hard either. Consider the prime
factorization of n:
n = p1 · p2 · p3 · ... · pk
every p1 ’th number smaller than n is divisible by p1 .
every p2 ’th number smaller than n is divisible by p2 .
...
every pk ’th number smaller than n is divisible by pk .
Therefore, φ(n) = n · (1 − 1/p1 ) · (1 − 1/p2 ) · ... · (1 − 1/pk ).
Arithmetics II
11
In essence, we’re simply removing all the numbers that aren’t co-prime to n. The
Y
1
above is often written as an Euler product: n · (1 − )
p
p|n
5.4
Modular multiplicative inverse
There are several ways to calculate the inverse of an element in group. We will now
show two ways to do it - one using Square-and-Multiply, and one using Extended
Euclidean.
To start with, we will show yet one example of how the Bézout identity can be
useful. A modular multiplicative inverse to a number n only exists if n and p are
co-prime, that is GCD(n, p) = 1. That means the Extended Euclidean Algorithm
solves the equation nx + py = 1, which modulo p is the same as nx ≡ 1 (mod p).
That is, the x that the Extended Euclidean Algorithm calculates is the inverse to
n in the integer ring modulo p.
We know that for a modular inverse to exists, GCD(n, p) = 1, so p must be coprime to all n. This means that p is a prime, and we can make use of Fermat’s
theorem. Since we get from Fermat that np−1 ≡ 1 (mod p), we easily see that
n · np−2 ≡ 1 (mod p), which means that the inverse of n is np−2 , and we can easily
calculate np−2 by Square-and-Multiply.
5.5
Vandermonde Matrix
Theorem 5.6 A Vandermonde matrix is always invertible
A Vandermonde matrix is a matrix with the following structure:


1 α1 α12 · · · α1n−1
1 α2 α22 · · · α2n−1 


1 α3 α2 · · · αn−1 
3
3


.
..
..
.. 
..
 ..
.
.
.
. 
2
n−1
1 αm αm · · · αm
(7)
If you let A be a Vandermonde matrix, and solve a system of equations
Pn−1 Au = y,
you will actually find the coefficients of the polynomial P (x) = i=0 ui · xi . In
this case, yi should be the y-value of the function at the point αi .