Advanced Algebraic Algorithms
on Integers and Polynomials
Analysis of Algorithms
Prepared by
John Reif, Ph.D.
Integer and Polynomial
Computations
a) Newton Iteration: application to division
b) Evaluation and Interpolation via Chinese
Remaindering
Main Lecture Material:
Algorithms on Integers
Advanced Lecture Material:
Extension of Algorithms to Polynomials
Taylor Expansion
(x-x 0 )2
f (x) = f (x 0 ) + (x - x 0 ) f ' (x 0 ) +
f " (x 0 ) + ...
2
Taylor Expansion (cont’d)
• To find root f(x), use Newton
Iteration:
f (x i )
xi +1 =xi -
f '(x i )
• Example:
1
– To find reciprocal of x choose f ( y ) = 1 -
1
y=
find root
x
f ( yi )
yi +1 = yi = yi (2 - yi x)
f '( yi )
xy
Application of Newton Iteration to
Reciprocal of an Integer
• Input integer x, accuracy bound k
• Initialize y0 = 2- n if x has n bits
for i=1 to k do
yi +1 ¬ yi (2 - yi x)
output yk where 1 - yk x £
1
2
proof let error e k = 1 - yk x
2k
then e k +1 = 1 - yk +1 x = 1 - xyk (2 - yk x)
2k
= (e k ) = (e 0 ) = 2
2
-2k
1
since e 0 £
2
Application of Newton Iteration to
Reciprocal of an Integer (cont’d)
• Theorem Integer Reciprocal can
be computed to accuracy 2-n in
O(log n) integer mults and
additions.
Steven Cooks’s Improvement
(His Harvard Ph.D. Thesis)
• Key Trick: Since 1 - yk x £
1
2
2k
we need only compute yk up to
2k+1 bit accuracy of error on kth stage
• Total Time Cost
log n
c × å M (2k +1 ) £ O( M (n))
k =0
• Where M(n) is time cost to multiply
two n bit integers
Other Applications of Newton
Iterations on Integers
• O(M(n)) time algorithms:
• Quotient + divisor of integer
division
• Square root
• Sin, cosine, etc.
• Used in practice!
Advanced Topic: Extension to
Polynomial Reciprocal
Definition of Polynomial Reciprocal
x 2n-2
• Reciprocal ( P ( x)) =
= polynomial r(x)
P( x)û
ë
• where r ( x) × p( x) = x
2 n-2
+ e ( x)
and (x) has degree < n-1
Algorithm: Reciprocal (P(x))
• Input polynomial P ( x) =
n -1
i
a
x
å i
i =0
degree n-1, n is power of 2
1
[1] if n  1 then return
else
a0
[2] r1 (x)  Reciprocal (P1 ( x)) where P1 ( x ) 
[3] R ( x)  2r1 ( x) x (3/ 2) n2  r1 ( x) 2 P( x)
R( x)
[4] return r ( x)  n2
x 

n 1
ax
i n / 2
i
i
n
2
Proof of the Reciprocal Algorithm
• Theorem: The Algorithm Correctly
Computes Reciprocal(P(x))
• Proof by induction
1
basis n = 1 Þ P( x) = a0 so r ( x) =
a0
n
2
inductive step let P ( x) = P1 ( x) x + P2 ( x)
n
n
where deg (P1 ) = - 1, deg (P2 ) £ - 1
2
2
Proof Algorithm Computes
Reciprocal (P(x)) (cont’d)
• By induction hypothesis, if
r1 ( x) = reciprocal ( P1 ( x)) then
r1 ( x) × P1 ( x) = x n - 2 + e1 ( x)
n
where e1 ( x) has degree < - 1
2
Proof Algorithm Computes
Reciprocal (P(x)) (cont’d)
• At line [3] we compute
R( x) ¬ 2r1 ( x) x
(3/ 2) n - 2
- r1 ( x) P( x)
2
n
2
since P( x) = P1 ( x) x + P2 ( x)
P( x) R( x) = 2r1 ( x) P1 ( x) x 2 n - 2 + 2r1 ( x) P2 ( x) x (3/ 2) n - 2
æ
ö
- ç r1 ( x) p1 ( x) x + r1 ( x) p2 ( x) ÷
è
ø
n
2
2
Proof Algorithm Computes
Reciprocal (P(x)) (cont’d)
Substituting x
n-2
+e1 (x) for r1 ( x) p1 ( x),
we get
æ
ö
R( x) p( x) = x
- ç e1 ( x) x + r1 ( x) p2 ( x) ÷
è
ø
= x3n - 4 - 0( x 2 n -4 )
R( x)
But r( x) = n - 2
so
x û
ë
3n-4
n
2
r ( x ) × p ( x ) = x 2 n - 2 - o( x n - 2 )
2
Modular Arithmetic
• Assume relatively prime P0, P1, …, Pk-1
k -1
• Let
p = Õ pi
given x, 0 < x < p
i =0
x ¬¾®
( x0 , ..., xk -1 ) where
1-1
xi = x mod pi for i = 0, ..., k -1
Applications to Arithmetic
Compute u op v by computing for i=0,...,k-1
w i = u i op vi mod pi
where u i = u mod pi
vi = v mod pi
op  {+, -, }
• But doesn’t extend to division
(overflow problems)
Super Moduli Computation
• Input p0, p1, …, pk-1, where pi < 2b
• Output Super modular Tree:
Super Moduli Computation (cont’d)
• Time Cost
k 
i

 iM 2b
i 0  2 
 O ( M (k b) log k )
log k
 
Algorithm Residue Computation
k -1
• Input
x, 0 £ x < P = Õ Pi
i =0
• Output x0, x1, …, xk-1, when
xi =x mod Pi
i=0, and k-1
• Recursive algorithm
[1] compute quotient and remainders:
x = q1v1 + r1 , v1 =
( k -1) / 2
Õ
i =0
Pi
k -1
Õ
x = q2 v2 + r2 , v2 =
i=
( k -1)
+1
2
Pi
Algorithm Residue Computation
(cont’d)
[2] recursively compute
(k - 1)
(2.1) r1 mod Pi for i = 0,...,
2
(k - 1)
(2.2) r2 mod Pi for i =
+ 1,..., k - 1
2
[3] output for i = 0, 1, …, k-1
(k - 1)
ì
ïïr1 mod Pi for i £ 2
x mod Pi = í
ïr mod P for i > (k - 1)
i
ïî 2
2
Time Cost for Residue Computation
• Let D(n) = time cost for integer division
= O(M(n))
• Total Time for input size n = k· b
ænö
T (n) = 2T ç ÷ + 2kD(b) + O(n)
è2ø
ænö
£ 2T ç ÷ + O( M ( n))
è2ø
£ O( M (n) log n)
Proof of Algorithm for Residue
Computation
• Idea of Proof of algorithm
Uses fact:
if x = q v + r and v mod Pi = 0,
then
X mod Pi = r mod Pi
Advanced Topic: Residue
Computation on Polynomials
• Input moduli P0(x), P1(x), …, Pk-1(x)
assume each degree < d
and relatively prime
• Algorithm uses similar Super modular
Tree tre , but using polynomials rather
than integers
Advanced Topic: Residue
Computation on Polynomials (cont’d)
• Output for i=0, …, k-1
Qi(x) = Q(x) mod Pi(x)
Q(x) has degree < kd
• Theorem The Residue Computation
can be done in time O(M(n) log n)
where n = k· d
• Proof Idea use same algorithm as in
integer case
Advanced Topic: Multipoint
Evaluation of Polynomials by
Residue Computation
• Input polynomial f(x) degree n-1 and
points x0, x1, …, xn-1
[1] for i=0, …, n-1 let Pi(x) = (x-xi)
[2] By Residue Algorithm Computer
for
i=0, …, n-1
f(xi) = f(x) mod Pi(x)
[3] output f(x0), …, f(xn-1)
• Time Cost
O(M(n) log n),
= O(n(log n)2)
Polynomial Interpolation
• Input
evaluation points x0, …, xn-1
values y0, …, yn-1
• Output P(x) where yk = P(xk)
for k=0, …, n-1
Polynomial Interpolation (cont’d)
• Interpolation formula:
n -1
P( x) = å yk ak Õ ( x - xi )
k =0
i¹k
• Where
1
ak =
Õ ( xk - xi )
i¹k
Proof of Polynomial Interpolation
• Proof uses identities:


 ak  ( x  xi )  mod( x  xk )  1
 ik



 ak  ( x  xi )  mod( x  x j )  0 for j  k
 ik

Using Chinese Remaindering for
Integer Interpolation
• Input relatively prime P0, P1, …, Pn-1
and yi ∈ {0,…Pi-1} for i=0,…,n-1
• Problem compute y < P0 P1 … Pn-1
yi = y mod Pi
i=0, …, n-1
s.t.
Using Chinese Remaindering for
Integer Interpolation
• Generalized Interpolation Formula:
n -1
y = å yk ak Õ Pi
k =0
• Where
i¹k
ak = Õ si ,k
and
i¹k
si ,k = ( Pi ) -1 mod Pk
• proof
ì1 j = k
æ
ö
ç ak Õ Pi ÷ mod Pj = í0 j ¹ k
è i¹k ø
î
Advanced Topic: Preconditioned
Interpolation
Preconditioned Case assumes
coefficients {ak | k=0, …, n-1} precomputed
• Use Divide & Conquer
n -1
y = å yk ak Õ Pi
k =0
i¹k
ö
æ ( n-1) / 2
ö æ n -1
n -1
n -1
= ç å yk ak Õ Pi ÷ + ç å yk ak Õ Pi ÷
÷
ç k =0
÷ ç ( n -1)
i
¹
k
i
¹
k
ç
÷ ç k = +1
÷
i
=
0
i
=
0
è
ø è 2
ø
Preconditioned Interpolation
(cont’d)
é
ù
( n -1)
ú
æ ( n -1) / 2 ö ê n -1
= ç Õ Pi ÷ ê å yk ak Õ Pi ú
ú
i¹k
è i =0
ø ê k = ( n -1) +1
( n -1)
ê 2
ú
i=
+1
2
ë
û
æ n -1
ö é ( n -1) / 2
ù
( n -1) / 2
+ ç Õ Pi ÷ ê å yk ak Õ Pi ú
ç ( n -1)
÷ ê k =0
ú
i
¹
k
ç i = +1 ÷ ê
úû
i =0
è 2
øë
Time Cost for Preconditioned
Interpolation
ænö
ænö
T (n) = 2T ç ÷ + 2M ç ÷
è2ø
è2ø
= O( M (n) log n)
• Assuming {a0, …, an-1} precomputed
Precomputation of {a0, …, an-1}
n -1
1) Compute
P = Õ Pi
i =0
2) Compute bk where
bkPk = P mod (Pk)2
by Residue Computation O(M(n) log n)
3) Compute ak = (bk)-1 mod Pk by Extended
GCD algorithm
Proof of Precomputation of {a0, …, an-1}
• proof
since b k Pk = P mod (Pk )
2
then P = d (Pk ) + b k Pk
so
ÕP = d P + b
i
k
i¹k
k
so b k = Õ Pi mod Pk
i¹k
2
Precomputation of {a0, …, an-1} for
Polynomial Interpolations
•
Here Pi = (x-xi) for i=0, …, n-1
n-1
Q(x)
bk 
where Q(x)=  (x-x j )
(x - x k )
j=0
Q(x) - Q(x k )
=
since Q(x k ) = 0
(x - x k )
d Q(x) 
=
dx  x = x
k
 reduces to multipoint evaluation of
derivative of Q(x) O(M(n) log n) time!
Conclusion
• Polynomial and Integer Computations use
similar divide and conquer techniques to
solve:
1. Multiplication
2. Division
3. Interpolation and evaluation
• Open Problem
Reduce from time O(M(n)log n) to O(M(n))
Newton Iteration and
Polynomial Computation
Analysis of Algorithms
Prepared by
John Reif, Ph.D.