Applied Symbolic Computation (CS 300) Karatsuba’s Algorithm for Integer Multiplication Jeremy R. Johnson Applied Symbolic Computation 1 Introduction • Objective: To derive a family of asymptotically fast integer multiplication algorithms using polynomial interpolation – – – – – – – Karatsuba’s Algorithm Polynomial algebra Interpolation Vandermonde Matrices Toom-Cook algorithm Polynomial multiplication using interpolation Faster algorithms for integer multiplication References: Lipson, Cormen et al. Applied Symbolic Computation 2 Karatsuba’s Algorithm • • Using the classical pen and paper algorithm two n digit integers can be multiplied in O(n2) operations. Karatsuba came up with a faster algorithm. Let A and B be two integers with – A = A110k + A0, A0 < 10k – B = B110k + B0, B0 < 10k – C = A*B = (A110k + A0)(B110k + B0) = A1B1102k + (A1B0 + A0 B1)10k + A0B0 Instead this can be computed with 3 multiplications • T0 = A0B0 • T1 = (A1 + A0)(B1 + B0) • T2 = A1B1 • C = T2102k + (T1 - T0 - T2)10k + T0 Applied Symbolic Computation 3 Complexity of Karatsuba’s Algorithm • Let T(n) be the time to compute the product of two n-digit numbers using Karatsuba’s algorithm. Assume n = 2k. T(n) = Θ(nlg(3)), lg(3) ≈ 1.58 • T(n) ≤ 3T(n/2) + cn ≤ 3(3T(n/4) + c(n/2)) + cn = 32T(n/22) + cn(3/2 + 1) ≤ 32(3T(n/23) + c(n/4)) + cn(3/2 + 1) = 33T(n/23) + cn(32/22 + 3/2 + 1) … ≤ 3iT(n/2i) + cn(3i-1/2i-1 + … + 3/2 + 1) ... ≤ cn[((3/2)k - 1)/(3/2 -1)] --- Assuming T(1) ≤ c ≤ 2c(3k - 2k) ≤ 2c3lg(n) = 2cnlg(3) Applied Symbolic Computation 4 Divide & Conquer Recurrence Assume T(n) = aT(n/b) + Θ(n) • T(n) = Θ(n) • T(n) = Θ(nlog(n)) [a = b] • T(n) = Θ(nlogb(a)) [a < b] [a > b] Applied Symbolic Computation 5 Polynomial Algebra • • Let F[x] denote the set of polynomials in the variable x whose coefficients are in the field F. F[x] becomes an algebra where +, * are defined by polynomial addition and multiplication. m n A( x) = ∑ ai x , B( x) = ∑ b j x i i =0 j j =0 m+n C ( x) = A( x) B ( x) = ∑ ck x , k k =0 c k = ∑a b k =i + j i j = min( k , m ) ∑a b i i = max( 0 , k − n ) Applied Symbolic Computation k −i 6 Interpolation • A polynomial of degree n is uniquely determined by its value at (n+1) distinct points. Theorem: Let A(x) and B(x) be polynomials of degree m. If A(αi) = B(αi) for i = 0,…,m, then A(x) = B(x). Proof. Recall that a polynomial of degree m has m roots. A(x) = Q(x)(x- α) + A(α), if A(α) = 0, A(x) = Q(x)(x- α), and deg(Q) = m-1 Consider the polynomial C(x) = A(x) - B(x). Since C(αi) = A(αi) B(αi) = 0, for m+1 points, C(x) = 0, and A(x) must equal B(x). Applied Symbolic Computation 7 Lagrange Interpolation Formula • Find a polynomial of degree m given its value at (m+1) distinct points. Assume A(αi) = yi ⎛ ( x −α j ) ⎞ ⎟y A( x) = ∑i =0 ⎜ ∏ ⎜ j ≠i (α −α ) ⎟ i i j ⎠ ⎝ m • Observe that ⎛ (α k −α j ) ⎞ ⎟y = A(α k ) = ∑i =0 ⎜ ∏ ⎜ j ≠i (α −α ) ⎟ i i j ⎠ ⎝ m Applied Symbolic Computation y k 8 Matrix Version of Polynomial Evaluation • Let A(x) = a3x3 + a2x2 + a1x + a0 • Evaluation at the points α, β, γ, δ is obtained from the following matrix-vector product ⎡ ⎡ A(α ) ⎤ ⎢1 ⎢ A( β )⎥ ⎢1 ⎥=⎢ ⎢ ⎢ A(γ ) ⎥ ⎢1 ⎥ ⎢ ⎢ ⎣ A(δ ) ⎦ 1 ⎣ α β γ δ 1 1 1 1 α β γ δ 2 2 2 2 ⎤⎡ ⎤ α 3⎥ a 0 ⎢ ⎥ ⎥ β ⎢ a1⎥ 3⎥ γ ⎥ ⎢⎢a2⎥⎥ 3⎥ ⎢⎣ a3 ⎥⎦ δ ⎦ Applied Symbolic Computation 3 9 Matrix Interpretation of Interpolation • Let A(x) = anxn + … + a1x +a0 be a polynomial of degree n. The problem of determining the (n+1) coefficients an,…,a1,a0 from the (n+1) values A(α0),…,A(αn) is equivalent to solving the linear system ⎡ A(α 0) ⎤ ⎡ 1 ⎥ ⎢ ⎢ ⎢ A(α 1) ⎥ = ⎢ 1 ⎢ ... ⎥ ⎢... ⎥ ⎢ ⎢ ⎢⎣ A(α n)⎥⎦ ⎢⎣ 1 α α 1 0 1 1 ... α 1 n ... ... α ⎤⎥ ⎡⎢a ⎤⎥ α ⎥⎥ ⎢ a ⎥ n 0 n 0 1 1 ... ⎢ ... ⎥ ⎥ ⎥ n ⎢ ... α n ⎥⎦ ⎢⎣a3⎥⎦ ... Applied Symbolic Computation 10 Vandermonde Matrix ⎡1 ⎢ 1 ⎢ V (α 0 ,...,α n) = ⎢ ... ⎢ ⎢⎣ 1 α α 1 0 1 1 ... ⎤ α 0n⎥ ... α 1 ⎥ ... ... ⎥ ⎥ n ... α n ⎥⎦ ... n α det(V ) = ∏ (α −α ) i< j 1 n j i V(α0,…, αn) is non-singular when α0,…, αn are distinct. Applied Symbolic Computation 11 Polynomial Multiplication using Interpolation • Compute C(x) = A(x)B(x), where degree(A(x)) = m, and degree(B(x)) = n. Degree(C(x)) = m+n, and C(x) is uniquely determined by its value at m+n+1 distinct points. • [Evaluation] Compute A(αi) and B(αi) for distinct αi, i=0,…,m+n. • [Pointwise Product] Compute C(αi) = A(αi)*B(αi) for i=0,…,m+n. • [Interpolation] Compute the coefficients of C(x) = cnxm+n + … + c1x +c0 from the points C(αi) = A(αi)*B(αi) for i=0,…,m+n. Applied Symbolic Computation 12 Interpolation and Karatsuba’s Algorithm • Let A(x) = A1x + A0, B(x) = B1x + B0, C(x) = A(x)B(x) = C2x2 + C 1x + C 0 • Then A(10k) = A, B(10k) = B, and C = C(10k) = A(10k)B(10k) = AB • Use interpolation based algorithm: – – – – Evaluate A(α), A(β), A(γ) and B(α), B(β), B(γ) for α = 0, β = 1, and γ =∞. Compute C(α) = A(α)B(α), C(β) = A(β) B(β), C(γ) = A(γ)B(γ) Interpolate the coefficients C2, C1, and C0 Compute C = C2102k + C110k + C0 Applied Symbolic Computation 13 Matrix Equation for Karatsuba’s Algorithm • Modified Vandermonde Matrix ⎡ C (0) ⎤ ⎡1 0 0⎤ ⎡c0 ⎤ ⎢ C (1) ⎥ = ⎢1 1 1⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ c1 ⎥ ⎢⎣C (∞)⎥⎦ ⎢⎣0 0 1⎥⎦ ⎢⎣c2 ⎥⎦ • Interpolation ⎡c0 ⎤ ⎡ 1 0 0 ⎤ ⎡ ⎤ A 0 B0 ⎢ ⎥ ⎢ ⎢ ⎥ ⎥ ⎢ c1 ⎥ = ⎢− 1 1 − 1⎥ ⎢( A0 + A1)(B0 + B1)⎥ ⎢c ⎥ ⎢⎣ 0 0 1 ⎥⎦ ⎢ ⎥ A1 B1 ⎣ ⎦ ⎣ 2⎦ Applied Symbolic Computation 14 Integer Multiplication Splitting the Inputs into 3 Parts • Instead of breaking up the inputs into 2 equal parts as is done for Karatsuba’s algorithm, we can split the inputs into three equal parts. • This algorithm is based on an interpolation based polynomial product of two quadratic polynomials. • Let A(x) = A2x2 + A1x + A0, B(x) = B2x2 + B1x + B, C(x) = A(x)B(x) = C4x4 + C3x3 + C2x2 + C1x + C0 • Thus there are 5 products. The divide and conquer part still takes time = O(n). Therefore the total computing time T(n) = 5T(n/3) + O(n) = Θ(nlog3(5)), log3(5) ≈ 1.46 Applied Symbolic Computation 15 Asymptotically Fast Integer Multiplication • We can obtain a sequence of asymptotically faster multiplication algorithms by splitting the inputs into more and more pieces. • If we split A and B into k equal parts, then the corresponding multiplication algorithm is obtained from an interpolation based polynomial multiplication algorithm of two degree (k-1) polynomials. • Since the product polynomial is of degree 2(k-1), we need to evaluate at 2k-1 points. Thus there are (2k-1) products. The divide and conquer part still takes time = O(n). Therefore the total computing time T(n) = (2k-1)T(n/k) + O(n) = Θ(nlogk(2k-1)). Applied Symbolic Computation 16 Asymptotically Fast Integer Multiplication • Using the previous construction we can find an algorithm to multiply two n digit integers in time Θ(n1+ ε) for any positive ε. – logk(2k-1) = logk(k(2-1/k)) = 1 + logk(2-1/k) – logk(2-1/k) ≤ logk(2) = ln(2)/ln(k) → 0. • Can we do better? • The answer is yes. There is a faster algorithm, with computing time Θ(nlog(n)loglog(n)), based on the fast Fourier transform (FFT). This algorithm is also based on interpolation and the polynomial version of the CRT. Applied Symbolic Computation 17