Lecture 3
Tuesday, February 10, 2015
[With the help of free online resources]
Divide and Conquer
Divide and Conquer?
• Split the entire range into smaller manageable parts.
• Solve each part separately.
• Combine the results to get result for the original larger problem.
Array of size 𝑛
Divide
among 𝑝
cores
and
solve
𝑛/𝑝
𝑛/𝑝
𝑛/𝑝
𝑛/𝑝
𝑛/𝑝
𝑛/𝑝
Array of size 𝑛
𝑛
𝑓 𝑥 = 𝑎0 +
𝑎𝑖 cos
𝑖=1
𝑛/𝑝
𝑛𝜋𝑥
𝐿
𝑛/𝑝
𝑛/𝑝
𝑛/𝑝
Merge
Divide and Conquer
Recursive Divide and Conquer?
• Doing Divide and Conquer recursively.
• i.e., First divide into two parts, then divide each of those two parts into two more smaller
parts and so on until reach a small enough size.
• cilk_for is also implemented as a recursive divide and conquer.
• Splits the loop range in a divide and conquer way
Array of size 𝑛
Split
into
2 parts
𝑛
𝑛
Array of size 2
𝑛
Array of size 4
log 𝑛
Array of size 2
𝑛
Array of size
Array of size 4
𝑛
4
Array of size
𝑛
4
………
Array of
size 2
Array of
size 2
Array of
size 2
Array of
size 2
Array of
size 2
Array of
size 2
Array of
size 2
Array of
size 2
……… 𝑛/2
of them
Computing sum of 𝑛 numbers
// Sequential sum of n numbers
// Parallel sum of n numbers
function sum(𝐴(1. . 𝑛))
function sum(𝐴(1. . 𝑛))
if (𝑛 = 1) return 𝐴(1)
if (𝑛 = 1) return 𝐴(1)
else
else
𝑛
2
x = sum(𝐴(1. . ))
𝑛
𝑛
2
x = cilk_spawn sum(𝐴(1. . ))
𝑛
y = sum(𝐴(2 + 1. . 𝑛))
y = sum(𝐴(2 + 1. . 𝑛))
return (𝑥 + 𝑦)
cilk_sync
return (𝑥 + 𝑦)
𝑛
+ log 𝑝
𝑝
𝑊𝑜𝑟𝑘 𝑎𝑛𝑑 𝑆𝑒𝑟𝑖𝑎𝑙 𝑡𝑖𝑚𝑒: 𝑇1 = 𝑂(𝑛)
𝐶𝑜𝑠𝑡: 𝑃𝑎𝑟𝑎𝑙𝑙𝑒𝑙 𝑡𝑖𝑚𝑒: 𝑇𝑝 = 𝑂
Homework: Implement it and show the performance.
Computing sum of 8 numbers
Merge Sort:
Sort n numbers recursively using
divide and conquer.
𝑀𝑒𝑟𝑔𝑒𝑆𝑜𝑟𝑡 𝐴, 𝑝, 𝑟 //𝑠𝑜𝑟𝑡 𝑡ℎ𝑒 𝑒𝑙𝑒𝑚𝑒𝑛𝑡𝑠 𝑖𝑛 𝐴[𝑝 … 𝑟]
1. 𝑖𝑓 (𝑝 < 𝑟) 𝑡ℎ𝑒𝑛
3. 𝑀𝑒𝑟𝑔𝑒𝑆𝑜𝑟𝑡(𝐴, 𝑝, 𝑞)
4. 𝑀𝑒𝑟𝑔𝑒𝑆𝑜𝑟𝑡(𝐴, 𝑞 + 1, 𝑟)
2. 𝑞 = (𝑝 + 𝑟) / 2
5. 𝑀𝑒𝑟𝑔𝑒(𝐴, 𝑝, 𝑞, 𝑟)
𝑃𝑎𝑟𝑀𝑒𝑟𝑔𝑒𝑆𝑜𝑟𝑡(𝐴, 𝑝, 𝑟) //sort the elements in A[p…r]
1. 𝑖𝑓 𝑝 < 𝑟 𝑡ℎ𝑒𝑛
2. cilk_spawn 𝑀𝑒𝑟𝑔𝑒𝑆𝑜𝑟𝑡 𝐴, 𝑝, 𝑞
4.𝑀𝑒𝑟𝑔𝑒𝑆𝑜𝑟𝑡 𝐴, 𝑞 + 1, 𝑟
5.𝑞 = (𝑝 + 𝑟) / 2
6.𝑀𝑒𝑟𝑔𝑒 𝐴, 𝑝, 𝑞, 𝑟
7.𝑠𝑦𝑛𝑐
Picture: Wikipedia
Divide and Conquer
Fibonacci number:
In mathematics, the Fibonacci numbers or Fibonacci sequence are the
numbers in the following integer sequence
1 1 2 3 5 8 13 21 ….
int fib(int n)
F(n) = F(n - 1) + F (n - 2) int fib(int n)
{
{
}
if (n < 2) return n;
int x = cilk_spawn fib(n-1);
int y = fib(n-2);
cilk_sync;
return x + y;
if (n < 2) return n;
int x = fib(n-1);
int y = fib(n-2);
return x + y;
}
Matrix-multiplication
Iterative-MM ( Z, X, Y ) // X, Y, Z are n × n matrices,
// where n is a positive integer
1. for i ← 1 to n do
3.
Z[ i ][ j ] ← 0
4.
for k ← 1 to n do
2.
for j ← 1 to n do
5.
Z[ i ][ j ] ← Z[ i ][ j ] + (X[ i ][ k ] * Y[ k ][ j ])
Par-Iterative-MM ( Z, X, Y ) // X, Y, Z are n × n matrices,
// where n is a positive integer
1. parallel for i ← 1 to n do
3.
Z[ i ][ j ] ← 0
4.
for k ← 1 to n do
2.
parallel for j ← 1 to n do
5.
Z[ i ][ j ] ← Z[ i ][ j ] + X[ i ][ k ] ⋅ Y[ k ][ j ]
Source: http://www.mathsisfun.com/algebra/matrix-multiplying.html
http://algebra.nipissingu.ca/tutorials/matrices.html
Links for c/c++ tutorial
• https://www.youtube.com/watch?v=rk2fK2IIiiQ
• https://www.youtube.com/watch?v=Rub-JsjMhWY
• https://www.youtube.com/watch?v=S3t-5UtvDN0
• https://www.youtube.com/watch?v=KNFv4DZ4Mp4
• https://www.youtube.com/watch?v=c5gg9F8h8Fw&list=PLfVsf4Bjg79
CZ5kHTiQHcm-l2q8j06ofd (probably by this time you know all)
• Source: google!
Group Homework1
[1] . Teach yourself c/c++
• Watch these online videos and other resources freely available in Google
and write a 10 page group report on c/c++ basic data types,
for loops, input, output, arrays and metrics with dynamic allocation,
and function.
•
•
•
•
•
https://www.youtube.com/watch?v=rk2fK2IIiiQ
https://www.youtube.com/watch?v=Rub-JsjMhWY
https://www.youtube.com/watch?v=S3t-5UtvDN0
https://www.youtube.com/watch?v=KNFv4DZ4Mp4
https://www.youtube.com/watch?v=c5gg9F8h8Fw&list=PLfVsf4Bjg79CZ5kHTiQHcml2q8j06ofd (probably by this time you know all)
[2]. Write all the codes discussed in the class so far. Compile them and generate
output. Vary n and take the time using the cilktime function.
Report the change in running time with n.
Fix n to the largest and vary # of cores and report the running time.