Example of usage of Prefix Sum
Example of usage of Prexix Sum
Compacting an Array
0 0 0 e1 0 0 e2 0 0
A
●
0 e3
B e1 e2 e3 0 0 0 0 0 0 0 0
Compacting an Array
Given an array A with many zeroes, compact it
–
0 0 0 e1 0 0 e2 0 0
1
2
A
S
●
Output an array B such that all the values in A not
zero are at the beginning of B
Initialize B with zeroes
●
If A[i] != 0 then B[????] = ????
Example of usage of Prexix Sum
Compacting an Array
0 0 0 e1 0 0 e2 0 0
1
2
●
B e1 e2 e3 0 0 0 0 0 0 0 0
Compacting an Array
0 0 0 e1 0 0 e2 0 0
1
2
A
S
Given an array A with many zeroes, compact it
–
●
0 e3
3
Output an array B such that all the values in A not
zero are at the beginning of B
●
Example of usage of Prexix Sum
A
S
B e1 e2 e3 0 0 0 0 0 0 0 0
Given an array A with many zeroes, compact it
–
Any idea on the solution (first in sequential)?
0 e3
3
●
Output an array B such that all the values in A not
zero are at the beginning of B
0 e3
3
B e1 e2 e3 0 0 0 0 0 0 0 0
Given an array A with many zeroes, compact it
–
Output an array B such that all the values in A not
zero are at the beginning of B
Hint: use an extra (binary) array S such that
–
S[i] == 0
if A[i] == 0
●
Initialize B with zeroes
–
S[i] == 1
if A[i] != 0
●
If A[i] != 0 then B[S[i]] = A[i]
How can we use S?
How would you do it in parallel?
Example of usage of Prefix Sum
Example of usage of Prexix Sum
Compacting an Array
Compacting an Array
A
S
0 0 0 e1 0 0 e2 0 0
1
2
●
B e1 e2 e3 0 0 0 0 0 0 0 0
Given an array A with many zeroes, compact it
–
●
0 e3
3
Output an array B such that all the values in A not
zero are at the beginning of B
Hint2: compute the prexif sum of S
–
S = [0, 0, 0, 1, 1, 1, 2, 2, 2, 2, 3]
And now?
A
S
0 0 0 e1 0 0 e2 0 0
1
2
0 e3
3
B e1 e2 e3 0 0 0 0 0 0 0 0
Algorithm COMPACT
1. assign value 1 to ei and value 0 to the others
2. compute the PREFIX SUMS of these values, and store the
results on S
3. begin
time of Step 3????
for 1 ≤ i ≤ n pardo
begin
B(i): = 0; if A(i) ≠0 then B( S(i) ):=A(i)
end
end
Example of usage of Prefix Sum
Compacting an Array
0 0 0 e1 0 0 e2 0 0
1
2
A
S
B e1 e2 e3 0 0 0 0 0 0 0 0
0 e3
3
Algorithm COMPACT
1. assign value 1 to ei and value 0 to the others
2. compute the PREFIX SUMS of these values, and store the
results on S
3. begin
constant time!
for 1 ≤ i ≤ n pardo
begin
B(i): = 0; if A(i) ≠0 then B( S(i) ):=A(i)
end
end
Prefix Sums
●
●
X = [2, 1, 4, 6, 9, 2, 1, 5]
Let us reduce the problem to a n/2 array Y, where
each element of Y is the sum in pairs of the
elements in X
–
●
Y = [3, 10, 11, 6]
Hint: Let us assume to have the prefix sum of Y
–
Z = [3, 13, 24, 30]
Can we compute the prexif sums of X from Z?
Prefix Sums
●
●
To understand the parallel solution, let us start from
describing it sequentially
Prefix Sums
●
●
We will first present a recursive solution
X = [2, 1, 4, 6, 9, 2, 1, 5]
Let us reduce the problem to a n/2 array Y, where
each element of Y is the sum in pairs of the
elements in X
–
Ideas?
Y = [3, 10, 11, 6]
●
Hint: Let us assume to have the prefix sum of Y
●
PrefixX =
–
Z = [3, 13, 24, 30]
[2, Z[?], ....]
Prefix Sums
●
●
X = [2, 1, 4, 6, 9, 2, 1, 5]
Let us reduce the problem to a n/2 array Y, where
each element of Y is the sum in pairs of the
elements in X
–
Prefix Sums
●
●
Y = [3, 10, 11, 6]
X = [2, 1, 4, 6, 9, 2, 1, 5]
Let us reduce the problem to a n/2 array Y, where
each element of Y is the sum in pairs of the
elements in X
–
Y = [3, 10, 11, 6]
●
Hint: Let us assume to have the prefix sum of Y
●
PrefixX =
How can we use Y?
–
Z = [3, 13, 24, 30]
[2, Z[1], ....]
Prefix Sums
●
●
X = [2, 1, 4, 6, 9, 2, 1, 5]
Let us reduce the problem to a n/2 array Y, where
each element of Y is the sum in pairs of the
elements in X
–
●
●
X = [2, 1, 4, 6, 9, 2, 1, 5]
Let us reduce the problem to a n/2 array Y, where
each element of Y is the sum in pairs of the
elements in X
Y = [3, 10, 11, 6]
●
Hint: Let us assume to have the prefix sum of Y
●
PrefixX =
–
Prefix Sums
–
Y = [3, 10, 11, 6]
●
Hint: Let us assume to have the prefix sum of Y
●
PrefixX =
Z = [3, 13, 24, 30]
–
[2, Z[1], ?]
Z = [3, 13, 24, 30]
[2, Z[1], Z[1]+X[3], Z[2], Z[2]+X[5], ....]
Prefix Sums
●
●
X = [2, 1, 4, 6, 9, 2, 1, 5]
Let us reduce the problem to a n/2 array Y, where
each element of Y is the sum in pairs of the
elements in X
–
●
●
●
●
X = [2, 1, 4, 6, 9, 2, 1, 5]
Let us reduce the problem to a n/2 array Y, where
each element of Y is the sum in pairs of the
elements in X
Y = [3, 10, 11, 6]
Hint: Let us assume to have the prefix sum of Y
–
Prefix Sums
–
Hint: Let us assume to have the prefix sum of Y
●
PrefixX =
Z = [3, 13, 24, 30]
PrefixX =
[2, Z[1], Z[1]+X[3], ?]
Y = [3, 10, 11, 6]
●
–
Z = [3, 13, 24, 30]
[2, Z[1], Z[1]+X[3], Z[2], Z[2]+X[5], Z[3], Z[3]+X[7], ....]
Prexif Sums
Prefix Sums
●
●
X = [2, 1, 4, 6, 9, 2, 1, 5]
1. if n = 1 then s(1) := x(1) return
Let us reduce the problem to a n/2 array Y, where
each element of Y is the sum in pairs of the
elements in X
2. for 1≤i≤n/2 pardo y(i) := x(2i - 1) * x(2i)
–
●
Y = [3, 10, 11, 6]
Hint: Let us assume to have the prefix sum of Y
–
●
(parallel version)
3. recursively compute the prefix sums of y(1), ...,
y(n/2) and store them in z(1), ..., z(n/2)
4. for 1≤i≤n pardo
i. if i even then s(i) :=z (i/2)
Z = [3, 13, 24, 30]
ii.if i =1then s(1):= x(1)
PrefixX =
[2, Z[1], Z[1]+X[3], Z[2], ?]
iii.if i odd then s(i) := z(i-1/2)*x(i)
end
Prefix Sums via Doubling
X
Y
3 5 2 3 4 6 3 1
8
Y
5
10
13
3
4
8
14
Y
10 13 17 23 26 27
8
13 23
13
27
27
S
Z
27
●
Z
27
Z
T(n) = ????
Another interesting technique that can be used to
solve the Prexif Sums is the Doubling
–
Iterative
–
A processing technique in which accesses or actions
are governed by increasing powers or 2
–
That is, processing proceeds by 1, 2, 4, 8, 16, etc.,
doubling on each iteration
Prefix Sums
X
Y
3 5 2 3 4 6 3 1
8
Y
5
10
13
3
4
8
14
Y
10 13 17 23 26 27
8
13 23
13
27
27
Z
27
●
Z
27
T(n) = T(n/2) + O(1)
S
Z
At the first step, each X[i] is added to X[i+1]
–
X = [4, 9, 5, 2, 10, 6, 12, 8]
–
X1 = [4, 13, 14, 7, 12, 16, 18, 20]
T(n) = O(log n)
How would you continue?
W(n) = ????
Prefix Sums
X
Y
Y
Y
3 5 2 3 4 6 3 1
8
5
10
13
4
14
27
T(n) = T(n/2) + O(1)
3
8
10 13 17 23 26 27
8
13 23
13
27
27
27
S
Z
Z
Z
T(n) = O(log n)
●
work-optimal!!
W(n) = W(n/2) + O(n)
●
At the first step, each X[i] is added to X[i+1]
–
At any time if an index exceeds n, the operation is
supressed
–
X = [4, 9, 5, 2, 10, 6, 12, 8]
–
X1 = [4, 13, 14, 7, 12, 16, 18, 20]
At the second step, each X[i] is added to X[i+2]
–
X2 = [4, 13, 18, 20, 26, 23, 30, 36]
W(n) = O(n)
Next step?
Prefix Sums
Prefix Sums by Doubling
* Operation supressed
●
●
–
At any time if an index exceeds n, the operation is
supressed
–
X = [4, 9, 5, 2, 10, 6, 12, 8]
–
X1 = [4, 13, 14, 7, 12, 16, 18, 20]
At the second step, each X[i] is added to X[i+2]
–
●
# contains final sum
At the first step, each X[i] is added to X[i+1]
X2 = [4, 13, 18, 20, 26, 23, 30, 36]
Doubling Time:
–
At step k, X[i] is added to X[i+2k-1]
p = n-1
Prefix Sums by Doubling
Tp = O(log n)
Prefix Sums by Doubling
* Operation supressed
●
●
What about the Work (total number of operations)?
–
At the first step: ???? operations
–
At the second step: ???? operations
–
At the third step: ???? operations
–
At the kth step: ???? operations
Total: ???? operations
How many steps do we need to finish?
Prefix Sums by Doubling
* Operation supressed
Prefix Sums by Doubling
# contains final sum
●
●
p = ????
Tp = O(????)
What about the Work (total number of operations)?
–
At the first step: n-1 operations
–
At the second step: ???? operations
–
At the third step: ???? operations
–
At the kth step: ???? operations
Total: ???? operations
Prefix Sums by Doubling
●
What about the Work (total number of operations)?
●
What about the Work (total number of operations)?
–
At the first step: n-1 operations
–
At the first step: n-1 operations
–
At the second step: n-2 operations
–
At the second step: n-2 operations
–
At the third step: ???? operations
–
At the third step: n-4 operations
–
At the kth step: n-2k-1 operations
–
●
Prefix Sums by Doubling
th
At the k step: ???? operations
Total: ???? operations
●
Total: Σ k=0..log n-1 (n-2k) =
Prefix Sums by Doubling
●
●
What about the Work (total number of operations)?
Prefix Sums by Doubling
●
What about the Work (total number of operations)?
–
At the first step: n-1 operations
–
At the first step: n-1 operations
–
At the second step: n-2 operations
–
At the second step: n-2 operations
–
At the third step: n-4 operations
–
At the third step: n-4 operations
–
At the kth step: ???? operations
–
At the kth step: n-2k-1 operations
Total: ???? operations
●
Total: Σ k=0..log n-1 (n-2k) =
(n-1) + (n-2) + .... + (n-2log n - 1) =
Prefix Sums by Doubling
●
●
What about the Work (total number of operations)?
Prefix Sums by Doubling
●
What about the Work (total number of operations)?
–
At the first step: n-1 operations
–
At the first step: n-1 operations
–
At the second step: n-2 operations
–
At the second step: n-2 operations
–
At the third step: n-4 operations
–
At the third step: n-4 operations
–
At the kth step: n-2k-1 operations
–
At the kth step: n-2k-1 operations
Total: ???? operations
●
Total: Σ k=0..log n-1 (n-2k) =
(n-1) + (n-2) + .... + (n-2log n - 1) =
(n log n) – (1 + 2 + 4 + 2log n - 1) =
Prefix Sums by Doubling
●
●
List Ranking
What about the Work (total number of operations)?
–
At the first step: n-1 operations
–
At the second step: n-2 operations
–
At the third step: n-4 operations
–
At the kth step: n-2k-1 operations
INPUT
C
S
e3
1
e1
n
2
e2
ø
1
3
n
CONTENT
SUCCESSOR
Total: Σ k=0..log n-1 (n-2k) =
(n-1) + (n-2) + .... + (n-2log n - 1) =
OUTPUT
(n log n) – (1 + 2 + 4 + 2log n - 1) =
Array R such that R(i) is equal to the distance (rank) of item C(i)
from the end of the list.
(n log n) – (2 log n – 1)
Prefix Sums
e1
R:
Algorithm
Work
Time
Sequential
N
N
Recursive
N
Log N
Doubling
N log N – (2 log N – 1)
Log N
●
e2
1
1
List Ranking
Given a linked list, stored in an array, compute the
distance of each element from the end (either end)
of the list
–
●
●
e1
R:
e4
e5
1
1
1
e6
e7
1
e8
1
NIL
At the beginning we initialize an array R (rank), that
will contain the rank of each element
–
●
Idea
e3
1
That is, the distance of each element from the end of
the list
e2
1
Idea
e3
e4
e5
1
1
1
e6
1
e7
e8
1
NIL
Problem is similar to prefix sums, using all 1’s to
sum
Called Pointer Jumping (not doubling) when using
pointers
Don’t destroy original list!
●
At the beginning, there are two elements with the
correct rank....which
which ones????
ones