CS4402
Due: 9th of April 2026
Problem Set 2
CS9635
Submission instructions on last page
Guideline. For this second assignment, solve one of the two problems below. The first
one is the one which was originally posted on March 9. The second one is closer to the
exercises we have encountered in Assignment 1 and Quiz 2.
Problem 1.
The goal of this problem is to realize a CUDA implementation of the PRAM algorithm
seen in class for univariate polynomial multiplication.
To simplify the implementation, we will make the same assumptions as in our PRAM
algorithm:
1. The two input polynomials have the same degree, namely n − 1
2. The integer n is a power of 2.
Your CUDA program should use multiple thread-blocks and allow n to reach (at least) 210 .
Note that the PRAM model does not have a concept of thread-block. So the n2 processors
of the PRAM algorithm can translate to n thread-blocks with n threads, or, even better,
n2 /t thread-blocks with t threads.
The input polynomials should be randomly generated on the CPU, then transferred to the
GPU. The product computed on the GPU should be transferred back to the CPU. Similarly
to the simple examples seen in class, your CUDA program should include a C function which
verifies that the result of the computations performed on the GPU are correct.
The input polynomials should have random coefficients in the range 0 · · · p − 1 where p is
a small prime number, say p = 103. Then, all products and sums of coefficients should be
computed modulo p. This will prevent from overflow and will make the verification feasible.
Question 1. [60 points] Realize a first version of your CUDA program using n thread-blocks
with n threads.
Question 2. [40 points] Modify your CUDA program such that it can work with n2 /t thread-
blocks with t threads, where t ∈ {64, 128, 256, 512}.
Problem 2.
In Quiz 2, for Exercises 9 and 10, we considerec a 2-way divide-and-conquer operating
on an array A of length n (where n is a power of 2, say n = 2k ) and returning another array
A′ of length n. As you may have noticed, the array A′ can be obtained by multiplying the
Hadamard matrx H2k with the array A (regarded as a column vector).
In fact, the algorithm of Quiz 2 can be transformed to an iterative form, which is more
suitable for an implementation on GPU. This algorithm is actually very famous and called
1
the Fast Walsh–Hadamard Transform and has numrous applications ranging from machine
learning to quantum computing. The Wikipedia page has Python code for the iterative
version of that algorithm. It is not hard to see that each of the for-loop can be executed in
parallel. In other words, if we were targeting a Cilk implementation, we could replace each
for with a cilk for.
Since we are targeting a GPU implementation, it is natural that each iteration of the
while loop translates to a call to a one-dimensional kernel with N thread-blocks, each of
them working on 2h coefficients of A, where
N :=
len(A)
.
2h
This would work, up to one important question: how many coefficients of A should be
handled by each thread of a thread-block? The answer depends on h. If h ≤ t holds, where
t is the maximum number of threads per thread-block, which is typically 256, 512 or 1024,
then one thread handles at most one coefficient. Otherwise each thread must handle multiple
coefficients.
Question 1. [60 points] Based on the above discussion, write a first CUDA program, imple-
mentating the iterative version of the Fast Walsh–Hadamard Transform as presented
in the Python code of he Wikipedia page. This code should:
1. transfer an input array from CPU to GPU
2. perform the calculations on the GPU,
3. transfer the output array back to CPU.
Your code should also have a CPU counterpart that you use to verifiy the calculations
of the GPU code.
Question 2. [40 points] When h is small in comparison to T , say h < t/2, each thread-block
under-utilizes the streaming processor it is running on, in particular when h is less
than a warp size (typically 32). To resolve that performance issue, thus when h < t/2
t
inner for-loops
holds, each thread-block should exceute 2h
1. Modify your CUDA program to implement this optimization,
2. Collct running times of both your optimized CUDA program and its CPU counterpart.
2
Submission instructions.
Format: Tests and a Makefile (for compiling and running) are required. Please provide
a README describing how to compile and test your code. Please submit source code
only! All the files should be archived using the UNIX tar command.
Submission: The assignment should be submitted via OWL.
Collaboration. You are expected to do this assignment on your own without assistance
from anyone else in the class. However, you can use literature and if you do so, briefly
list your references in the assignment. Be careful! You might find on the web solutions
to our problems which are not appropriate. For instance, because the parallelism model
is different. So please, avoid those traps and work out the solutions by yourself. You
should not hesitate to contact me if you have any questions regarding this assignment.
I will be more than happy to help.
Marking. This assignment will be marked out of 100. A 10 % bonus will be given if your
submission is clearly organized. Messy assignments (no Makefile, no README, no
tests) will receive a 10 % malus in addition to possible deductions (e.g. if the code
cannot be compiled).
3