430.758 (003)
Topics in Signal Processing
(Computational Imaging with
Optimization and Deep Learning)
Sample Midterm Problems
Se Young Chun, PhD
Associate Professor
Department of Electrical & Computer Engineering
Seoul National University
April 12, 2022
Intelligent Computational imaging Lab (ICL)
Q. Show the details: If f and g are Lipschitz continuous functions on R, then h(x) = f(x)g(x) is a Lipschitz continuous
function on R.
Q. For f(x) = (1/2)∥Ax − y∥22 we have ∥∇f(x) − ∇f(z)∥2 = ∥Aʹ(Ax − y) − Aʹ(Az − y)∥2 = ∥AʹA(x − z)∥2 ≤ ∥AʹA∥2 ∥x − z∥2. Then,
find a Lipschitz constant of ∇f. Discuss whether your answer is the best Lipschitz constant for that ∇f or not.
Q. The Fair potential is ψ(z) = δ2 (|z/δ| − log(1 + |z/δ|)) for some δ > 0. Is it Lipschitz continuous? Find the Lipschitz
constant of the derivative of it.
Q. Consider the complex LS cost function Ψ(x) = (1/2)∥Ax − y∥2 , where A ∈ CM×N and x ∈ CN and y ∈ CM . This function
is not differentiable on CN. Show that g = Aʹ(Ax − y) is a best ascent direction for Ψ at x. (Hint: the equality condition
for Cauchy-Schwarz inequality?)
Q. Assume that the definition for the proximal operator is given. Determine analytically the proximal operator for the
2-norm. Then, determine the proximal operator for the (nonconvex) 0-norm.
Q. Solve the LASSO problem as a constrained LS problem using the gradient projection (GP) method. (Hint: one can
use the positive and negative parts of x separately).
Intelligent Computational imaging Lab (ICL)
2
Q. Descriptions on the gradient descent, the steepest descent, the heavy ball method, the Nesterov’s fast gradient
method, the Newton’s method (no more memorization than these. OGM is too much for memorizing, but I may ask
something about OGM by providing related formular).
Q. For the LS problem with Ψ(x) = (1/2)∥Ax − y∥2, describe the ideal preconditioner using ordinary GD and
preconditioned GD. Why is it the ideal preconditioner?
Q. See Lecture 6, page 11 (a quadratic majorizer for the LASSO problem). For the quadratic majorizer for (1/2)∥Ax −
y∥2, find c1 and c2.
Q. Describe the ISTA, the proximal gradient method (PGM) and the IHT.
Q. For the cost function of the form f(x) = ∑$
!"# hi([Ax]i), derive a quadratic majorizer of it using additive separability
assuming that hi are convex.
Q. Derive the ML-EM using MM approach.
Q. Describe the differences between maximum curvature and optimal curvature (Huber’s majorizer).
Intelligent Computational imaging Lab (ICL)
3
Q. Consider f(x) = 3 ∥x∥1+XC(x), C={x ∈ RN: ∥x∥∞ ≤5}. Find the proximal operator of f. Describe the proximal point
algorithm.
Q. Describe the fast proximal gradient method (FPGM) or FISTA (POGM is too complicated to memorize).
Q. Find the block coordinate minimization (BCM) for the following joint cost function: (1/2)∥Ax − y∥22 + b ((1/2)∥x −
D z∥22 + c ∥z∥1). Find the BCD for the same problem.
Q. Given N × L data matrix X, consider the following dictionary learning optimization problem: argminD minZ Ψ(D,Z),
Ψ(D,Z)=(1/2)|X−DZ|F2+β∥vec(Z)∥0, where D is the set of N × K matrices having unit-norm columns. The drawback of
the atom-wise BCM approach is that it is not conducive to parallel computing.
(a) Write down a MM (or equivalent PGM) update for D. (This update is mentioned in the course notes without
any formula given.)
(b) An alternative is to make a BCD approach where one updates, say, two atoms of D together. Thinking of the
partition D = [D1:2 D3:K ] write a concise expression for an update of D1:2 that is guaranteed to decrease Ψ. Discuss
what advantage this approach has over (a).
Intelligent Computational imaging Lab (ICL)
4
Q. Describe the variable splitting, the Lagrangian function, and augmented Lagrangian method.
Q. Find the convex conjugate of the affine function.
Q. Examples in Lecture notes 12, 13.
Q. Describe the Lagrangian dual function, the duality gap and the ADMM.
Q. Describe the linearized augmented Lagrangian method.
Q. Describe the primal-dual hybrid gradient method.
Intelligent Computational imaging Lab (ICL)
5
Questions
Any questions?
sychun@snu.ac.kr
Intelligent Computational imaging Lab (ICL)
6