NUS – School of Computing
CS6234 Advanced Topics in Algorithms
𝑳𝟎 Gradient Minimization
Abdelhak Bentaleb (A0135562H), Lei Yifan (A0138344E),
Ji Xin (A0138230R), Dileepa Fernando (A0134674B),
Abdelrahman Kamel (A0138294X)
Outline
• Introduction (Abdelhak)
• 𝐿0 gradient minimization using coordinate descent optimization
(Ji Xin)
• 𝐿0 gradient minimization using alternating optimization and
region fusion (Abdelrahman & Dileepa)
• Experiments & Applications (Lei Yifan)
7/1/2016
L0 Gradient Minimization
2
Introduction
Abdelhak
7/1/2016
L0 Gradient Minimization
3
Introduction
• 𝐿0 norm
• 𝐿0 gradient
• 𝐿0 gradient minimization
• Why it is NP hard
• Alternating optimization
7/1/2016
L0 Gradient Minimization
4
Introduction
𝑳𝟎 Norm
• 𝑳𝒑 norm for vector x is
𝑥
𝑝 =
𝑝
𝑥𝑖 𝑝 , 𝑝 ∈ ℝ
𝑖
• A pth root of a summation of all elements to the pth power
• 𝑳𝟎 norm is
𝑥
0 =
0
𝑖
𝑥𝑖
0
• Mathematically, not a norm
• because there is a presence of zeroth-power (00) and zeroth-root in it
• For applicability reasons, 𝐿0 norm can be redefined
𝑥 0 = #(𝑖│𝑥𝑖 ≠ 0)
• total number of non-zero elements in a vector
• Wide applications on discrete functions/signals
7/1/2016
L0 Gradient Minimization
5
Introduction
𝑳𝟎 Gradient
• Recall: 𝐿0 norm is the number of non-zero elements
• 𝑳𝟎 gradient is the number of non-zero gradients
• Example:
8
6
6
4
2
20 points
0
3 zero gradients -2
17 non-zero gradients
𝐿0 gradient = 17
1
1
2
4
3
1
0
2
3
4
5
-1
6
-1
3
2
8
3
1
9
10
11
12
13
14
2
2
17
18
3
1
0
15
16
-1
19
20
Discrete Function
5
1
1
2
3
0
0
1
-5
7
5
4
-3
5
-1
6
2
7
1
8
2
2
9
10
3
0
11
-1
12
-2
13
0
14
-2
15
-1
16
-1
17
18
1
19
20
-2
Gradients
7/1/2016
L0 Gradient Minimization
6
Introduction
𝑳𝟎 Gradient Minimization
• Convert input function 𝐼  Output function 𝑆
Minimize the number of non-zero gradients (𝑳𝟎)
in 𝐼 to get 𝑆
while maintaining the similarity
between 𝐼 and 𝑆
as possible
• To minimize 𝐿0 gradient is an optimization problem
7/1/2016
L0 Gradient Minimization
7
8
6
6
Ad-hoc Example
4
1
2
2
3
1
1
0
0
-2
4
2
3
4
5
-1
6
-1
7
3
2
8
3
1
9
10
11
12
13
14
2
2
17
18
3
1
0
15
16
-1
19
20
Discrete Function
Original Function I
5
• 𝐿0 gradient = 17
5
1
1
2
3
0
0
1
-5
4
-3
5
-1
6
2
7
1
8
2
2
9
10
3
0
11
-1
12
13
3
3
-2
1
0
14
-2
15
-1
16
-1
17
18
19
20
2
2
2
2
-2
Gradients
Removing some small gradients
6
4
2
2
5
2
2
2
5
2
0
0
1
2
3
4
-2
5
-1
6
-1
7
Output Function S
• 𝐿0 gradient = 8
5
8
9
10
11
12
13
0
0
0
14
15
16
17
18
19
20
0
0
2
0
0
0
15
16
17
18
19
20
Discrete Function with less non-zero gradients
10
0
-10
1
0
0
2
3
0
4
-2
5
-1
3
6
7
0
8
3
9
0
0
10
11
0
12
-2
13
14
-3
Gradients
7/1/2016
L0 Gradient Minimization
8
Introduction
It is NP Hard
• Discrete
• Non-convex
• Non-derivative
• Can’t use traditional optimization techniques (such as gradient
descent)
• So, we need to use Alternating Optimization to get an Approximate
Solution
• (See appendix A for detailed proof)
7/1/2016
L0 Gradient Minimization
9
Introduction
Alternating Optimization – Overview
• Iterative method that generates a sequence of improving
approximate solutions for a problem.
• An optimization problem can be written as:
• minimize some objective function 𝒇(𝒙) given a set of data points x
• Partition x ∈ 𝑆 into 𝑝 non-overlapping parts 𝑦𝑙 such that
This is some kind of
local optimization
7/1/2016
L0 Gradient Minimization
10
Introduction
Alternating Optimization – Overview
7/1/2016
L0 Gradient Minimization
11
Introduction
Alternating Optimization – Overview
• For each iteration 𝒌 (until convergence)
• For each partition of data 𝒚𝒍
• Find 𝒚𝒍 values for next iteration such that 𝒇(𝒙) is minimized sufficiently
• i.e., optimize one partition while fixing all other partitions
7/1/2016
L0 Gradient Minimization
12
Introduction
Alternating Optimization – Overview
7/1/2016
L0 Gradient Minimization
13
𝑳𝟎 Gradient Minimization (1)
Using Coordinate Descent
Ji Xin
7/1/2016
L0 Gradient Minimization
14
𝑳𝟎 Gradient Minimization (1)
𝑳𝟎 Gradient Minimization (1)
• Recall: L0 gradient is the number of non-zero gradients
• Original function 𝐼  Desired output 𝑆
• Denote the gradient of 𝑆 as 𝛻𝑆
• The original function usually consists of discrete points (e.g.,
signals for 1D, pixels for 2D) in real-world applications.
• Thus, 𝛻𝑆 can be computed by subtraction between neighbor
points.
• For 1D signal, one way to calculate the gradient at point 𝑖 is
𝛻𝑆𝑖 = 𝑆𝑖 − 𝑆𝑖−1 + |𝑆𝑖+1 − 𝑆𝑖 |
7/1/2016
𝐿0 Gradient Minimization
15
2D Formulation
𝑳𝟎 Gradient Minimization (1)
• Original image 𝐼  smoothed output 𝑆
• Gradient for pixel p is 𝛻𝑆𝑝 = (𝛻𝑆𝑝𝑥 , 𝛻𝑆𝑝𝑦 )𝑇 where each dimension
can be treated as a 1D situation independently.
• The L0 gradient is
𝛻𝑆 0 = 𝑝 | 𝛻𝑆𝑝𝑥 + 𝛻𝑆𝑝𝑦 ≠ 0
• Objective function:
𝑓 = 𝑆 − 𝐼 2 + 𝜆 · 𝛻𝑆 0
• λ controls the weight between L2 norm measure (input-output
similarity) and L0 gradient (smoothness)
7/1/2016
𝐿0 Gradient Minimization
16
𝑳𝟎 Gradient Minimization (1)
Solver
• Introduce auxiliary variables 𝛿, 𝛿 is initially the gradient of the input
image.
• Each pixel p of 𝛿 is denoted by 𝛿𝑝 = (𝛿𝑝𝑥 , 𝛿𝑝𝑦 )𝑇 , corresponding to 𝛻𝑆𝑝𝑥
and 𝛻𝑆𝑝𝑦 .
• Objective function:
𝑓 = 𝑆−𝐼
2
+𝜆∙ 𝛿
0 + 𝛾 ∙ 𝛻𝑆 − 𝛿
2
• 𝛾 controls the similarity between 𝛿 and their corresponding gradient
𝛻𝑆, 𝛾 > 0
• Can be solved by the following two subproblems
7/1/2016
𝐿0 Gradient Minimization
17
𝑳𝟎 Gradient Minimization (1)
Subproblem 1: Fix 𝜹 and Compute 𝑺
𝑓 = 𝑆−𝐼
2
+λ∙ 𝛿
• Objective function
𝑓 = 𝑆−𝐼
2
0 + 𝛾 ∙ 𝛻𝑆 − 𝛿
+ 𝛾 ∙ 𝛻𝑆 − 𝛿
2
2
• This is a quadratic function, global minimum 𝑓 achieved
by gradient descent. (Let 𝛻𝑓 = 0, compute 𝑆)
7/1/2016
𝐿0 Gradient Minimization
18
𝑳𝟎 Gradient Minimization (1)
Subproblem 2: Fix 𝑺 and Compute 𝜹
𝑓 = 𝑆−𝐼
• Objective function
2
+λ∙ 𝛿
0 + 𝛾 ∙ 𝛻𝑆 − 𝛿
2
𝜆
𝑓 = 𝛻𝑆 − 𝛿 + ∙ 𝛿 0
𝛾
• Compute optimal pixel-wise objective function at a time
• Pixel-wise objective function
𝜆
2
𝑓𝑝 = 𝛻𝑆𝑝 − 𝛿𝑝 + ∙ 𝛿𝑝 0
𝛾
2
7/1/2016
𝐿0 Gradient Minimization
19
𝑳𝟎 Gradient Minimization (1)
Subproblem 2: Fix 𝑺 and Compute 𝜹
• Compute optimal 𝛿𝑝 for every pixel p
• Objective function
𝑓𝑝 = 𝛻𝑆𝑝 − 𝛿𝑝
7/1/2016
0
=
1
𝜆
+ ∙ 𝛿𝑝
𝛾
0
2
0
𝛿𝑝
2
𝑓𝑝 = 𝛻𝑆𝑝 − 𝛿𝑝
𝑓𝑝 = 𝛻𝑆𝑝
𝜆
𝜆
2
+ =0+ ,
𝛾
𝛾
𝐿0 Gradient Minimization
𝛿𝑝 ← 𝛻𝑆𝑝
20
𝑳𝟎 Gradient Minimization (1)
Subproblem 2: Fix 𝑺 and Compute 𝜹
• Compute optimal 𝛿𝑝 for every pixel p
• Objective function
𝑓𝑝 = 𝛻𝑆𝑝 − 𝛿𝑝
• Case 1: Condition
𝜆
𝛾
≥ 𝛻𝑆𝑝
• Case 2: Condition
< 𝛻𝑆𝑝
𝜆
+ ∙ 𝛿𝑝
𝛾
0
2
• 𝑓𝑝 can achieve the minimum 𝛻𝑆𝑝
𝜆
𝛾
2
2
2
when 𝜹𝒑 = (𝟎, 𝟎)𝑻 .
.
λ
𝛾
• 𝑓𝑝 can achieve the minimum when 𝜹𝒑 = 𝜵𝑺𝒑 .
7/1/2016
𝐿0 Gradient Minimization
21
𝑳𝟎 Gradient Minimization (1)
Subproblem 2 – Combined Result
• Minimum 𝑓𝑝 achieved under the condition
𝛿𝑝 =
0, 0
𝑇, 𝜆
𝛾
≥ 𝛻𝑆𝑝
𝛻𝑆𝑝 , 𝑜𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒
2
.
• Corresponding 𝑓𝑝
𝑓𝑝 =
𝛻𝑆𝑝
2
, 𝑤ℎ𝑒𝑛 𝛿𝑝 = 0,0
𝜆
, 𝑤ℎ𝑒𝑛
𝛾
𝛿𝑝 = 𝛻𝑆𝑝
𝑇
.
• So, the global objective function achieves by aggregating the objective
function of every pixel.
7/1/2016
𝐿0 Gradient Minimization
22
𝑳𝟎 Gradient Minimization (1)
Algorithm
• Input: image 𝐼, smoothing weight λ, parameters 𝛾, 𝛾0 and rate κ
• Initialization: 𝑆 ← 𝐼, 𝛾 ← 𝛾0 , 𝑖 ← 0
• repeat
• With 𝑆 , solve for 𝛿
(𝑖)
(𝑖)
in Eq. 𝛿𝑝 =
0, 0
𝑇 𝜆
,
𝛾
≥ 𝛻𝑆𝑝
2
for every pixel.
𝛻𝑆𝑝 , 𝑜𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒
• With 𝛿 (𝑖) , solve 𝑆 (𝑖+1) by gradient descent. • 𝛾 starts form small 𝛾0 , in this case 2𝜆
• 𝛾 ← κ𝛾, 𝑖 + +.
• 𝛾 ends with 𝛾𝑚𝑎𝑥 , 𝛾𝑚𝑎𝑥 is set to 1e5
• until 𝛾 ≥ 𝛾𝑚𝑎𝑥
• Output: smoothed image 𝑆
7/1/2016
• 𝛾 is multiplied by κ every iteration
• κ controls the growth of 𝛾
• finally becomes the approximation of
the gradient of the output image
𝐿0 Gradient Minimization
23
𝑳𝟎 Gradient Minimization (2)
Abdelrahman & Dileepa
7/1/2016
L0 Gradient Minimization
24
Objective Function
𝑳𝟎 Gradient Minimization (2)
• Original function 𝐼
Desired output 𝑆
• We need to minimize number of non-zero gradients in 𝐼 to get 𝑆
• while maintaining the similarity between 𝐼 and 𝑆 as possible
Maintaining
similarity
7/1/2016
L0 Gradient Minimization
Minimizing
gradients
Parameter to control the 𝐿0 gradient
Larger 𝜆 means smaller 𝐿0 gradient
25
Objective Function
𝑳𝟎 Gradient Minimization (2)
• Removing a gradient depends on the neighbors of some point
• For example, in 1D, point 𝑆𝑖 affects 2 gradients (𝑆𝑖 − 𝑆𝑖−1 ) and (𝑆𝑖+1 − 𝑆𝑖 )
• So, rewrite the previous function as summation over all points
• If we have 𝑴 points
Maintaining
similarity
𝜆 is divided by 2 as each
gradient is calculated twice
7/1/2016
L0 Gradient Minimization
Minimizing
gradients
Sum of all gradients from
point 𝑖 to all its neighbors 𝑁𝑖
26
Neighborhood
𝑳𝟎 Gradient Minimization (2)
• What are the neighbors of some point 𝑆𝑖
• Some examples:
𝑺𝒊
𝑺𝒊
7/1/2016
L0 Gradient Minimization
𝑺𝒊
27
𝑳𝟎 Gradient Minimization (2)
Alternating Optimization
• Recall that 𝐿0 minimization is NP-hard
•
•
•
•
So, we need to apply alternating optimization
How:
Optimize for each pair of neighboring points at a time (partition)
So, for one pair, the minimization step becomes
𝑆𝑖 and 𝑆𝑗 are some pair of
neighbor points
7/1/2016
L0 Gradient Minimization
28
Minimization Step
𝑳𝟎 Gradient Minimization (2)
• Now, we need to solve this minimization step
• Let’s consider two cases:
• When 𝑆𝑖 ≠ 𝑆𝑗
• When 𝑆𝑖 = 𝑆𝑗
7/1/2016
L0 Gradient Minimization
29
𝑳𝟎 Gradient Minimization (2)
Minimization Step – Case 1
• Case 1: When 𝑆𝑖 ≠ 𝑆𝑗
• 𝑆𝑖 − 𝑆𝑗
0
= 1 because the gradient 𝑆𝑖 − 𝑆𝑗 ≠ 0
1
• So, 𝑓 becomes
• And, 𝑆𝑖 = 𝐼𝑖 and 𝑆𝑗 = 𝐼𝑗 give us the minimum value of 𝑓 = 𝜆
7/1/2016
L0 Gradient Minimization
30
𝑳𝟎 Gradient Minimization (2)
Minimization Step – Case 2
• Case 2: When 𝑆𝑖 = 𝑆𝑗
• 𝑆𝑖 − 𝑆𝑗
0
= 0 because the gradient 𝑆𝑖 − 𝑆𝑗 = 0
0
• So, 𝑓 becomes a simple quadratic equation in one variable 𝑆𝑖
See Appendix B for
detailed solution
𝟐
• And, 𝑺𝒊 = 𝑺𝒋 = (𝑰𝒊 + 𝑰𝒋 )/𝟐 give us the minimum value of 𝒇 = 𝑰𝒊 − 𝑰𝒋 /𝟐
7/1/2016
L0 Gradient Minimization
31
Fusion Criterion
𝑳𝟎 Gradient Minimization (2)
• Based on Case 1 & 2, we have 2 possible minimum values of 𝑓
• To solve the minimization step, we need to know which is smaller
• Combined together, we have the solution:
• This is called the Fusion Criterion
7/1/2016
L0 Gradient Minimization
32
𝑳𝟎 Gradient Minimization (2)
Example – Fusion Criterion
• Assume 𝝀 = 𝟑
5
4
3
2
1
5
4
2
𝐼𝑖
𝐼𝑗
1
2
4
𝐼𝑖 − 𝐼𝑗
2
2
=2<𝜆
𝐼𝑖 + 𝐼𝑗
𝑆𝑖 = 𝑆𝑗 =
=3
2
0
3
2
1
3
3
𝑆𝑖
𝑆𝑗
1
2
0
𝑳𝟎 = 𝟏
𝑳𝟎 = 𝟎
5
5
4
3
2
1
4
𝐼𝑖
𝐼𝑗
1
𝐼𝑖 − 𝐼𝑗
2
2
=8>𝜆
𝑆𝑖 = 𝐼𝑖 = 1 ,
𝑆𝑗 = 𝐼𝑗 = 5
0
3
2
1
𝑆𝑖
𝑆𝑗
1
0
1
7/1/2016
5
5
2
𝑳𝟎 = 𝟏
1
L0 Gradient Minimization
2
𝑳𝟎 = 𝟏
33
Region Fusion
𝑳𝟎 Gradient Minimization (2)
• Up to now,
we considered minimizing 𝐿0 for pairs of points,
𝑮𝒊
• For alternating optimization, it is better to optimize
regions or groups of points together at a time
𝑰𝒊
• For faster convergence
• Lets assume, for each point 𝐼𝑖 we have a
group of points 𝑮𝒊 of 𝒘𝒊 points
that is fused together to the same value 𝑆𝑖
𝑮𝒊
𝑺𝒊
𝑺𝒊
𝑺𝒊
• Example: in the figure 𝑤𝑖 = 3
7/1/2016
L0 Gradient Minimization
34
Region Fusion
𝑳𝟎 Gradient Minimization (2)
• For generality, in 2D and 3D functions, we must consider
the number of gradients 𝒄𝒊,𝒋 between pairs of neighboring
groups 𝑖 and 𝑗
• 𝒋 ∈ 𝑵𝒊
• Example: in the figure,
𝑐1,2 = 𝑐2,1 = 5,
𝑤1 = 10
• Note that 𝒘𝒊 and 𝒄𝒊,𝒋 are initially set to 𝟏.
7/1/2016
L0 Gradient Minimization
35
Region Fusion
𝑳𝟎 Gradient Minimization (2)
• After adding the notion of groups, the minimization step becomes:
• Now, 𝑌𝑖 and 𝑌𝑗 are the average value of the original function in the
regions 𝐺𝑖 and 𝐺𝑗, respectively
• 𝑤𝑖 , 𝑤𝑗 = number of points in 𝐺𝑖 and 𝐺𝑗
• 𝛽 is to control the number of gradients between 𝐺𝑖 and 𝐺𝑗
• 𝑐𝑖,𝑗 = number of gradients between 𝐺𝑖 and 𝐺𝑗
• The above equation can be solved in the exact same manner used for
Minimization Step
7/1/2016
L0 Gradient Minimization
36
𝑳𝟎 Gradient Minimization (2)
Region Fusion
• Solution:
• 𝐵 = (𝑤𝑖 𝑌𝑖 + 𝑤𝑗 𝑌𝑗 )/(𝑤𝑖 + 𝑤𝑗 )
weighted average of the two groups
𝐺𝑖 and 𝐺𝑗 .
• This criterion is used to decide
whether to fuse the group 𝐺𝑖 into
the group 𝐺𝑗 or not.
7/1/2016
L0 Gradient Minimization
37
Region
Fusion
Algorithm
7/1/2016
=1
𝐿0 Gradient Minimization
38
𝑳𝟎 Gradient Minimization (2)
Initialization of the Algorithm
7/1/2016
𝐿0 Gradient Minimization
39
𝑳𝟎 Gradient Minimization (2)
Fusion
Step for Gi
with Gj
7/1/2016
𝐿0 Gradient Minimization
40
𝑳𝟎 Gradient Minimization (2)
Fusion Step Explained
• Decision for merging is taken by comparing objective function
change
• A neighbour is merged and it’s information is also merged
• After deleting merged group, Indexes are updated
• Number of groups is updated
• Loop goes through the neighbours of the merged group for more
fusions
7/1/2016
𝐿0 Gradient Minimization
41
𝑳𝟎 Gradient Minimization (2)
Reduction of the objective function
• Claim : The sum of non zero gradients removed from the merged
neighbours after any fusion steps, is a lower bound for the total
non zero gradients removed
• Sub Claim: At the end of 𝛽 = 0 iteration,
1. ∀𝑝 < 𝑀 , 𝐼𝑝 = 𝑆𝑝
2. For any two neighbours 𝑝1 , 𝑝2 ∈ 𝐼, 𝐼𝑝1 = 𝐼𝑝2 ⟹ 𝐺 𝑝1 = 𝐺(𝑝2 )
(Here 𝐺 𝑥 is the group which pixel 𝑥 belonging to)
7/1/2016
𝐿0 Gradient Minimization
42
𝑳𝟎 Gradient Minimization (2)
Reduction of the objective function
Why the sub claim is correct?
1. When 𝛽 = 0 ,two groups 𝐺𝑖 , 𝐺𝑗 are merged only if 𝑌𝑖 = 𝑌𝑗
i.e.(𝑤𝑖 𝑤𝑗 𝑌𝑖 − 𝑌𝑗
2
≤ 0)
1. New value after merging does not change because, weighted
average of equal values is the value itself. (Objective function is
not changed)
2. Since each pixel is visited at least once, if two pixels have similar
values, they should be merged to the same group.
7/1/2016
𝐿0 Gradient Minimization
43
𝑳𝟎 Gradient Minimization (2)
Reduction of the objective function
When 𝛽 > 0,
• Initially, no two neighbouring groups have zero gradients
(sub claim)
• In the first iteration for 𝛽 > 0 , zero gradients are introduced for
the neighbours considered (fusion step- 𝐺𝑖 and 𝐺𝑗 )
• Some side neighbours (𝐺𝑘 ) may also introduce zero gradients.
• In a later iteration, they may result in non-zero gradients
• Non-zero gradients of the side neighbours does not exceed the
initial count
7/1/2016
𝐿0 Gradient Minimization
44
𝑳𝟎 Gradient Minimization (2)
Reduction of the objective function (upper bound)
• An upper bound for the objective function exists
• It keeps on decreasing from the initial value 𝜆||∇S||0
• Amount of reduction at each fusion step is at least ,
𝜆ci,j − wi wj ||Yi − Yj||2 /(wi + wj)
7/1/2016
𝐿0 Gradient Minimization
45
Choice of β
𝑳𝟎 Gradient Minimization (2)
• β=0 case, groups neighbours with zero gradient
• Increasing β in small intervals gives a very smooth solution but it
consumes time.
7/1/2016
𝐿0 Gradient Minimization
46
Choice of β
𝑳𝟎 Gradient Minimization (2)
• For an extreme example, let’s say β’s first non zero value is 𝜆.
• Then very large gradients can be merged resulting a larger
objective function.
• If β starts from a small value, small gradients are merged and after
merge a large gradient between two input pixels may be converted
into much larger gradient between two groups containing the
input pixels.
• Then the inputs pixels are not grouped, resulting in a smaller
objective function value.
7/1/2016
𝐿0 Gradient Minimization
47
𝑳𝟎 Gradient Minimization (2)
Choice of β
Assume β 𝒊𝒔 𝒄𝒉𝒐𝒔𝒆𝒏 𝒔. 𝒕. 𝒑𝒊𝒙𝒆𝒍𝒔 𝟏 𝒂𝒏𝒅 𝟐 𝒂𝒓𝒆 𝒏𝒐𝒕 𝒎𝒆𝒓𝒈𝒆𝒅 𝒇𝒐𝒓 𝒔𝒎𝒐𝒐𝒕𝒉 β , 𝒂𝒏𝒅 𝒂𝒓𝒆 𝒎𝒆𝒓𝒈𝒆𝒅 𝒇𝒐𝒓 𝒏𝒐𝒏 − 𝒔𝒎𝒐𝒐𝒕𝒉 β
Output – smooth beta
Input
Output – non-smooth beta
7
7
7
6
6
6
5
5
5
4
4
4
3
3
3
2
2
2
1
1
1
0
0
0
Pixel position
1
2
3
Pixel position
1
2
3
F= 1.5
7/1/2016
𝐿0 Gradient Minimization
Pixel position
1
2
3
F= 14
48
Applications
Lei Yifan
7/1/2016
L0 Gradient Minimization
49
Applications
Demo
L0 norm
L2 norm
7/1/2016
L0 Gradient Minimization
50
Applications
Experiment
Objective Function 𝜆 = 0.01
6000
5000
4000
3000
L0 norm
2000
1000
L2 norm
0
0
10
20
30
40
50
60
70
80
90
100
# of iterations
7/1/2016
L0 Gradient Minimization
51
Applications
Experiment – Comparison
input
The result of 1st
algorithm.
The result of 2nd
algorithm.
input
7/1/2016
L0 Gradient Minimization
The result of 1st
algorithm.
The result of 2nd
algorithm.52
Applications
Experiment – Comparison
input
The result of 1st
algorithm.
The result of 2nd
algorithm.
input
7/1/2016
L0 Gradient Minimization
The result of 1st
algorithm.
The result of 2nd
algorithm.
53
Applications
Example Applications
• Due to its unique property, it can be used in many application.
1.
2.
3.
4.
5.
6.
7/1/2016
Edge Enhancement
Color Quantization
JPEG artifact removal
3D Mesh Denoising
Underdetermined linear system
…
L0 Gradient Minimization
54
Applications
Edge Enhancement
• Edge extraction is very useful in many application. However,
sometimes the edge of the original input is not sufficient enough.
To address this problem, L0 minimization method could be used.
Original input
7/1/2016
Extracted Edge from
original input
Image after
L0-minimazition
L0 Gradient Minimization
Extracted Edge from
Image after L0-minimazition
55
Applications
Color Quantization
• L0 gradient can also be used to do color quantization. Color
quantization is really helpful for image compression.
Input, # of colors is 41290
7/1/2016
Output, # of colors is 38
L0 Gradient Minimization
56
Applications
JPEG Artifact Removal
• It is with high probability that there could be some artifact when
compressing image especially for Clip-Art. This algorithm can
address this problem well.
7/1/2016
L0 Gradient Minimization
57
Applications
3D Mesh Denoising
7/1/2016
L0 Gradient Minimization
58
Applications
Underdetermined Linear Systems
• Solve underdetermined linear system, if a sparse solution is
needed.
• To solve Ax=v, we can minimize
𝑓 𝑥 = 𝐴𝑥 − 𝑣 + 𝜆 𝑥
0
• We can use similar strategy to optimize this objective function.
7/1/2016
L0 Gradient Minimization
59
Conclusion
• We started by introducing some background.
• Then, we presented L0 gradient minimization using two
algorithms.
• Coordinate descent optimization.
• Alternating optimization with region fusion.
• Also, we presented some experiments and comparisons between
the two algorithms.
• Finally, we showed the applicability of the algorithms on some
problems.
7/1/2016
L0 Gradient Minimization
60
Thank You!
Appendix A
Proof: 𝑳𝟎 Minimization is NP Hard
• The solution for 𝐿0 gradient minimization for some vector 𝑥 is given by: min 𝑥
• The definition of 𝑥
0
0
s. t. 𝐴𝑥 − 𝑏 < ϵ
is number of non-zero entries in 𝑥.
• Consider the simplest case where 𝑥
0
= 1 but you don't know the location of that non-zero entry.
• We have 𝑁1 possibilities and to find the solution we have to examine the values of all the
unique minimizer.
• Similarly, if 𝑥
0
= 𝑘, you need to search
𝑁
𝑘
possibilities. i.e., the algorithm grows as
𝑁
𝑘
𝑁
1
possibilities for finding the
with increase in 𝑘.
• Since 𝑘 is not known a priori, you have to check for all the 𝑁 possible values of 𝑘.
• Hence, the complexity of the algorithm is
𝑁
𝑖=1
𝑁
𝑖
• This is NP hard.
7/1/2016
L0 Gradient Minimization
62
Appendix B
Solving Minimization Step
𝑓 = 𝑆𝑖 − 𝐼𝑖
2
+ 𝑆𝑖 − 𝐼𝑗
2
𝑓′ = 0
2 𝑆𝑖 − 𝐼𝑖 + 2 𝑆𝑖 − 𝐼𝑗 = 0
4𝑆𝑖 − 2𝐼𝑖 − 2𝐼𝑗 = 0
(𝐼𝑖 +𝐼𝑗 )
𝑆𝑖 =
2
7/1/2016
L0 Gradient Minimization
63
References
1. Xu, L., Lu, C., Xu, Y., and Jia, J., "Image smoothing via L0 gradient
minimization." ACM Transactions on Graphics (TOG), Vol. 30, No. 6, ACM,
2011.
2. Cheng, X., Zeng, M., and Liu, X., "Feature-preserving filtering with L0 gradient
minimization." Computers & Graphics, 38, 150-157, 2014.
3. Nguyen, Rang M. H., and Brown, M. S., "Fast and Effective L0 Gradient
Minimization by Region Fusion." International Conference on Computer
Vision (ICCV), 2015.
7/1/2016
L0 Gradient Minimization
64