Lecture 13
Shape from Shading
Shape from Shading
Looking at finding normal, not distance
Normal: Describe the shape
Assuming point light source is far away
p,q are unknowns
I
p  q 1
2
2
2
  kap  kbq  k 
2
2
Ellipse
2
SFS: Data Constraint
Given an intensity value I, (p,q) are constrained to be an ellipse
I
p  q 1
2
2
2
  kap  kbq  k 
2
2
q
p
3
Self Occluding
If know self occluding, we can estimate normal.
Normal for self occluding edges are perpendicular to
the edge in 2D
4
SFS
In human : tends to assume that light is above
5
SFS: Classical Work
Horn and Schunk
Use (p,q)
Problem: Self Occluding
p,q undefined
At occluding edge
dz/dx
p ∞, q∞
To rectified this,
sphere coordinates (,) are used
6
Photometric Stereo
2/3 lights
-Given L1, Observe I1
-Given L2, Observe I2
-Given L3, Observe I3
Have 3 lights at different directions with distance ∞
-Turn on L1, take pictures
-Turn on L2, take pictures
-Turn on L3, take pictures
7
Photometric Stereo
I(x,y) = kdId(N.L)
= N.L
 = Albedo = whiteness
8
Photometric Stereo
Ii   (xlix  y liy z l iz )
4 unknowns : albedo and normal
 I1   I1 x
I   I
 2   2x
 I 3   I 3 x
known
I1 y
I2y
I3 y
known
I1z    x 


I 2 z    y 
I 3 z    z 
unknown
9
Photometric Stereo
I = L.N
  x 
 I1 
     L1  I 
 y
 2
  z 
 I 3 
N x , N y , N z   2 N x2   2 N y2   2 N z2   2 ( N x2  N y2  N z2 )
 2  
1
10
Photometric Stereo
2
3
 
 4 
  4  9 16  29
2

N  3
4

29 

29 
29 

11
Remaining Topic
•Genetic Algorithm
•Neural Network
•Motion Processing
•Structure
•Tracking (Fovea – Center Of Vision)
•Camera Calibration
•Object Recognition Models
12
Genetic Algorithm
Optimization Technique Based on “Survival of Fittest”
1. Population Size – Fixed
Each solution is an individual
2. Fitness of Individual – Fitness of “Chromosome” (small part)
“Goodness”
3. Crossover – Combining Parts of 2 or more individuals
New child is put into population. May die.
4. Mutation – “Random change done to Part of an Individual”
13
Genetic Algorithm
Charls Darwin – “Survival of Fitness”
proposed how species develop
Moth
-
Industrial Revolution
Yellow
99%
70%
20%
Black
1%
30%
80%
14
Genetic Algorithm
Algorithm:
1.
Create a random solution of 20 individuals
Sort by fitness { (1,__), (2, __), (3, __), …. }
2.
R = Random(0,1)
if (R<0.9) /* Cross Over */
Use 2 or more solutions to create new one
else
Create a new random Individual
Insert new individual in to population if it is fitter than the worst one
Repeat Until the top 20 does not change
15
Problem Statement: Input
• The Order Book consists of Many Instances of a Set of Patterns
• Each Piece of garment can be placed at 0 degrees or 180 degrees
16
Problem Statement: Output
W
L
• Place all the pieces in the order book to use minimum fabric
length on a fixed width roll
• Method: Genetic Algorithm
17
About Genetic Algorithms


An optimization method that mimics the evolution of life
using concepts of survival of the fittest within a fixed
population.
Parts of Genetic Algorithm:





Individual – a solution with known fitness
Population – set of individuals forming the genetic pool
Fitness Function – a measure of the goodness of an individual
Creating a New Individual through Reproduction:
1. Self Replication
2. Crossover
3. Mutation
Output, result reported as the most fit individual in population.
18
Related Work:
Pargas and Jain 1993 [8]
R. P. Pargas and R. Jain. A Parallel Stochastic
Optimization Algorithm for Solving 2D Bin Packing
Problems. IEEE Int. Conf. on A.I. for Applications. 1993.






A stochastic approach to bin packing 2-D figures
Similar to genetic algorithms and simulated annealing
Has 80% efficiency on regular shapes
Tested on objects that can be packed to 100% efficiency
For garments, efficiency cannot be 100%
We compare to human expert for benchmark
19
Related Work:
Roussel and Mouche 1993 [9]
G. Roussel and S. Maouche, “Improvements About
Automatic Lay-Planning For Irregular Shapes on Plain
Fabric” IEEE Proc. Systems Man and Cybernetics, System
Engineering, 1993.
 Shape layout problem for garment pieces
 Use a heuristic tree search algorithm called -admissible
algorithm
 Reasonable results
 Too much time spent back-tracking
 Tends to have a local minimum problem
20
Related Work:
Ismail and Hon [11]
H.S. Ismail and K.B. Hon. The Nesting of twodimensional Shapes Using Genetic Algorithms.
Proceedings of The Institution of Mechanical Engineers
Part B, Journal of Engineering Manufacture. 1995.




Minimize Raw Mat for cutting 2-D pieces
Uses Genetic Algorithms and Heuristics
Simple shapes used
Raw Material width not fixed, unrealistic
21
Related Work:
Bounsaythip et al [10]
C. Bounsaythip, S. Maouche and M. Neus. Evolutionary
Search Techniques Application in Automated LayPlanning Optimization Problem. IEEE Int. Conf. on
Intelligent Systems for the 21st Century, 1995.
 Minimize unoccupied space by Evolutionary Algorithm




Shape representation by Comb Code
Efficiency Measure: Length of Raw Mat (Fabric).
Used pant garment pieces, regular shapes
Good Results due to regular shapes used.
22
Related Work:
Bounsaythip and Maouche [12]
C. Bounsaythip and S. Maouche. Irregular Shape Nesting
And Placing With Evolutionary Approach”. IEEE Int.
Conf. On Systems Man and Cybernetics. 1997.
 Use the comb code representation for each garment piece




Shape placement represented as hierarchical tree
Allow orientations of 0, 90, 180, and 270 degrees
Crossover: combine parts of tree, removing redundancy
Results presented on relatively simple shapes, making it
difficult to assess the efficiency achieved
23
Placing Items from Order Book
1
4
8
2
5
3
6
9
7
10
11 12
13
14
• The Order Book consists of Many Instances of a Set of Patterns
• Each Piece of garment can be placed at 0 degrees or 180 degrees
• Each garment piece is polygonal
• Smooth contours (splines) approximated by convex hull first
24
Placement Method: Check for
Overlaps
Convex Polygons Assumed
Upon placing a new polygon, must check that:
1. NO vertex is INSIDE other polygons
2. NO other polygon’s vertex is INSIDE
25
Our Genetic Algorithm
1.
2.
3.
Set and Randomly Initialize the Population of size P = 15
Divide Individual into chromosome strips. Compute strip efficiency.
Crossover, Mutation, and Selection:
Repeat
If random (0, 1) < crossover probability
Create a New Individual by Crossover:
Repeat
Sample for a chromosome from the solution biased by efficiency
Recompute chromosome efficiency based on Order Book balance
Until No Efficient Chromosome Available
Fill Remaining Solution Randomly by Order Book balance
Else
Create New Individual by Mutation: Fill Randomly from Order Book
Insert the solution into the new population
Check for Survival of Fittest P = 15
Until(Population has converged)or(No improvement in Best Solution)
26
Step 1:
Initial Random Population
An Individual Solution:
where
5
3
4
2
1
7
S = [(F1, O1), (F2, O2),…,(Fn, On), L)]
S - completed order book
Fi - garment piece number
Oi - orientation at 0 or 180 degrees
L - Length of Fabric used
S1 =
S2 =
S3 =
S4 =
S5 =
S6 =
S7 =
S8 =
S9 =
S10 =
[(5, 1), (3, 0), (4, 1), (1, 1), (2, 0), 15]
[(1, 0), (3, 0), (2, 1), (3, 0), (4, 0), 17]
[(5, 1), (3, 0), (7, 1), (1, 1), (2, 0), 20]
[(1, 0), (5, 0), (4, 0), (3, 1), (2, 0), 21]
[(1, 0), (3, 0), (5, 0), (2, 1), (4, 0), 23]
[(4, 0), (5, 0), (1, 1), (2, 0), (3, 0), 25]
[(3, 1), (4, 0), (5, 1), (2, 0), (1, 0), 27]
[(5, 1), (4, 0), (3, 1), (2, 1), (1, 0), 29]
[(3, 0), (2, 0), (1, 0), (5, 1), (4, 1), 32]
[(4, 1), (3, 0), (1, 1), (2, 1), (5, 1), 34]
27
Step 2: Determine Strips
Lstr1
Lstr2
Lstr3
Each Strip is determined by
largest piece along column.
W
L
Efficiency of each strip is
Computed, used for crossover
28
Crossover:
Sampling Within Population

 Lmax  L 
  1
LB   B  1
 Lmax  Lmin 

P=
LB
 LB
29
Crossover:
Sampling for Efficient Strips

 E  E min 
  1
E B   B  1
 E max  E min 

P=
EB
 EB
• After sampling from population for 4 best solutions,
Crossover takes place using efficient strips first
30
Crossover:
Efficiency Decreases over Time
• Efficiency decreases as Order Book fills up
• Once Efficiency too low, fill remaining items randomly
high
lowered
31
Mutation:
New Random Individual
• When Efficiency low, fill randomly is partly like mutation
• Mutation by generating a new random solution
• This new individual usually does not survive
• Our results are compared with layout by human expert
32
Experimental Result 1:
Rectangles
6 Sets in Order Book
G.A.: 1:15 Hrs.
33
Rectangles by Human Expert
Manually: 0:25 Hrs.
34
Experimental Result 2:
Shirt Pieces
10 Sets in Order Book
35
Shirt Pcs. (10 orders) by G.A.
G.A.: 51:08 Hrs.
36
Shirt Pcs. (10 orders) by Human
Manually: 1:25 Hrs.
37
Experimental Result 3:
Multi-Edged Shapes by G.A.
3 Sets in Order Book
38
Multi-Edged G.A. vs. Human
G.A.: 0:57 Hrs.
39
Finding Crossover Number
62
E%
60
3
1
2
4
58
5
56
54
52
0
5
10
15
20
25
30
35
40
Iterations
45
50
55
60
65
70
75
40
Finding Good Population Size
62
60
E%
30
25
15
58
10
56
20
54
52
50
0
5
10
15
20
25
30
35
40
Iterations
45
50
55
60
65
70
75
41
Discussion and Conclusion
 Main Problem: successive crossover reduces efficiency of good strips.
high
lowered
 Efficiency 3-5% lower than human expert, while taking more time.
 Efficiency depends on the complexity of the piece, making it hard to
compare results among researchers.
42