PuzzleSolver
Joe Zeimen
Monday, April 29, 13
Goal
Monday, April 29, 13
General Outline
• Scan Pieces
• Find pieces
• Extract edges
• Compare edges
• Assemble Puzzle
• Generate output image
Monday, April 29, 13
Thresholding
Monday, April 29, 13
Find Contours
• use cv::findContours() on the binary image.
• Filter out contours smaller than an expected puzzle piece size.
• Provides shape informaFon
• Provides a mask to produce the final output image.
5
Monday, April 29, 13
Find the corners
• cv::goodFeaturesToTrack()
• cv::cornerSubPix()
6
Monday, April 29, 13
Edges and classifica7on
7
Monday, April 29, 13
Comparing Edges
First and Last
Used Matches from Angry Birds Puzzle
Best: 0.453672
Last Used: 0.865829
8
Monday, April 29, 13
Assembly Algorithm
– Goal: Join pieces together to result in lowest cost.
– 4nn! combinaFons where n is the number of pieces
n
combina(ons
24
1.74641E+38
48
9.83532E+89
104
4.23677E+228
9
Monday, April 29, 13
Assembly Algorithm
• Compare every pair of edges, store in a sorted list.
• While the puzzle is not finished:
– Pop the next best match off the list
– Merge the 2 pieces using the 2 edges from the list
• Merge(piece 1, edge 1, piece2, edge 2)
– If piece 1 and piece 2 are in same set return
– If merging piece 1 and piece 2 result in overlap return
– merge piece 1 and piece 2
10
Monday, April 29, 13
Best Matches List
Cost
...
Piece 1 Edge 1
Piece 2
Edge 2
0.453672
12
2
13
1
0.539882
14
1
23
0
0.557301
4
0
9
1
0.589648
11
0
23
2
0.598454
0
0
20
2
0.61856
3
3
20
1
0.625861
5
3
9
0
0.630817
11
2
22
3
...
...
...
...
11
Monday, April 29, 13
Combine Pieces: 12
Monday, April 29, 13
Combine Pieces: 12
Monday, April 29, 13
Combine Pieces: 12
Monday, April 29, 13
Combine Pieces: 12
Monday, April 29, 13
Side numbering conven7on
3
0
2
1
13
Monday, April 29, 13
How the computer does it:
Want to merge piece 5, edge 0 with piece 6 edge 2
4
LocaFons
RotaFons
5
6
9
-­‐1
0
1
3
0
0
14
Monday, April 29, 13
How the computer does it:
Want to merge piece 5, edge 0 with piece 6 edge 2
4
LocaFons
RotaFons
9
5
-­‐1
0
1
3
0
1
6
0
Rotate the matrix set with 5 in it
14
Monday, April 29, 13
How the computer does it:
Want to merge piece 5, edge 0 with piece 6 edge 2
5
LocaFons
RotaFons
-­‐1
6
4
9
0
1
3
0
0
15
Monday, April 29, 13
How the computer does it:
Want to merge piece 5, edge 0 with piece 6 edge 2
5
LocaFons
RotaFons
-­‐1
6
4
0
9
1
1
3
0
0
Update the rotaFon matrix
15
Monday, April 29, 13
How the computer does it:
Want to merge piece 5, edge 0 with piece 6 edge 2
5
LocaFons
RotaFons
-­‐1
6
4
9
2
0
1
0
0
16
Monday, April 29, 13
How the computer does it:
Want to merge piece 5, edge 0 with piece 6 edge 2
5
LocaFons
RotaFons
-­‐1
6
4
2
9
2
0
1
0
0
Rotate 6
16
Monday, April 29, 13
How the computer does it:
Want to merge piece 5, edge 0 with piece 6 edge 2
5
LocaFons
RotaFons
-­‐1
6
4
9
2
0
1
0
0
17
Monday, April 29, 13
How the computer does it:
Want to merge piece 5, edge 0 with piece 6 edge 2
5
LocaFons
RotaFons
-­‐1
6
4
2
1
9
0
0
2
0
Update 6’s rotaFon matrix
17
Monday, April 29, 13
How the computer does it:
Want to merge piece 5, edge 0 with piece 6 edge 2
5
LocaFons
RotaFons
-­‐1
6
4
9
2
0
1
0
2
18
Monday, April 29, 13
How the computer does it:
Want to merge piece 5, edge 0 with piece 6 edge 2
5
LocaFons
-­‐1
6
4
9
2
2
1
0
6
4
9
+
RotaFons
5
2
0
1
0
=
2
19
Monday, April 29, 13
Saving the Image
• Given the locaFons an rotaFons of the soluFon, the pieces can be put into place using affine transforms.
• Can figure out all corner locaFons except lower right. • Use the 3 corners to calculate the transformaFon matrix.
• Scan horizontally then verFcally using the corners computed previously to place the next piece. 20
Monday, April 29, 13
Other Solved Puzzles
21
Monday, April 29, 13
Other Solved Puzzles
48 Pieces
Monday, April 29, 13
22
Other Solved Puzzles
23
Monday, April 29, 13
Other Solved Puzzles
Back of Toy Story puzzle, numbered based on soluFon by hand.
Monday, April 29, 13
24
104 piece puzzle (back)
Monday, April 29, 13
25
Ques(ons?
Code and images at:
github.com/jzeimen/PuzzleSolver
26
Monday, April 29, 13