Introduction
 Augmented Reality is a view of a real‐world environment that has been augmented by computer input.  Motivation:
 Seemed like an interesting field
 Becoming more common  Enjoy playing chess
Project Goals
 Augment images of a game of chess
 Decided to draw possible moves for chess pieces
 Would like to consider rules of chess as well
 Originally considered the idea of real‐time implementation
Previous Work
 Two main resources
 Gambit: A Robust Chess‐Playing Robotic System (3D)



Locate the board
Piece Recognition
 Square Detector (HOG features)
 Piece Detector
 Color Detector
 Piece Classifier (SIFT)
Game Playing
Previous Work
 Shogi Project (2D)
 Pose Estimation Algorithm

Used four colored markers on corners
 Template Matching to Identify Pieces


NCC with sample images
Peaks assumed to be piece locations
 Augment Arrows on Pieces
Implementation
 Camera Calibration  Obtain Intrinsic Parameters
 Pose Estimation
 Used same method as Shogi Project
 Piece Recognition
 SIFT
 Template Matching
Camera Calibration
 Matlab’s camera calibration toolbox
 10 pictures of chess board
 Originally used 5
 Focal Length:  fc = [ 2736.25403 2761.31401 ] ± [ 19.46801 19.78666 ]
 Pixel error: err = [ 1.75785 1.70676 ]
Camera Calibration
Extracted corners
200
Yc (in camera frame)
400
600
800
1000
1200
1400
1600
dY
O dX
1800
500
1000
1500
Xc (in camera frame)
2000
2500
Pose Estimation
 Based on finding four colored markers on the corners of the chess board
 Used pose estimation algorithm from class once location of four corners was known
 Theoretically easy method of automatic pose detection
 Proved to be annoying
 Values for colors change largely from small changes in lighting or angle
 Poor detection while keeping thresholds constant, only 4 out of 8 images correctly determined
Pose Estimation (Bad Example)
Pose Estimation (Good Example)
Piece Recognition
 Started with an attempt to use SIFT
 Very little progress despite many attempts
 Changed code to match from multiple training images
 Attempted angled views and overhead views
 Attempted to crop images for easier matching
 Decided to try other methods after multiple failures
 Template matching (NCC)
Piece Recognition
4
4
1
8
3
3
5
2
6
6
7
8
2
7
1
5
Piece Recognition
2
1
5
3
6
3
1
4
6
7
7 4
5
2
Piece Recognition
5
5
4
3
2
4
1
2
3
1
Piece Recognition
 Second method: Template matching using normalized cross‐correlation
 Currently in the process of implementing
 First attempt caused zero potential interest points to show up  Probably need to look at thresholds
Final Result
 Biggest obstacle is 3D object recognition
 Overhead view oddly difficult for chess pieces
 Pose estimation based on colored markers proved to be a hassle
Areas to Improve
 Work on color algorithm or try corner/Hough methods
 3D Piece Recognition  Implement AR Functionality
Questions?