Summary of the different techniques and methods tried on the sample images
The goal of the project is to develop a tool which analyzes the self assembly
process. This is done by evaluating some metrics. In order to gain some understanding as
to what information can be extracted from the images, some Computer Vision algorithms
have been tested on the images. Using the knowledge from this exercise metrics can be
decided. Below is a summary of the same.
Fig 1a & 1b: Sample images of the cubes
Preprocessing
This stage involves converting the image to grayscale, background subtraction,
contrast enhancement, Edge detection using Sobel/canny, binary conversion etc. The
right mix of these steps is decided by what the main algorithm wants as input.
Fig 2a: Edges using 3x3 Sobel filter
Fig 2b: Binary image after edge detection
Enhancing contrast can be done in different ways, usually equalizing or normalizing the
histogram is employed.
Morphological operations
There are many morphological operators and this is no index of them but what
can be useful to us are dilation, erosion, adding and subtracting.
Fig 3a: Dilate operator applied 2 times
Fig 3b: Erode operator applied 2 times
General Algorithms
Some well known CV algorithms were tested on these images to see if we can
extract any information.
> Hough Transform: This transform in its generalized form can detect any arbitrary
shapes. The principle behind this is that a shape is parameterized in some way and now
the image is transformed in to the domain of these parameters and then we search for best
match for the given shape. The most common implementations are for lines, circles and
ellipses. The Hough transform for lines was tested on the images and some interesting
results are seen.
Fig 4a: Binary Image of sample
Fig 4b: Lines detected on inverted binary
There is the scope for using generalized Hough Transform for detecting squares but for
that, the square has to parameterized in a unique way and we have transform the image
into the parameter domain and search for best matches. This approach may be tried in
course of the work with proper parameterization of squares or cubes.
> Harris Corners : Using an implementation of the algorithm explained in C.G. Harris
and M.J. Stephens. "A combined corner and edge detector", Proceedings Fourth Alvey
Vision Conference, Manchester. pp 147-151, 1988 we can detect corners in an image. For
us this may be quite interesting since corners represent squares and may lead to further
definition of what state the system is if we can detect squares or extract some other kind
of information from the corners.
Fig 5: Corners detected using Harris corner detection
> Hit and Miss: This is also a CV algorithm which gives highlights pixels when a mask is
placed over it and the corresponding pixels are matching. For detecting the 4 corners of a
square we need 4 runs on the image with 4 different masks. If the orientation of the
square is different then corresponding masks have to be designed. Since Harris corners
algorithm was giving better results, this approach was not further tested.
Conclusion:
At the time of writing this summary, the goals have become more clearer and hence forth
all energies will be focused on trying to get the metric ‘entropy’ in the system by clearly
identifying cubes and their alignment; while all these methods explained so far may come
useful.