Advanced Practical Course: Sensor-enabled
Intelligent Environments
Barcode-based Object Recognition
Final Presentation
Presented by:
Nacer KHALIL
Supervised by:
Dejan PANGERCIC
1
Table of content
I- Overall project goal
II- Autofocus
III- Bacode decoding
IV- information retrieval
V- Barcode localization
VI- Conclusion
2
II-Autofocus
How autofocus works

Active vs passive autofocus
Courtesy of howstuffworks.com
3
II-Autofocus
(continued)
4
II- Autofocus
Implementation in the project

Used camera: Logitech QC PRO 9000

Driver used: ROS::uvc_camera

Problem: Autofocus is not supported by the driver

Solution:


Autofocus was added to uvc_camera driver
Autofocus algorithm was taken from GUVCVIEW
software and integrated within uvc_camera driver
5
II- Autofocus
result
6
III-Barcode decoding
How Zbar works
12
10
8
Column 1
Column 2
Column 3
6
4
2
0
Row 1
Courtesy of Jeff Brown
Row 2
Row 3
Row 4
7
IV-Information retrieval





Barcoo is a product information store that has a
database composed of 7 million commercial
objects.
Access to this database was granted to us.
Communication to the database is done through
HTTP protocol.
Request: an http link containing the barcode
Response: XML file containing all information about
the object
http://www.barcoo.com
8
IV- Information retrieval
Barcoo request response example

Request:
http://www.barcoo.com/api/get_product_complete? Pi=73705207908 &pins=ean&amp
;format=xml&source=ias-tum
Response:
We are parsing for:
- Image
- product name
- category
- producer
9
V- Barcode localization
Techniques used

Techniques used to find the barcode region of
interest
–
Blob-based barcode localization
–
Parallel line-based localization
–
Adjacent line-based localization
10
V- Barcode localization
Blob-based localization(working example)
11
V- Barcode localization
Blob-based localization (not working example)
12
V- Barcode localization
Adjacent line-based localization
13
V-Barcode localization
How adjacent line-based localization works
Take picture
Picture
fragmentation
Elimination of
inacceptable
intervals
Computation
of jeffries
distances
Selection of
best ROI
14
V-Barcode localization
Adjacent line-based approach explanation
- Take picture
- Convert to grayscale
- Parameters: interval size, min/max # of transitions, max Jeffrie’s value, min # of rows
per ROI
Image
matrix
Transitions
matrix
Eliminated
intervals
255
15
56
54
84
165
75
0
250
20
60
84
120
0
240
97
248
18
61
0
13
51
15
85
246
17
55
70
55
52
0
200
1
0
2
2
2
2
1
0
1
2
2
2
1
0
2
1
2
2
1
0
2
1
2
2
1
0
-1
-1
-1
-1
1
0
1
-1
-1
-1
1
0
-1
1
-1
-1
1
0
-1
1
-1
-1
15
V-Barcode localization
Adjacent line-based approach explanation (continued)
Jeffrie ’s
distance
matrix
Eliminated
intervals
matrix
Final
matrix
0,2
1
5,2
8,4
5,3
1,3
1,2
2
2,4
2,4
6,7
1
0,5
1
3,2
0,1
8,4
2,4
1
0
-1
-1
-1
-1
1
0
1
-1
-1
-1
1
0
-1
1
-1
-1
1
0
-1
1
-1
-1
0,2
1
-1
-1
-1
-1
1,2
2
0
-1
-1
-1
0,5
1
-1
0,1
-1
-1
16
IV- Barcode localization
Adjacent line-based localization - results
17
Open Source Code
Packages list:
- zbar_barcode_reader_node
- zbar_qt_ros
- uvc_camera
- barcode_detection
Repositories:
- http://code.cs.tum.edu/indefero/index.php//p/seie2011fall/source/tree/HEAD/kh
alil
- http://code.cs.tum.edu/indefero/index.php//p/ias-perception/source/tree/master/
18
Conclusion


Project is composed of three parts:

Barcode localization

Implementation of autofocus

Information retrieval of objects
Future work:

Creation of the barcoo ontology and storage on
KnowRob

Integration and testing on PR2

Integration with object modeling center
19
Demonstrations of the project in the
kitchen lab after the presentations end
20