Projects
CS 661
DAS 02, Princeton, NJ
• OCR Features and Systems
– Degradation models, script ID, Bilingual OCR, Kannada OCR, Tamil OCR,
mp versus hw checks, traffic ticket reading
• Handwriting Recognition
– Stochastic models, holistic methods, Japanese OCR
• Classifiers and Learning
– Multi-classifier systems
• Layout Analysis
– Skew correction, geometric methods, test/graphics separation, logical
labeling
• Tables and Forms
– Detecting tables in HTML documents, use of graph grammars, semantics
•
•
•
•
Text Extraction
Indexing and Retrieval
Document Engineering
New Applications
– CAPTCHA, Tachograph chart system, accessing driving directions
ICDAR 03, Edinburgh, UK
•
•
•
•
•
•
•
•
•
•
•
•
•
Multiple Classifiers
Postal Automation and Check Processing
Document Understanding
HMM Classifiers
Segmentation
Character Recognition
Graphics Recognition
Non-Latin Alphabets- Kanji/Chinese, Korean/Hangul,
Arabic/Indian
Web Documents, Video
Word Recognition
Image Processing
Writer Identification
Forms and Tables
Project Assignments
Faisal Farooq
Multilingual Digital Library- Indexing, Retrieval, Script
discrimination
Swapnil Khedekar
Multilingual document layout analysis, OCR
Kompalli Surya
Multilingual OCR using HMMs
Lei Hansheng
Off-line and on-line handwriting integration and matching
Sumit Manocha
Fingerprint image enhancement and minutiae extraction
Lin Yu-Hsuan **
Multiple Classifier Combination- multiple modlaities
Praveer Mansukhani
Interactive Handwriting Recognition Model
Amalia Rusu
Handwritten Captchas
Sutanto Adi **
Indirect biometric data extraction from medical forms
Multilingual Digital Library
Control Panel
Query Result
Telugu and Arabic modules under development
Query Input
Multilingual DIA and OCR
Text/Image Separation
Intervals between peaks
Line Separation
• Ascenders & descenders interfering with lines
• Region-growing approach
• In Devanagari, single word is a single
connected component
• Grow regions using horizontally adjacent
components
Word Separation
• In Devanagari, all characters in a word are glued
together by Shirorekha
• Vertical Projection profile easily separates words
Multilingual OCR using HMMs
Continuous Attributes
grapheme
pos
orientation
Down
cusp
3.0
-90o
Up loop
Down arc
angle
Stochastic Model
Observations
Integrating Online and Offline
Handwriting Recognition
Structural Features
BAG
End
Loops
Junction
End
Turns
Loop
Feature Extraction and Ordering
Critical node: removal disconnects a connected component.
Loops
End
Turns
End
Junction
Turns
Loop
2-degree critical nodes keep feature ordering from left to right.
Left
Component
Right
Component
Fingerprint Enhancement and
Feature Extraction
Fingerprint Recognition
Orientation maps and minutiae detection
Preprocessing Operations
•Image
Enhancement
Filtering
•Image
Segmentation
•Correlation
among fingers
Multiple Classifier Systems
Combination and Dynamic Selection
[Govindaraju and Ianakiev, MCS 2000]
image
WR 1
Lexicon
Top 50
<55
WR 2
Top 5
•Optimization problem
•Combinatorial explosion in
•arrangement of recognizers
•lexicon reduction levels
WR 3
+
1
Lexicon Density
[Govindaraju, Slavik, and Xue, IEEE PAMI 2002]
Lexicon 1
Lexicon 2
Me
He
So
To
In
Me
Memo
Memory
Memoirs
Mellon
Interactive Handwriting
Recognition
Handwriting Recognition
Context
Ranked
Lexicon
Multiple Choice Question
Context
Ranked
Lexicon
Interactive Models
[McClelland and Rumelhart, Psychological Review, 1981]
ABLE
A
TRAP
TRIP
Words
N
T
Letters
Features
Handwritten CAPTCHAs
“CAPTCHAs”:
Completely Automated Public Turing
Tests
to Tell Computers & Humans Apart
• challenges can be generated & graded automatically
(i.e. the judge is a machine)
• accepts virtually all humans, quickly & easily
• rejects virtually all machines
• resists automatic attack for many years
(even assuming that its algorithms are known?)
NOTE: the machine administers, but cannot pass the test!
L. von Ahn, M. Blum, N.J. Hopper, J. Langford, “CAPTCHA: Using
Hard AI Problems For Security,” Proc., EuroCrypt 2003, Warsaw,
Poland, May 4-8, 2003 [to appear].
Yahoo!’s present CAPTCHA:
“EZ-Gimpy”
• Randomly pick:
one English word, deformations, degradations,
occlusions,
colored backgrounds, etc
• Better tolerated by users
• Now used on a large scale to protect various
services
• Weaknesses: a single typeface, English lexicon
Indirect Biometrics from Medical
Forms Images
The Biometrics Spectrum
Hard biometrics
Soft biometrics
Derived
biometrics
Face
Age
Eye :Retina &
Iris
Ethnicity
Text/News
Nationality
WWW
Fingerprint
Hand
Geometry
Handwriting
Speech
DNA
Indirect
biometrics
Driver’s
License
Build
Medical
Records
Gait
INS Forms
Mannerisms
Writing style
(Semantic)
•Biometric Consortium (www.biometrics.org) lists
several products:
–Faces (30); Fingerprints (50); Hand
geometry (30); Handwriting (5); Iris (5);
Multimodal (6); Retinal (2); Vein (3); Voice
(22); Other (20)
–NONE on soft biometrics
–NONE on the fusion of indirect and
derived biometrics
NYS EMS PCR Form
NYS PCR Example
Thousands are filed a day.
Passed from EMS to Hospital.
PCR Purpose:
– Medical care/diagnosis
– Legal Documentation
– Quality Assurance
EMS Abbreviations
COPD Chronic Obstructive Pulmonary
Disease
CHF Congestive Heart Failure
D/S
Dextrose in Saline
PID
Pelvic Inflammatory Disease
GSW Gunshot Wound
NKA No known allergies
KVO Keep vein open
NaCL Sodium Chloride
Medical Text Recognition and
Data Mining