Projects CS 661 DAS 02, Princeton, NJ • OCR Features and Systems – Degradation models, script ID, Bilingual OCR, Kannada OCR, Tamil OCR, mp versus hw checks, traffic ticket reading • Handwriting Recognition – Stochastic models, holistic methods, Japanese OCR • Classifiers and Learning – Multi-classifier systems • Layout Analysis – Skew correction, geometric methods, test/graphics separation, logical labeling • Tables and Forms – Detecting tables in HTML documents, use of graph grammars, semantics • • • • Text Extraction Indexing and Retrieval Document Engineering New Applications – CAPTCHA, Tachograph chart system, accessing driving directions ICDAR 03, Edinburgh, UK • • • • • • • • • • • • • Multiple Classifiers Postal Automation and Check Processing Document Understanding HMM Classifiers Segmentation Character Recognition Graphics Recognition Non-Latin Alphabets- Kanji/Chinese, Korean/Hangul, Arabic/Indian Web Documents, Video Word Recognition Image Processing Writer Identification Forms and Tables Project Assignments Faisal Farooq Multilingual Digital Library- Indexing, Retrieval, Script discrimination Swapnil Khedekar Multilingual document layout analysis, OCR Kompalli Surya Multilingual OCR using HMMs Lei Hansheng Off-line and on-line handwriting integration and matching Sumit Manocha Fingerprint image enhancement and minutiae extraction Lin Yu-Hsuan ** Multiple Classifier Combination- multiple modlaities Praveer Mansukhani Interactive Handwriting Recognition Model Amalia Rusu Handwritten Captchas Sutanto Adi ** Indirect biometric data extraction from medical forms Multilingual Digital Library Control Panel Query Result Telugu and Arabic modules under development Query Input Multilingual DIA and OCR Text/Image Separation Intervals between peaks Line Separation • Ascenders & descenders interfering with lines • Region-growing approach • In Devanagari, single word is a single connected component • Grow regions using horizontally adjacent components Word Separation • In Devanagari, all characters in a word are glued together by Shirorekha • Vertical Projection profile easily separates words Multilingual OCR using HMMs Continuous Attributes grapheme pos orientation Down cusp 3.0 -90o Up loop Down arc angle Stochastic Model Observations Integrating Online and Offline Handwriting Recognition Structural Features BAG End Loops Junction End Turns Loop Feature Extraction and Ordering Critical node: removal disconnects a connected component. Loops End Turns End Junction Turns Loop 2-degree critical nodes keep feature ordering from left to right. Left Component Right Component Fingerprint Enhancement and Feature Extraction Fingerprint Recognition Orientation maps and minutiae detection Preprocessing Operations •Image Enhancement Filtering •Image Segmentation •Correlation among fingers Multiple Classifier Systems Combination and Dynamic Selection [Govindaraju and Ianakiev, MCS 2000] image WR 1 Lexicon Top 50 <55 WR 2 Top 5 •Optimization problem •Combinatorial explosion in •arrangement of recognizers •lexicon reduction levels WR 3 + 1 Lexicon Density [Govindaraju, Slavik, and Xue, IEEE PAMI 2002] Lexicon 1 Lexicon 2 Me He So To In Me Memo Memory Memoirs Mellon Interactive Handwriting Recognition Handwriting Recognition Context Ranked Lexicon Multiple Choice Question Context Ranked Lexicon Interactive Models [McClelland and Rumelhart, Psychological Review, 1981] ABLE A TRAP TRIP Words N T Letters Features Handwritten CAPTCHAs “CAPTCHAs”: Completely Automated Public Turing Tests to Tell Computers & Humans Apart • challenges can be generated & graded automatically (i.e. the judge is a machine) • accepts virtually all humans, quickly & easily • rejects virtually all machines • resists automatic attack for many years (even assuming that its algorithms are known?) NOTE: the machine administers, but cannot pass the test! L. von Ahn, M. Blum, N.J. Hopper, J. Langford, “CAPTCHA: Using Hard AI Problems For Security,” Proc., EuroCrypt 2003, Warsaw, Poland, May 4-8, 2003 [to appear]. Yahoo!’s present CAPTCHA: “EZ-Gimpy” • Randomly pick: one English word, deformations, degradations, occlusions, colored backgrounds, etc • Better tolerated by users • Now used on a large scale to protect various services • Weaknesses: a single typeface, English lexicon Indirect Biometrics from Medical Forms Images The Biometrics Spectrum Hard biometrics Soft biometrics Derived biometrics Face Age Eye :Retina & Iris Ethnicity Text/News Nationality WWW Fingerprint Hand Geometry Handwriting Speech DNA Indirect biometrics Driver’s License Build Medical Records Gait INS Forms Mannerisms Writing style (Semantic) •Biometric Consortium (www.biometrics.org) lists several products: –Faces (30); Fingerprints (50); Hand geometry (30); Handwriting (5); Iris (5); Multimodal (6); Retinal (2); Vein (3); Voice (22); Other (20) –NONE on soft biometrics –NONE on the fusion of indirect and derived biometrics NYS EMS PCR Form NYS PCR Example Thousands are filed a day. Passed from EMS to Hospital. PCR Purpose: – Medical care/diagnosis – Legal Documentation – Quality Assurance EMS Abbreviations COPD Chronic Obstructive Pulmonary Disease CHF Congestive Heart Failure D/S Dextrose in Saline PID Pelvic Inflammatory Disease GSW Gunshot Wound NKA No known allergies KVO Keep vein open NaCL Sodium Chloride Medical Text Recognition and Data Mining