The « Intelligent Camera »
Objectives
The objective of this collaboration is twofold :
-
-
a long term scientific collaboration on the problem of feature extraction for character
recognition and text localization. This could be extended to feature extraction on more
general objects such as iconic representations, faces, panoramic views (well known
buildings) …that will enable more comprehensive understanding on perception and
recognition processing.
a European project (IST- Information Science and Technology) for the development of
a tourist assistant. This system could be embedded in a digital camera or a palm
computer with a camera. The primary objective is the recognition and translation of
Chinese or Latin characters pointed by the user’s camera. An industrial partner has to
be found in the near future. It will have to integrate the different parts of the system.
Introduction
Tourist may be helped by a translation system of foreign characters that can be encountered
when touring in a foreign city. We will consider the cases of Chinese to French or English
translation and Chinese to French as well. The system will be composed of a high resolution
image sensor and of one computing system and of an HMI (Human-Machine Interface)
device.
Tsinghua University has already developed an OCR system for Chinese printed and
handwritten characters (OCR Lab. of Pr. Ding) but also a translation system for Chinese to
French and English and Portuguese and vice versa (Lab. of Pr. Mao).
The TSI (Signal and Image processing) department of the French National School for
telecommunications (ENST) has a document analysis activity (text line extraction, fax
interpretation, handwriting characters recognitions, electronic document encoding…).
The ETIS Lab (Image and Signal processing) of the Cergy Pontoise University and the French
National School in Electronics and Applications (ENSEA) works on object recognition and
indexation (feature extraction, object localization, segmentation…), robot vision, hardware for
machine vision and modelization of human perception.
Tasks
The following tasks will be performed. Most of them will benefit from the interaction with the
user through an HMI device.
1. Geometric corrections of the image (perspective view point, rotation, scaling…).
2. Text or object localization and string extraction (lines or columns, words, …).
3.
4.
5.
6.
7.
Feature extraction and selection (definition and learning of the features as well).
Character and object recognition (similarity measure and decision).
Word recognition (using a language model only or a dictionary).
Translation
Algorithm optimisation according to hardware requirements (type of DSP or specific
VLSI).
We propose that the work could be split in the following way:
Tsinghua OCR Lab. : 2) 3) 4) mainly for Chinese characters
Tsinghua Speech and Language Information Processing Research Center: 5) and 6)
TSI Lab. At ENST Paris: 2) for string extraction 3) for latine characters
ETIS Lab. At Cergy Pontoise (Cergy Pontoise University and ENSEA) 1) 2) for « object »
localization 3) 4) for a global object recognition method (Chinese, Latin and objects) 7)
Planning
a. Listing of existing algorithms, characterization according to time computation,
robustness, precision, size of database and learning processes.
b. Algorithms integration in the system related to point c)
c. HMI definition related to point b) taking account ergonomic aspects.
d. Algorithm optimisation according to hardware requirements.
e. HMI and system validation
Agreement:
The University of Tsinghua (and in particular the department of Electronic Engineering, OCR
Lab. of Prof. DING Xiaoqing and the Speech and Language Information Processing Research
Center of Prof. MAO Yuhang) and the Euro-Chinese University without wall (and in
particular the CNRS URA-820 laboratory of ENST represented by Dr. Laurence
LIKFORMAN and the University of Cergy Pontoise (UCP) represented by Prof. Alain LE
PICHON and the ETIS Lab./CNRS UPRES A 2235 of UCP and ENSEA represented by
Philippe GAUSSIER) agree to collaborate on “The Intelligent Camera ” project in general and
specifically on the following short term actions:
- search for financial support from governmental and industrial sources,
- search for industrial partners,
- fair exchange of information between the partners,
- exchange of researchers.
Prof. DING Xiaoqing
Prof. Ph. GAUSSIER
Prof. MAO Yuhang
Dr L. LIKFORMAN
Prof. A. LE PICHON