MACHINE TRANSLATION OF INDIAN SIGNS
FOR ENDOCRINOLOGIST
Jagriti Mishra1*, Gouri Sankar Mishra1,2, Kiran Ravulakollu1,3, Ravi Rastogi1,4,K.M Rafi2,5
1,
Department of Computer Science & Engineering, School of Engineering & Technology,
Sharda University, Greater Noida, INDIA
2
GS, SAKSOI, New Delhi
1*
Corresponding Author: jagritimp@gmail.com
2gourisankar.mishra@sharda.ac.in
3kiran.ravulakollu@sharda.ac.in
4ravi.rastogi@sharda.ac.in
5 kmrafi1@gmail.com
Scientists are now trying to develop translators for specific
domains to assist hearing impaired people.
Abstract—India being the second most populated country in
the world with over a billion population and over a million
hearing impaired and diabetes disease patients, a translation
system which can translate a given input into sign languages
can be used to disseminate information to the million hearing
impaired patients. Such people find it difficult to access
information in common places like hospitals and railway
stations. A translation system which can convert English into
Indian Sign Languages can be developed to help such people.
II. RELATIONAL WORK
Computer is an integral part of human life now-a-days.
In day-to-day life human depends upon computers for their
various needs, right from laundry to dissection of human
body, computers are involved in everything. Scientists and
engineers are now trying to make computers understand
natural languages.
Machine Translation (MT) is an innovative paradigm
that promotes the advancement of science and technology to
built smart environments.
It advocates an invisible
technological support layer of information processing to
improve the quality of life. In this paper we discuss. Present
study contains possibilities of integrating concept related to
natural language processing (NLP) certain aspects of
integrating English languages to sign language.
III. PAPER ORGANIZATION
In section A we have introduced Natural Language,
Machine Language, ISL [1] and in section B we have
mentioned Includes Problem Foundation, Methodology.
Section A
1. Natural Language Processing
Keywords—MT, Sign, NLP, ISL, Linguistics
I. INTRODUCTION
(NLP) is the computerized approach foranalysing text that
is based on both a set of theories and computational
techniques for analysing and representing naturally
occurring texts achieving human technologies. Natural
LanguageProcessing is a theoretically motivated range of like language processing for a range of tasks or
applications. The goal of NLP is to accomplish human-like
language processing.
Natural Language Processing (NLP) is the ability of
computers of computer programs to understand the human
language. Machine Translation is a field of Natural
Language Processing which deals with automated
translation. It is the process by which computer software is
used to translate a text from one natural language (such as
English) to another (such as Spanish or Sign language).
Sign Linguistics, on the other hand, is a relatively
new field of study in the area of Linguistics, and the
combination - machine translation of sign languages - is
less than twenty years old [15].Sign languages as languages
have not been studied as extensively as spoken languages,
and there is still much to be learned about them. Sign
Linguistics, on the other hand, is a relatively new field of
study in the area of Linguistics, and the combination
machine translation of sign languages - is less than twenty
years old [15].
2. Machine Translation
The objective of Machine Translation is to restore the
meaning of original text into translated verse. In general the
translation process has two levels:
• Metaphrase – Metaphrase means word to word
translation. The translated version will have literal
translation for each word in text. But, translated text may
not necessarily convey meaning of original text, i.e. the
semantics of original text may be lost.
• Paraphrase – The translated text will contain the core
of the original text but may not necessarily contain the
word –to-word translation
With the requisite of a translation system able of converting
an input into Sign Language and accompaniment of
adequate advantages of developing such a system,
1
languages, there is free and easy communication”, “Sign
language is the mother tongue of the Deaf”, and the like
[12].
Scientists are trying to develop systems which can
translate a given set of input text into an equivalent desired
source language. One such system is developed by
Purushottamkar et al [2]. They reported a cross-modal
translation system from Hindi strings to Indian Sign
Language (ISL) [11].
Though the number of deaf people learning ISL is
rising there are only few people in the society who
can converse with them in sign languages. Due to the
paucity of people who can converse in ISL, the pain
and trouble of the deaf and mute community still
prevails[10]. A lot of problem is faced by the deaf
and mute people during their visits to the doctors.
a)Machine Translation Architecture
Machine translation system can be loosely grouped into
three basic designs: Direct, Transfer, or Interlinguashown
in Fig 1.2(a). Direct systems base their processing on the
individual words of the source language string. In direct
systems translation is obtained without performing any
kind of syntactic analysis on the source input. In transfer
systems, the input text is analyzed for syntax to some
extent and then a special set of transfer rules are employed
to read the information of the source language structure and
produce a corresponding syntactic or semantic structure in
the target language. In Interlingua systems, the text is
analyzed and transformed into intermediary language
which is independent of any of the languages involved in
translation. The translated verse for the target language is
then derived through this intermediate representation. [3]
Section B
1. Problem Description
A translation system converting English text into Indian
Sign Languages can be developed [9]. Developing a system
dealing with all the possible domains in medical sciences is
a tedious work. The development can be done domain wise.
A system converting English text into Sign Languages for
possible use by Endocrinologist, i.e. the doctor who
specializes in dealing with Thyroid patients, can be
developed. [6][7]The doctor will type his text that he wants
to convey to the patient and the input text will be converted
into an equivalent sign language output.[13][14] The deaf
can interact with the doctor by writing his problems
or else a system using pattern recognition can be
developed to translate the pictorial input into
comprehensive sequence of text or speech.
Fig.1.2Machine Translation Architecture
A translation machine can be developed to convert
English text into Indian Sign Language (ISL)[4][5]and
such a natural language processing system can lessen
the pain and difficulties of deaf and mute citizens of
India as shown in Fig 1.2(a).
2. Methodology
The translation system can be developed using
python environment [6] which is usually the preferred
programming platform for developing expert systems
involving a lot of user inputs as shown in Fig.2.
Direct or transfer architecture can be used to
develop the required translation system. The following
approach can be adopted to achieve the goal:-
Fig. 1.2(b)Sign Representations
3. Indian Sign Languages
India estimated that there are over a million people who
are profoundly deaf and approximately 10 million hard of
hearing people, in India. These figures are extrapolated
from the number of people who are deaf and hard hearing
in western nations. It would be realistic to believe that the
actual number of people who are deaf and hard of hearing
must be much higher,
Increasing awareness about the nature of sign languages
is evidenced in statements such as: “Through sign
2
Languages, i.e. ‘Subject-Object- Verb’. Then the translated
text can be achieved.
c) Design
The system will take English text as input. The doctor
can type the question or suggestion in a provided box. The
box can show suggestions while the doctor types his input.
The output shown to the patient can be in the form of
pictures projected on a TV screen or a Computer screen.
The images can be projected through animation or through
captured photographs.
Front end which will take text as input may look like the
following figure Fig.3:-
Fig 2Methodology
a) Phase 1
First phase of work includes collection of data for
developing the translation system. For the development of
the proposed work, all the possible conversations among
diabetic patients and doctor should be available. I have
collected all the possible conversations between an
Endocrinologist and a patient from various hospitals
and doctors. Conversation between a doctor and patient
includes the questions that a doctor asks, and the
suggestions doctor gives to his patient[8].
A sample of questions and answers has been mentioned
below:Questions 1. Do you often feel hungry?
2. Are you eating enough but still losing
weight?
3. Are you gaining weight?
4. Do you often suffer from fatigue?
Suggestions1. Avoid mango
2. Avoid beet roots
3. Avoid potato
4. Consume fenugreek powder with milk or water
5. Reduce salt intake
6. Reduce sugar intake
The next target is to understand the grammar of Indian
Sign Languages, which has also been achieved.
Fig. 3Translation System for Endocrinologist
d) Challenges
Collection of all the possible patient- endocrinologist
conversation is a daunting task. A lot of doctors have to be
consulted in order to make sure that all the data has been
collected.
Understanding grammar of ISL- The amount of work
done in India to develop Indian Sign Language is really
inadequate. It is difficult to find research papers and books
for Indian Sign Language.
Many English words do not have an assigned sign – The
words which do not have an assigned signed in Indian Sign
Language have to be spelled.
Differences in dialects – India are a country where
language changes after every mile. The differences in
dialects can be observed in Indian Sin Languages also.
IV. CONCLUSION
In this work a translation System is being developed which
will convert English text into Indian Sign Languages for
possible use by Endocrinologists. Hearing impaired people
feel a lot of trouble when they have to access information
in common places like hospitals and railway stations. Only
few people can know how to express their thoughts using
sing languages. Such a translation system can be an aid for
hearing impaired people in public places. Diabetes is a
prevailing disease in India. Many diabetic patients must be
b) Phase II
Phase II involves development of the translation system.
The input text will be parsed for its syntax which will then
be tokenised. Tokenisation refers to breaking of the input
text into small units called tokens. These small units can be
words. Afterbeing broken into smaller units, a parsed tree
will be established according to the grammar of English,
i.e. ‘Subject- Verb- Object’. Then the English parse tree
will be translated according to the grammar of Indian Sign
3
[12] Zeshan U. “Gebärdensprachen des indischenSubkontinents.
[Sign languages of the Indian Subcontinent]” Munich:
Lincom Europa, 2000.
hearing impaired. Diabetic patients need to visit their
doctors on a regular basis. The patient can write his
problems to the doctor or a system can be developed which
can capture images and can convert them into English text.
The proposed system will convert the text typed by the
doctor into Indian Sign Language understandable the
hearing impaired patient.
[13] Alison Wray, Stephen Cox, Mike Lincoln and Judy
Tryggvason, (2004) “A formulaic approach to translation at
the post office: reading the signs", Language and
Communication, 24: 59-75.
[14] UlkrineZeshan “Sign Language in Indo-Pakistan- A
Description of Signed Languages”, John Benjamins B.V.,
2000
V. FUTURE SCOPE
[15] Morrissey, S., and Way, A. An example-based approach to
translating signlanguage.
The proposed system deals with a specific domain. It
will only translate sentences used by endocrinologists. The
system can be extended to deal with various domains of
medical and health sciences. The system can include
domains like cardiology, neurology, nephrology, etc.
References
[1] Dasgupta, T., &Basu, A. (2008). An English to Indian Sign
Language Machine Translation System
www.cse.iitd.ac.in/embedded/ assistech/Proceedings/P17.pdf
[2] Kar, P., Reddy, M., Mukherjee, A., and Raina A. M. (2007)
“INGIT: Limited Domain Formulaic Translation from Hindi
Strings to Indian Sign Language” International Conference
on Natural Language Processing (ICON), Hyderabad.
[3] Mathew P. Huenerfauth (2003) “A Surveyand Critique of
American Sign Language Natural Language Generation and
Machine Translation Systems ”, University Of
Pennysylavania.
[4] Shangeetha, R.K.; Valliammai, V.; Padmavathi, S., (2012)
"Computer vision based approach for Indian Sign Language
character recognition," Machine Vision and Image
Processing (MVIP), 2012 International Conference on , vol.,
no., pp.181,184, 14-15.
[5] Sinha, S. (2007) “ A skeletal grammar of Indian sign
language”. PhD thesis
[6] Stokoe, W. C. (1960) “ Sign language structure: an outline of
the visual communication systems of the American deaf”
Silver Spring, MD: Linstok Press.
[7] Tariq, M.; Iqbal, A.; Zahid, A.; Iqbal, Z.; Akhtar, J. (2012)
"Sign language localization: Learning to eliminate language
dialects," Multitopic Conference (INMIC), 2012 15th
International , vol., no., pp.17,22, 13-15 Dec. 2012
[8] Tmar, Zouhour; Othman, Achraf; Jemni, Mohamed, (2013)
"A rule-based approach for building an artificial EnglishASL corpus," Electrical Engineering and Software
Applications (ICEESA), InternationalConference on, vol.,
no., pp.1, 4, 21-23.
[9] Ulrike Zeshan, Madan M. Vasishta, MeherSethna (2005)
“Implementation of Indian Sign LanguageI in Educational
Settings”, Asia Pacific Disability Rehabilitation Journal, Vol
16.
[10] Banerji JN. India. “International Reports of Schools for the
Deaf”, 18-19.Washington City: Volta Bureau, 1928.
[11] Cross J. “Toward a standardized sign language for India”.
Gallaudet Today, 1977; 8(1): 26-29.
4