Development of NE Wordnet: An Integrated - cfilt

advertisement

Development of NE Wordnet:

An Integrated Wordnet for Languages of the

North-East India

Assamese & Bodo by

Utpal Saikia

Biswajit Brahma

Dibyajyoti Sarmah

Dept. of Computer Science & Information Technology

Gauhati University

INTRODUCTION

 NE Wordnet Project for Assamese & Bodo started in 2009.

 NE Wordnet Project for Assamese & Bodo have been developed with expansion approach with the Original Hindi Wordnet structure against the IDs and Concept of Hindi Wordnet.

NE Wordnet Development Project outcomes till now

Validation

 All the Assamese and Bodo Wordnet activities have been reviewed by the Professors of the Department of

Assamese, Modern Indian Language & Bodo of Gauhati

University as well as other invited resource persons.

Contd….

 The developed NE Wordnet structured in the form of Database, integrated with interactive Interface is ready for different NLP research and Development. Different NLP application and research related works already started using the NE WordNet.

 Automatic Bilingual Dictionary Construction: Assamese-Bodo

Dictionary Construction : Prototype developed at Gauhati University.

 Web based Automatic Multilingual Dictionary Construction:

Assamese-Bodo-Nepali-Hindi-English Dictionary Construction: Full

Web based System ready: By Gauhati University Team.

 Intelligent Document Categorizing System: Prototype Developed and

Tested at Gauhati University: Research Paper already accepted for

GWA-2010.

NE Wordnet Development Project outcomes till now

Following are the glosses which are completed in Assamese language till now:

 common Synset completed = 11579

Pan Indian Synset all Completed

Universal Synset (Total= 7168) completed = 7147

Adjective Synset Completed = 2376 (Total = 3605)

Adverb Synset Completed = 174 (Total= 209)

Verb Synset Completed = 1588 (Total = 1798)

Language Specific completed = 127 (Total =1000)

Total linked Number =24,338

NE Wordnet Development Project outcomes till now

Following are the glosses which are completed in

Bodo language till now:

 common synset completed = 11522

Pan Indian synset all Completed

Universal Synset (Total= 7168) completed = 7143

Adverb Synset Completed = 192 (Total= 209)

Adjective Synset Completed = 2473 (Total = 3605)

Verb Synset Completed = 1752 (Total = 1798)

Synset Ranker = 34264 (34378)

Language Specific = 74

Total linked number = 24,493

Problems Faced During Development for Assamese

Synset related:

In common synsets of Assamese and Bodo, a few number of synsets do not have proper Assamese word to represent. So they are not entered yet. Those left synsets have been send to the expert committee to review.

Expansion from Hindi/English:

The main challenge in expansion approach is in one to one mapping.

Problems Faced During Development for Bodo

 Challenges in Expansion

Bodo is a developing language. It does not have a very strong linguistic resource. Also literature resource is very limited. The language does not have enough vocabulary, and new and new words are being discovered, coined and added. As a result, the development of Bodo Wordnet faces typical and frequent problems, and overcoming the problems to accommodate expansion of the

Hindi Wordnet with one to one mapping has been a big challenge

Workshop/conference organized and participated by the member groups:

1.Global Wordnet Conference in IIT, Mumbai from 31st Jan.-4th Feb. 2010

2. Indo Wordnet Conference in Amrita University, Coimbatore, in June,

2010

3. NE Wordnet Workshop, Guwahati, Assam, 2010

4. Indo Wordnet Workshop, IIT Kharagpur, 2010

5.Attended Spell checker training, C-DAC Pune, 2010

6. Indo Wordnet Workshop, Shillong, 2011

7. CLIA developers workshop, C-DAC Pune, 2011

8. Multiword Expression Workshop, University of Kashmir, Srinagar, 2011

Tools, Applications & Research

 During this period, language specific tools have been developed .

Language specific Synset creation tools interface

 Multi_lingual_dictionary

[Online Bodo, Assamese and Hindi Language]:

Step1:

First select the language

Step2:

Type the word of the language

Step3:

When word automatically come then select the word

Step4:

After search the word

Published paper in conferences/journals/workshop

1.

A Novel Approach for Document Classification using Assamese

WordNet, Jumi Sarmah, Navanath Saharia and Shikhar K. Sarma,

Global Wordnet Conference (GWC), Japan, 2012

2.

3.

4.

Assamese Vocabulary and Assamese Wordnet Building: An Analysis,

Shikhar Kr. Sarma, Utpal Saikia, Mayashree Mahanta, Himadri

Bharali, Global Wordnet Conference (GWC), Japan, 2012

Foundation and Structure of Developing an Assamese Wordnet,

Shikhar Kr. Sarma, Moromi Gogoi, Rakesh Medhi, Utpal Saikia,

Global Wordnet Conference, IIT Bombay, 2010

A Wordnet for Bodo Language: Structure and Development, Shikhar

Kr. Sarma, Moromi Gogoi, Biswajit Brahma, Mane Bala Ramchiary,

Global Wordnet Conference, IIT Bombay, 2010

Published paper in conferences/journals/workshop

5.

6.

7.

8.

A Novel Approach for Document Classification using Assamese

WordNet, Jumi Sarmah, Navanath Saharia and Shikhar K. Sarma,

Global Wordnet Conference (GWC), Japan, 2012

Assamese Vocabulary and Assamese Wordnet Building: An Analysis,

Shikhar Kr. Sarma, Utpal Saikia, Mayashree Mahanta, Himadri

Bharali, Global Wordnet Conference (GWC), Japan, 2012

Foundation and Structure of Developing an Assamese Wordnet,

Shikhar Kr. Sarma, Moromi Gogoi, Rakesh Medhi, Utpal Saikia,

Global Wordnet Conference, IIT Bombay, 2010

A Wordnet for Bodo Language: Structure and Development, Shikhar

Kr. Sarma, Moromi Gogoi, Biswajit Brahma, Mane Bala Ramchiary,

Global Wordnet Conference, IIT Bombay, 2010

Contd …

9. Kinship Terms in Assamese Language, Shikhar Kumar Sarma, Utpal

Saikia, Mayashree Mahanta, Indo Wordnet Workshop, IIT, Kharagpur,

2010

10. Formation of Kinship Terms in Bodo Langauge, Shikhar Kr. Sarma,

Biswajit Brahma, Mane Bala Ramchiary, Indowordnet Workshop, IIT

Kharagpur, 2010

11. Architecture of a Spell Checker for An Indo-Aryan Language: Assamese,

Gogoi, Ambeswar. Shikhar Kr. Sarma and Kishore Baishya, International journal of Computational Linguistics, Volume (1): Issue (1), 2009

12. A case study of Dictionary Annotation As A Pre-procesing task to develop

Assamese Spell checker, Ambeswar Gogoi and Kishore Baishya, Making of Electronic Dictionary, Linguistic Data Consortium for Indian Languages,

CIIL Mysore, 2009

Conclusion

 Integration and collaboration of the man powers in the field of

Linguistics and Computing; Trained man power development in the field of NLP, Local Language Technology Development.

 Through this project a new breed of researchers in language technologies have been trained for proper skills and knowledge sets. As in these local languages the linguistic and Literature studies in formal education are with minimum computational linkage, and with no training/exposure for interlinking of linguistics and computing, the project facilitates in developing a team of interdisciplinary researchers. The project has contributed in expertise development and awareness creation in latest in machine translation, lexical semantics, cross lingual IR etc. in specific.

THANK YOU

Download