Monotrans: Human-Computer
Collaborative Translation
Crowdsourcing Translation with People Who Speak Only One
Language
Chang Hu, Ben Bederson, Philip Resnik
Human-Computer Interaction Lab
Computational Linguistics and Information Processing Lab
University of Maryland
Outline
• Why translation by monolingual people?
• How Monotrans works
• Research prototype
• Preliminary evaluation
Languages on Internet by Population
2000
Japanese
9%
the rest
29%
English
52%
Spanish
5%
Chinese
5%
Source: Global Reach,
Internet World Stats
Languages on Internet by Population
2005
2000
Japanese
9%
Spanish
5%
the rest
29%
English
52%
the rest
31%
English
32%
Chinese
5%
Japanese
8%
Spanish
8%
Chinese
21%
Source: Global Reach,
Internet World Stats
Languages on Internet by Population
2000
Japanese
9%
Spanish
5%
the rest
29%
English
52%
2009
2005
the rest
31%
English
32%
Chinese
5%
Japanese
8% Spanish
Chinese
21%
the rest
37%
English
28%
8%
Japanese
5%
Spanish
8%
Chinese
22%
Source: Global Reach,
Internet World Stats
A real-world problem:
International Children’s Digital Library
www.childrenslibrary.org
Machine Translation (MT)
(餐厅= restaurant, dining hall)
Large volume, cheap, fast
Unreliable quality
Professional Translators
High quality, but slow and expensive
(even for common language pairs)
Translation with the Crowd
Bottle neck: bilingual people
Translation
with the
Monolingual
Crowd
Translation
with
the Crowd
Wikipedia: 800 translators vs. 75,000 contributors
Affordability
Machine
Translation
Monolingual
Human
Participation
Amateur Bilingual
Human Participation
Professional Bilingual
Human Participation
Quality
Outline
• Why translation by monolingual people?
• How Monotrans works
• Research prototype
• Preliminary evaluation
Basic Idea
Source language speaker
Target language speaker
Original source
sentence
MT
Inaccurate
translation
Inaccurate back
translation
MT
Fluent
translation
Fluent, accurate
source sentence
MT
Et cetera…
An (Richer) Example
Pierre
Says: En général, on s'entend bien, tous les
deux.
(lit. In general, we get along together, the two
of us.)
Mary
Pierre
Mary
Says: En général, on s'entend bien, tous les
Sees: In general, it means well, both.
deux.
MT
(lit. In general, we get along together, the two
of us.)
Pierre
Mary
Says: En général, on s'entend bien, tous les
Sees: In general, it means well, both.
deux.
MT
(lit. In general, we get along together, the two
of us.)
Edits into: In general, it is about both of us.
Pierre
Mary
Says: En général, on s'entend bien, tous les
Sees: In general, it means well, both.
deux.
MT
(lit. In general, we get along together, the two
of us.)
Sees: En général, Il est à la fois de nous.(*)
MT
Edits into: In general, it is about both of us.
Pierre
Mary
Says: En général, on s'entend bien, tous les
Sees: In general, it means well, both.
deux.
MT
(lit. In general, we get along together, the two
of us.)
Sees: En général, Il est à la fois de nous.(*)
Edits into:
En général, nous nous entendons bien.
(lit. In general, we get along well.)
MT
Edits into: In general, it is about both of us.
Pierre
Mary
Says: En général, on s'entend bien, tous les
Sees: In general, it means well, both.
deux.
MT
(lit. In general, we get along together, the two
of us.)
Sees: En général, Il est à la fois de nous.(*)
MT
Edits into:
En général, nous nous entendons bien.
MT
Edits into: In general, it is about both of us.
Sees:
In general, we get along fine.
enrichment
(lit. In general, we get along well.)
En général
In general
Nous entendons
Get along
Pierre
Mary
Says: En général, on s'entend bien, tous les
Sees: In general, it means well, both.
deux.
MT
(lit. In general, we get along together, the two
of us.)
Sees: En général, Il est à la fois de nous.(*)
MT
Edits into:
En général, nous nous entendons bien.
MT
Edits into: In general, it is about both of us.
Sees:
In general, we get along fine.
enrichment
(lit. In general, we get along well.)
Pierre
Mary
Says: En général, on s'entend bien, tous les
Sees: In general, it means well, both.
deux.
MT
(lit. In general, we get along together, the two
of us.)
Sees: En général, Il est à la fois de nous.(*)
MT
Edits into:
En général, nous nous entendons bien.
MT
Edits into: In general, it is about both of us.
Sees:
In general, we get along fine.
enrichment
(lit. In general, we get along well.)
Edits into: In general, we get along well.
Pierre
Mary
Says: En général, on s'entend bien, tous les
Sees: In general, it means well, both.
deux.
MT
(lit. In general, we get along together, the two
of us.)
Sees: En général, Il est à la fois de nous.(*)
MT
Edits into:
En général, nous nous entendons bien.
MT
Edits into: In general, it is about both of us.
Sees:
In general, we get along fine.
enrichment
(lit. In general, we get along well.)
Sees: En général, nous nous entendons bien.
(lit. In general, we get along well.)
MT
Edits into: In general, we get along well.
Pierre
Mary
Says: En général, on s'entend bien, tous les
Sees: In general, it means well, both.
deux.
MT
(lit. In general, we get along together, the two
of us.)
Sees: En général, Il est à la fois de nous.(*)
MT
Edits into:
En général, nous nous entendons bien.
MT
Edits into: In general, it is about both of us.
Sees:
In general, we get along fine.
enrichment
(lit. In general, we get along well.)
Sees: En général, nous nous entendons bien.
(lit. In general, we get along well.)
Proposes to stop with current translation
MT
Edits into: In general, we get along well.
Agrees to stop with current translation
Monotrans Protocol
Outline
• Why translation by monolingual people?
• How Monotrans works
• Research prototype
• Preliminary evaluation
Web link
Image
Mark OK
Mark unclear
Outline
• Why translation by monolingual people?
• How Monotrans works
• Research prototype
• Preliminary evaluation
Preliminary Evaluation
• Older version of the UI (same protocol)
• Children’s book, Russian to Chinese
• 2 Russian speakers and 4 Chinese speakers
formed 4 Pairs*
• 1 hour per pair
Results
• 44 sentences (6 pages) worked on
• 28 sentences finished (≈ 4 pages)
• Overall translation speed: 50 words per hour

professional translator speed: 250 words per hour
Evaluation
14
12
12
Google Translate
Monotrans
# of sentences
10
9
8
7
6
6
6
5
4
4
4
3
2
0
0
none
little
much
most
Original meaning translated
all
Google Translate …
14
12
Google Translate
Monotrans
# of sentences
10
9
8
6
6
6
4
4
3
2
0
none
little
much
most
Original meaning translated
all
… Monotrans
14
12
12
Google Translate
Monotrans
# of sentences
10
9
8
7
6
6
6
5
4
4
4
3
2
0
0
None
Little
Much
Most
Original meaning translated
All
Where to from here?
• Larger and more formal validation of the protocol
• Richer annotations
✓Images
✓Web links
✓Marking correct spans
✓Marking incorrect spans
Paraphrase
Word clouds
…??
• Large-scale crowd support

(CrowdFlow talk @1:20PM)
Take-Away Message
• Monolingual translation can help large-scale
translation
• Translation with monolingual people is
actually feasible
Sponsors
Q&A
Thank You
Backup slides
Projected annotation
• Project information from one language to
another using word alignments as a bridge
• Illustration of how this has been done for
natural language annotation
[Kolak 2005]
Projected annotation
Tout le monde doit entendre
l'histoire de Cendrillon
Tout le monde doit entendre
l'histoire de Cendrillon
Tout le monde doit entendre
l'histoire de Cendrillon
=> Pilot experiment results:
Projected annotations helped
improve translation
MT
Everybody has heard the
business by Cinderella
MT
Everybody has heard
the business by
Cinderella
MT
Everybody has heard
the business by
Cinderella
Everybody has heard the
story about Cinderella
One of my examples
involves rmvng ll th
vwls frm th wrds nd
shwng tht th rdr cn stll
ndrstnd th sntnc.
Three Types of Errors
Tout le monde doit entendre l'histoire de Cendrillon.
MT
Everybody has hear
story about Cinderella
Everybody has heard the story about Cinderella
I. Detectable and Correctable Error
Pilot experiment results: Post-editing machine translation output
by monolingual people improves translation quality
Three Types of Errors
Tout le monde doit entendre l'histoire de Cendrillon.
MT
Everybody has hear
story about Cinderella
MT
Everybody has heard the
business by Cinderella
Communication needed
Everybody has heard the story about Cinderella
II. Detectable but not Correctable Error
Three Types of Errors
Tout le monde doit entendre l'histoire de Cendrillon.
MT
Everybody has hear
story about Cinderella
MT
Everybody has heard the
business by Cinderella
Everybody has heard the story about Cinderella
II. Detectable but not Correctable Error
Pilot experiment results: Communication through enrichment
channel can improve translation
Three Types of Errors
Tout le monde doit entendre l'histoire de Cendrillon.
MT
Everybody has hear
story about Cinderella
MT
Everybody has heard the
business by Cinderella
MT
Everybody loves the
story about Cinderella
Need more redundancy
Everybody has heard the story about Cinderella
III. Undetectable Error
Add more redundancy, reduce it to type I or type II
Prototype Evaluation
(1=unintelligible, 4=very intelligible)
(1=not translated, 5=full meaning)
System seems promising