From: Proceedings, Fourth Bar Ilan Symposium on Foundations of Artificial Intelligence. Copyright © 1995, AAAI (www.aaai.org). All rights reserved.
Bar Hillel
and Machine Translation: Then and Now
Sergei Nirenburg
Computing Research Laboratory
NewMexico State University
Las Cruces, NM88003, USA
sergei @crl.nmsu.edu
The nameof YehoshuaBar Hillel is knownto every student of machinetranslation. The late
philosopher richly deserves this recognition: he had been a central figure in the early
developmentof the field and contributed what should be considered the first set of sober
assessments for MTat the time of the widespreadgung-hoattitude to its prospects. Thefame is
probablydue to the correctness of Bar Hillel’s forecasts of the field’s future stumblingblocks.
Still, what is generally rememberedof Bar Hillel’s contributions to MTis but a small fraction
of his ideas and opinions on the subject.
There is an unsettling tendency amongmy colleagues in MT, natural language processing,
linguistics and AI tacitly to assumethat their predecessors in the field, not having at their
disposal either the latest machinesor the latest theories, were somehow
naive and "incomplete."
History of the subject starts for themwith the dissertation workof their thesis advisor. For such
people, Bar Hillel mayonly have the distinction of being the earliest widely quoted author in
MT.In reality, reading Bar Hillel can be very instructive for today’s scholars, especially as he
excelled in recognizing and assessing the intellectual evolution of entire fields (philosophy,
logic, linguistics, MT)and pointing out lacunaein the applicability of their findings.
This paper, first, takes the reader on a brief tour of Bar Hillel’s writings about MTand, second,
attempts an assessment of the current state of MTresearch in the spirit of Bar Hillel (though
without any claims for similar profundity).
1. Then
Bar Hillei’s central argumentcan be presented as the following sequence of brief theses. The
summariesare illustrated by a few quotations.
1. Bar Hillel believed that, in order to achieve fully automatichigh-quality machinetranslation
(FAHQMT),
machines must be able to process meaning. "The task of instructing a machinehow
to translate from one languageit does not and will not understandinto another language it does
not and will not understandpresents a real challengefor structural linguists, in that their thesis
that language can be exhaustively described in non-referential terms undergoes here an
experimentum
crucis. If, in a translation programme,somestep has to be taken whichdirectly or
indirectly dependson the machine’sability to understandthe text on whichit operates, then the
machinewill simply be unable to makethe step, and the whole operation will cometo a full
stop. (I have in mindpresent day machinesthat do not possess a semantic organ. Thesituation
might change in the not too distant future.)" SomeLinguistic Problems Connected With
Machine Translation. Aspects of Language, p.308. In this passage, Bar Hillel addresses
30O
BISFAI-95
From: Proceedings, Fourth Bar Ilan Symposium on Foundations of Artificial Intelligence. Copyright © 1995, AAAI (www.aaai.org). All rights reserved.
translation without treatment of meaning."Non-referential" means"uninterpreted" that is, free
of meaning. "It nowseems that for the purpose of computer-aided translation the semantic
structure of sentences to be translated has to be exhibited." The Outlook for Computational
Semantics. Aspects of Language,p.358.
2. The study of meaningin language is the realm of semantics, and there were at the time two
major traditions that could be drawn upon to provide solutions for MT.These were linguistic
semantics and logical semantics.
3. Contemporarylinguistic semantics (epitomized by the work of Katz and Fodor, 1963) was
found largely unpromisingby Bar Hillel because a) it wasnot adequately formalized; and b) its
apparatus neglected large parts of semantics such as non-compositional phenomena,and every
other phenomenonfor whose description it is necessary to rely overtly on "background
information," the knowledgeof the world. Note that Bar Hillel actually sawsomethingpositive
in Katz and Fodor’s program, in that he believed that linguistic semantics can overcomeits
shortcomingsby paying attention to issues not developedby Katz and Fodor: "It is very likely
that their shortcomings will further the field muchmore than their actual positive
achievements." Reviewof "The Structure of Language." Aspects of Language,p.175
4. Contemporarylogical semantics was dismissed by Bar Hillel in the context of MTbecause it
focused on artificial languages. "... [Rudolf Carnap]is thinking mostly in terms of constructed
languages..." The Outlook for ComputationalSemantics, Aspects of Language,p.359.
5. Bar Hillel believed that treatment of meaningcan only be based on a system of logic: first,
because for him only hypotheses formulated as logical theories had any scientific status and,
second, because inference rules necessary for MTcould only be based on logic.
6. At the same time, he considered such logical systems unattainable because they could not
workdirectly on natural language,using instead one of a numberof artificial logical notations.
"...The evaluation of arguments presented in a natural language should have been one of the
major worries.., of logic since its beginnings. However.... the actual developmentof formal
logic took a different course. It seemsthat ... the almost general attitude of all formal logicians
was to regard such an evaluation process as a two-stage affair. In the first stage, the original
language formulation had to be rephrased, without loss, in a normalized idiom, while in the
second stage, these normalized formulations wouldbe put through the grindstone of the formal
logic evaluator .... Withoutsubstantial progress in the first stage even the incredible progress
madeby mathematical logic in our time will not help us muchin solving our total problem."
Argumentationin Natural Language,Aspects of Language,pp. 202-203. Bar Hillei criticized the
methodologyof logical semanticists because they spent all their time on a partial task, without
concern for complete coverage of language phenomena."Onemajor prejudice.., is the tendency
to assign truth values to indicative sentences in natural languages and to look at those cases
where such a procedure seems to be somehowwrong..." Argumentation in Natural Language,
Aspects of Language,p. 203. There were in Bar Hillel’s time no extant proposals about howto
translate natural language into a formal language amenableto processing by logic. The reason,
according to Bar Hillel, was that such a translation process would have to rely on knowledge
about the world.
Ni~nburg 3~
From: Proceedings, Fourth Bar Ilan Symposium on Foundations of Artificial Intelligence. Copyright © 1995, AAAI (www.aaai.org). All rights reserved.
7. Thus, acquiring woddknowledgebecamea precondition for the success of an entire sequence
of enterprises culminating with machinetranslation. Bar Hillel considered this task infeasible,
and this was the ultimate reason for his well-knownpessimism about MTand computational
semantics. "It seems nowquite certain ... that with all the progress madein hardware,
programmingtechniques and linguistic insight, the quality of fully autonomousmecahnical
translation, evenwhenrestricted to scientific or technological material, will never approachthat
of qualified humantranslators and that therefore MTwill only under very exceptional
circumstancesbe able to competewith humantranslation. This "’pessimistic" evaluation is based
uponvarious considerations, only one of whichwill be presented here, and even this, for obvious
reasons, only very shortly and therefore dogmatically. Expert humantranslators use their
backgroundknowledge,mostly subconsciously, in order to resolve syntactical and semantical
ambiguities which machines will have either to leave unresolved or resolve by some
"mechanical" rule which will every so often result in a wrongtranslation." The Future of
MachineTranslation, Languageand Information, p. 182.
Bar Hillel’s famousexampleconcerned the following text: Little John was looking for his to),
box. FinaUyhe found it. The box was in the pen. John was very. happy. "Whyis it that a machine
... is ... powerless to determine the meaningof pen in our sample sentence within the given
paragraph? Theexplanation is extremely simple, and it is nothing short of amazingthat, to my
knowledge, this point has never been madebefore, in the context of MT... What makes an
intelligent humanreader grasp this meaningso unhesitatingly is ... his knowledge[of] the
relative sizes of pens, in the sense of writing implements,toy boxes and pens in the sense of
playpens... WheneverI offered this argumentto one of mycolleagues workingon MT,his first
reaction was: "But whynot envisage a system which will put this knowledgeat the disposal of
the translation machine?"Understandableas this reaction is, it is very easy to showits futility.
Whatsuch a suggestion amountsto, if taken seriously, is the requirement that a translation
machineshould not only be supplied with a dictionary but also with a universal encyclopedia.
This is surely utterly chimerical and hardly deserves any further discussion." Nonfeasibility of
FAHQMT,
Language and Information, p. 176.
8. The above line of reasoning led Bar HiUel to the conclusion that FAHQMT
should not be the
stated goal of MTresearchers, as significant practical advances could be obtained for less
demandingobjectives. "...[T]here is really no need at all to compromisein the direction of
reducing the reliability of the machineoutput. True enough, a smoothmachinetranslation looks
impressive... [but] it is muchsafer to compromise
in the other direction. Let us be satisfied with
a machineoutput which will every so often be neither unique nor smooth, which every so often
will present the post-editor with a multiplicity of renderings amongwhichhe will have to take
his choice, or with a text which, if it is unique, will not be grammatical.... Let the machine...
provide the post-editor with all possible help, present him with as manypossible renderings as
he can digest without becomingconfused by the embarrass de richesse ... but never let the
machinemakedecisions by itself on purely frequential reasons even if these frequencies can be
relied upon." Aimsand Methodsof MT.Languageand hlformation, p. 17 I.
3O2
BISFAI-95
From: Proceedings, Fourth Bar Ilan Symposium on Foundations of Artificial Intelligence. Copyright © 1995, AAAI (www.aaai.org). All rights reserved.
2.
Now
Our MTresearch group at CRLshares all of Bar Hillel’s premises, including that concerning
unattainability of a complete database of woddknowledge.Webelieve, however,that world
modelsare useful and feasible even whenthey are not completeor provably correct. This seems
to be in the spirit of Bar Hillel’s ownopinion concerningcompetenceand performancetheories:
"I have already voiced ... mymisgivings over the conception of Chomsky
and others that a more
or less completedevelopmentof a theory of competenceis a prerequisite for the developmentof
a theory of performance.That this could not be so can be seen simply fromthe fact that the very
adequacyof a particular theory of competencecan only be determinedon the basis of
performance, with or without theory" Reviewof John Lyons’ "Introduction to Theoretical
Linguistics". Aspects of Language,pp371-72.
Our approach to FAHQMT-oriented
research (e.g., Nirenburg et al., 1992, Onyshkevychand
Nirenburg, 1995) is based on the centrality of meaning and on a logical system underlying
meaning. The logical system is interpreted with the help of an ontology, a world knowledge
model.
However, at present, MTresearch and development at CRLand elsewhere, incorporates work
whoseobjectives cover the entire spectrum between theoretical and empirical linguistic
approaches, fully-automatic NIT and machine-aided humantranslation tools and, most
prominently, between "rule-based" and "corpus-based" approaches, where the latter label
applies to systemswhichtreat languageas an artifact (the sumof all texts written in it or at least
all texts available in electronic form) and apply general-purpose statistics-based methodsto
solve problems such as translation. The debate between the rule-based and corpus-based
approacheshas enlivened the field of computationallinguistics since about 1990. Onepossible
assessmentof the state of affairs in this debate follows.
Machinetranslation has been a fashionable field for at least forty years of its fifty-year history.
The reasons for this vary from R&D
glory to commercialpayoff. Over the years, an impressive
variety of methods have been used as the basis for translation programs. The problem has,
however,proved so complexthat the quality of the final result has not correlated significantly
with the methodchosen. Rather, it correlated with the amountof descriptive workon
languagethat wascarried out.
Of course, MTresearch has brought about significant side benefits. Entire scientific fields were
created largely due to MTefforts: witness the nascence of computationallinguistics. Often, MT
wasused as an application of choice for a variety of workersto test and attempt to corrborate
their theories of languageand of humanthinking capacity. It is characteristic that the final
report of the Eurotra project listed as its major success the creation of computational-linguistic
infrastructure in the countries of the European Communitydeemphasizing the fact that no
realistic MTsystem was built under its auspices. Manyfactors contributed to the lack of the
engineering achievementin this project, amongthem the relative lack of accent in Eurotra on
actual description and system building, with preference given to designing detailed formal
specifications of (largely syntactic) levels of analysis and their correspondingformalisms.
Nirenburg 303
From: Proceedings, Fourth Bar Ilan Symposium on Foundations of Artificial Intelligence. Copyright © 1995, AAAI (www.aaai.org). All rights reserved.
Is the Eurotra case prototypical for the entire field of MT?Oneof the problemswith the field
has been that the descriptive work is, frankly, rather monotonousand boring. This is why
attempts were made either to make it less boring (by adding an independently motivated
theoretical angle to the descriptive work)or to try to avoidit altogether.
The latter objective was mademanifest in a) attempts to use AI learning techniques or more
practical semi-automatic procedures for knowledgeacquisition and b) the application of
statistical methodsfor establishing cross-linguistic correspondences in lieu of language
description work. Theformer solution madeitself manifest in viewingMTas a testbed for one’s
favorite linguistic or computational-linguistic theories, such as the currently fashionable
"principle-based" approach to syntax. Machinetranslation is indeed a tempting avenue of
computational inquiry into modeling humanmental and language processes, and a numberof
approaches to NLPin AI dabbled in MTas a potential application. Knowledge-basedMTis a
direct offshootof the AItradition.
Themost remarkablefeature of the statistical methodsin MTis that they are not at all specific
to their subject matter --- the sametechniques applied to processing language could and are
used, for example,in the studies of the humangenome.
The current R&D-orientedMTapproaches, whether rule-based or statistical or hybrid, are based
on "imported" ideas. At the same time, the best systems on the market cannot boast muchby
wayof technological or scientific advances. Instead, they rely on brawn: huge, handcrafted
dictionaries and grammarsand a plethora of specialized translation routines. All of us are
curious to see howwell the R&D
approaches will workonce sufficient resources are allocated
for one or moreof themto reach the status of a product. Thequestion is: what kind of imported
techniques shows the most promise? The answer is not clearly obvious and is determined by
sociological (read: the vagaries of funding) as well as scientific and technologicaltrends.
The major scientific (or methodological)trend in the field is experimentingwith howwell the
statistics-oriented methodswill advancethe state of the art in MTwithout the need for massive
manual knowledgeacquisition.
Themajor technological trend in the field looking for the best waysof mixingthe statistical and
the "rule-based" methods.This author has been an early advocate of mixingsuch methodsat the
level of their final results, a methodcalled multi-engine MT.Other approaches seek a more
involvedinteraction, with statistics used not only during the process of MTbut also to support
developmentof backgroundresources (i.e., dictionaries and grammars).
Themajor sociological trend, at least in the US, is the emphasison a regimenof evaluations and
competitions amongMT(and, more broadly, NLP)systems. This promotes rigor and discipline
as well as conformity and search for local solutions, which are not necessarily the most
promising ones in the long run. Approaches that show a steady improvementare rewarded.
Approacheswith long gestation periods are punished.
Emphasison mixedapproachesis, for non-statisticians, a rearguard regrouping action, while for
statisticians (witness the evolution of the claims and practices of the CandideIBMMTgroup),
3O4 BISFAI-95
From: Proceedings, Fourth Bar Ilan Symposium on Foundations of Artificial Intelligence. Copyright © 1995, AAAI (www.aaai.org). All rights reserved.
search for any avenuefor improvingthe rather modestfinal results.
The knowledge-based and linguistics-based
methods will do good by regrouping and
concentrating on those tasks and situations in whichstatistical approachesfail to deliver. One
must, however, rememberthe lesson of computer chess: at present, the best chess-playing
systems are not terribly knowledgeableabout chess strategy and tactics but they consistently
beat AI-based programs and competeon equal terms with grandmasters. The $64,000 question
is: howmuchmorecomplexis humantranslation ability comparedto the humanchess-playing
ability? That is, for howlong will there be an opportunity to study language use through MT?If
statistical methodssucceed, rule-based MTmaygo the wayof the M-basedchess programs.
MTseems to be too complex a task to be fully accountable for by the current statistical
processing methods, even though these methods do not aspire to building representational
models of humanlanguage capacity and rely only on the input-output behavior of such models
(in MT,a text and its translation). In the final analysis, the open-endednessof languagewill
becomethe stumbling block for these methodsconceptually, just as logistically, the chronic
shortage of resources (bilingual corpora) mayprecipitate the swing of the pendulumof
R&D
fashion back to the mentalist campfrom its current behaviorist direction.
Howlong will this take? If history is any guide, such swings comeroughly every 30 years:
mentalism was in scientific ascendancy between 1960 and 1990, while behaviorism reigned, at
least in the US, for about thirty years prior to that. Of course, wecannot be certain that weare
witnessing this pendulumswing and not some other, unconnected development. Time will show.
Amoreintriguing thought is that, just possibly, the rule-based/corpus-baseddichotomyis not as
important as we currently think. Maybethe real problemof MTas technology is that it is not
generally understood howdifficult the problem actually is. The confident claims, madeby
newcomersto MT(including this author some fifteen years ago), help stoke the high
expectations of getting the desired result with a modestexpenditure. At the current level of MT
R&D,either the expectations should be loweredor the time scale of getting the results must be
significantly extended.
The above list of opinions, though quite current in 1996, would not have been entirely out of
place around 1960, the time at which Bar Hillel’s celebrated assessments of MTwere published.
I think the entire enterprise of MTwill continue to be reasonably successful (recurring
outlandish claims comingfrom confident newcomersnotwithstanding) as long as at least some
of Bar Hillel’s legacy of self-assessmentis preserved.
References
Bar Hillel, Y. 1964. Languageand Information. Reading, MA:Addison Wesley.
Bar Hillel, Y. 1970. Aspects of Language. Jerusalem: Magnes.
Katz, J. and J. Fodor. 1963. TheStructure of a SemanticTheory.Language,39, pp. 170-210.
Nirenburg, S., J.Carbonell, M.Tomita and K.Goodman. 1992. Machine Translation: A
Knowledge-Based Approach. San Mateo, CA: Morgan Kaufmann.
Onyshkevych, B. atad S. Nirenburg. 1995. A Lexicon for Knowledge-Based MT. Machine
Translation, 10, pp. 5-57.
Nirenburg 305