Evaluating semantic similarity and sameness in studies of polysemy

advertisement

Evaluating semantic similarity and sameness in studies of polysemy and synonymy

Jarno Raukko (U. Helsinki)

1

For a full version of the PPT, see handout distributed Oct 28, 2010.

SKY WEBPAGE VERSION

2

2. Are violin and fiddle synonyms?

EXPECTED ANSWER:

”Well, almost.”

(SYNONYMY)

Examples

1. Are thrifty and stingy synonyms?

EXPECTED ANSWER:

”Well, not quite.”

3. Does back have the same meaning in

My back hurts and

I came back?

EXPECTED ANSWER:

”Not at all. Different.”

4. Does back have the same meaning in

I came back and

I got it back?

EXPECTED ANSWER:

”Well, almost.”

(POLYSEMY)

3

Relevance of semantic similarity

(vs. difference)

• In synonymy: you expect similarity for a pair/(set) of items to be of interest

• In polysemy: primarily, you expect difference for a pair/(set) of items to be of interest;

secondarily, you group items according to similarity and difference

4

Yet…

SYNONYMY

SIMILARITY DIFFERENCE

(DEFAULT)

Synonymy is about similar meanings of different words.

But you are interested in the differences between near-synonyms.

POLYSEMY

Polysemy is about related (and therefore somehow similar) meanings of a word.

(DEFAULT)

Polysemy is about different (related) meanings of a word.

5

synonymy --- polysemy ?

• Dirk Geeraerts tomorrow in Helsinki:

”The problem of synonymy and the problem of polysemy are essentially the same”

• Dylan Glynn & Justyna Robinson (eds, in press)

Polysemy and Synonymy. Corpus methods and

applications in Cognitive Linguistics. Amsterdam:

Benjamins.

6

synonymy

WORD 1 WORD 2

If their semantic content is similar or the same, this is a case of synonymy.

If their semantic content is (very) different, a researcher of synonymy ignores this case.

7

polysemy

MEANING 1

OF WORD 1

MEANING 2

OF WORD 1

The starting point is that Word 1 has at least 2 (different) meanings.

If meanings 1 and 2 are very similar, this might be a case of vagueness.

If meanings 1 and 2 are totally different (and not related semantically), this might be a case of homonymy.

If meanings 1 and 2 are somewhat different but somehow relatable (or a bit similar), this is probably a case of polysemy.

henceforth W = word M = meaning

8

scale of similarity: synonymy

The meaning of W1 and W2 is…

THE SAME --------------------------------- DIFFERENT perfect nearweak synonymy synonymy synonymy full semiquasisynonymy synonymy synonymy

NOT

WORTH

DISCUSSION

9

scale of similarity: polysemy

The meaning of M1 and M2 (of W1) is…

THE SAME --------------------------------- very DIFFERENT two vaguep o l y s e m y instances ness of the same meaning two instances of the same meaning type

(ambiguity) instances of different (yet related) meaning types homonymy

10

Main question

• Is semantic similarity somehow different when we look at polysemy than we look at synonymy?

11

Differences so far

• Which is the default, similarity or difference?

• In synonymy, we idealize on the extreme of the scale, but mainly look at the part of the scale which is (fairly) close to the extreme.

• In polysemy, we operate pretty much on the whole scale, with focus on the middle.

12

synonymy --- polysemy ?

• when you study synonymy, the polysemy of the items gets in the way

– can you ever say “W1 and W2 are synonymous”?

– should you always say “M x synonymous?

of W1 and M y of W2” are

• when you study polysemy, you often use synonyms to talk about meanings

– “Are get ‘receive’ and get ‘arrive’ meanings of the same verb?”

13

synonymy --- polysemy ?

• Synonymy occurs when meaning is shared (but form differs)

• Polysemy occurs when form is shared (but meaning differs)

• Synonymy is a relational lexical-semantic property that unites

(parts of the semantic potential of) “accidentally” coinciding words

– The forms of words involved in the synonymy relationship are arbitrary (although the relationships might be nonarbitrary, cf. Levin this morning)

– The semantic value (that is shared) is motivating enough that two or more forms coincide on it

– It is typical that one meaning can be expressed with two different words.

14

synonymy --- polysemy ?

• Polysemy is a semantic property of one word at a time that unites meanings. The relationship between them is motivated, but it is only sometimes predictable.

– It is not accidental or arbitrary that words acquire polysemy. It is in their nature. :-)

– It is typical for semantic value to be flexible, extended, and

“multiplied”.

– Polysemy is about categorization, both between words (W1 covers a semantic territory) and within a word (M1 and M2 are categories too).

• One form : One meaning

• a principle that cognition may strive for / take as a default

• synonymy breaks it

• polysemy breaks it

15

synonymy --- polysemy ?

• The role of co(n)text

• You can evaluate synonymy in identical co(n)texts:

I like to play the fiddle in bars.

I like to play the violin in bars.

• Usually you evaluate polysemy in non-identical co(n)texts

I got to Zabriskie Point.

I got to a point in my life where…

• But you can use identical co(n)texts as well.

I got to be the last one.

I got to be the last one.

16

evaluating

• To study shades of semantic similarity, we need to evaluate it.

• A corpus cannot tell us if two instances are semantically similar

– It requires human judgement

• The main use of evaluating in this paper:

– How informants / test subjects / speakers evaluate the semantic similarity (or difference) of linguistic items in a more or less experimental setting

(e.g., similarity rating test)

≈ Data elicitation ≈ Population test

17

evaluating

• quantitative:

– Estimate the degree of synonymy

(or semantic distance between two meanings in polysemy)

• qualitative:

– Justify / explain / explicate the nature of / the reason for semantic similarity

18

evaluating takes place in real life as well

• synonymy (examples)

– in linguistic production, you e.g. estimate which of the near-synonyms might suit your needs best

– in comprehension, you e.g. estimate whether near-synonyms that you have encountered refer to the same semantic value

– in communication, when you negotiate meaning, you e.g. operate with synonymous alternatives

• polysemy (examples)

– in production, you e.g. apply words to new contexts

– in comprehension, you e.g. approximate meanings according to related meanings of the same word

– jokes often exploit polysemy

– polysemy may cause misunderstandings

– in communication, when you negotiate meaning, you e.g. cross-check with polysemy of other words

19

(Back to experiments/elicitation.)

Expected difference between synonymy and polysemy, 1

• If an informant is asked to rate the semantic similarity/difference of two words,

– the very fact that they are different words might cause her/him to presuppose that there is at least some semantic difference.

– Therefore, rating two words ”semantically identical” requires a marked choice.

– However, if the informant realizes that the researcher is after synonymy, then evaluating W1 and W2 as semantically similar is more likely.

20

Expected difference between synonymy and polysemy, 2

• If an informant is asked to rate the semantic similarity/difference of two meanings of one word,

– the very fact that they are uses/instances of the same word might cause her/him to presuppose that there is at least some semantic similarity.

– Therefore, rating two words ”semantically totally/very different” requires a marked choice.

– However, if the informant realizes that the researcher is after polysemy, then evaluating M1 and M2 as semantically different is more likely.

21

Factors that influence

• In both cases (synonymy and polysemy)

– it matters a great deal

• Which test (type) we use

• What the instructions (exact phrasings) are

• Whether there is an example rating given by the researcher

• What the selection of stimuli is

• What the linguistic context of each stimulus is

• Which types of cases have been placed in the

beginning of the test (or, the order in general)

22

Factors that influence

• Should we expect (total) consensus?

• No. There will be subjective differences.

• Why?

• The nature of semantics:

• Based on intersubjective convention

• Based on negotiation and flexibility

• Must allow for variability and variation

23

Examples from (more or less) experimental studies on synonymy and polysemy

24

Whitten & al. 1979 (synonymy)

• “Indicate the degree to which two words have the same meaning by writing a digit from 1 to 7.”

• 7 =excellent synonymy

• 1 = poor synonymy

• All 464 stimulus noun pairs were listed as synonyms in standard references.

• The rated degree of synonymy ranged from 6.79 to

2.24. The median was 5.08.

• If placed within context of nonsynonym pairs, the ratings for the low end might have been higher.

25

Whitten & al. 1979 (synonymy) cont’d

• Stimulus pairs at the high end:

• purchase – buy

• lawyer – attorney

6.79

6.78

• autumn – fall penny – cent

6.72

6.71

• taxi – cab 6.71

• Stimulus pairs close to the median

• college – university 5.12

• output – yield 5.10

• expert – authority 5.09

• effort – attempt

• servant – maid

• soldier – warrior

5.08

5.08

5.07

26

Whitten & al. 1979 (synonymy) cont’d

• Stimulus pairs at the low end:

• thunder – clap

• patient – invalid

• needle – spike

2.72

2.55

• visit – chat 2.52

• suburb – neighborhood 2.34

2.24

• Although instructions said that all stimuli are nouns, some of these are more common as verbs: buy, purchase, visit, chat

• The polysemy is obvious in many cases: fall, authority, clap, patient, invalid

27

Whitten & al. 1979 (synonymy) cont’d

• The main variable that they paid attention to was the order of the two stimuli: ½ of the informants got “forward order”, ½ got

“back order”.

– In 1979 one of their main aims was to study the structurings of the mental lexicon and lexical access.

– Example: purchase => buy 6.72

buy => purchase 6.86

– On average, perceived synonymy was affected by word order.

– For 21 word pairs, the effect of the order was significant.

28

Whitten & al. 1979 (synonymy) cont’d

• Some of the 21 word pairs where the order played a significant role in the rating of the degree of synonymy: motive => reason quarter => fourth mission => task era => age appetite => hunger nectar => honey aborigine => native

6.28

6.24

5.66

5.80

5.18

4.94

4.52

reason => motive fourth => quarter task => mission age => era

5.56

5.00

4.84

4.60

hunger => appetite honey => nectar

4.24

3.68

native => aborigine 3.22

• Generalization: a more specific, more academic, and less polysemous word prompts a positive synonymy judgement more readily than vice versa.

29

Whitten & al. 1979 (synonymy) cont’d

• Variance (between informants)

– Mostly .50–1.20 at the end of 50 most synonymous

• Exceptionally high variance at the high synonymy end:

murder => homicide 2.75 (cf. homicide => murder

1.03)

– Mostly 2.00–3.00 at the median of the scale

• Exceptionally low variance: province => territory 1.55

• Exceptionally high variance: congress => legislature 3.79

– Mostly 2.50–4.00 at the end of 50 least synonymous

• That is, there was little consensus at the lower end of the scale.

30

Raukko 1994 (polysemy)

• “Decide whether the word get carries the same meaning or two different meanings in the sentences.”

– 0 = the same meaning

– 2 = somewhat different meaning

– 4 = very different meaning

(heuristic post hoc: 4 might mean homonymy; 0 would refer to two instances of the same meaning type; typical polysemy would be 1...3)

31

Raukko 1994 (polysemy)(cont’d)

• Data from my 1994 test, see handout.

32

Comparisons so far

• Whitten & al. / synonymy

– scale 1...7 (1 = very different meaning, 7 = same meaning)

– synonymy ratings ranged 2.24...6.79

– median 5.08 (most pairs were viewed at least somewhat synonymous)

• Raukko / polysemy

– scale 0...4 (0 = same meaning, 4 = very different meaning)

– polysemy ratings ranged 0.45...3.13

– average rating 1.55, median 1.34 (most pairs were viewed as having fairly similar but not identical meaning)

33

Comparisons so far

• Whitten & al. / synonymy

– informants saw synonymy where they were supposed to

• Raukko / polysemy

– informants did not see large meaning difference for the most part => get is polysemous, not homonymous

– they saw some similarities, some differences, as predicted

=> they saw polysemy

• both

– differing degrees of similarity were apparent

– many ratings make sense, some don’t

– method is useful but there are skewing effects and irreliability in several details of the setting

34

Conclusions so far

• In both synonymy and polysemy studies, semantic intuitions vary.

• In both synonymy and polysemy studies, finding a scale of semantic similarity is useful.

– Cf. Sandra & Rice 1995: 125

• “[researchers of prepositional polysemy] cannot propose extremely fine-grained distinctions without bothering about empirical data”

• “language users’ mental representation [...] is [in fact] characterized by a high degree of granularity”

35

quantitative => qualitative

• Whitten & al’s and Raukko’s similarity rating tests did not include informants justifying and explaining their ratings.

• E.g., Liu (this symposium) reports tests with informants explaining their choices.

• In Raukko’s study, qualitative results come from other types of tests

– sorting test: (1) combine stimuli into categories, (2) give names to categories, etc.

– production test: (1) produce examples of the use of polysemy, (2) explain links you find between them, etc.

• Vanhatalo 2005

36

Vanhatalo 2005 (synonymy)

• her PhD, The use of questionnaires in exploring synonymy

• several types of tests

– choose most likely components

– rate components

– choose better alternative (cf. Liu)

– complete as sentences (only the word given)

– define typical frames

– spell out semantic differences

37

Vanhatalo 2005 (cont’d)

• several factors investigated

– 18 Finnish verbs of “nagging”, 17 Estonian verbs of nagging

• the gender and age of the portrayed speaker (the subject of “nag”)

• the degree of irritation of the portrayed speaker and hearer

• the volume of the vocal act

– 2-4 Finnish adjectives ‘important, central, crucial, significant’: open questions mainly

38

Vanhatalo 2005 (cont’d)

• main results (Vanhatalo 2005: 40-45): the questionnaire method

– helped to trace differences in the meaning and use of synonyms

• many differences not documented before in dictionaries

• sometimes consensus, sometimes deviation

• useful especially for large groups of semantically similar words

• (Vanhatalo did not use the method for placing synonyms on a scale of similarity)

• both open questions and ratings should be used

39

Vanhatalo 2005 (cont’d)

• main results (Vanhatalo) (cont’d)

– helped to find differences between related words in

Estonian and Finnish

– sociodemographic variables caused fairly little variation

• age and education affected a bit more than gender

• answers critique

40

Vanhatalo 2005 (cont’d)

• main results (Vanhatalo) (cont’d)

– when both corpus method and questionnaire method were applicable, they yielded similar results

• however, justification of results was different

• questionnaire method dug up semantic properties that corpus method could not

• in addition, can tackle low-frequency words

– results of questionnaire method can be utilized in the production of electronic dictionaries

41

Other studies of synonymy that employ experimental techniques

• Arppe & Järvikivi 2002, 2007

• Divjak & Gries 2008

• Liu, in this symposium

• Oversteegen, in this symposium

• etc.

42

polysemy / qualitative

• In experimental settings (e.g., the sorting test) :

– An informant gives a name to a meaning type, a category within polysemy

– An informant spells out the semantic link between two meanings

– An informant draws a hierarchy between macrotypes and microtypes (more general and more specific meaning types)

– An informant pinpoints at cases difficult to evaluate

43

• to conclude…

And…

44

Evaluating semantic similarity

• Both synonymy and polysemy operate on the scale of semantic similarity vs. difference.

• Knowing about the degree of similarity is one useful property of both.

• The way to find out about it is to use elicitation/experiments.

• There is deviation in informants’ ratings.

• A simple explanation: informants use different criteria for evaluation.

• Solutions: let them explicate the criteria.

use multiple methods.

45

Synonymy vs. polysemy

• Evaluating semantic similarity between the meanings of two separate words (synonymy) is a matter of evaluating the match between two separate ”semantic events”

– There should be mismatch, but there isn’t.

• Evaluating semantic similarity/relatedness/ difference between the meanings of one word

(polysemy) is a matter of comparing the applications of one single category.

– There should be match between the semantic events.

46

Synonymy vs. polysemy

• When you evaluate near-synonyms, you balance between (i) the ideal of what would constitute a perfect match and (ii) the nuances of the near-synonyms

• When you evaluate meanings of a polysemous word, you balance between (i) the assumption that some meaning should be shared and (ii) the actual semantic profile of the uses

47

Synonymy vs. polysemy

• In evaluating synonymy, the idealized equivalence can be taken from the semantic description of either of the two words.

• In evaluating polysemy, the common factor

(”core meaning”, ”shared meaning”) may be hard to find, or become too abstract.

Maybe the first task is easier?

48

General relevance

• ”Insights in the equality or similarity of meaning may shed light on meaning itself”

(Oversteegen / SKY 2010, Helsinki)

• The question of “identical meaning” is a crucial basis for e.g. typology and language comparisons: the problem of tertium comparationis

– Cf. Haspelmath’s plenary on Saturday

49

References

Arppe, Antti & Juhani Järvikivi 2007.

Every method counts –

Combining corpus-based and experimental evidence in the study of synonymy. Corpus

Lingustics and Linguistic Theory 3:

2: 131-159.

Colombo, Lucia & Giovanni B. Flores d’Arcais 1984. The meaning of

Dutch prepositions: a psycholinguistic study of polysemy. Linguistics 22: 51-98.

Divjak, Dagmar & Stefan Gries 2008:

Clusters in the mind? Converging evidence from near-synonymy in

Russian. The Mental Lexicon 3: 2:

188-213.

Geeraerts, Dirk – in this symposium

Liu, Dilin – in this symposium

Oversteegen, Eleonore – in this symposium

Raukko, Jarno 2003. Polysemy as flexible meaning: experiments with

English get and Finnish pitää. In

Brigitte Nerlich & al (eds) Polysemy.

Flexible patterns of meaning in

mind and language. 161-193.

CONTINUED...

50

References Author’s cont’d contact information

Sandra, Dominiek & Sally Rice 1995.

Network analyses of prepositional meaning: mirroring whose mind – the linguist’s or the language user’s?

Cognitive Linguistics 6: 89-130.

Vanhatalo, Ulla 2005. Kyselytestit

synonymian selvittämisessä (etc.) [The use of questionnaires in exploring synonymy, etc.] PhD thesis, U-Helsinki. http:/ethesis.helsinki.fi/julkaisut/hum/ suoma/vk/vanhatalo/kyselyte.pdf

Whitten, William B. II, W: Newton Suter, and Michael L. Frank 1979.

Bidirectional Synonym Ratings of 464

Noun Pairs. Journal of Verbal Learning

and Verbal Behavior 18: 109-127.

• e-mail:

See handout and list of participants.

• home postal address

See handout.

• affiliation

Department of Modern

Languages

Metsätalo (Unioninkatu 40 B)

FIN-00014 University of Helsinki

Finland

51

Download