RUNNING HEAD: HEAVY-NP-SHIFT REFERENCE COMPREHENSION Put in last position something previously unmentioned: word order effects on referential expectancy and reference comprehension Jennifer E. Arnold and Shin-Yi C. Lao University of North Carolina at Chapel Hill Address all correspondence to: Jennifer Arnold UNC Chapel Hill Dept. of Psychology Davie Hall #337B CB #3270 Chapel Hill, NC 27599-3270 jarnold@email.unc.edu ABSTRACT Research has shown that the comprehension of definite referring expressions (e.g., the triangle) tends to be faster for given (previously mentioned) referents, compared to new referents. This has been attributed to the presence of given information in the consciousness of discourse participants, (e.g., Chafe, 1994) suggesting that given is always more accessible. We establish a novel finding: a new bias occurs during the on-line comprehension of the direct object in heavyNP-shifted word orders, e.g. Put on the star the…. This order tends to be used for new direct objects; canonical unshifted orders are more common with given direct objects. Thus, word order provides probabilistic information about the givenness or newness of the direct object. Results from eyetracking and gating experiments show that the traditional given bias only occurs with unshifted orders; with heavy-NP-shifted orders comprehenders expect the object to be new, and comprehension for new referents is facilitated. Language comprehension requires the successful identification of the intended referent for all referring expressions. But the linguistic content of such expressions is notoriously underspecified. Listeners must figure out, for example, which paper the speaker means in I read the paper, or which female is intended in I talked to her. It has been suggested that reference comprehension can nevertheless succeed because listeners focus their attention on some entities more than others; these focused entities are then privileged during reference interpretation (e.g., Garnham, 2001; Garrod & Sanford, 1994; Gordon, Grosz, & Gilliom, 1993; MacDonald & MacWhinney, 1990; McDonald & MacWhinney, 1995; Sanford & Garrod, 1981). There is substantial evidence that reference comprehension is facilitated for referents that are discourse-old or given (i.e., already mentioned in the discourse), compared with those that are discourse-new, especially given referents occurring in prominent syntactic and thematic positions (see, e.g., Almor, 2000; Arnold, 2001; Arnold, Eisenband, Brown-Schmidt, & Trueswell, 2000; Chambers & Smyth, 1998; Clark & Sengul, 1979; Dahan et al., 2002; Foraker & McElree, 2006; Gernsbacher et al., 1989; Järvikivi et al., 2004; Stevenson et al., 1993; Stewart, Pickering, & Sanford, 2000). This research leads to the impression that the accessibility of discourse entities occurs as a result of how recently or prominently they were mentioned. Yet most research on reference comprehension has used stimuli with canonical word orders (for exceptions see Kaiser & Trueswell, 2004; Staub, Clifton, & Frazier, 2006). Here we provide the first evidence for reference comprehension biases during noncanonical heavy-NPshifted word orders (Kimball, 1973), e.g. Put on the table the large bag of cat food that I just bought. Our results show that given and prominently mentioned information is not always accessible, and new information can even be more accessible than given. The possible accessibility of new information follows from the prediction that referent accessibility is a function of how expected a referent is (Arnold, 1998, 2001; Arnold et al., in press; Arnold & Tanenhaus, 2007). Variations in word order provide a promising test of this prediction, because speakers tend to produce utterances with given information before new (e.g., Arnold, Wasow, Losongco & Ginstrom, 2000; Bock & Irwin, 1980; Haviland & Clark, 1974; Gundel, 1988; Ward & Birner, 2001; Wasow, 2002). This means that by the end of an utterance, listeners may expect reference to new information or something that was mentioned in a nonprominent syntactic position. This prediction is particularly clear when listeners hear a low-frequency order like the heavy-NP-shift. This order results when the direct object (e.g., the large bag of cat food that I just bought) is not produced in its canonical position, but rather is produced after another postverbal constituent, on the table. This construction occurs very rarely (about 3% of the time with the verb put, Wasow, 2002), and generally when the direct object is either relatively discoursenew or long, compared with the other constituents (Arnold et al., 2000; Hawkins, 1994; Stallings, MacDonald, & O'Seaghdha, 1998; Wasow, 1997; Wasow, 2002). Therefore, once listeners have heard enough of an utterance to recognize that it is heavy-NP-shifted, they may expect the upcoming direct object to be new, facilitating reference to new entities. By contrast, in canonical unshifted constructions, listeners tend to be biased towards given and prominently positioned information. For example, most adults tend to perceive Valentina as the pronoun referent in sentences like (1). (1) Valentina greeted Francisca when she got home. The accessibility of Valentina in this situation has been attributed to the prominence of grammatical subject position, the prominence of first-mention position, and the structural parallelism with the position of the pronoun (e.g., Arnold, Eisenband, Brown-Schmidt, & Trueswell, 2000; Chambers & Smyth, 1998; Gernsbacher & Hargreaves, 1988; Stevenson et al., 1993). Referent accessibility is also identified with focusing constructions like clefts and acoustic prominence (Arnold, 1998; Almor, 2000; Foraker & McElree, 2006). Accessible entities are particularly facilitated when the referring expression is extremely underspecified, for example a pronoun (e.g., Gordon, Grosz, & Gilliom, 1993). But even noun-phrase anaphors can show a bias towards previously-mentioned entities. For example, Almor (1999) reported faster reading times for noun-phrase anaphors with focused antecedents. During spoken language comprehension, unaccented noun phrases are preferentially associated with the most highly focused entity in the discourse (Arnold, 2007; Dahan et al., 2002), while accented noun phrases are either biased towards previously-mentioned but unfocused entities (Dahan et al., 2001), or equally biased towards focused and unfocused entities (Arnold, 2007). The picture that emerges from this literature is that the listener's focus of attention is drawn towards things that have been previously mentioned, especially those mentioned in a prominent position. One explanation is that given information is accessible because it is in the listener's consciousness (Chafe, 1994), suggesting that given information is always more accessible. However, recent evidence demonstrates that previously unmentioned objects can become accessible when reference to a new object is relatively expected. Arnold, Tananhaus, Altmann & Fagnano (2004) examined the comprehension of disfluent speech (e.g., Now put thee uh…), which occurs more frequently with references to new information than given. Thus, disfluency provides information that the speaker is relatively likely to be referring to something new. Arnold et al. monitored listeners eye movements during comprehension of references to given and new information. With fluent instructions there were more looks to given objects that matched the initial segment of the target noun, whereas disfluent instructions resulted in more early looks to a new object. In a related finding, Arnold, Hudson-Kam & Tanenhaus (2007) found that disfluency created a bias to interpret disfluent expressions as co-referential with novel objects (the funny squiggly shape…), rather than familiar objects (the ice cream cone). Does referential expectancy also vary as a function of word order? We tested this question by examining listeners’ eye movements while they viewed displays like in Figure 1. Listeners heard pairs of instructions, e.g., Put the red triangle with a squiggle on the heart. Now put on the moon the red triangle with… The second instruction referred to something that was either given (previously mentioned) or new (unmentioned), in an unshifted or heavy-NP-shifted word order. All target objects (e.g., the red triangle with a squiggle) had a color-matched competitor object (e.g., the red triangle with a tree). This created a temporary ambiguity at the onset of the referring expression, where the input (the red triangle with) was consistent with both a given and a new object. By looking at eye movements during this ambiguous region, we can assess their biases towards given and new referents in different conditions (for similar methodology see Allopenna et al., 1998; Arnold et al., 2004; Arnold et al., 2007; Dahan et al., 2002). All instructions used the verb put, which obligatorily takes a direct object NP. This means that when comprehenders encountered the post-verbal prepositional phrase (e.g., Put on the heart), they knew unambiguously that they were hearing the heavy-NP-shifted word order. By contrast, the unshifted word order is so frequent that listeners were likely to initially assume this order in the absence of evidence to the contrary (Staub et al., 2006). [INSERT FIGURE 1 ABOUT HERE] Because the heavy-NP-shifted word order is rare, we wanted participants to believe that instructions had been produced naturally, which may be critical for making the inference that the motivation for the shifted order was the newness of the direct object. We therefore followed the technique developed by Arnold et al. (2004), telling participants that the instructions had been recorded by naïve speakers giving instructions to live addressees. In actuality the instructions were recorded by the first author. The stimuli from the on-line experiment were first used in an off-line gating task to establish that the expectancy of the direct object differed for shifted and unshifted word orders. 16 participants from UNC Chapel Hill heard digitally truncated versions of the instructions. The first sentence was always heard in its entirety. In critical items the instruction was cut off before the target NP was fully disambiguated (see Table 1). Participants identified what they thought the speaker was about to say next, by circling a picture of the object on a piece of paper. Fillers were truncated late enough to allow correct identification of the target (e.g., Now put on the sun the grey triangle with a d-), to encourage participants to attend to the input. 24 experimental items were combined with 36 fillers. [INSERT TABLE 1 ABOUT HERE] For both the short and long fragments, participants were more likely to choose the given object in the unshifted than shifted condition (unshifted/short: 30%, shifted/short: 17%; unshifted/long: 46%; shifted/long: 35%). That is, the given object was more expected in the unshifted order. Analyses of variance revealed the crucial main effect of word order (F1 = 5.28, p < .05, 2p = .26; F2 = 4.410, p < .047, 2p = .16), as well as main effect of fragment length (F1 = 14.392, p <.005; F2 = 14.571, p < .005). There were no interactions. In sum, the heavy-NP-shifted word order decreases the expectancy of something given as the referent of the direct object noun phrase, and increases the expectancy of something new. If listeners can use this expectancy during on-line reference comprehension, their eye movements should reveal a preference to look at the new referent during the ambiguous portion of the noun phrase. ON-LINE EXPERIMENT Method Participants. 48 students at the University of North Carolina at Chapel Hill participated in exchange for course credit. 4 were excluded from analysis: 1 for track loss, 1 because the contingent analysis had too many missing cells, and 2 who reported attentional deficits on a voluntary questionnaire. This left 44 participants in the analysis. All participants were native speakers of English with normal or corrected-to-normal vision. Materials and Procedure. Participants viewed a display like Figure (1). Their eye movements were monitored with an Eyelink II eyetracker, while they followed auditory instructions to move one or more of the four objects in the middle of the screen. These four objects were novel figures designed to have medium-length descriptive names, like The red triangle with a squiggle. Each object was composed of a simple shape (triangle, circle, square, cross, diamond, ring) and a superimposed modifier shape (e.g., squiggle, tree). The basic shape was the same for all four objects in each display; two modifier shapes were crossed with two colors to result in four different objects. There were four un-moveable destination objects in the corners that were present on all trials (arrow, heart, sun, moon). Each trial had two instructions (see Table 2). The first established one object as given. The second, critical, instruction used either the heavy-NP-shifted or unshifted word order, and asked the participant to move either the given object, or the new competitor (the other object with the same color). The expression of interest was the direct-object noun phrase in the second instruction. One recording of the context instruction was used for the two given conditions, and one was used for the two new conditions. The second instruction was cross-spliced so that both unshifted conditions used the same frame Now put the {color} {object} with a…., on the {location}, and both heavy-NP-shifted conditions used the same frame Now put on the {location} the {color} {object} with a….. That is, the given and new instructions were identical except for the modifier word. Furthermore, the exact same fragment for the target instruction was used across all four conditions (e.g., the red triangle with a). The modifier word was pronounced with an accent in the new condition, and without an accent in the given condition. This preserved the natural correlation between givenness and accenting, which has been shown to affect eye movements in this paradigm (see Arnold, 2007; Dahan et al., 2002). However, by placing the emphasis on only the final modifier word (as is natural for complex noun phrases), all four conditions had an identical ambiguous portion of the noun phrase. There were 24 experimental items, rotated through the four conditions and assigned to 4 lists in a Latin square design. There were 36 fillers, which were like the experimental stimuli in having two instructions, the second of which was heavy-NP-shifted in half the items. The 24 matched fillers pictured the same kinds of objects as the experimental stimuli, but the second instruction always referred to one of the color-contrasting objects (half the time the one with the same modifier, and half the time the one with a different modifier). This removed any expectation that the second instruction would refer to one of the objects matching the color of the object moved in the first instruction (as was always the case in the experimental items). There were an additional 12 filler items that presented four different objects with different colors and modifier types (e.g., green diamond with a bucket, purple square with a bucket, yellow diamond with a bucket, yellow circle with a bunch of lines); on half of these the second instruction referred to the given object from the first instruction, and on half it referred to a new, not-colormatched object. There were three practice items prior to the experiment (one heavy-NPshifted/new; one unshifted/new, and one unshifted/given). Eye movements were monitored with an Eyelink II eyetracker. The visual and auditory stimuli were presented on a PC computer running the ExBuilder software (Longhurst, 2006), which randomized the order of presentation of trials. The location of display objects placed the color-matched objects horizontal from each other, and the two objects with the same modifier shape vertical from each other. The target occurred equally in all four positions on each list, and at equal rates for each condition across all four positions across all lists. The first instruction always moved an object to the adjacent destination, keeping the object in the same corner. The second instruction moved either the same object or its color competitor to one of the locations on the other half of the screen. [INSERT TABLE 2 ABOUT HERE] Results and Discussion As predicted, the heavy-NP-shifted instructions led participants to look at the new colormatched object more often than the unshifted instructions did, starting at the onset of the critical noun. Eye movements were analyzed in terms of looks to objects, which begin at the onset of a saccade and continue until the next saccade. Looks were categorized in terms of each object on the screen; i.e., a saccade to another part of the same object counted as part of the same look. All reported statistical analyses consider eye movements occurring after the onset of the color word, and prior to the mention of the modifier word (e.g., red triangle with a). This eliminates any effects of the accenting on the modifier word, which is confounded with the given/new manipulation. Since eye movements in response to linguistic input typically appear about 200300 msec after the onset of the critical word (e.g., Arnold et al., 2004; Dahan et al., 2002), the region was defined as 300 msec after the color word onset until 300 msec after the modifier onset. Figure 2 reveals how participants directed their looks over time to display objects. These graphs collapse across target referent (given vs. new), since we are primarily interested in participants' biases during the initial, ambiguous portion of the referring expression, at which point the target identity is unknown. Trials are only included if the participant was not already looking at either of the color-matched objects at the onset of the critical region (t = color word + 300 msec; 66% of total). In the unshifted condition, fixations to both given and new color-matched objects rose initially, followed by a large advantage for the given object. By contrast, in the heavy-NP-shifted condition there was a steady advantage for the new color-matched object throughout the ambiguous region. Importantly, the contrast between word order conditions occurs within the ambiguous region of the referring expression, demonstrating that listeners integrate word order with givenness as they try to anticipate which red triangle is being referred to. The effect of word order on preference for given or new referents is reflected in an analysis of looks to the competitor object during the ambiguous region, which reveals more given competitor looks in the unshifted condition, and more new competitor looks in the shifted condition (Table 3). ANOVAs revealed the critical interaction between shifting and target referent (F1(1,43) = 5.75, p < .05, 2p = .16; F2 (1,23) = 8.30, p < .01, , 2p = .44), as well as a main effect of word order (F1(1,43 = 7.26, p < .05; F2(1,21) = 4.87, p <.05). Two items were excluded from the items analysis because of missing cells. The same pattern obtained in an analysis of the full dataset (including trials where the color-matched objects were fixated at baseline (interaction: F1(1,43) = 5.54, p < .05, 2p = .11; F2 (1,23) = 8.21, p < .005, 2p = .26; main effect of word order: (F1(1,43 = 19.66, p < .001; F2(1,23) = 24.82, p <.001). In all analyses, trials with more than 33% trackloss during the critical region were excluded from analysis (less than 3% of all trials). Thus, looks to the given object dropped in the shifted condition, compared with the unshifted condition. This parallels the findings from the gating task, which revealed that word order influenced the expectancy of given and given objects. [INSERT FIGURE 2 ABOUT HERE] GENERAL DISCUSSION The results show that word order modulates the accessibility of given and new information as listeners generate on-line hypotheses about what an expression refers to. This constitutes the first demonstration that heavy-NP-shifting influences on-line reference comprehension. Related evidence for how word order influences accessibility comes from Finnish, in which the less frequent object-verb-subject word order led to the expectation that the postverbal subject noun phrase would refer to something not previously mentioned (Kaiser & Trueswell, 2004). The effect of heavy-NP-shifting on accessibility supports the proposal that listeners interpret referential expressions in real time by preferentially considering referents that are relatively expected in the current context (e.g., Arnold & Tanenhaus, 2007). These findings join other evidence for the facilitation of expected referents, for example through disfluency (Arnold et al., 2004), the affordances of available objects for particular actions (Chambers et al., 2002), or the semantic constraints of verbs on possible direct objects (Altmann & Kamide, 1999; Kamide et al., 2003). Our findings specifically support the prediction that reference comprehension can be facilitated for both given and new information. The bias toward new information with the heavyNP-shifted word order falls in contrast with the wealth of existing studies showing that given and prominently mentioned information is more accessible than new. This shows that comprehenders are sensitive to fine-grained contingencies about likely patterns of reference: given is more likely early in an utterance, but new is more likely late in an utterance, especially with the specialized shifted construction. Thus, referential accessibility is not just the result of how recently or how prominently an entity has been mentioned, but instead is modulated by its likelihood of mention in a particular position. ACKNOWLEDGEMENTS Many thanks to Jordan Todd and Glenn Kern for their help with stimulus preparation, and to Bob McMurray for the use of his data analysis program. This research was funded by NIH grant HD41522 to J. Arnold. REFERENCES Allopenna, P. D., Magnuson, J. S. & Tanenhaus, M. K. (1998). Tracking the time course of spoken word recognition using eye movements: Evidence for continuous mapping models. Journal of Memory and Language, 38, 419-439. Altmann, G. T. M. & Kamide, Y. (1999). Incremental interpretation at verbs: Restricting the domain of subsequent reference. Cognition, 73(3), 247-264. Almor, A. (2000). Constraints and mechanisms in theories anaphor processing. In M. W. Crocker, M. Pickering, & C. Clifton, Jr. (Eds.), Architectures and mechanisms for language processing (pp. 341-354). New York, NY: Cambridge University Press. Arnold, J. E. (1998). Reference form and discourse patterns (Doctoral dissertation, Stanford University, 1998). Dissertation Abstracts International, 59, 2950. Arnold, J. E. (2001). The effect of thematic roles on pronoun use and frequency of reference continuation. Discourse Processes, 31, 137–162. Arnold, J. E., & Tanenhaus, M. K. (2007). Disfluency effects in comprehension: How new information can become accessible. In E. Gibson, & N. Perlmutter, (Eds), The processing and acquisition of reference, Cambridge, MA: MIT Press. Arnold, J. E., Brown-Schmidt, S., & Trueswell, J. C. (in press). Children’s use of gender and order-of-mention during pronoun comprehension. Language and Cognitive Processes. Arnold, J. E., Hudson-Kam, C., & Tanenhaus, M. K. (2007). If you say ‘‘thee uh-’’ you’re describing something hard: The on-line attribution of disfluency during reference comprehension. Unpublished manuscript, University of North Carolina at Chapel Hill. Arnold, J. E., Tanenhaus, M. K , Altmann, R., & Fagnano, M. (2004). The old and thee, uh, new. Psychological Science, 15(9). 578-582. Arnold, J. E., Wasow, T., Losongco, A., & Ginstrom, R. (2000). Heaviness vs. newness: The effects of structural complexity and discourse status on constituent ordering. Language, 76, 28–55. Arnold, J.E. (2007). THE BACON not the bacon: How children and adults understand accented and deaccented noun phrases. Unpublished manuscript, University of North Carolina at Chapel Hill. Birner, B. J., & Ward, G. (1994). Uniqueness, familiarity, and the definite article in English. In Proceedings of the Twentieth Annual Meeting of the Berkeley Linguistics Society. Berkeley: Berkeley Linguistics Society, p. 93-102. Bock, K., Irwin, D. E., & Davidson, D. J. (2004). Putting First things First. In Henderson, J. M., & Ferreira, F. (Eds.), The interface of language, vision, and action: Eye movements and the visual world. New York: Psychology Press, p. 249-278. Bower, G. H. & Morrow, D. G. (1990). Mental models in narrative comprehension. Science, 247(4938), 44-48. Chafe, W. L. (1994). Discourse, consciousness, and time. Chicago: Chicago University Press. Chambers, C. G; Tanenhaus, M. K., & Eberhard, K. M. (2002). Circumscribing referential domains during real-time language comprehension, Journal of Memory and Language, 47(1), 30-49. Chambers, C., & Smyth, R. (1998). Structural parallelism and discourse coherence. Journal of Memory and Language, 39, 593–608. Clark, H. H., & Sengul, C. J. (1979). In search of referents for nouns and pronouns. Memory & Cognition, 7, 35-41. Dahan, D., Tanenhaus, M. K. & Chambers, C. G. (2002). Accent and reference resolution in spoken language comprehension. Journal of Memory and Language, 47, 292-314. Foraker, S. & McElree, B. (2006). The role of prominence in pronoun resolution: Active versus passive representations. Journal of Memory and Language, 56, 357-383. Gernsbacher, M.A., & Hargreaves, D.J. (1988). Accessing sentence participants: The advantage of first mention. Journal of Memory and Language, 27, 699–717. Gernsbacher, M.A., Hargreaves, D.J., & Beeman, M. (1989). Building and accessing clausal representations: The advantage of first mention versus the advantage of clause recency. Journal of Memory and Language, 28, 735–755. Gordon, P. C., Grosz, B. J. & Gilliom, L. A. (1993). Pronouns, Names, and the Centering of Attention in Discourse. Cognitive Science, 17, 311-347. Gundel, J. K. (1988). Universals of topic-comment structure. In Hammond, M., Moravcsik, & Wirth, J. R. (Eds.), Studies in syntactic typology. Amsterdam: Benjamins, p. 209 – 239. Gundel, J. K., Hedberg, N., & Zacharski, R. (1993). Cognitive status and the form of referring expressions. Language, 69, 274–307. Haviland, S. E. & Clark, H. H. (1974). What's new? Acquiring new information as a process in comprehension. Journal of Verbal Learning and Verbal Behavior, 13, 512-521. Hawkins, J. A. (1994). A performance theory of order and constituency. Cambridge: Cambridge University Press. Kaiser, E., & Trueswell, J. C. (2004). The role of discourse context in the processing of a flexible word-order language. Cognition, 94, 113–147 Kehler, A. (2002). Coherence, Reference, and the Theory of Grammar. Stanford: CSLI Publications. Ross, J. R. (1967). Constraints on variables in syntax. Ph.D. dissertation, MIT. Sanford, A. J., & Garrod, S. C. (1981). Understanding written language. Chichester: Wiley. Stallings, L. M., MacDonald, M. C. & O'Seaghdha, P. G. (1998). Phrasal Ordering Constraints in Sentence Production: Phrase Length and Verb Disposition in Heavy-NP Shift. Journal of Memory and Language 39, 392-417. Staub, A., Clifton, C. Jr., Frazier, L. (2006). Heavy NP shift is the parser's last resort: Evidence from eye movements. Journal of Memory and Language, 54, 389-406. Stevenson, R. J., Crawley, R. A., & Kleinman, D. (1994). Thematic roles, focus and the representation of events. Language and Cognitive Processes, 9, 473-592. Stewart, A. J.; Pickering, M. J.; Sanford, A. J. (2000). The time course of the influence of implicit causality information: Focusing versus integration accounts. Journal of Memory and Language, 42, 423-443. Ward, G. & Birner, B. J. (2001). Discourse and Information Structure. In Schiffrin, D., Tannen, D., & Hamilton, H., Handbook of Discourse Analysis. Oxford: Basil Blackwell. p. 119137 Wasow, T. (1997). End-weight from the speaker’s perspective. Journal of Psycholinguistic Research, 20, 347–361. Wasow, T. (2002). Postverbal Behavior. Stanford, CA: CSLI Publications. Wasow, T., & Arnold, J. E. (2003). Postverbal constituent ordering in English. In G. Rohdenburg & B. Mondorf (Eds.), Determinants of grammatical variation in English (pp.120-154). Berlin: Mouton de Gruyter. [40 REFERENCES] TABLES Table 1. Sample auditory instructions for the gating task. Condition Unshifted S1: Context Put the red triangle with a squiggle on the heart. S2: Short fragment Now put … S2: Long fragment Now put the red triangle with…. Heavy-NP-shifted S1: Context Put the red triangle with a squiggle on the heart. S2: Short fragment Now put on the moon… S2: Long fragment Now put on the moon the red triangle with…. Table 2. Sample auditory instructions for the on-line experiment. Condition Unshifted/given Put the red triangle with a squiggle on the heart. Now put the red triangle with a squiggle on the sun. Unshifted/new Put the red triangle with a tree on the heart. Now put the red triangle with a squiggle on the sun. Heavy-NP-shifted/given Put the red triangle with a squiggle on the heart. Now put on the sun the red triangle with a squiggle. Heavy-NP-shifted/new Put the red triangle with a tree on the heart. Now put on the sun the red triangle with a squiggle. Table 3. Percentage of time spent looking at the competitor (color-matched object), starting 300 msec after the onset of the color word until 300 msec after the onset of the modifier. Data include only trials with no baseline looks to given or new critical objects. Condition % SE Unshifted/given competitor 28.8 3.1 Unshifted/new competitor 22.7 2.4 Heavy-NP-shifted/given competitor 16.6 2.1 Heavy-NP-shifted/new competitor 22.7 2.4 FIGURES FIGURE CAPTIONS Figure 1. Sample visual display for on-line and gating experiments. The top two triangles in this example are red; the bottom two are blue. All corner shapes are yellow. Figure 2a (Unshifted condition) and Figure 2b (Heavy-NP-shifted condition). Starting 300 msec after the onset of the color word, proportion looks over time to the given and new critical objects (i.e., those that match the color of the auditory referring expression), and to unrelated (non-colormatched) objects. Data include only trials with no baseline looks to given or new critical objects. Q uickTim e™ and a G r aphics decom pr essor ar e needed t o see t his pict ur e. Q uickTim e™ and a G r aphics decom pr essor ar e needed t o see t his pict ur e. Q uickTim e™ and a G r aphics decom pr essor ar e needed t o see t his pict ur e. Q uickTim e™ and a G r aphics decom pr essor ar e needed t o see t his pict ur e. Q uickTim e™ and a G r aphics decom pr essor ar e needed t o see t his pict ur e. Q uickTim e™ and a G r aphics decom pr essor ar e needed t o see t his pict ur e. Q uickTim e™ and a G r aphics decom pr essor ar e needed t o see t his pict ur e. Q uickTim e™ and a G r aphics decom pr essor ar e needed t o see t his pict ur e. Figure 1. Sample visual display for Experiments 1 and 2. The top two triangles in this example are red; the bottom two are blue. All corner shapes are yellow. given given Heavy-NP-Shifted new new unrelated 50% 40% 40% 30% 30% time from onset color word 1164 972 876 780 684 588 1164 1068 972 876 780 684 0% 588 0% 492 10% 396 10% 492 20% 396 20% 300 % looks 50% 300 % looks unrelated 1068 Unshifted time from onset color word Figure 2a (Unshifted condition) and Figure 2b (Heavy-NP-shifted condition). Starting 300 msec after the onset of the color word, proportion looks over time to the given and new critical objects (i.e., those that match the color of the auditory referring expression), and to unrelated (non-colormatched) objects. Data include only trials with no baseline looks to given or new critical objects.