Comparing the Efficacy of Different Signaling Techniques Robert K. Atkinson Division of Psychology in Education Arizona State University robert.atkinson@asu.edu Lijia Lin & Caroline Harrison Division of Psychology in Education Arizona State University ABSTRACT: This experiment examined which type, or a combination, of attention-gaining and attention-directing techniques foster learning in a multimedia learning environment. One hundred and sixty-nine participants were randomly assigned to the cells of a 3 x 3 factorial design where the first factor was attention-gaining techniques (none, highlighting, or flashing) and the second factor was attention-directing techniques [none, realistic/concrete pointer (i.e., human hand with pointing finger), or symbolic/abstract pointer (i.e., arrow)]. The results revealed that participants assigned to the static attention-gaining condition (highlighting) outperformed their peers in the other two conditions (no attention-gaining technique and flashing) on learning outcome measures. Participants in the hand and arrow attention-direction conditions produced significantly higher scores on the posttest than their peers that were not provided these attention-directing techniques. Introduction Information is presented through different modalities (auditory and visual modality) to help learners construct their mental representations in a complex multimedia environment. Due to the limited capacity of working memory (Miller, 1956), learners cannot attend to all of the incoming information in such a complex environment. Instead, they will visually search what they consider relevant and important information to match the information coming from the auditory channel. Such cognitively high-visual-search activity has been found to result in poor performance (Jeung, Chandler & Sweller, 1997). This may be explained by cognitive load theory (Paas, Renkl, & Sweller, 2003; Sweller, van Merrienboer, & Paas, 1998), which assumes three subcomponents—intrinsic load, extraneous load and germane load. (The intrinsic load cannot be altered for a designated material, whereas the extraneous load is unnecessary and can be altered by appropriate instructional design. Once the extraneous load is lowered, learners can engage more for understanding, thus fostering germane load.) High-visual-search activities in the multimedia environment impose high extraneous load and low germane load, which do not foster understanding. From the perspective of cognitive load theory (Paas, Renkl, & Sweller, 2003; Sweller, van Merrienboer, & Paas, 1998), signaling is one effective way to reduce extraneous cognitive load (Mayer & Moreno, 2003; Wouters, Paas, & van Merriënboer, 2008) and thus foster germane cognitive load. Based on the three essential processes specified in the cognitive theory of multimedia learning (Mayer, 2005), the signaling principle states that emphasizing relevant information by means of visual indicators (e.g., typographical signals such as bolding, flashing, pointers, color coding of related information) gains and directs attention, helps learners organize information by making explicit relations that might otherwise have to be inferred, and helps them integrate related information into one coherent mental representation (Mautone & Mayer, 2001). Therefore, in order to enhance deep and meaningful learning, the issue of how to gain and direct learner’s attention to the important information becomes crucial. Signaling to Gain Attention The signaling technique, by adding non-content information to the learning environment, is intended to enhance learner’s attention which leads to essential cognitive processing. There are quite a few signaling techniques to gain learners’ attention, such as coloring, highlighting and flashing. Although in Harp and Mayer’s paper-based instruction, no effect of signaling (highlighting) was found (Harp and Mayer, 1998), research studies in the computer-based environment have shown positive results of signaling compared to non-signaling in the past decades. Coloring technique was found to promote learning and reduce cognitive load in some experiments (Jamet, Gavota & Quaireau, 2008; Kalyuga, Chandler, & Sweller, 1999). The cognitive function of electronic flashing was investigated in the experiment conducted by Jeung, Chandler, & Sweller (Experiment 2, Jeung, Chandler, & Sweller, 1997). Participants studying a complex geometric diagram accompanied with narration and visual indicators (flashing) outperformed their counterparts who were presented with materials with no visual indicators (either a diagram with on-screen statements or a diagram with narrated statements). In another experiment, cued animation (highlighting relevant parts by darkening all other parts on graphics) was found to enhance learners’ comprehension on cardiovascular system (de Koning, Tabbers, Rikers, & Paas, 2007). A weak signaling effect was also found in a non-scientific context (using materials to teach instructional design strategies) (Tabbers, Martens, & van Merriënboer, 2004). Therefore, based on the previous literatures, we expect that object-related attention-gaining techniques will effectively promote learning in a scientific multimedia context (e.g., the human cardiovascular system). Regarding the effectiveness of different attention-gaining techniques (e.g., highlighting vs. flashing), the results were uncertain. From the cognitive perspective, we expect that the dynamic manipulation (flashing) may be more effective to enhance attention than the static highlighting due to its dynamism. Signaling to Direct Attention Signaling has the cognitive function to direct learner attention by pointing to the relevant information in the learning environment. Most research on external attention pointers involves the investigation of gestures of animated pedagogical agent in multimedia learning environment. Animated pedagogical agents are lifelike characters that complement multimedia instruction by providing instruction through verbal and nonverbal modes of communication. According to social agency theory (Mayer, 2003; Atkinson, Mayer & Merrill, 2005), the social cues (voice, locomotion, gaze and gestures) provided by the animated agent are expected to enhance human-computer interaction—learners in such learning environments will perceive to have conversations with a human by interacting with the animated agent. Research studies revealed that the animated pedagogical agent not only has a social impact on learning and perception in multimedia environment, but also has cognitive functions provided by the agents’ specific features, such as gaze, gestures, locomotion. The experiments conducted by Atkinson (Atkinson, 2002) shed some light on this issue. The animated agent, which is a non-anthropomorphic agent (Peedy the Parrot), delivered instructional explanations and solutions steps of worked examples while using nonverbal cues such as gestures and gaze to direct the learner’s attention. Results showed that there was an image effect— visually presenting an animated agent gesturing and glancing to the material with narration is more effective to promote learning than presenting voice only. Atkinson thus pointed out that the nonverbal cues (gaze and gestures) of the animated agent may have the same function as a visual indicator to direct learners’ attention, which was also found in previous research (Jeung et al, 1997). Nevertheless, literature reveals that there is a mixed effect of presenting the agent with gesture—positive effect was found in experiments conducted by Lusk and Atkinson (Lusk, & Atkinson, 2007) and Baylor and Kim (Baylor & Kim, 2009), whereas negative effect was found in some other experiments (Choi & Clark, 2006 and Craig, Gholson, & Driscoll, 2002). Therefore, systematically investigating specific features of the animated agents (e.g. human hand pointing) is needed, as some potential positive effects of agent features may well be suppressed by other irrelevant and in fact harmful features, adding up to a zero effect, if the agent is presented to its maximum amount. Lusk and Atkinson (2007) investigated the effect of the degree of agent embodiment in the context of worked examples —presenting a full agent to direct learner attention by narration, gesture and gaze, a minimum agent only providing verbal explanations and no agent. The results tended to favor the fully embodied agent, which employed attention directing techniques. However, as the whole body of the agent was presented in both fully embodied and minimally embodied conditions, it is unconvincing to contribute the result to the effect of agent pointing. Therefore, in the present experiment, we will investigate the effect of only presenting a human hand pointer, no agent’s head and body, compared to the non-pointing condition. In Mautone and Mayer’s study (Mautone, & Mayer, 2001), signaled animation using arrows failed to affect learning compared to non-signaled condition. Such failure of the abstract signaling device (arrow) remind us to not only look into the issues of complexity of graphics and prior knowledge level (Mautone, & Mayer, 2001), but also consider the issue from the social-motivational perspective—such abstract attention pointer may not engage learner into the learning activities. A realistic attention pointer (e.g., a human hand pointer) may engage the learner more. Nevertheless from the cognitive perspective, it may be argued that it is the attention-directing function that affects learning, rather than the realism. Therefore, a deictic human hand may have the same cognitive function as an abstract arrow. Research has suggested that instruction should be designed parsimoniously (Choi, & Clark, 2006) and should exclude extraneous elements (Harp & Mayer, 1998; Mayer, & Moreno, 2002). In the present experiment, we will investigate which type of attention-directing technique (concrete pointing finger vs. abstract arrow) is more effective to promote learning, and whether such attention-directing techniques are effective in multimedia environment. Overview of the experiment The purpose of the experiment is to examine which type, or a combination, of attention-gaining and attention-directing techniques foster learning in a multimedia learning environment. Independent variables Attention-gaining techniques and attention-directing techniques are the two independent variables manipulated in this experiment. It was designed to investigate the signaling principle by testing the cognitive effectiveness of using (A) target-related, attention-gaining techniques, that is, (A1) no attention attention-gaining techniques (A2) versus highlighting (static attention getter) versus (A3) flashing (dynamic attention getter). In addition, this experiment will also explore the use of (B) attention-directing techniques to target, that is, (B1) no pointer versus (B2) realistic/concrete pointer (i.e., human hand with pointing finger) versus (B3) symbolic/abstract pointer (i.e., arrow). Research Questions In this experiment, three research questions will be addressed: 1) What type of attention-gaining techniques (flashing or highlighting) is most effective for facilitating learning in a multimedia environment? 2) What type of attention-directing techniques (concrete image/pointing finger or abstract image/arrow) is most effective for facilitating learning in a multimedia environment? 3) Is there an interaction between attention-gaining and attention-directing techniques? From a cognitive perspective, both attention-gaining and attention-directing techniques are expected to improve learning outcomes and reduce subjective cognitive load. The experiment will assess the comparative effectiveness of these two categories, used separately and in combination. Secondly, this experiment will assess the comparative effectiveness of static versus dynamic object attention getters (highlighting vs. flashing), the expectation being that a dynamic manipulation will elicit higher attention and result in better cognitive processing. Thirdly, this experiment will compare the effectiveness of realistic versus abstract attention pointers, the expectation being that cognitively, there will be no difference. From a socio-motivational perspective, the use of a human body part (hand) with a deictic function (finger pointing to target) is expected to enhance viewer engagement due to realism (as compared to a symbolic deictic, i.e., arrow). Method Participants and Design 169 participants (91 female and 78 male) from a large southwestern university volunteered to participate in the experiment. They were primarily undergraduates from education, music, and social sciences. They were drawn from educational psychology and introductory computer courses in the College of Education as well as through recruiting from around the campus. Participants were either paid a small stipend ($20US) or were given class extra credit. Participants were randomly assigned to one of nine treatment groups as formed by the cells of a 3 x 3 factorial design: 1) a static circle to gain attention, 2) a flashing circle to gain attention, 3) an arrow to direct attention, 4) a hand to direct attention, 5) a static circle with an arrow, 6) a static circle with a hand, 7) a flashing circle with an arrow, 8) a flashing circle with a hand, or 9) a control group with no additional attention gaining or directing cues. Materials The instructional material consisted of a multimedia lesson on the cardiovascular system created in Visual Basic. It included embedded 2-D images and animations presented as Adobe Flash files. The lesson portion of the environment included 24 separate screens of instruction. Each of these screens included content presented through narration accompanied by a Flash file containing images or animations to illustrate the audio content. All of the animations were presented on the left side of the screen with the navigational and animation controls presented on the right side of the screen. Learners were also presented with their location within the environment with the section number and the screen number at the top. The program and content were adapted from a previous project created by Atkinson et al (?). It covered objectives related to the circulatory system such as the structure and function of the heart, blood, circulatory system, and material exchange. Before starting the lesson content, learners were provided with a tutorial screen about the environment. This screen gave a brief explanation of the program and demonstrated the animation and navigational controls without any accompanying narration or animation. The instructional information within the lesson was divided into five sections with a varied number of screens. Navigational aids allowed learners to control the pace of the instruction. They were able to take as much time as they needed with each screen. On each content screen learners could stop and start the narration and animation as often as needed. Animation and narration controls were located to the right of the animation. The same button controlled both animation and narration in order to keep them synchronized. Within each instructional section, learners could go back to previous pages or go forward to the next page. There was no limit as to how often they could view a screen within a section. However, once a section had been completed learners could not go back to review its content. Navigation controls were located at the bottom right of each screen below the animation and sound controls. Immediately following each section of instruction, learners were given four multiple choice practice questions on the content just presented. Each question had five alternate choices. Each question was presented individually on the left portion of the screen, replacing the animations. The navigation and animation controls were not available during the practice. The primary purpose of these practice questions was to engage the learners and encourage their attention to the material rather than to test their learning. For this reason a researcher told the learners about the practice questions prior to the start of the lesson. Once learners reached the practice they could not go back and review any of the prior instructional material. They could not proceed to the next question or section of material until they had answered the question being presented. When the learner was ready, they would click on the Next button to record the answer. Students were then given immediate feedback on each practice item. This was presented with a pop-up message box to allow learners to review and process the feedback. If they chose the correct answer they were presented with the message “Yes! That’s Correct”. If the answer that they chose was incorrect they were given the correct answer as part of the feedback. Learners could review the question and feedback for as long as necessary. When the learner was ready they would click on the OK button to continue to the subsequent practice item, to begin the next section, or to complete the program. All conditions included identical narration and practice within the environment. In addition, the content and primary graphics of the narrations were also the same across all conditions. However, eight of the conditions included either an attention directing device of a hand (Figure 1) or an arrow (Figure 2), an attention gaining device of a static or flashing circle (Figure 3), or combinations of both (Figure 4) overlaid on the original visuals. Figure 1 Hand directing attention Figure 2 Arrow directing attention Figure 3 Circle gaining attention Figure 4 Combination of devices Dependent Measures Before instruction, learners were presented with a pretest to measure their prior knowledge of the cardiovascular system content. This pretest consisted of 20 multiple choice questions and the questions were presented one at a time. Participants were asked to read each statement and select the best answer from four possible choices. They could not return to their previous questions. They were not given any feedback on their answer. Examples of pretest questions include: 1. The purpose of the heart is to a. remove wastes from the blood. b. make new blood. c. pump blood through the body. d. transfer heat to the rest of the body. 2. Which of the following is most similar to the heart? a. A pump because it pushes blood through the body. b. A hose because blood travels in tubes. c. A cup because it is open on the top. d. A broom because it cleans the blood. Learning outcomes were measured by the use of a posttest created for the material presented in the lesson. The posttest consisted of 20 multiple choice items based on the content. As with the pretest, participants were given a statement and asked to select the best choice from four possible answers. Again, questions were presented one at time, there was no backtracking, and no feedback was provided. Following are sample questions from the posttest. 1. The valves in the heart are like a. gates. b. windows. c. closets. d. chambers. 2. The right ventricle pumps blood to the a. left atrium. b. right atrium. c. left ventricle. d. lungs. Text log files were created by the program while participants worked through the learning environment. The log files were automatically created and updated as participants proceeded through the program. They recorded the date and start time, participant experiment ID number, total time spent in the program, the answer selected and whether it was correct for each testing or practice item, the total correct responses for the entire lesson, and responses to the attitude questionnaires. In addition to measuring prior knowledge and subsequent learning, participants completed several questionnaires to measure their subjective cognitive load. These instruments were adapted from similar instruments used by Linek, Gerjets, and Sheiter (2007) in their study using human voices. All of the attitude measures were standardized to use an 8 point numbering scale, otherwise they were unchanged. Cognitive load was measured by subjective rating technique. Participants were asked to self-report their perceived cognitive load through five questions using an 8-point Likert scale. These five questions are 1) How much mental and physical activity was required (e.g., thinking, deciding, calculating, remembering, looking, searching etc.)? That is, was the learning task easy (simple, forgiving) or demanding (exacting or unforgiving)? 2) How hard did you have to work in your attempt to understand the contents of the learning environment? 3) How successful do you think you were in your attempt to understand the content of the learning environment? 4) How much effort did you have to invest in order to navigate the learning environment (e.g., for deciding between different hyperlinks, finding your way around)? and 5) How stressed (insecure, discouraged, irritated, annoyed) did you feel during the learning task? Procedure The experiment was conducted in a computer lab. Participants were asked to sign a consent form stating they agreed to participate and specify whether they were either paid or receiving extra credit for participating in the experiment. After that, each participant was seated at an individual cubicle, facing a computer. Participants were debriefed by a researcher about the goal and procedures of the experiment. Then, they started the pretest on the computer. Once the pretest was completed, each participant was given a randomly assigned experiment ID number to start the computer-based lesson. The intention of using the experiment ID number is to preserve the anonymity of each participant. A posttest and a questionnaire were administered, once participants completed the computer-based lesson. Upon completion of the posttest and the questionnaire, the participants were thanked and were either paid or received course extra credit. The log file from the sessions was then preserved for analysis. Scoring Participants’ answers from the pretest, posttest, and practice items were automatically scored by the environment and recorded as part of the text log file created with each session. The given answer was recorded then compared to the correct answer and assigned either 1 if correct or 0 if incorrect. Using this information the number of correct responses for the pretest, posttest, each section’s practice items, and all practice items within the lesson were totaled and logged. The number corresponding to the selection on the scales for the attitude questions was also recorded into the log. The log files were then imported into a Microsoft Excel spreadsheet. This allowed for formatting and calculating the reverse coding and scales for the attitude questions. The data was then copied into SPSS for complete data analysis. Results To analyze the posttest, a 3 x 3 ANCOVA was used with the pretest as a covariate. The first factor was target-related, attention-gaining techniques (none, highlighting, or flashing). The second factor was attentiondirecting techniques to target [none, realistic/concrete pointer (i.e., human hand with pointing finger), or symbolic/abstract pointer (i.e., arrow)]. Significant main effects were followed up with Fisher’s LSD post-hoc pairwise comparison procedure. A significant main effect was found for target-related, attention-gaining techniques, F (2, 159) = 6.45, MSE = 6.32, p < .01. Pairwise post hoc tests revealed that the participants assigned to the static attention-gaining condition (highlighting) outperformed their peers in the other two conditions: no attention-gaining technique and the flashing approach. There was also a main effect for attention-directing techniques, F(2, 159) = 3.10, p < .05. Pairwise post hoc tests indicated that the participants in the hand and arrow attention-direction conditions produced significantly higher scores on the posttest than their peers that were not provided these attention-directing techniques. There was no significant interaction between the two factors. The same results were reported for the cognitive load measure. The static attention-gaining condition reported low perceived cognitive load than their peers in the other two conditions: no attention-gaining technique and the flashing approach. The participants in the hand and arrow attention-direction conditions reported lower perceived cognitive load than their peers that were not provided these attention-directing techniques attention-directing techniques. While participants provided with the hand pointer did outperform their peers on the posttest, they performed comparably to their peers provided with arrows as attention-directing guides. There were also no differences on measures of cognitive load or intrinsic motivation. Conclusion and Discussion From the cognitive perspective, in a complex multimedia environment, attention directing and attentiongaining techniques may promote cognitive processing due to enhanced learner attention. The results of the experiment support the use of these techniques—both object-related attention getters and external attention pointers improved learning outcomes while reducing subjective cognitive load. These findings are consistent with previous studies (de Koning, Tabbers, Rikers, & Paas, 2007, Jeung, Chandler, & Sweller, 1997, Kalyuga, Chandler, & Sweller, 1999). Also, these findings suggest that applying the non-content signaling devices in the multimedia environment does not introduce extraneous elements and seductive details, which may be harmful to learning. However, the dynamic flashing condition did not elicit higher attention and result in better cognitive processing than the static highlighting condition, which is out of our expectation. Based on the findings of de Koning et al (de Koning, Tabbers, Rikers, & Paas, 2007), we explain that learners in the flashing condition may obtain some non-signaled information, which is not reflected in the posttest. By holding both signaled and nonsignaled information in their working memory, learners’ cognitive load in the flashing condition would be at a higher level than that in the highlighting condition. This indicates that learners studying the dynamic attention getters do not perceive the content more accurately than the static condition. Therefore, dynamic visual representations should be designed clearly to be perceived by the viewers, according to apprehension principle (Tversky, Morrison, & Betrancourt, 2002). From the socio-motivational perspective, the use of a human body part (hand) with a deictic function (finger pointing to target) was expected to enhance learner engagement due to realism as compared to a symbolic deictic, (i.e., arrow). On the other hand, from a cognitive perspective, we expect that there is no difference in performance between the realistic versus abstract attention pointers (hand vs. arrow). The findings of this experiment confirm our expectation from a cognitive perspective. We explain that only presenting part of human body of the animated pedagogical agent may not provide sufficient social cues to promote learning. These findings support the agent image effect on an opposite direction. From the cognitive perspective, we conclude that it is the function of pointing that affects learning outcome and cognitive load, not the degree of realism of the pointer. This implicates that instruction should be designed parsimoniously. Finally, we must take several limitations into consideration. First, we assume that enhanced attention to the selected information improves learning. And our findings indeed support this assumption. However, according to Mayer, there are three essential processes for active learning: selecting relevant information, organizing selected information and integrating selected information with prior knowledge (Mayer, 2005). Learners may have just retained relevant information from their enhanced attention and deep understanding was not been fostered. Therefore, in order to address this issue, in future research, it would be better to test both retention and transfer knowledge which can clearly address whether signaling has promoted deeper understanding. Second, the present experiment was conducted in a controlled laboratory setting, which limits the generalizability of the findings. It may be valuable to replicate the experiment in future in a real classroom setting, where there are more attention distracters. References Atkinson, R. K. (2002). Optimizing learning from examples using animated pedagogical agents. Journal of Educational Psychology, 94, 416-427. Atkinson, R. K., Mayer, R. E., & Merrill, M. M. (2005). Fostering social agency in multimedia learning: Examining the impact of an animated agent’s voice. Contemporary Educational Psychology, 30(1), 117-139. Baylor, A. L., & Kim, S. (2009). Designing nonverbal communication for pedagogical agents: When less is more. Computers in Human Behavior, 25(2), 450-457. Choi, S., & Clark, R. E. (2006). Cognitive and affective benefits of an animated pedagogical agent for learning english as a second language. Journal of Educational Computing Research, 34(4), 441-466. Craig, S. D., Gholson, B., & Driscoll, D. M. (2002). Animated pedagogical agents in multimedia educational environments: Effects of agent properties, picture features and redundancy. Journal of Educational Psychology, 94(2), 428-434. de Koning, B. B., Tabbers, H. K., Rikers, R. M. J. P., & Paas, F. (2007). Attention cueing as a means to enhance learning from an animation. Applied Cognitive Psychology.Special Issue: A Cognitive Load Approach to the Learning Effectiveness of Instructional Animation, 21(6), 731-746. Harp, S. F., & Mayer, R. E. (1998). How seductive details do their damage: A theory of cognitive interest in science learning. Journal of Educational Psychology, 90(3), 414-434. Jamet, E., Gavota, M., & Quaireau, C. (2008). Attention guiding in multimedia learning. Learning and Instruction, 18(2), 135-145. Jeung, H., Chandler, P., & Sweller, J. (1997). The role of visual indicators in dual sensory mode instruction. Educational Psychology, 17, 329-433. Kalyuga, S., Chandler, P., & Sweller, J. (1999). Managing split-attention and redundancy in multimedia instruction. Applied Cognitive Psychology, 13, 351-371. Lusk, M. M., & Atkinson, R. K. (2007). Animated pedagogical agents: Does their degree of embodiment impact learning from static or animated work examples? Applied Cognitive Psychology.Special Issue: A Cognitive Load Approach to the Learning Effectiveness of Instructional Animation, 21(6), 747-764. Mautone, P. D., & Mayer, R. E. (2001). Signaling as a cognitive guide in multimedia learning. Journal of Educational Psychology, 93, 377-389. Mayer, R. E. (2005). Cognitive theory of multimedia learning. In R. E. Mayer (Ed.), The cambridge handbook of multimedia learning. (pp. 31-48). New York, NY, US: Cambridge University Press. Mayer, R. E., Sobko, K., & Mautone, P. D. (2003). Social cues in multimedia learning: Role of speaker's voice. Journal of Educational Psychology, 95(2), 419-419. Mayer, R. E., & Moreno, R. (2002). Animation as an aid to multimedia learning. Educational Psychology Review, 14(1), 87-87. Mayer, R. E., & Moreno, R. (2003). Nine ways to reduce cognitive load in multimedia learning. Educational Psychologist, 38(1), 43-52. Miller, G. A. (1956). The magical number seven plus or minus two: some limits on our capacity for processing information. Psychological Review, 63, 81-97. Paas, F., Renkl, A., & Sweller, J. (2003). Cognitive load theory and instructional design: Recent developments. Educational Psychologist, 38(1), 1-4. Sweller, J., van Merrienboer, Jeroen J. G., & Paas, F. G. W. C. (1998). Cognitive architecture and instructional design. Educational Psychology Review, 10(3), 251-296. Tabbers, H. K., Martens, R. L., & van Merriënboer, J. J. G. (2004). Multimedia instructions and cognitive load theory: Effects of modality and cueing. British Journal of Educational Psychology, 74(1), 71-81. Tversky, B., Morrison, J. B., & Betrancourt, M. (2002). Animation: Can it facilitate? International Journal of Human-Computer Studies, 57(4), 247-262. Wouters, P., Paas, F. G. W. C., & van Merriënboer, J. J. G. (2008). How to optimize learning from animated models: A review of guidelines based on cognitive load. Review of Educational Research, 78(3), 645-675. Zahn, C. J., & Hopper, R. (1985). Measuring language attitudes: The speech evaluation instrument. Journal of Language and Social Psychology, 4(2), 113-123.