Developing and Assessing Your Students’ Language Proficiency Mississippi Bend Area Education Agency Workshop 1 November 2010 Paul Sandrock Wisconsin Department of Public Instruction Email: s.paul.sandrock@dpi.wi.gov Outline: 1. Matching the means of assessment with the purpose behind assessment 2. Motivating students with the thematic focus for a unit of instruction 3. Building repertoire for assessing the interpretive mode of communication 4. Expanding ideas for assessing the presentational mode of communication 5. Designing rubrics to focus students’ improvement 6. Identifying what counts in assessing the interpersonal mode of communication: Exploring how to gather evidence of interpersonal communication Creating rubrics to capture what counts in interpersonal communication 7. Involving students in their learning Paul Sandrock – Mississippi Bend AEA Workshop – November 2010 - Page 2 Planning Curriculum for Learning World Languages WI Department of Public Instruction Products – Practices – Perspectives Nature of Language Reinforce and further knowledge of other disciplines Concept of Culture Life-long learning Beyond the school setting Acquire information and distinctive viewpoints Paul Sandrock – Mississippi Bend AEA Workshop – November 2010 - Page 3 Context: According to Helena Curtain (University of Wisconsin-Milwaukee) Cognitively engaging Intrinsically interesting Culturally connected And (added by Sandrock & Clementi): Communicatively purposeful Interpretive Communication How does Jai Scott … ? Create Understanding Check for Understanding Interpretive Communication Is NOT Interpretive Communication IS Translation Hunt for trivial details Glossed readings; teaching all new vocabulary words first Reading, listening, or viewing from the “reader’s” perspective only Reading word-for-word Paul Sandrock – Mississippi Bend AEA Workshop – November 2010 - Page 4 Interpretive Mode of Communication Intermediate Low - Unit Summative Task: Complete a fact sheet about the area you have selected to visit in the target country, using Internet resources, identifying the benefit of visiting that destination Formative Assessments Summative Assessments 1. After examining a website, identify correct information given “Either-Or” choices Learning Checks 1. Word splash: given key words, create summary of travel article 2. Check off information that is actually found in article (practice skimming and scanning) 2. Fill in information from website 1. From 8 elements that could be found in the article; locate those that are actually in the article and tell one detail about what was said. 3. Read story, teachers holds #, students identify illustrations 3. Receive sheet of ten photographs; listen to descriptions and identify the photograph being described (write the number of the description on the photo) 2. Summarize the same news stories from three different perspectives/sources 3. Reflection: What this means to me? Why would you choose this travel destination to visit? Interpretive Mode of Communication Novice Intermediate Pre-Advanced Literal Comprehension Key word recognition Important words/phrases Main idea detection Main idea detection Supporting details detection Main idea detection Supporting details detection Interpretive Comprehension Word inferences Concept inferences Organizing principle(s) Author/Cultural perspectives Paul Sandrock – Mississippi Bend AEA Workshop – November 2010 - Page 5 INTERPRETIVE TASK – Intermediate Level I. Supporting information. For each of the following, circle the letter of each logical detail that is actually mentioned in the article write the information that is given in the article in the space provided Note to teacher: Provide 5 correct statements that support the main idea and 3 distracters. A. Foods that make your body strong - ___________________________________________ B. Names of vitamins to buy at the health food store - _______________________________ C. Suggestions for losing weight - _______________________________________________ D. A recommended cholesterol level - ____________________________________________ E. Ways to limit your salt intake - ________________________________________________ F. Types of effective physical exercise - ___________________________________________ G. Ways to relax before going to bed - ____________________________________________ H. Sources to contact for good diet tips - __________________________________________ II. Meaning from context. Based on this passage, write what the boldface words probably mean in English. . 1. 2. 3. bajos en grasa (1st paragraph) - ____________________________________________ mejores hábitos alimenticios (4th paragraph) - ________________________________ los ejercicios más relajantes (5th paragraph) - _______________________________ III. Inferences. Answer the following questions by providing as many reasons as you can. Your responses may be in English or in Spanish. Note to teacher: Write two open-ended questions “why do you think that”, “why is it important that”, “what might be the effect of”, etc., that require inference on the part of the student. Here is where you are asking students to tie in cultural knowledge, relating practices and products to perspectives. This is where your classroom instruction ties into assessment, asking students to apply what they’ve learned. 1. What problems might a person have if he or she doesn’t eat a variety of foods? Use details from the article to support your answer. 2. Why is it important to have a regular exercise program? Explain using information from the article and noting anything you feel is stressed more in Argentina than in the U.S. IV. Main idea. Answer the following question in one sentence in English: What is the author’s point (main idea) in this article? Source: ACTFL – Integrated Performance Assessment Handbook (www.actfl.org) Paul Sandrock – Mississippi Bend AEA Workshop – November 2010 - Page 6 How does you prepare your students for the presentational task? Learning Checks Formative Assessments Presentational Communication Is NOT Summative Assessment Presentational Communication IS Negotiated communication One-way communication (requiring a higher degree of accuracy) Random Practiced, rehearsed, polished, edited (often using a script, rough draft) Unplanned Organized (content and flow matter) Speaking or writing in a vacuum An awareness of audience (formal/informal; cultural context) Reliance on circumlocution (may be) improved by using a bilingual dictionary or spell-check Talking or writing only for the teacher Maintaining attention of the intended audience Presentational Communication Intermediate Low - Unit Summative Task: Create a commercial and a flyer to showcase the region you have selected Learning Checks 1. Write brief descriptions of illustrations on a single topic (preparation for creating brochure) 2. Finish the sentence 3. Ticket to Leave Formative Assessments 1. Tell story back from the illustrations as a summary 2. Graded rough draft (giving feedback on organization) 3. Send a postcard/letter back to host family Summative Assessments 1. Memorized dialogue, skit, or play 2. PowerPoint presentation 3. Letter of application to represent your school Paul Sandrock – Mississippi Bend AEA Workshop – November 2010 - Page 7 THINKING ABOUT RUBRICS When is a rubric an effective and efficient way to find out what students know? How can a rubric be an effective and efficient feedback mechanism to improve students’ performance? Designing rubrics backwards 1. Start with the ends (performance goals) 2. Brainstorm characteristics of the best performance. Identify the elements you are emphasizing 3. Cluster into categories (groupings) 4. Describe each element in terms of the performance you want to see Describe each category so the students know what makes: • a highly effective performance • an acceptable, moderately effective performance • a performance approaching your expectations 5. Revise in terms of what will help students set targets to improve Guiding Principles for Creating Rubrics • Take out the non-negotiables • Don’t just count (it’s not about quantity; it’s quality) • Provide clear descriptors • Push students’ performance toward the next level (show what they could do) Key Elements for Presentational Task Rubric Category Descriptive Elements Paul Sandrock – WAFLT Workshop – November 2009 - Page 8 Performance Guidelines for K-12 Learners (Plain Talk Summary) Comprehensibility Comprehension Novice Predictable; Rehearsed; Imitation; Rely on more than just verbal cues Intermediate Express own thoughts; Recognize patterns; Understood by teacher and peers PreAdvanced Control of tenses; Continuity with fewer pauses; Detailed / depth of expressions Gestures; Gist; Cognates; Familiar topics Branch out from known to unknown; Longer and more complex; Contextual clues Details with inferences; Cultural nuances; Can clarify and transfer to another context Source: Participants at New Jersey IPA Workshop (2003) Language Control Accurate with memorized material; Fall back on formulas; Inaccurate when trying to create More inferences; Can get meaning from the unfamiliar; Put sentences together Vocabulary Everyday words; Pantomime; Occasionally use native language; Memorized; Limited in number (lots of cognates used) More daring and experimentation (mistakes); Aware of sentence structure; Use affirmative/ negative and context clues Capable of Culturallycomplexity; authentic Still present expressions; tense-bound; Variety and use More idiomatic of idioms; usage Use target language to define target language Cultural Awareness Imitate; Recognize similarities; Use native culture except if memorized target culture (e.g., gestures) Communication Strategies Visual clues; Repetition; Previous (background) knowledge Recognize differences; Make comparisons Expansion; Circumlocute; Inferences Analyze perspectives; Use appropriate gestures; Use idioms Sustain use of target language; Degree of analysis/depth; Refining and detailed Paul Sandrock – Mississippi Bend AEA Workshop – November 2010 - Page 9 What are the characteristics of Interpersonal Communication? One-way communication Two-way communication Spontaneous Following up Memorized Indicating interest Focused on message Helping partner Taking turns Focused on accuracy Using various means to get meaning across Asking for clarification Other characteristics you noticed: Intermediate Low - Unit Summative Task: Discuss with a partner the places in classmates’ commercials, explaining which you want to visit and why Learning Checks Formative Assessments Summative Assessments 1. Information Gap Pair Activity: controlled tasks (Map or Clock partners) 1. Questions in an envelope (want to use the fewest out of the envelope) 1. Work with partner to retell story or summarize an article on travel 2. Post-It Note Feedback 2. Identify how much you and partner have in common about things you like to do on trips 2. Identify ways that you and your partner could provide service on a trip to address a global issue 3. TALK scores 4. Keep conversation going for two minutes 3. Find someone who … (ask as many follow-up questions as possible about trips taken) Paul Sandrock – Mississippi Bend AEA Workshop – November 2010 - Page 10 Learning Checks: TALK Scores Shrum, J. L. & Glisan, E. W. (2005). Teacher’s Handbook: Contextualized Language Instruction (3rd ed.).Boston: Heinle Target Language Accurate Listens Kind Student A + + + + Student B √ – + + Student C – √ – + Formative Assessment: Scored Discussion (4 Person Conversation - Based on ideas from Greta Murray, Medford,WI) Move from: 1–3–5 Move to: Asks random questions Follows up with logical questions Only answers the question asked Contributes additional information Responds, but rarely initiates Contributes personal insights to enhance discussion and draw in others Comments are not relevant Stays on topic Rubric Design: Holistic or Analytic? Characterized by: Useful for: Limited by: Holistic Criteria are combined into a single descriptive paragraph Broad program checkpoints; Students see if they have performed at the targeted level (or above or below it) Analytic Criteria are identified and described separately in categories Feedback to focus on specific elements of the performance; Students can see strengths and weaknesses; Provides student with feedback to guide improvement and emphasis (personal goals) Requires strategic decisions about Requires strategic decisions as to the how to balance various criteria in weight of various criteria especially if coming up with the single holistic used for a letter grade decision of performance level Paul Sandrock – Mississippi Bend AEA Workshop – November 2010 - Page 11 Feedback Mechanisms: Sample Checklist: Task: Novice Level – Interpersonal – Find out who has the busiest schedule. Performance Criteria: 1. 2. 3. 4. 5. 6. 7. 8. 9. I can do this on my own I can do this I cannot do with some this help (from a student or the teacher) I can use numbers I can use words for classes I can use words for activities I can understand my partner I can say how many times, how often, how frequently I have or do various things I can ask some questions I can provide some description I can use expressions to show that I agree or disagree with what my partner says I can say how I am feeling (emotions) Even at a more advanced level, a checklist provides a framework for helping students know the expectations and how close they come to meeting or exceeding them. Task: Pre-Advanced Level – Interpersonal – Find out who has the busiest schedule. Performance Criteria: 1. Questions include a variety of topics and interests beyond just school 2. Many questions are open-ended to encourage more interaction in the conversation 3. Questions lead to in-depth exploration of the topic, beyond mere facts; asks for more detail, further explanation 4. Demonstrates careful listening by using expressive reactions and appropriate follow-up questions Achieves the target and more (Consistently) Achieves the target (Frequently) Achieves the target (Minimally) Falls short of the target Paul Sandrock – Mississippi Bend AEA Workshop – November 2010 - Page 12 Sample “I Can” Statements – Interpersonal Mode (From CLVisa, a Linguafolio project of Concordia Language Villages) Categories: Targeting Novice Level Targeting Intermediate-Low Targeting Intermediate Level I can … I can … I can … Carry on a conversation asking and answering simple questions Ask simple questions and provide brief answers Begin and continue a conversation, using long sentences with lots of description Ask and answer a wide variety of questions, giving reasons and explanations, and asking follow-up questions Share detailed descriptions with a wide variety of descriptive words and phrases A. Carry on Carry on a simple conversation conversation using memorized phrases B. Ask and answer Answer simple questions with a single word or two C. Provide description Describe with 2 or 3 descriptors Share a simple description of a person, place, or thing D. Express preferences Talk with someone about things I like and dislike Discuss personal preferences and feelings, giving simple reasons for those preferences Discuss preferences, with detailed reasons and explanations to support my opinions E. Extend context for expressing self Make myself understood in a simple way, and understand the other person when he/she talks slowly and clearly and can help me understand Say when I don’t understand and ask people to repeat or speak more slowly Talk about things that have happened or are going to happen Support my opinions and make recommendations with detailed explanations, arguments, and reaction to the topic being discussed Ask for a definition or explanation and restate what has been said in another way to make sure that I understand F. Ask for clarification Ask someone to explain again in simpler language and repeat to make sure I understand Targeting PreAdvanced (Intermediate High) Level I can … Begin and effortlessly continue a long conversation Ask and answer a wide variety of questions requiring extended explanations and follow-up discussions Share detailed, effective descriptions of people, places, or things; explain a complicated problem or dilemma and negotiate to arrive at a solution Comfortably participate in a spontaneous discussion and express, discuss, and defend my opinions with only minor pauses and hesitations Convey and understand finer shades of meaning by using a wide range of expressions to adapt to the demands of the conversation Ask for more information and clarification and suggest alternative words and paraphrase to ensure understanding Paul Sandrock – Mississippi Bend AEA Workshop – November 2010 - Page 13 Rubric Development: Targeting the Proficiency Level: Interpersonal: Maintaining and Sustaining the Conversation Novice Interme-diate Responds when Mainly in a Uses circumlocuAsks follow-up prompted or reactive mode, tion to stay in the questions and asked, usually to sticks to direct conversation; inserts rejoinders high-frequency answers; able to maintains to maintain the cues; initiation is initiate conversation with conversation and limited to conversation “give and take” vs. stay on topic memorized with a few turn-taking; words or phrases original uses informational questions; questions (who, generally uses what, where, when) yes/no questions Interpersonal: Asking for Clarification Novice Says “Huh?” or Repeats what Provides uses gestures was not perceived and facial understood; meaning; repeats expressions to asks speaker and adds a tag show to repeat or question to show confusion slow down understanding Interpersonal: Comprehensibility Novice Intermediate Sticks to Tries to use Some success in known words new phrases using new and combinaphrases and tions; rarely combinations successful Asks simple questions to clarify meaning Some errors that may require repetition to be understood Pre-Advanced Helps partner by Provides own Uses transition supply-ing words responses to phrases to signal or finishing prompt partner changes in the sentences or (models conversation’s thoughts responses to help direction; works partner respond) with partner to explore topics in depth Intermediate Provides alternatives, examples, to point out what is not understood; asks for a definition or explanation Paraphrases to verify what was understood Pre-Advanced Control of language Some errors; decreases as topics errors seldom and time frames interfere with expand beyond present comprehension and familiar Pre-Advanced Adds own interpretation to verify what was understood Errors are not noticeable; no interference with comprehension Paul Sandrock – Mississippi Bend AEA Workshop – November 2010 - Page 14 Resources: Annenberg/CPB Library Teaching Foreign Languages K-12: A Library of Classroom Practices: http://www.learner.org/channel/libraries/tfl/ The Teaching Foreign Languages K-12 video library and professional development guide bring to life the Standards for Foreign Language Learning in the 21st Century. Illustrates effective instruction and assessment strategies in K-12 classrooms around the country. 27 teachers and their students; eight languages; full range of competency levels. Teaching Foreign Languages K-12: Workshops http://www.learner.org/resources/series201.html A video workshop for K-12 teachers; 8 half-hour video programs, workshop guide, and website; graduate credit available. Leading researchers and practicing teachers discuss how the standards play out in day-to-day classroom situations. Workshop guide available online and in print, plus interactive activities on the web. Wisconsin Educational Communications Board World Language Assessment: Get in the Mode! http://www.ecb.org/worldlanguageassessment/ Seven video segments (15 minutes each) take teachers into K-12 classrooms to explore issues and examples of assessing world languages. Focuses on the three modes of communication (interpretive, presentational, and interpersonal), includes web resources, provides a framework of formative and summative assessments for use at any grade level. Foreign Language Educators of New Jersey (FLENJ) Consortium for Assessing Performance Standards http://flenj.org/CAPS/?page=parent / This web site showcases the work of teachers in four New Jersey school districts: 70 assessment tasks, rubrics developed for these tasks, samples of student work for some tasks, and answers to frequently asked questions (funded by a Foreign Language Assistance Program grant awarded in September 2003). Fairfax County Public Schools (VA) Performance Assessment for Language Students (PALS) http://www.fcps.edu/DIS/OHSICS/forlang/PALS/ Fairfax County Public Schools teachers have been developing a performance assessment program since 1995. Teachers and students field tested rubrics and modified them to reflect the language development of students within an academic setting. The assessment tasks, both formative and summative, and accompanying rubrics are available on the website. Paul Sandrock – Mississippi Bend AEA Workshop – November 2010 - Page 15 Request for Work Demonstrating Global Competence Dear Colleague: You are invited to participate in an exciting new project to transform the teaching and assessing of key skills in students of all ages. EdSteps is collecting samples of work that demonstrate Global Competence as part of a ground breaking effort to assess student performance using real examples of work done by students and professionals from across the nation and throughout the world. Why Global Competence? th Today’s students are entering into a world vastly different from that of the 20 century. Schools and communities need to prepare their young people to succeed in this new global environment. For our students’ well-being, the vitality of our communities and the welfare of our entire nation we must cultivate the knowledge, skills and passion in students that will enable them to recognize the challenges and opportunities of an interconnected world and contribute to its improvement. Better approaches to teaching and assessing student work that address issues of global significance both world wide and in their own neighborhood are essential to a world-class education system. EdSteps: A Call for Work Demonstrating Global Competence EdSteps seeks work done by students and professionals from the U.S. and around the world that demonstrates global competence: the knowledge, skills and disposition to understand and act creatively on issues of global significance. EdSteps will publish the work online to show examples of global competence at various levels of expertise. Once the work is published, it will help teachers strengthen instruction and better evaluate student work and provide examples for students to help them strive for greater achievement. To accomplish these goals, we need your help and professional judgment. We invite students to submit works that address one or more of the critical global issue areas of our time, such as: • Environmental Sustainability • Health and Human Development • Population Growth and Transition • Human Rights • Economic Development • Cultural Identity and Diversity • Global Conflict and Cooperation • Media and Technology Your students should identify and investigate a specific problem or opportunity within one of these issue areas, or another critical global issue area that is meaningful to them. The work may be created for a variety of purposes: to analyze, describe, critique, explain, persuade, tell a story, express an opinion, offer an artistic interpretation, etc., in various forms: • Written document (essay, story or screenplay, “Op-Ed”, poetry, etc.) • A work of digital media (short video, PowerPoint, audio slide show, etc.) • A work of art or design (painting, sculpture, architectural design, song, etc.) www.edsteps.org A project of the Council of Chief State School Officers (CCSSO) Paul Sandrock – Mississippi Bend AEA Workshop – November 2010 - Page 16 We invite your students to submit work that: 1. Presents and weighs evidence on issues of global significance or important local issues faced by others in the world 2. Considers their own and others’ perspectives 3. Communicates ideas clearly and appropriately to diverse audiences 4. Illustrates options for responsible action to improve a situation, concern or condition. The Global Competence Matrix Investigate the World Students investigate the world beyond their immediate environment. Recognize Perspectives Students recognize their own and others’ perspective. Communicate Ideas Students communicate their ideas effectively with diverse audiences. Students can: • Generate and explain the significance of locally, regionally or globally focused researchable questions. Students can: • Recognize and articulate their own perspective(s) on situations, events, issues or phenomena and identify the influences on that perspective. Students can: • Recognize that diverse audiences may perceive different meanings from the same information. • Identify, collect and analyze the knowledge and evidence required to answer questions using a variety of international sources, media and languages. • Weigh, integrate and synthesize evidence collected to construct coherent responses that are appropriate to the context of issues or problems. • Develop an argument based on compelling evidence that considers multiple perspectives and draws defensible conclusions. • Articulate and explain perspectives of other people, groups or schools of thought and identify the influences on those perspectives. • Explain how the interaction of ideas across cultures influences the development of knowledge and situations, events, issues or phenomena. • Use appropriate language, behavior and strategies to effectively communicate, both verbally and nonverbally, with diverse audiences. • Explain how effective communication impacts understanding and collaboration in an • Select and effectively use appropriate technology and media to communicate with diverse audiences. • Articulate how the consequences of differential access to knowledge, technology and resources affect the quality of life and influences perspectives. www.edsteps.org Take Action Students translate their ideas and findings into appropriate actions to improve conditions. Students can: • Recognize their capacity to advocate for and contribute to improvement locally, regionally, or globally. • Identify opportunities for personal and collaborative action to address situations, events, issues or phenomena in ways which can make a difference. • Assess options for action based on evidence and the potential for impact, taking into account varied perspectives and potential consequences for others. • Act creatively and innovatively to contribute to improvement locally, regionally or globally both personally and collaboratively. Paul Sandrock – Mississippi Bend AEA Workshop – November 2010 - Page 17 What Makes the Right Travel Destination for Me? Unit Level Summative Performance Assessments Novice Level: Where would I like to travel? Interpretive Presentational Listen to a travelogue: Write a brief website match photos to each description for 5 places of description heard interest (where, hours open, admission, etc.) Interpersonal In pairs, look at 5 photos and discuss likes and dislikes, deciding which 2 places to visit on your last day in the target country Intermediate Level: Why would I choose a particular destination? Interpretive Presentational Interpersonal Complete a fact sheet about Create a commercial and a Discuss with a partner the the area you have selected flyer to showcase the region places in classmates’ to visit in the target country, you have selected commercials, explaining using Internet resources, which you want to visit and identifying the benefit of why visiting that destination Pre-Advanced (Intermediate-High) Level: How could travel help my career options and qualifications? Interpretive Interpersonal Presentational Investigate places for study, Evaluate with a partner the Write a letter to apply for an travel, or work in the target career advantages you could internship, explaining how country; identify how they gain by studying, traveling, the experience will fit into would be helpful to four or working in the target your career plans, and how careers you are exploring country you have prepared for it Advanced Level: How does travel change the way I look at the world? Interpersonal Presentational Interpretive Discuss stereotypes of the Write a newspaper editorial Read a work of literature target culture that you know explaining one U.S. cultural with a strong sense of place; you see differently now; phenomenon that is likely to describe the cultural identify ideas you want to be misinterpreted by tourists influences shaping the main investigate while in the from the target country idea or conflict target country Paul Sandrock – Mississippi Bend AEA Workshop – November 2010 - Page 18 What is the target for performance? Think about the unit you are planning: what is the engaging Thematic Focus? What are the Summative Assessment targets for Interpretive, Interpersonal, and Presentational communication? Do these assessments engage and motivate students? Interpretive Performance Presentational Performance Interpersonal Performance Activities to Prepare for Summative Tasks: (Formative Assmts. & Learning Checks) Interpretive Practice Presentational Practice Interpersonal Practice Knowledge Students Need to be Successful on these Performance Assessments: Language Functions Vocabulary Structures