USING AUTHENTIC ASSESSMENT: TASKS, RUBRICS AND LANGUAGE COMPETENCIES Lina Lee University of New Hampshire December 6, 2014 llee@unh.edu Let’s Discuss … (1) What types of classroom assessment do you use? (2) Do you include four language skills? If so, give a few examples. (3) Do you include an oral component? Why and why not? (4) How effective is the way you measure student language abilities? Any challenges? Three Fundamental Questions (1) What do you believe about language learning? (1) What approaches or methods do you use? Are they effective? (1) What types of activities do you use? Are they meaningful? Teaching Practices (1) Grammar-driven approach (2) In BETWEEN (1) and (3) (3) Communication, PERFORMANCEBASED use of language Traditional Design • Vocabulary + Grammar • Practice • Quiz • Practice more • Culture • Chapter test • Feedback PPP Cycle – Framework • Presentation • By the teacher • Practice • Pattern drills with specific grammar points • Production • Controlled language exercises • Focus on accuracy and then develop fluency Traditional Quiz Fill in each blank with the correct form of the present tense verb. Yo _______ (caminar) por el parque y mi amigos ________ (practicar) frisbi en la playa. Nosotros _________(escribir) composiciones y Luis _________ (leer) el texto de español … Another Traditional Quiz Choose the correct answers. (1) ¿ ________ hora es? a. Quién b. Qué c. Dónde (2) ¿ ________ te llamas? a. Qué b. Cuándo c. Cómo (3) ¿ ________ años tienes? a. Cuántos b. Cómo c. Cuál (4) ¿ ________ vives? a. ¿Cómo b. Dónde c. Quién Creating a Dialogue María: ¡Hola! ¿Cómo ________? Gabi: Muy bien. ¿Y tú? María: Regular. ¿_________ clase ahora? Gabi: No, tengo clase hasta la 1:00. ¿ ________ tomar café conmigo? María: Claro que sí. ¿ ________ quieres ir? Gabi: Vamos a Dunkin’ Donuts. Question-Answer Type Modelo Juan: ¿Vas a la escuela? María: No, no voy a la escuela. 1. 2. 3. 4. ¿Vas a la biblioteca? ¿Va tu amigo a la playa? ¿Van tus amigos a la plaza? ¿Vamos al parque? Language Output Write a dialogue about what you and your friend did last week. Be sure to use complete sentences. 1. 2. 3. 4. Buy Christmas gifts Study for a exam See Halloween decorations Attend a birthday party Traditional Testing • Grammar is main key to L2 Learning. • Focus on form NOT meaning • Without meaningful contexts • Lack of authenticity and creativity • Drills and after drills are not sufficient. • Overuse of a specific L2 item • Teacher as “grammar police” • Exhausting and boring • Fail to perform outside the “box” Communicative Competence • Fluency • How well? • Accuracy • How accurate? • Pragmatics • How appropriate? How to carry a good conversation at dinner? Observation: Linguistic accuracy should not be the primary focus. Areas of Assessment • Communicative Competence (Canale & Swain, 1980) • Linguistic Competence • Socio-linguistic Competence • Discourse Competence • Strategic Competence Question: Are your students equally strong in all four competences? Communicative Language Teaching • Meaningful use of the target language • • • • • • Interaction in real-life situations Authentic and functional use of L2 Fluency before accuracy Integration of four skills and culture Learner-centered classroom Beyond classroom learning Authentic Assessment • Performing real-world tasks • Application of knowledge and skills • Focus on meaning not form • Within meaningful contexts • Creating new meanings • An ongoing process • Effective grading system & practices Making Languages Come to Life Tasks Michael Long (1985) “A piece of work undertaken for oneself or others … things people do in everyday life.” Jane Willis (2002) “A task is an activity where the target language is used by the learner for a communicative purpose in order to achieve an outcome.” Key: “Promoting Learning by Doing” Task is an essential concept in • Real world • Doing ‘things’ in everyday life (e.g., go shopping) • Language teaching and learning • Use of the target language for authentic communication • Second language acquisition • Input, interaction, negotiation and output (conditions) • Assessment of language competence • Fluency + accuracy + pragmatics Create Authentic Assessment • Two documents • National Standards – “Communication” • ACTFL Proficiency Guidelines – “Levels” • Four steps • Identify your goals • Select authentic tasks • Identify the criteria for each task • Create rubrics Goal: National Standards “Knowing how, when, and why to say what to whom” ACTFL Proficiency Guidelines (2012) Distinguished Linguistic Skills + Cultural Knowledge Oral Proficiency Interview (OPI) http://actflproficiencyguidelines2012.org Expected Language Proficiency ACTFL Proficiency Levels • Intermediate High (College major without study abroad) • Intermediate Mid/Low (4 years - high school; 6 years - grade 7-12) • Novice High/Mid/Low (2 years - high school) Doing Tasks in Real Life #1 Doing a Variety of TASKS • Give three examples for each level • Beginning • Intermediate • Advanced • Identify language skills • Listening, reading, writing and speaking • Share your answers with … #2 Doing a Variety of TASKS (1) Give directions or instructions (2) Listen to weather forecast or news online (3) Send e-mails to friends or colleagues (4) Read daily newspapers or recipes (5) Make a hotel reservation (6) Deliver a paper in a national conference (7) Read travel brochures (8) Write a complaining letter (9) Talk to your friend about your last weekend (10) Listen to a podcast about your famous artist Integrated Performance + Task • A shift from focusing on teaching to learning • Authentic communication In real world context • Moving beyond single skills • Tasks should incorporate three modes of communication. • Tasks should be interrelated and built upon one another. After Graduation: Job Search (1) Complete the Career Interest Inventory • http://www.okcareertech.org/cac/Pages/Career_cluster/Caree rClustersInterestSurveySPANISH_rev.pdf With the results, use the websites to find jobs that meet your 3 highest recommendations • http://www.infoempleo.com/ • http://www.computrabajo.com.ar/ Real world Tasks (2) With your partner, compare the jobs that you have chosen and why (3) Create a resume for your first choice job Novice • Giving simple; limited learned information • Task “Describe the weather in New Hampshire” • “Hace” weather expressions • Vocabulary—seasons, months, geography • To be – ser Three Modes of Communication • Interpretive • Listen to a TV weather report (e.g., CNN; BBC) • Read weather report online http://www.rtve.es/eltiempo/ • Interpersonal • Discuss the weather in ________. • Compare and contrast the weather in two places. • Presentational • Create a weather forecasting in _______. • Record the broadcast in VoiceThread or YouTube. Intermediate • Description in the present tense using basic vocabulary; uncomplicated tasks; survival skills with some creativity • Task “Describe/Talk about your daily routine” • Reflexive verbs (regular and irregular) • Connectors (before, after, first, then, …) Advanced • Narration and description in the present, past and future time with details; formal vs. informal • Task “Describe your best Thanksgiving” • Preterit/imperfect Superior • Supporting an opinion/hypothesizing and presenting arguments • Task “No Child Left Behind” • Sophisticated vocabulary; extended discourse and argumentation #3 “My Dream Trip to …” • Level: Intermediate • Objectives • Tasks • Interpretive • Interpersonal • Presentational • Evaluation criteria #4 INPUT – Comprehension 1) Do you teach listening and reading skills? 2) How often do your students listen or reading in L2? 3) What kinds of activities do you use? 4) Do you include these two skills in your assessment? Give a few examples. http://www.youtube.com/watch?v=4K11o19YNvk&list=PL00C11E2EF1F9843E “We acquire language in only one way: when we understand messages; that is when we understand what we hear or what we read, when we understand the message.” Krashen Listening / Reading Comprehension • Schema Theory • Prior knowledge • “Restaurant” • Anticipate and predict incoming information • Top-down approach • Global comprehension • Bottom-up approach • Detailed information Strategies for INPUT comprehension Interpretive Communication • A lot of INPUT • Global understanding and then details • Use of authentic materials • Author’s perspective and cultural context • Use of context cues • Not translation nor learning about new words or expressions 3 Modes Source: Discovering French Bleu Mc Dougal LIttell Testing Reading Comprehension http://web.uvic.ca/hrd/span100/main_reading.htm Interpretive Tasks • • • • • • • • Maps Songs Poems Stories Catalogs and Ads Folktales Fairy tales Movie trailers • Charts • • • • • • • Checklists Key Word Recognition True / False Questions Drawing responses Graphic organizers Resources • French and Spanish http://www.ver-taal.com/index.htm • French and Spanish http://miscositas.com/index.html • Various languages http://wlteacher.wordpress.com • Spanish http://www.laits.utexas.edu/spe/ • Spanish http://zachary-jones.com/zambombazo/ http://www.ver-taal.com/index.htm http://www.thefrenchpodcast.com/index.php http://www.bbc.co.uk/languages/ #5 Interpersonal Exchange 1) How do you teach interpersonal skills? 2) What types of activities do you use? Give a few examples. 3) Do you include speaking and writing in your assessment? Your examples? 4) Do you use digital tools to help students develop interpersonal skills? Interpersonal Communication • Two-way exchange • Not asking questions by the instructor • Interactive; negotiation of meaning • Input + feedback + pushed output • Spontaneous; not memorized dialogues • Focusing on the message not overly concerned about accuracy • Communication strategies (e.g., circumlocution; body language; geatures) I think Kate is pregnant … What do you mean? Clarification check Interpersonal Tasks • Pictures; drawings • Real life situations • Controversial issues • Problem-solving • Theme-based topics • Role-play • Interview • Information-gap • Tell difference • Debate • Panel discussion • Survey Interview Activities • Small Group ( 3-4 students) or walk around the class • Worksheet • Asking questions using a worksheet; vocabulary practice • Ex:谁,什么,在哪儿,有没有,会不会 Name 1 2 3 4 你的老师是谁? 你在哪儿学中文? 你有没有中文书? 你会不会说日文? Survey Activity • Worksheet • Students walk around the class and ask questions until they fill out all the information on the worksheet • Ex: 去过, 吃过, 喝过,看过,用过. 名字 1 2 3 去过中国 用过毛笔 喝过烧仙草 吃过臭豆腐 Find the Difference Small group activities: A/B ask each other until they both have the same objects in both rooms. A B Information Gap Activities • Students Working in pairs • Student A has the information student B does not have and they will share their information by asking each other. • Weather, restaurant menu, airplane/train timetable, person’s schedule, etc. • Target grammar: 几点/ 什么时候/ 什么时间/ 多久了 北京到上海 起飞时间 降落时间 广州到南京 香港到广州 西安到上海 成都到长沙 Role-Play You are in a clothing store and have $150 to spend Card A: You are customer Card B: You are clerk • • • • • • • You are looking for gifts for three people; your mom, brother and boyfriend Ask for suggestions Ask for prices Ask him/her to wrap the gifts Thanks him/her • • Offer to help Tell him/her what is on sale Suggestions: a bottle of perfume for mom, a pair of pants for brother, a watch for boyfriend Go to the customer service Digital Tools • Google Voice; Google Hangouts http://ditchthattextbook.com/wp-content/uploads/2013/03/google-voicecall.mp3 • Skype • Wikispaces • Blogger Synchronous & Asynchronous Computer-Mediated Communication http://jaredseminar997.blogspot.com/ • AudioBoom (pair or group recordings) • VoiceThread (interactive comments) • Voxopop (Talk discussion board) Presentational Communication • One way communication • Awareness of audience • Not just talking or writing for teachers • Maintain audience’s attention • Organized, rehearsed and practiced • Edited and polished Presentational Tasks • Descriptive • Narrative • Demonstrative • Explanatory • Transformative • • • • • • Personal experience Tell a story Instructions Provide evidence Give a talk Convince someone to adapt a point of view Performing a Task • Create a podcast and talk about “your best friend” • Use descriptive adjectives and comparisons • más/menos … que; tanto/a/s … como • A minimum of one minute voice recording • Listen to my voice (Model) • Guided questions: Who is your best friend?; How is she/he?; How old is he/she?, etc. Presentational Mode Presentational Mode https://audioboom.com/boos/2282461-mi-mejor-amigo-y-yo Performing a Task • Create a blog entry and write about the celebration of July 4 th in 2014 • Use both the preterit and imperfect • A minimum of 120 words • Guided questions: What did you do? Where did you go? What was the weather like? Why was the day special or not special? Presentational mode through writing Presentational Tools • Voki (voice recordings) http://e- languageexemplars.wikispaces.com/Voki • Glogster (interactive posters) • Storybird (digital stories) http://voicethread.com/?#q+el+perro+Frederico.b3205219.i16999401 • Animoto (videos) • Audioboom (voice recordings) http://audioboo.fm/Milana • Audio Dropboxes (voice recordings) http://clear.msu.edu/teaching/online/ria/ • VoiceThread (interactive presentations) http://spanish402people.wikispaces.com/VoiceThread+presentations • Prezi (fancy PowerPoint shows) #3 “Project-Based Assessment” 1) Do you use project-based learning (PBL)? 2) What types of projects do you use? Give a few examples. 3) Do you find them useful? Why and why not? Some Examples … • The menu project • Authentic dishes of the target culture • The children story book • Create a set of characters in the target country • The ABC book project (one semester) • Focus on a target city (e.g., Quebec) • The intercultural exchange project • Spanish-American telecollaborative exchange https://sinbarreras.wikispaces.com/Nuestros+Blogs A Quote from a Spanish Teacher “When designing my unit, I wrote my learning outcomes based on the 3 modes of communication and proficiency guidelines, then developed assessments around them …” - Señora B http://senorab1972.wordpress.com/2011/12/19/integrated-performanceassessment/ “La Comunidad” • Interpretive task • Understand a written description of a monument of Segovia, Spain • Interpersonal task • Complete an info-gap activity with a map of Segovia • Presentational task • Write a description of a monument in __. Interpretive Task You are meeting your friend in Segovia and you are trying to find some interesting sites to visit. You find this text about Alcázar de Segovia. Read the text to find out more about the castle and what you need to know to visit. http://www.alcazardesegovia.com/index.htm Interpersonal Task You are meeting your friend from Madrid in Segovia tomorrow but the maps that you printed online cut off the names of some of the monuments. Your friend has half of the names and you have the other. However, your friend is in Madrid. Because your friend does not understand English. You must speak Spanish to get the missing names of monuments on the map. Before you begin, you should • Greet your friend • Ask how your friend is doing • Try to find the names of the missing monuments by asking and responding to questions about each structure Presentational Task You loved Segovia. You want to send your host family a postcard to describe your experience while there. Choose your favorite site in Segovia and describe it in your postcard to the host family. “La Vida Escolar” • Interpretive task • Understand a written description of a school curriculum • Interpersonal task • Conversation with a Mexican exchange student • Presentational task • Create a foreign language program brochure Interpretive Task http://www.colegioespanoldetijuana.com/ Interpersonal Task Mario es un estudiante de intercambio que ha llegado recientemente de México. Le gustaría saber información acerca de tu escuela. Comparte información con él mencionando aspectos de tu vida escolar que te gustan o que no te gustan. Tu charla es de 2 minutos. Digital tools: Skype; Google Hangouts Presentational Task A group of high school students are coming to visit your university. With your partner, create a foreign language program brochure and make a PowerPoint presentation to present the program http://www.stocklayouts.com/Templates/Free-Templates/FreeBrochure-Templates-Download.aspx #6 Grading and Rubrics What grading criteria do you use to measure interpersonal skills? writing skills? presentational skills? Six Components for Rubrics 1. 2. 3. 4. 5. 6. Comprehensibility Comprehension Language control Vocabulary use Discourse (flow) Cultural awareness (sociolinguistic issues) http://www.fcps.edu/is/worldlanguages/pals/index.shtml#steps Holistic Scoring Rubrics 3. Exceeds expectations Message effectively communicated; rich vocabulary; highly accurate; fully covered content 2. Meets expectations Message generally comprehensible; appropriate vocabulary; some patterns of errors; adequate content 1. Does not meet expectations Message communicated with difficulty; inappropriate vocabulary; patterns of errors; insufficient information Teacher Feedback “Your description lacked details to keep your audience interested. Try to write more complex sentences that include more colorful adjectives and phrases that help the audience to envision what you are describing.” Source: http://www.carla.umn.edu “Three” is the Key • Three modes of communication Interpretive, interpersonal, presentational • Three real world tasks within one ‘theme’ “Jour de la Terre” (environmental issues) • Three proficiency levels Novice, intermediate, advanced • Three criteria for scoring rubrics Exceeds, meets or does not meet expectations “US Boys: Jane Lee” • Interpretive task • Interpersonal task • Presentational task http://www.youtube.com/watch?v=5ih1p2Jn-EE Toni Theisen-Loveland High School “Caillou se va a la playa” • Interpretive task • Interpersonal task • Presentational task http://www.youtube.com/watch?v=p2dQmSChFPE Interpersonal Task Useful Resources Examples of IPAs from CARLA – University of Minnesota http://www.carla.umn.edu/assessment/vac/CreateUnit/e _1.html World Language Assessment: Get in the Mode! http://www.ecb.org/worldlanguageassessment Consortium for Assessing Performance Standards http://flenj.org/CAPS/?page=parent%20/ Teaching Foreign Languages K-12: A Library of Classroom Practices http://www.learner.org/libraries/tfl/ More Resources • Project-based Learning http://www.edutopia.org/project-based-learning • Ohio Department of Education http://education.ohio.gov/Topics/Ohio-s-New-LearningStandards/Foreign-Language/World-LanguagesResources/Ohio-Foreign-Language-Model-Assessment-Project • French and Spanish Tasks with Rubrics http://jfmueller.faculty.noctrl.edu/toolbox/examples/tasks_hi ghschool_foreignlanguages.htm Useful Book The Keys to Assess Language Performance: Teacher’s Manual by Paul Sandrock Available at actfl.org