Effective schools use a recursive cycle of assessment and analysis to examine student performance from data generated to drive instruction. Standards Instructio n Assessmen t Reflective teachers carefully analyze state standards to understand the skills and processes needed to demonstrate mastery of the content. Standards Instru ction Assess ment Students are seldom asked to demonstrate “How do you know?” or “How did you get that answer?” or “Why do you think so?” or “Show me proof that answer is correct.” Standards Instructi on Assessme nt This lack of cognitive follow-through in our classrooms leads to shallow thinking and encourages students to simply try to guess the correct multiple choice response. Standards Instructi on Assessme nt In real-world situations, students must be able to apply knowledge and use thinking strategies to analyze, synthesize, and evaluate information. Standards Instructi on Assessme nt This performance gap can be analyzed through a constructed-response which demonstrates student’s skill level as well as processes with the the concept. Standards Instructi on Assessme nt Reading Selection Title___________________________________ Text Type: Test Item Question Constructed Response Text Evidence Student Achievement Through Constructed Response Teacher Toolkit Developed by: Marie Heath, M.Ed. Richardson ISD Secondary ESL Specialist Marie.heath@risd.org Marie Heath 2014 Page 1 Constructed Response Introduction Any good carpenter knows having the right tools makes the job at hand easier. Hopefully this toolkit will make it easier for you to “build” and use constructed response assessments. The materials in the toolkit are organized so that you will gain an understanding for what is constructed response. Research based evidence as to why it is important to use constructed response. Also, why a constructed response assessment will warrant the time and energy required to design and use it in your daily teaching. You will learn how to design constructed response assessments for students at your grade level. There is rigid method to creating a good constructed response assessment, but there are components that you should try to include in each one. You will find included a template to use to ensure that you consider each of these components Finally, the scoring of constructed response assessments is addressed. So dig in and find the right “tools” for the job of teaching and assessing with constructed response in an authentic way! Marie Heath 2014 Page 2 What is constructed response assessment? A constructed response assessment is one that asks students to produce an answer, not just select one. Here are some characteristics of a constructed response assessment: • Constructed response, as its name implies, asks students to construct, create, or, in other words, “do” something to build their answer to a question or problem. • Constructed response does not ask students to choose a correct answer form several possibilities, match terms with definitions, or decide whether a statement is true or false. • Constructed response assessment may take longer for students to complete than selected response, but the benefits will be worth it. • The justification for a constructed response assessment can take many forms: Written justification Model Labeled diagram Graph Computation Explanation Graphic organizer Connection to Prior Learning Diagram Formula Marie Heath 2014 Quotes from the text Timeline Page 3 Why use constructed response assessments? Constructed response assessment is a quality learning opportunity for students and a more authentic assessment tool for teachers. The importance of constructed response assessment: • Constructed response entails students to write what they know which writing in the content area increases understanding and helps support student learning. • Writing what you know demands thinking and writing is an active process. Students are not able to remain passive. • It requires higher level thinking to respond to the question. • When requiring students to justify their response, it activates metacognition and students learn how to “think about their thinking.” • It organizes and clarifies a student’s thoughts. • Writing will increases retention because an additional learning mode has been employed. • Students are not just listening; they are also kinesthetically involved • Helps students gain total picture of concept or process. • Constructed response enables the teacher to see immediately who doesn’t understand and what isn’t understood. • Teachers are able to see where the breakdown may be in learning whether it is the skill, process, or vocabulary. • Using CR provides students with less test items, but with deeper thinking and understanding of the concepts. Marie Heath 2014 Page 4 How to create your own constructed response assessments. A constructed response question should focus on the standard being tested to produce a clear understanding of what students have learned Strand/Text Type: The strand should be identified to help students to have a clear construct and to create a schema of the organization of the content area. Response: • • Student’s response to the question Students use the author’s words from the question to respond • Student must respond in complete sentences • Response should be clear and simple • Mastery of the skill is identified Justification: • • • • • • • Marie Heath 2014 Student’s justification explaining why their response is correct Students must give specific evidence to justify their response Students my include reflections from teachings from the classroom or real world knowledge Students make connections from prior learning Graphic representations can be used to demonstrate understanding Mastery of the process is identified through students justification If processes are used, they can be demonstrated through formulas, computations, graphs, etc. Page 5 CR Products The following are some ideas of products that students may use in their justification section of their constructed response. This list is not all-inclusive, but it will give you some ideas when considering the type of skills you want your students to demonstrate. • • • • • • • • • • Written explanation to justify their response Venn Diagrams for comparisons and classifications Lists to organize information with a related topic Labeled diagram to display understanding of whole-to-part relationships Solution to a problem to demonstrate the process used in solving the problem • Model to give a visual representation of ideas and concepts • Formulas used to calculate a response • Word associations to provide understanding of vocabulary words. Can be in the form of synonyms, antonyms, cognates, and for ELL students, occasional native language words to clarify understanding if very limited in their English. Reflections from previous instruction pertaining to the topic Connections to real world events Graphs to summarize data Timeline to display information chronologically Cause and Effect to identify the results from an event Marie Heath 2014 Page 6 Constructed Response Development Process Clearly identify the concept or skill you want to assess. Decide what you want your students to “construct”. What will the justification include? Ex. Written response, labeled graph, diagram, etc. Whenever possible, Design a real-world prompt connected to your students’ lives to engage them and provide a context for the assessment. Create the task you will ask your students to complete. Be clear as to what the product should look like and include. Ex. Text, stimulus, graph, chart, excerpt, etc. Create a scoring rubric which will include the points for skill demonstrated in the constructed response as well as points for their justification/process used to construct a response. Score the responses using the rubric focusing on the skill and process standard being tested. Grammar and lack of vocabulary to express ideas should not be counted off unless it is part of the assessed skill. If students construct an incorrect answer, they should be allowed to review and correct their response while providing an explanation to their erroneous thinking. This enables students the opportunity to utilize the highest cognitive level of thinking (evaluation) while giving students the opportunity to gain deeper understanding of the concept. Marie Heath 2014 Page 7 Constructed Response Template Write the standard you will be assessing? What are the key concepts your students will need to understand for this response? Is there a process they will need to use to justify their response? What are the steps needed to demonstrate understanding of the process? 1. 2. 3. 4. 5. 6. What are the key vocabulary words that need to be used to demonstrate understanding of the concept? • • • • • • Determine how many points will be assigned to the different levels of proficiency and areas. Strand: Correct__________ Incorrect_________ Constructed Response: Advanced ______________ Satisfactory_____________ Unsatisfactory___________ Marie Heath 2014 Evidence/Justification/Process: Advanced______________ Satisfactory_____________ Unsatisfactory___________ Page 8 Scoring Constructed Responses To prevent scoring of student’s constructed response from becoming subjective, a rubric is needed to set the criteria. Rubrics should address the concepts and skills you wanted to assess with the given students tasks. The rubric is an important part of the constructed response assessment for several reasons: 1. Unlike selected response, true/false, matching, or multiple choice assessments, a constructed response will not always be clearly right or wrong. A well-designed rubric makes the job of scoring CR assessments quicker and easier. The more specific the rubric, the less room there will be for subjective assessing. 2. You can never predict all possible responses and with some responses it will be difficult to determine whether the student really understands the concept or not. Creating a rubric with various possible student responses will help to identify knowledge of the concept. 3. Points can be varied according to the area of focus. More or less points can be weighed in either the response section or the justification section according to the level of cognition, understanding, and verbal communication of the concept being tested. Marie Heath 2014 Page 9 Steps to Creating a CR Rubric Consider the concept, skill, and/or standard being tested. Category section: Students should be able to identify the category/strand in which they are being assessed. Response section: (Level 0-4) Decide what important content ideas you want your students to know and express in their response. Create a list of acceptable and possible responses. Justification/Evidence section: (Level 0-Advanced High) Consider the thinking process that will be needed to provide evidence of mastery of the concept. Will it only be a written explanation or will a skill be included to demonstrate a process. Decide how many total points the CR will be worth overall and how many of the possible correct responses the student must include for an advanced score, a satisfactory score, and an unacceptable score. It is recommended to use the STAAR performance descriptors in your content area as a guide to the different point values assigned. (http://www.tea.state.tx.us/student.assessment/staar/performance-standards/) Marie Heath 2014 Page 10 Constructed Response Proficiency Levels LEVEL Advanced High This response shows understanding of the content, question, and/or problem. The response is insightful, integrates knowledge, and demonstrates powerful application. The application shows powerful evidence of higher order thinking skills. Concepts are accurate and well supported. There are no misconceptions. The response is comprehensive. LEVEL Advanced This response shows some understanding of the content, question, and/or problem. The response includes appropriate application that demonstrates evidence of higher order thinking skills. The application shows some evidence of higher order thinking skills. Concepts are accurate and supported. There are no interfering misconceptions. The response may not develop all parts equally. LEVEL Satisfactory The response shows knowledge of the content, question, and/or problem. The response is acceptable with some key ideas. The response shows little evidence of application. The response includes some basic ideas. The response provides little or no support. There are minimal misconceptions. LEVEL Unsatisfactory This response shows minimal knowledge of the content, question, and/or problem. The response is related to the question, but is inadequate. The response includes incomplete or fragmented ideas or knowledge. There may be significant misconceptions. LEVEL 0 The response is completely incorrect or irrelevant. There may be no response. Marie Heath 2014 Page 11 What is constructed response? From the words of Ha, a former 6th grade student… “My teacher Ms. Heath taught me an incredible way of answering questions without answer choice. She told me it was simple. You have to respond your answer then you have to tell how do you know that’s the answer or where did you get that answer. All of Ms. Heath student do it. It’s called constructive response. First you have to read the story (any story). Second you read the question (make sure you understand it). Third you answer the question in a complete sentence by starting using the author’s words. Forth you have to tell why is that the answer, where did you get the answer or how did you know if that your correct answer to your question for your evidence. That is how I do it to pass my test every day. I get a lot of high grades because of the constructive response.” H T Bibliography Heath, Marie, 2013. Constructed Response for Higher Thinking. TxTESOL, Dallas, Tx. Heath, Marie.; Pflug, M. 2013 Eliminating the Achievement Gap. UTA, Arlington, Tx. Heath, Marie. 2013 Closing the Achievement Gap in the Content Areas. CAST, Converence, Desoto, Tx. Heath, Marie. 2013 Student Achievement through Constructed Response. TCTELA Conference. Dallas, Tx. Heath, M.; Pflug, M,. 2013. Constructed Response in Science. CAST Institute. Desoto, Tx. Kilgo, Margaret. 2011-12. Data Driven Decisions Reading. Reading Workshop, Austin, Tx. Kilgo, Margaret. 2012-2013. Assessment Workshop Reading. Reading workshop, Austin, Tx. Maryland Department of Education School Improvement Site. Spencer, Pete. Constructed Response in Science Toolkit. Michigan. Tankersley, Karen. 2007. Tests that teach: Using standardized tests to improve instruction. ASCD Marie Heath 2014 Page 12 Chapter 7 Adapting Materials for English Language Learners Please note: The term limited English proficient (LEP) and English language learner (ELL) are both used to describe students who are not native speakers of English. The term ELL is more frequently used in recent literature in the field and as such is used more frequently in this guide. However, the terms are interchangeable. The published materials available to a teacher are never wholly adequate. All groups of learners are unique, with different needs and interests that cannot be met by a single textbook written for a generalized audience. As such, teachers find that they must go searching for the materials that they need or create new ones. This is especially true of teachers in blended classrooms, where the texts used are often not entirely appropriate for ELLs. While creating new materials from time to time is probably unavoidable, it is not reasonable to expect a teacher to create all the class texts (that is, readings, handouts, worksheets, etc.) himself. Previous chapters in this series have explored how CTE instructors can modify their teaching for a blended classroom. This chapter discusses how the instructor can modify course materials in order to make them more accessible - easier to read and understand for English language learners (ELLs). A short reading passage and its adaptation for ELLs are included as an example of the application of some strategies and techniques introduced here. Improve Readability CTE teachers can make some texts more accessible for ELLs without changing the wording at all. The techniques outlined can draw attention to key parts of the text, allowing ELLs to grasp the most important information, even though they may not be able to read it in its entirety. The techniques, which involve changing the appearance of UNT in partnership with TEA, Copyright ©. All rights reserved. 1 the text or changing the instruction and activities built around the text, are useful for readings that are too long to re-write or adapt in other ways. • Highlight key information Call ELLs’ attention to key words, concepts, and the main ideas of a text by highlighting them. Highlighting can be done with the use of different colored inks or by underlining, circling, italics, or bold type. • Preview vocabulary Identify the vocabulary in a passage that is the most important and/or the most difficult. Difficult words include those that are uncommon, particularly long, or very technical. In addition to highlighting these words, they can be glossed (defined) and discussed before students do the reading and reviewed afterwards. • Use pictures and props Find photos, illustrations, and items that can be used before and after a reading assignment to help clarify meaning for ELLs. Photos and illustrations can sometimes be found on the internet, in pamphlets, and in other textbooks. For example, for a reading on electrical safety, show pictures of the items and practices in the text or bring in the tools and safety items themselves to show and use in a demonstration. • Use graphic organizers and outlines Graphic organizers are visual devices that teachers use to make words and concepts more easily accessible for all learners, not just ELLs. These organizers can be used before a reading to help students brainstorm answers to a question related to the text and to access their background knowledge on a topic. They can be used during a reading to make concepts clearer or after a reading as well to review and expand upon the concepts in the text. Graphic organizers include timelines, flow charts, Venn diagrams, and word webs. Teachers can find free and customizable graphic organizers on Web sites for educators such as the following: http://www.teachervision.fen.com/graphic-organizers/printable/6293 UNT in partnership with TEA, Copyright ©. All rights reserved. 2 • Increase “white space,” font size, and use of headings Even if you do not have the time to change the words in a text, you can make it more accessible by changing its formatting. If you have an electronic copy of the reading, chunk the parts of the reading into small sections with bolded headlines. Leave more empty space between each section and between the lines of the text. This is often referred to as “white space,” and it can help a text look less dense and more inviting for ELLs, as well as help them more quickly locate and take in main ideas. If the font used in the text is small, make it a little larger. • Tape the text ELLs learn best when the same information is presented in different ways. Consider asking a native English-speaking student to read a passage out loud into a recorder (or do it yourself) and provide tapes along with the text for ELLs to take home. Adapt Texts Short texts can be adapted for the ELLs in a blended classroom. In this sense, to adapt means to change the wording of the text in order to make it easier to read. It is important to note that adaptation simplifies the language in a text, but not the content. ELLs need access to grade-level content. Adaptation increases this access without simplifying the concepts contained in the text. Instructors are encouraged to adapt only those texts that are relatively short (between 1-5 paragraphs) and central to the themes of the lesson. The following strategies can guide the CTE instructor in modifying important class texts for ELLs. • Identify main ideas and key words In preparation for adapting a text, read it a few times. Identify its main ideas and the vocabulary that is key to those main ideas. Underline these or list them separately as points of reference – these items must be clear and salient in the adapted version of the text. It is also recommended that you write an outline of your adaptation before UNT in partnership with TEA, Copyright ©. All rights reserved. 3 you begin to write to ensure that ideas flow logically and that all important points are covered. • Be consistent The structure of the adapted passage should be clear and consistent. It is recommended that you begin each paragraph or segment with a clear topic sentence and follow that with a few supporting sentences that provide more detail. This practice can provide important predictability and clarity for the ELL. • Shorten sentences Whenever possible, break long, complex sentences into two or more sentences. Make sentences short and straightforward and eliminate unnecessary words or clauses. For example, consider the following sentence about first aid for burns: After the burn becomes cooler, apply a moisturizer or aloe vera gel to the burn to reduce scratching and itching and to keep the skin moist. This sentence could be broken up into three shorter sentences without any loss of meaning: Let the burn cool. Then, put on a moisturizer or aloe vera gel. Moisturizer or gel will keep the skin moist (a little wet) and reduce scratching and itching. • Simplify grammar Grammar can also be simplified. To some extent, this may happen naturally when you shorten sentences. In addition, use simple tenses (e.g., I ate vs. I have eaten), difficult-to-define words like nevertheless and insomuch, and the active voice (e.g., Tom ate the apple.) instead of the passive voice (e.g., The apple was eaten by Tom). • Simplify vocabulary Replace long, uncommon, or otherwise difficult words with shorter, simpler words when possible. For example, ascertain could be replaced with find out; obstruct could be replaced with stop. Other words that are not central to the meaning of the passage can be omitted altogether. UNT in partnership with TEA, Copyright ©. All rights reserved. 4 • Rephrase complex ideas If a sentence or passage is too difficult to superficially modify, you may need to completely rewrite it. If that’s necessary, don’t lose the main idea of the original text. • Clarify In addition to simplifying and eliminating words and sentences, you may also have to add some clarifying language as well. Where necessary, add examples to illustrate a point or give the meaning of a word in parentheses. • Consider text appearance When adapting a text, do not forget to utilize the previously mentioned techniques for improving the appearance and accessibility of a text. Incorporate plenty of “white space,” create bolded headings for different sections of the text, and increase the font size slightly for better readability. Example of Text Adaptation The following is a brief passage on first aid that might appear in a book or article used in class. After that, an adaptation of this passage is presented. As you read both texts, try to identify the strategies that were used in the adaptation process. Original Text: A bystander is often the first to encounter an emergency situation. If you are the first to reach an ill or injured individual, immediately apply the basics of first aid. First, if another person is present, ask him to call or locate help. If you are not trained in first aid and another person is not present, you may decide that you need to leave the injured party and seek help. If you stay, ascertain that the victim is out of danger without putting yourself in harm’s way. If the victim must be moved, do so with extreme care. Otherwise, refrain from moving him, due to the possibility of spinal injury. Next, if you are trained to do so, apply basic first aid: clear the victim’s airway, check for respiration, and perform rescue breathing. Then, check for circulation and, if necessary, perform CPR. If the victim is bleeding profusely, apply pressure to control it as much as possible. UNT in partnership with TEA, Copyright ©. All rights reserved. 5 Adapted Text: First Aid Part A. Instructions: Discuss the meanings of the following key words with your teacher: injured first aid hazards to clear pressure Part B. Instructions: If you see a person who is injured (hurt) or sick, apply first aid: • Get help first Call 911 or send another person for help. If you are alone, decide if you should leave to get help. • Look around for hazards Make sure the victim (the hurt person) is out of danger. Make sure you are safe, too. • Do not move the victim. Do not move the victim. He may have injuries to his neck or spine. If you need to move the victim out of danger, move him very carefully. = spine UNT in partnership with TEA, Copyright ©. All rights reserved. 6 • Apply first aid If you are trained, apply first aid: - CLEAR the airways (the nose, mouth, and throat). Make sure nothing is blocking the victim’s breathing. • - CHECK for breathing. - DO rescue breathing. - CHECK for circulation. - DO CPR, if necessary. Apply pressure If the victim is bleeding a lot, apply pressure (press down) to control it. Other Strategies In addition to the changing the appearance or the wording of difficult texts, instructors can employ other strategies to increase ELLs’ access to course content. • Decrease the amount of work Once you have identified the main ideas of a lesson, consider lessening the work load on ELLs. For example, ask them to read a little less than their native speaking classmates or do a little less homework, while making sure that the reading and homework they are required to do contain the main ideas and key information from the lesson. • Increase opportunities for practice, discussion, and review While you may decrease the amount of input for ELLs, you can increase the ELLs’ opportunities to interact with the text and the new ideas and information it contains. Create more opportunities to practice the new information learned in the text, through UNT in partnership with TEA, Copyright ©. All rights reserved. 7 class or small group discussions, review exercises, debates, role plays, demonstrations, and other activity types. • Work with an ESL teacher ESL teachers and other school staff trained to work with ELLs can be excellent resources to the CTE instructor. Take advantage of opportunities to work with these professionals. They can provide assistance and guidance in the adaptation of materials, suggest or even help design activities to help ELLs practice and review the content of texts, and may even have materials to share. Conclusion ELLs needs access to the same content as their English-speaking peers in a blended classroom. However, the complex or technical language in which classroom texts are written can be a barrier to that access. Through adaptations that change the appearance and/or wording of a text, as well as through the instruction and appropriate activities, the CTE instructor can help ELLs understand and learn important course content. UNT in partnership with TEA, Copyright ©. All rights reserved. 8 References Burkart, G.S., & Sheppard, K. (2002). Content-ESL across the USA: A training packet. Washington, DC: Center for Applied Linguistics. Carrasquillo, A.L., & Rodriguez, V. (1996). Language minority students in the mainstream classroom. Buffalo, NY: Multilingual Matters LTD. Echevarria, J., Vogt, M.E., & Short, D. (2004). Making content comprehensible to English language learners: The SIOP model (2nd. ed.). Boston: Allyn & Bacon. Gillespie, M.K. (1996). Learning to work in a new land: A review and sourcebook for vocational and workplace ESL. Washington, DC: Center for Applied Linguistics. Irujo, S. (2006). Please don’t tell me that I have to create all my own materials! Retrieved June, 3, 2009, from http://www.coursecrafters.com/ELLOutlook/2006/jan_feb/ELLOutlookITIArticle6.htm Keller, J.J., & Associates, Inc. (2000). Keller’s official OSHA construction safety handbook. Neenah, WI: J.J. Keller & Associates. Note: Clipart is courtesy of Discovery Education School’s Clipart Gallery and is available for use by teachers at http://school.discoveryeducation.com/clipart/category/hlth0.html and Teacher Files.com at http://www.teacherfiles.com/clip_art_symbols_signs.htm UNT in partnership with TEA, Copyright ©. All rights reserved. 9