Common Core State Standards: Opportunity for Reform or Same Old, Same Old…? P. David Pearson UC Berkeley February 2011 Acknowledgements Karen Wixson Standards and Assessment Sheila Valencia Assessment Freddy Hiebert Complexity My Relationship with CCS • Member of the Validation Committee • Background work on text complexity with a grant from Gates Foundation • Long (and occasionally checkered) history with standards going back to – – NBPTS: Standards for Teaching IRA/NCTE Standards • Research and development work on assessment Just to remind us College and Career Readiness Standards Common Core State Standards (grade by grade) Assessments to measure their mastery 10 recurring standards for College and Career Readiness Show up grade after grade In more complex applications to more sophisticated texts Across the disciplines of literature, science, and social studies Affordances of the CCS 1. 2. 3. 4. 5. 6. 7. An uplifting vision based on our best research on the nature of reading comprehension Focus on results rather than means Integrated model of literacy Reading standards complement cognitive theory and NAEP Elaborated theory of text complexity Shared responsibility (text in subject matter learning) Lots of meaty material in writing and language standards An exercise Take one of the CCR standards and trace it out across all the grade levels to see how it changes 1. An Uplifting Vision: ELA CCSS Students who meet the Standards readily undertake the close, attentive, reading that is at the heart of understanding and enjoying complex works of literature. They habitually perform the critical reading necessary to pick carefully through the staggering amount of information available today in print and digitally. They actively seek the wide, deep, and thoughtful engagement with high-quality literary and informational texts that builds knowledge, enlarges experience, and broadens world views. They reflexively demonstrate the cogent reasoning and use of evidence essential to both private deliberation and responsible citizenship in a democratic republic. 2. Focus on results rather than means Why? Leave a place for each lower level to add its own signature Some decisions about means really are local Appropriate role for a larger body politic Balance between our goals and our methods From the ELA Standards Document… By emphasizing required achievements, the Standards leave room for teachers, curriculum developers, and states to determine how those goals should be reached and what additional topics should be addressed. Thus, the Standards do not mandate such things as a particular writing process or the full range of metacognitive strategies that students may need to monitor and direct their thinking and learning. Teachers are thus free to provide students with whatever tools and knowledge their professional judgment and experience identify as most helpful for meeting the goals set out in the Standards.” 3. Integrated Model of Literacy Two views of integration Integrated Language Arts Integration between ELA and disciplines The CCSS are better on the interdisciplinary than on the ELA integration Corresponds to the actual uses to which reading and writing are put. Reading, writing, and language always serve specific purposes Reading and writing, not generically, But about something in particular The something in particular What reading, writing and language look like in a domain The information for a particular topic or unit or chapter The information in a particular text Language Arts Social Studies Science Mathematics Our current view of curriculum A model I like: Tools by Disciplines Academic Disciplines……….. Science Social Studies Mathe- Literature matics Reading Writing Language Early: Tools dominate Academic Disciplines……….. Science Social Studies Mathematics Literature Reading Writing Language Later: Disciplines dominate Academic Disciplines……….. Science Social Studies Mathe Literature matics Reading Writing Language Weaving is even a better metaphor than a matrix Writing Language Reading math literature Social studies Science 4. Comprehension Complements Other Important Efforts NAEP Rand view of Comprehension NAEP Locate and Recall Interpret and Integrate Critique and Evaluate Common Core Key ideas and details Craft and structure Integration of knowledge and ideas Range and level of text complexity CCSS NAEP Key ideas and details Locate and Recall Craft and structure Interpret and Integrate Integration of knowledge Critique and Evaluate and ideas Range and level of text complexity Complexity is specified but implicit not explicit Consistent with Cognitive Views of Reading Kintsch’s Construction-Integration Model Build a text base Construct a “situation” model Put the knowledge gained to work by applying it to novel situations. Consistent with Cognitive Views of Reading Kintsch’s Construction-Integration Model Locate andbase Recall What the text says Build a text Integrate and Interpret Construct a “situation” model What the text means the textitdoes Critique andgained Evaluate Put the knowledge to work What by applying to novel situations. These consistencies provide… Credibility Stretch Research “patina” 5. Elaborated Theory of Text Complexity Why text complexity? The gap for college and career readiness Jack Stenner’s (lexile guy) depiction of the 200 lexile gap 6. Shared Responsibility English and Subject Matter What we said before, reading and writing are always situated in a topic and a purpose. Knowledge fuels comprehension and writing. Reading and writing, along with experience and instruction, fuel knowledge. Reading and writing and language work better when they are “tools” for the acquisition of Knowledge Insight Joy Why sharing now? The gap for college and workplace readiness The increasing demands of an informational society Finally addressing a problem that has always been there Increasing awareness among disciplinary scholars April 23, 2010 edition of Science. 7. Lots of meaty material in writing and language All of the good vocabulary skills and content that we often claim for reading? As much of an issue for oral language and writing as for reading. Writing Media Argumentation: Claim-evidence-warrant Form follows function: we write with particular structures to achieve particular purposes As important for comprehension as it is for composition Constraints, Dilemmas, and Puzzles? Can we manage the text complexity issue? 2. How do we disarm the “We already do all this” stance? 3. How do we avoid a canon of texts? 4. What do we do about assessment? 1. Text Complexity Can we really make up the gap? If we are really honest, we’ll acknowledge that in our current “dumbed down” world, we have LOTS of kids who can’t handle the texts we currently give them What makes us think that we can up the ante without promoting even greater angst among students and teachers? Doesn’t text complexity have to be calibrated at an individual level? Independent-Instructional-Frustration level What are we going to do about text complexity in Grades K-3? Lexiles are highly unstable at prior to grade 3 Broaden our notions of Text Complexity— Appendix A Qualitative evaluation of the text Levels of meaning, structure, language conventionality and clarity, and knowledge demands Quantitative evaluation of the text Readability measures and other scores of text complexity Matching reader to text and task Reader variables (such as motivation, knowledge, and experiences) and task variables (such as purpose and the complexity generated by the task assigned and the questions posed) Grapes of Wrath (9-10 Complexity Band) Qualitative Measures Levels of Meaning There are multiple and often implicit levels of meaning within the excerpt and the novel as a whole. The surface level focuses on the literal journey of the Joads, but the novel also works on metaphorical and philosophical levels. Structure The text is relatively simple, explicit, and conventional in form. Events are largely related in chronological order. Language Conventionality and Clarity Although the language used is generally familiar, clear, and conversational, the dialect of the characters may pose a challenge for some readers. Steinbeck also puts a great deal of weight on certain less familiar words, such as faltering. In various portions of the novel not fully represented in the excerpt, the author combines rich, vivid, and detailed description with an economy of words that requires heavy inferencing. Knowledge Demands The themes are sophisticated. The experiences and perspective conveyed will be different from those of many students. Knowledge of the Great Depression, the “Okie Migration” to California, and the religion and music of the migrants is helpful, but the author himself provides much of the context needed for comprehension. Quantitative Measures The quantitative assessment of The Grapes of Wrath demonstrates the difficulty many currently existing readability measures have in capturing adequately the richness of sophisticated works of literature, as various ratings suggest a placement within the grades 2–3 text complexity band. A Coh Metrix analysis also tends to suggest the text is an easy one since the syntax is uncomplicated and the author uses a conventional story structure and only a moderate number of abstract words. (The analysis does indicate, however, that a great deal of inferencing will be required to interpret and connect the text’s words, sentences, and central ideas.) No matter how many indicators we have in place, teacher judgment will have to be used in particular cases. What we really need are even more instructional scaffolds, so we can answer the Reader-Task Considerations These are to be determined locally with reference to such variables question, under what conditions support as a student’s of motivation, knowledge, and experiences as well as purpose and the complexity of the task assigned and the questions can particular students read the text? posed. Recommended Placement Though considered extremely easy by many quantitative measures, The Grapes of Wrath has a sophistication of theme and content that makes it more suitable for early high school (grades 9–10), which is where the Standards have placed it. In this case, qualitative measures have overruled the quantitative measures. And we are going to need a whole new theory of text complexity for grades K-3? 2. How do we prevent the “threre’s nothing new” response? If educators do the mapping at a fairly general level, they will conclude that we already do all of these things. Almost any current set of state standards will map onto these standards at the 60-80% level, especially if we include the foundational skills. Have to examine the entirety of these standards, regard them as an integrated system of pedagogy. Start with the ELA Common Core State Standards (CCSS) Three main sections K−5 (cross-disciplinary) 6−12 English Language Arts 6−12 Literacy in History/Social Studies, Science, and Technical Subjects Shared responsibility for students’ literacy development Three appendices • • • A: Research and evidence; glossary of key terms B: Reading text exemplars; sample performance tasks C: Annotated student writing samples Karen Wixson Learn How to Read the ELA CCSS The standards are meant to be read as an integrated ELA program The Reading standards should be read with the complexity information in Appendix A and with the exemplary works that comprise each complexity band found in Appendix B The Writing standards should be read with the writing samples in Appendix C, which illustrate how good is good enough for each genre, grade by grade The Language standards should be read with the skills ladder in Appendix A which illustrates when skills should be introduced/mastered In sum, a standard “alignment” exercise should take into account not just the grade level standards alone, but also how the appendices help define these standards PLUS what comes before and after each grade band ELA CCSS 6-12 The opportunity of a lifetime… We are poised, with these standards in hand, to achieve integration both within the language arts and between ELA and the disciplines Do we have the chutzpah and commitment to take advantage of this moment? 3. The Textual Canon Dilemma The tyranny of the example: if it was good enough to illustrate the sort of thing we should be doing, then we should do it! No one will read this disclaimer… Given space limitations, the illustrative texts listed above are meant only to show individual titles that are representative of a wide range of topics and genres. (See Appendix B for excerpts of these and other texts illustrative of K–5 text complexity 4. The VAST unknown: CCSS and Assessment Assessments will make or break the CCSS movement This is where we decide whether the movement is Opportunity for reform Or Same old, same old If assessments are not changed, these standards will not make an iota of difference in teaching and learning The Players in the Assessment Game Partnership for the Assessment of Readiness for College and Careers: PARCC Smarter Balanced Assessment Consortium: SBAC. State Assessments NAEP Testing Industry Constraints Common Core Standards Assessment consortia frameworks Smarter Balanced Assessment Consortium (SBAC) Partnership for the Assessment of Readiness for College and Careers (PARCC) Different audiences and purposes (summative/formative/diagnostic) Affordances Learning progressions Computer adaptive testing Automated and distributed scoring Improved psychometric tools Through-Course, Interim/Benchmark Assessment Visions PARCC Signal & model good instruction Rich & rigorous performance tasks SBAC Empirically validate descriptions of learning progressions Through-Course Comprehension Assessments & Learning Progressions Should: reflect the interactive and multidimensional nature of comprehension assess readers’ abilities to understand, learn from, and use text to accomplish specific purposes provide transparent models of the demands of skilled reading across a range of grades, disciplines, tasks provide strong and informative predictors of success in college, careers, and K-12 From Pearson, Valencia, and Wixson An assumption/prediction? Whatever model we develop, it is likely to be a hybrid model. Item format Efficiency Multiple Choice Constructed Response Instructional Validity Performance Tasks Deeper Learning Passage issues Length and authenticity Disciplines—Literature, Science and History My focus Given ourvast experience with MC and CR, I’ll focus on performance tasks… Except to say that well developed theories of mc items, along with equally well-developed theories about classes of distractors, are really important to decision validity and the information value of test items. We need to learn something from each and every response a student makes, not just the right ones. Performance Tasks: Why bother? External validity College ready Career ready Curricular validity Powerful learning Deeper learning Consequential validity What curricular activities will it lead teachers and students toward? Compare the PARCC and SMARTER Proposals to ELA CCSS Two Salient Issues from the CCSS Text Complexity PARCC proposes to create a “text complexity diagnostic tool” SMARTER doesn’t consider directly Reading across the Disciplines PARCC addresses indirectly through sample items SMARTER makes general references, but nothing specific PARCC Attention to Discipline/Genre Through-Course Assessments (Interim/Benchmark) After roughly 25% of instructional time (ELA-1) After roughly 50% instructional time (ELA-2) After roughly 75% instructional time (ELA-3 Sample Extended Constructed Response Items for ELA-1 and ELA-2 Through-Course Assessments Measure the most fundamental capacity essential to achieving college and career readiness according to the CCSS: the ability to read increasingly complex texts, draw evidence from them, draw logical conclusions and present analysis in writing. ELA-1 & ELA-2 assessments include up to 2 extended constructed response items For ELA-3, students have extended time to identify or read relevant research materials and compose written essays. Students then publicly present the results of that research and writing, answering questions or engaging in debate, so teachers can assess their speaking and listening skills using common rubric Provide actionable data and useful models of student work teachers can use to plan and adjust instruction Sample Extended Constructed Response Items (taken from CCSS) 9-10 ELA, Informational--Students analyze how Abraham Lincoln in his “Second Inaugural Address” unfolds his examination of the ideas that led to the Civil War, paying particular attention to the order in which the points are made, how Lincoln introduces and develops his points, and the connections that are drawn between them. 11-12, ELA, Informational--Students delineate and evaluate the argument that Thomas Paine makes in Common Sense. They assess the reasoning present in his analysis, including the premises and purposes of his essay. Sample Extended Constructed Response Items (taken from CCSS) 11-12, ELA, Drama--Students compare two or more recorded or live productions of Arthur Miller’s Death of a Salesman to the written text, evaluating how each version interprets the source text and debating which aspects of the enacted interpretations of the play best capture a particular character, scene, or theme. 11-12, ELA Poetry--Students cite strong and thorough textual evidence from John Keats’s “Ode on a Grecian Urn” to support their analysis of what the poem says explicitly about the urn as well as what can be inferred about the urn from evidence in the poem. Based on their close reading, students draw inferences from the text regarding what meanings the figures decorating the urn convey as well as noting where the poem leaves matters about the urn and its decoration uncertain. Sample Extended Constructed Response Items (taken from CCSS) 11-12, Informational Texts: Science--Students analyze the concept of mass based on their close reading of Gordon Kane’s “The Mysteries of Mass” and cite specific textual evidence from the text to answer the question of why elementary particles have mass at all. Students explain important distinctions the author makes regarding the Higgs field and the Higgs boson and their relationship to the concept of mass Resources www.commoncore.org (not the “official” website, provides curriculum “maps”) Lee, C. D. & Spratley, A. (2010). Reading in the disciplines: The challenges of adolescent literacy. New York, NY: Carnegie Corp. Example of a New Standards Task from mid 1990s Man and His Message MLK 6-8 days, depending on class time Culminating task: write an essay based upon choosing one of several prompt options. Pearson Texts Encountered A video about the Civil Rights Movement entitled, A Time for Justice. An article about the Civil Rights Movement entitled, Confrontations. An article about Ghandi from Scholastic's SEARCH magazine. An oral rendition of King's Letter from a Birmingham Jail. Printed versions of other King speeches. An excerpt from a Time magazine account of the Rodney King riots in East Central Los Angeles. Two CNN video accounts of the riots: Rage of Despair and Roots of the Problem. Tasks Completed over the Period collaboratively complete separate cognitive webs on key concepts from the readings (Martin Luther King, Civil Rights Movement, Non-Violent Resistance). keep an ongoing log/chart of emerging learnings from all the different texts (written, oral, or video). answer straightforward "assignment-like" questions. compare the similarities and differences between King and Ghandi in a modified Venn diagram display. the culminating essay EVERYTHING can be scored Affordances Has the look and feel of powerful or deeper learning Engages students in workplace like behaviors, including social behaviors Expands our conceptualization of what counts as a text High capacity for engagement: interest and relevance Maps onto many of the Common Core Standards for reading in History Could build professional community of teachers around implementation and scoring Constraints Whose work is it anyway? The inevitable dilemma of collaboration Not just reading (video and audio texts) The usual suspects for performance tasks Task generalizability Scoring costs Domain coverage What counts for which standards Example of a MEAP Inspired Pilot Task circa 2000 for a Local Michigan District School-wide Comprehension Assessment Instructionally embedded (took a week out of the LA block) Multiple text Listening and reading Reliance on multiple choice questions Individual texts Cross texts Written Response to Reading Position taken in response to the prompt question Support from personal experience Support from texts Counts for both writing and reading comprehension depending on the rubric used Listening: Sister Anne’s Hands Multiple Choice Question Stems facts, relationships, inferences This story is mostly about… Sister Anne showed determination when she said… What did Sister Anne mean when she said, “For me, I’d rather open my door enough to let everyone in”? The children learned much from Sister Anne. This selection tells us that… Kate Shelly and the Midnight Express Multiple Choice Question Stems facts, relationships, inferences An important lesson of this story is… How are Kate and her mother different? In this selection, how do you know Kate showed determination and bravery when crossing the Des Moines River Bridge? Because Kate followed through, how would you predict she will face problems in the future? What dialogue does the author use to show you Kate has determination? How do you know this story takes place in the past? A Day’s Work Multiple Choice Question Stems facts, relationships, inferences By showing determination, Francisco… An important lesson from this selection is… In this selection, why did Francisco and Grandpa leave the weeds? This selection is not only about determination, it is also about… Why did the author have Grandpa and Francisco speak in Spanish? Cross Text Mult Choice Stems facts, relationships, inferences What important advice would both Grandpa and Kate give? In both reading selections you read about main characters who… How are Francisco and Kate different? How were the characters rewarded for showing determination and following through? Applying Ideas to a Task If you were trying to do something that was very hard, and you did not think you could get it done, would you keep trying or quit? Use examples from the two stories we read to support your decision. Scoring Answers questions by making connections bet ween readings and u sing ideas from both readings to support position taken Answers questions and u ses ideas from at least one story to support position taken. Answers question and refers to ideas in on e text Answers question or responds to theme Writing in Response to Reading Point Score 6 The student clearly and effectively chooses key or important ideas from each reading selection to support a position on the question and to make a clear connection between the reading selections. The point of view and connection are thoroughly developed with appropriate examples and details. There are no misconceptions about the reading selections. There are strong relationships among ideas. Mastery of language use and writing conventions contributes to the effect of the response. Affordances In the direction of powerful and deeper learning, but… Only one task for rubric-based scoring Pretty good coverage of a range of cognitive targets vis a vis question types. Constraints Does the reliance on MC format compromise its position vis a vis powerful and deeper learning? Limited to a single discipline—literature Limited to a single genre—narrative Limited to a single medium—text Looking Ahead Lots of dilemmas to manage Back to the future and déjà vu all over again Take advantage of new technologies, tools, and understandings Dilemmas to Manage Social nature of embedded tasks Domain coverage Enabling skills or just the big outcomes Dependence/independence across Standards/cognitive targets/items Issues of equity across populations, especially ELL and LD populations Déjà vu all over again Build on what worked Face the music on Intertask generalizability Scoring reliability and cost Take advantage of new tools and technologies Learning progressions (see SBAC) But they are hard and different in reading Discipline, topic, and text play a MAJOR role in shaping item difficulty We’ll just have to see how things scale in IRT models Computerized scoring, but… Easily corruptible Will clever kids learn how to school the systems? Computer adaptive testing Garbage in-Garbage out FINAL THOUGHT ABOUT ASSESSMENT The promise of the ELA CCSS will not be realized unless we create a new generation of reading assessments that capitalize on the knowledge gained in recent decades and the visions for the future. To Summarize Lots to Like Affordances of the CCS 1. 2. 3. 4. 5. 6. 7. An uplifting vision based on our best research on the nature of reading comprehension Focus on results rather than means Integrated model of literacy Reading standards complement cognitive theory and NAEP Elaborated theory of text complexity Shared responsibility (text in subject matter learning) Lots of meaty material in writing and language standards Lots to Worry about Constraints, Dilemmas, and Puzzles? Can we manage the text complexity issue? 2. How do we disarm the “We already do all this” stance? 3. How do we avoid a canon of texts? 4. What do we do about assessment? 1. Old Chinese Proverb May you live in interesting times…