EQuIP Rubric & Quality Review
Training Session:
ELA/Literacy Grades 3-12
1
Session Goals
Use the EQuIP quality review process to determine the quality and alignment of
lessons and units to the Common Core State Standards (CCSS) in English language
arts (ELA)/literacy
During this session, reviewers will:
• Develop their abilities to use EQuIP criteria to provide observations about CCSSaligned instructional materials and make suggestions for improvement
• Develop a common understanding of the EQuIP quality review process
• Develop a common understanding of the rating scale and descriptors for the four
rubric dimensions and the rating categories and descriptors for overall ratings
• Develop their abilities to use EQuIP criteria, rating scales and rating descriptors to
accurately rate instructional materials
2
EQuIP Quality Review:
Principles & Agreements
1. CCSS: Before beginning a review, all members of a review team are familiar with the
CCSS.
2. Inquiry: Review processes emphasize inquiry rather than advocacy and are
organized in steps around a set of guiding questions.
3. Respect & Commitment: Each member of a review team is respected as a valued
colleague and contributor who makes a commitment to the EQuIP process.
4. Criteria & Evidence: All observations, judgments, discussions and recommendations
are criterion and evidence based.
5. Constructive: Lessons/units to be reviewed are seen as “works in progress.”
Reviewers are respectful of contributors’ work and make constructive observations
and suggestions based on evidence from the work.
6. Individual to Collective: Each member of a review team independently records
his/her observations prior to discussion. Discussions focus on understanding all
reviewers’ interpretations of the criteria and the evidence they have found.
7. Understanding & Agreement: The goal of the process is to compare and eventually
calibrate judgments to move toward agreement about quality with respect to the
CCSS.
3
EQuIP Quality Review:
Process & Dimensions
EQuIP Quality Review Process
The EQuIP quality review process is a collegial process that centers on the use of
criteria-based rubrics for English language arts (ELA)/literacy and mathematics. The
criteria are organized into four dimensions:
The Four Dimensions
1. Alignment to the depth of the CCSS;
2. Key shifts in the CCSS;
3. Instructional supports; and
4. Assessment.
As educators examine instructional materials against the criteria in each dimension, they
are able to use common standards for quality and generate evidence-based commentary
and ratings on the quality and alignment of materials.
4
Using the Electronic Quality Review Rubric
PDF Form
5
Using the Quality Review Rubric PDF Form
For each dimension:
•
Select the checkbox for each
criterion for which clear and
substantial evidence is found.
•
Make observations and
suggestions related to criteria
and evidence.
•
Determine a rating for each
dimension based on checked
criteria and observations.
For Dimension I:
•
Use alignment rating to
determine whether to
proceed with review.
6
Giving Feedback
Writing effective feedback is vital to the EQuIP Quality Review Process. Below are the four
qualities of effective feedback.
•
Criteria-based: Written comments are based on the criteria used for review in each
dimension. No extraneous or personal comments are included.
•
Evidence Cited: Written comments suggest that the reviewer looked for evidence in the
lesson or unit that address each criterion of a given dimension. Examples are provided
that cite where and how the criteria are met or not met.
•
Improvement Suggested: When improvements are identified to meet criteria or
strengthen the lesson or unit, specific information is provided about how and where such
improvement should be added to the material.
•
Clarity Provided: Written comment are constructed in a manner keeping with basic
grammar, spelling, sentence structure and conventions.
Example 1: ELA/Literacy
This unit clearly targets two standards, which are noted in the overview on p. 3. The
activities throughout the unit correspond to this list of targeted standards. There are
possibilities for alignment with other supporting standards, but the focus chosen by
the developer is clear. The purpose of instruction is clearly stated and the unit
contains multiple and well integrated opportunities for speaking, listening, reading
and writing via discussion, worksheets and close readings. After reviewing the texts,
it is clear that they are sufficient quality, complexity and scope for the purpose of
the unit to make claims based on textual evidence. Too, the unit provides
opportunities to build the students’ content knowledge and their understanding of
reading and writing in social studies by weaving in reading from primary sources for
the Women’s Suffrage movement.
Is this feedback criteria-based?
Was evidence cited?
Was there an improvement suggested?
Is clarity provided?
Example 1: Feedback
Criteria-based: Yes
Evidence Cited: Yes
Improvement suggested: Partial
Clarity Provided: Yes
•
This is effective feedback. The reviewer mentions two standards and cites their evidence,
“which are noted in the overview.” It would be more effective if the reviewer named the
standards specifically.
•
The reviewer continues to use the alignment criteria to provide effective feedback about
why the instructional materials meet the criteria: “The purpose of instruction is clearly
stated and the unit contains multiple and well integrated opportunities for speaking,
listening, reading, and writing via discussion, worksheets and close reading.”
•
The single improvements mentioned, “ there are possibilities for other alignment,” could
be more effective if it suggested at least one of those possibilities.
•
The written comments are constructed in a manner in keeping with basic grammar,
spelling, sentence structure and conventions.
Example 2: ELA/Literacy
The lesson targets three standards, which are highlighted in the lesson cover page. The first
activity targets the first standard listed. The second activity to compare and contrast
character traits, although an instructive exercise, is not aligned to the standards listed and, in
this context, the instructional purpose of the compare and contrast activity is unclear. The
texts are very engaging and could be fun to do as a read-aloud as well.
Is this feedback criteria-based?
Was evidence cited?
Was there an improvement suggested?
Is clarity provided?
Example 2: Feedback
Criteria-based: Partial
Evidence Cited: Partial
Improvement suggested: No
Clarity Provided: Partial
•
This could be more effective feedback. The reviewer mentions two standards and cites their
evidence. It would be more effective if the reviewer named the standards specifically. The reviewer
also mentions personal comments that are not related to the criteria.
•
The reviewer mentions the second activity and describes why it does not meet the criteria stating,
“although an instructive exercise, is not aligned to the standards listed and, in this context, the
instructional purpose of the compare and contrast activity is unclear. The texts are very engaging and
could be fun to do as a read-aloud as well,” but fails to offer the improvement that would help to
meet the criteria. A suggestion for improvement could be to revise the activity so that there is better
alignment to the standard.
•
The feedback could be more clear and the written comments are constructed in a manner in keeping
with basic grammar, spelling, sentence structure and conventions.
Quality Review Steps
Step 1. Review Materials
• Record the grade and title of the lesson/unit on the Quality Review Rubric PDF
• Scan to see what the lesson/unit contains and how it is organized
• Read key materials related to instruction, assessment and teacher guidance
• Study and measure the text(s) that serves as the centerpiece for the lesson/unit,
analyzing text complexity, quality, scope and relationship to instruction
Step 2. Apply Criteria in Dimension I: Alignment to the Depth of the CCSS
• Identify the grade-level CCSS that the lesson/unit targets
• Closely examine the materials through the “lens” of each criterion
• Indicate each criterion for which clear and substantial evidence is found
• Record input on specific improvements needed to meet criteria or strengthen alignment
• Compare observations and suggestions for improvement
12
Quality Review Steps
Step 3. Apply Criteria in Dimensions II–IV
• Examine the lesson/unit through the “lens” of each criterion
• Indicate each criterion met and record observations and feedback
When working in a group, individuals may choose to compare observations and suggestions
for improvement after each dimension or wait until each person has rated and recorded all
input for Dimensions II–IV.
Step 4. Apply an Overall Rating and Provide Summary Comments
• Individually review comments for Dimensions I–IV, adding/clarifying comments as
needed
• Individually write summary comments on the Quality Review Rubric PDF
When working in a group, individuals should record summary comments prior to
conversation.
Step 5. Compare Overall Ratings and Determine Next Steps
• Note the evidence cited to arrive at summary comments and similarities and differences
among reviewers. Recommend next steps for the lesson/unit and provide
recommendations for improvement to developers/teachers.
13
EXAMPLE:
Common Unit for Review — ELA/Literacy
Grade 8 — “Making Evidence-Based Claims”
This state is working with vendors to create full instructional units and, ultimately,
comprehensive, year-long curricular materials aligned to the CCSS. This state uses the
CCSS, the Tri-State Rubrics and the Publishers’ Criteria to guide development. Teachers
will be able to use these materials for classroom instruction or model their own planning
after these examples.
Note: This unit was submitted to EQuIP for review in October 2012 as an in-progress
work. The unit has been revised by the developers in response to feedback from the EQuIP
quality review process. Do not distribute the version of this unit used in EQuIP trainings.
See the revised lesson/unit on the Engage NY website:
www.engageny.org/resource/making-evidence-based-claims-units-ccss-ela-literacygrades-6-12.
14
EXAMPLE:
Step 1. Review Materials
• Record the grade and title of the lesson/unit on the Quality Review Rubric PDF
– Grade 8, “Making Evidence-Based Claims”
• Scan to see what the lesson/unit contains and how it is organized
–
–
–
–
–
Unit overview
Lessons with instructional notes and assessment opportunities
Checklists and rubrics
Student handouts
Model handouts
• Read key materials related to instruction, assessment and teacher guidance
– Unit outline (p. 2 of overview)
– Alignment (p. 3 of overview and pp. 1, 7, 15 and 15 of lesson)
– Assessment (p. 3 of overview and at the end of each lesson)
– Instructional notes (in each lesson)
15
EXAMPLE:
Step 1. Review Materials
• Study and measure the text(s) that serves as the centerpiece for the lesson/unit,
analyzing text complexity, quality, scope and relationship to instruction
–
–
–
–
“Ain’t I a Woman” measures 770L
“Equal Rights for Women” measures 1170L
“Wimbledon Has Sent me a Message” measures 1130L
The texts contain complex ideas, figurative and academic language, and distinct
points of view
16
Criteria for Dimension I: Alignment
to the Depth of the CCSS
The lesson/unit aligns with the letter and spirit of the CCSS:
1. Targets a set of grade-level ELA/literacy standards .
2. Includes a clear and explicit purpose for instruction.
3. Selects text(s) that measure within the grade-level text complexity band and are
of sufficient quality and scope for the stated purpose (i.e., present vocabulary,
syntax, text structures, levels of meaning/purpose, and other qualitative
characteristics similar to CCSS grade-level exemplars in Appendices A and B).
A unit or longer lesson should:
4. Integrate reading, writing, speaking and listening so that students apply and
synthesize advancing literacy skills.
5. (Grades 3–5) Build students’ content knowledge and their understanding of
reading and writing in social studies, the arts, science or technical subjects
through the coherent selection of texts.
11
Dimension Rating and Descriptive Scales To
Synthesize Judgment
Rating Scale for Dimensions I–IV:
3: Meets most to all of the criteria in the dimension
2: Meets many of the criteria in the dimension
1: Meets some of the criteria in the dimension
0: Does not meet the criteria in the dimension
Descriptors for Dimensions I–IV:
3: Exemplifies CCSS Quality — meets the standard described by criteria in the dimension,
as explained in criterion-based observations
2: Approaching CCSS Quality — meets many criteria but will benefit from revision in
others, as suggested in criterion-based observations
1: Developing toward CCSS Quality — needs significant revision, as suggested in
criterion-based observations
0: Not representing CCSS Quality — does not address the criteria in the dimension
18
EXAMPLE:
Step 2. Apply Criteria in Dimension I: Alignment
The lesson/unit aligns with the letter and spirit of the CCSS:
 1. Targets a set of grade-level ELA/literacy standards.
 2. Includes a clear and explicit purpose for instruction.
 3. Selects text(s) that measure within the grade-level text complexity band and are of
sufficient quality and scope for the stated purpose (i.e., present vocabulary, syntax, text
structures, levels of meaning/purpose, and other qualitative characteristics similar to
CCSS grade-level exemplars in Appendices A and B).
A lesson or longer unit should:
 4. Integrate reading, writing, speaking and listening so that students apply and synthesize
advancing literacy skills.
 5. (Grades 3–5) Build students’ content knowledge and their understanding of reading
and writing in social studies, the arts, science or technical subjects through the coherent
selection of texts.
NOTE: Reviewers should be able to refer to evidence to support pattern of
checkboxes selected.
19
EXAMPLE:
Step 2. Apply Criteria in Dimension I: Alignment
Observations/Feedback and Rating
This unit clearly targets two standards on p. 4 of the overview. The activities throughout the
unit correspond to this list of targeted standards (e.g., p. 8). There are possibilities for other
alignment, but the focus chosen by the developer is clear.
The texts measure 770L, 1130L and 1170L. After reading the texts, it is clear that they are of
sufficient quality and scope for the purpose of the unit — making claims based on textual
evidence. The texts also cohere around the theme of women’s rights.
The purpose of instruction is clearly stated, and the unit contains multiple and well-integrated
opportunities for speaking, listening, reading and writing (discussion, worksheets, close
readings).
Rating: 3
Approaching CCSS Quality — meets many criteria but will benefit from revision in
others, as suggested in criterion-based observations
20
EXAMPLE:
Step 2. Apply Criteria in Dimension I: Alignment
Compare Criterion-Based Checks, Observations and Feedback
• What is the pattern within our team in terms of the criteria we have checked?
• Do our observations and feedback reference the criteria and evidence (or lack of
evidence) in the instructional materials?
• Does our feedback include suggestions for improvement(s)?
21
Criteria for Dimension II:
Key Shifts in the CCSS
The lesson/unit addresses key shifts in the CCSS:
1. Reading Text Closely: Makes reading text(s) closely, examining textual evidence and
discerning deep meaning a central focus of instruction.
2. Text-Based Evidence: Facilitates rich and rigorous evidence-based discussions and
writing about common texts through a sequence of specific, thought-provoking and textdependent questions (including, when applicable, questions about illustrations, charts,
diagrams, audio/video and media).
3. Writing from Sources: Routinely expects that students draw evidence from texts to
produce clear and coherent writing that informs, explains or makes an argument in
various written forms (notes, summaries, short responses or formal essays).
4. Academic Vocabulary: Focuses on building students’ academic vocabulary in context
throughout instruction.
Criteria for Dimension II:
Key Shifts in the CCSS
A unit or longer lesson should:
5. Increasing Text Complexity: Focus students on reading a progression of complex texts
drawn from the grade-level band. Provide text-centered learning that is sequenced,
scaffolded and supported to advance students toward independent reading of complex
texts at the college- and career-ready level.
6. Building Disciplinary Knowledge: Provide opportunities for students to build knowledge
about a topic or subject through analysis of a coherent selection of strategically
sequenced, discipline-specific texts.
7. Balance of Texts: Within a collection of grade-level units a balance of informational and
literary texts is included according to guidelines in the CCSS (p. 5).
8. Balance of Writing: Include a balance of on-demand and process writing (e.g., multiple
drafts and revisions over time) and short, focused research projects, incorporating digital
texts where appropriate.
23
Dimension Rating and Descriptive Scales To
Synthesize Judgment
Rating Scale for Dimensions I–IV:
3: Meets most to all of the criteria in the dimension
2: Meets many of the criteria in the dimension
1: Meets some of the criteria in the dimension
0: Does not meet the criteria in the dimension
Descriptors for Dimensions I–IV:
3: Exemplifies CCSS Quality — meets the standard described by criteria in the dimension,
as explained in criterion-based observations
2: Approaching CCSS Quality — meets many criteria but will benefit from revision in
others, as suggested in criterion-based observations
1: Developing toward CCSS Quality — needs significant revision, as suggested in
criterion-based observations
0: Not representing CCSS Quality — does not address the criteria in the dimension
24
EXAMPLE:
Step 3. Apply Criteria in Dimension II: Key Shifts in the CCSS
The lesson/unit addresses key shifts in the CCSS:
 1. Reading Text Closely: Makes reading text(s) closely, examining textual evidence and
discerning deep meaning a central focus of instruction.
 2. Text-Based Evidence: Facilitates rich and rigorous evidence-based discussions and
writing about common texts through a sequence of specific, thought-provoking and textdependent questions (including, when applicable, questions about illustrations, charts,
diagrams, audio/video and media).
 3. Writing from Sources: Routinely expects that students draw evidence from texts to
produce clear and coherent writing that informs, explains or makes an argument in
various written forms (notes, summaries, short responses or formal essays).
 4. Academic Vocabulary: Focuses on building students’ academic vocabulary in context
throughout instruction.
25
EXAMPLE:
Step 3. Apply Criteria in Dimension II: Key Shifts in the CCSS
A unit or longer lesson should:
 5. Increasing Text Complexity: Focus students on reading a progression of complex texts
drawn from the grade-level band. Provide text-centered learning that is sequenced,
scaffolded and supported to advance students toward independent reading of complex
texts at the college- and career-ready level.
 6. Building Disciplinary Knowledge: Provide opportunities for students to build
knowledge about a topic or subject through analysis of a coherent selection of
strategically sequenced, discipline-specific texts.
 7. Balance of Texts: Within a collection of grade-level units a balance of informational
and literary texts is included according to guidelines in the CCSS (p. 5).
 8. Balance of Writing: Include a balance of on-demand and process writing (e.g., multiple
drafts and revisions over time) and short, focused research projects, incorporating digital
texts where appropriate .
26
EXAMPLE:
Step 3. Apply Criteria in Dimension II: Key Shifts in the CCSS
Observations/Feedback and Rating
The unit is structured throughout to support students in reading texts closely. The questions
and activities make text the focus of instruction and develop a clear lesson progression that
develops skills and causes students to search for textual evidence. The texts work well
together to deepen content knowledge about the women’s rights movement.
As students move from the EBC worksheets to writing a developed claim, they engage in a
process of writing their ideas and refining and developing these ideas over time as they read
several texts.
Importantly, there are missed opportunities to develop academic vocabulary.
Rating: 2
Approaching CCSS Quality — meets many criteria but will benefit from revision in others, as
suggested in criterion-based observations
27
EXAMPLE:
Step 3. Apply Criteria in Dimension II: Key Shifts in the CCSS
Compare Criterion-Based Checks, Observations/Feedback and Rating
• What is the pattern within our team in terms of the criteria we have checked?
• Do our observations and feedback reference the criteria and evidence (or lack of
evidence) in the instructional materials?
• Do our ratings correspond to the rating and descriptors on the rubric?
28
Criteria for Dimension III:
Instructional Supports
The lesson/unit is responsive to varied student learning needs:
1. Cultivates student interest and engagement in reading, writing and speaking about texts.
2. Addresses instructional expectations and is easy to understand and use.
3. Provides all students with multiple opportunities to engage with text of appropriate
complexity for the grade level; includes appropriate scaffolding so that students directly
experience the complexity of the text.
4. Focuses on challenging sections of text(s) and engages students in a productive struggle
through discussion questions and other supports that build toward independence.
5. Integrates appropriate supports in reading, writing, listening and speaking for students
who are English language learners, have disabilities or read well below the grade level
text band.
6. Provides extensions and/or more advanced text for students who read well above the
grade level text band.
29
Criteria for Dimension III:
Instructional Supports
A unit or longer lesson should:
7. Include a progression of learning where concepts and/or skills advance and deepen over
time (may be more applicable across the year or several units).
8. Gradually remove supports, requiring students to demonstrate their independent
capacities (may be more applicable across the year or several units).
9. Provide for authentic learning, application of literacy skills, student-directed inquiry,
analysis, evaluation and/or reflection.
10. Integrate targeted instruction in such areas as grammar and conventions, writing
strategies, discussion rules, and all aspects of foundational reading for grades 3–5.
11. Indicate how students are accountable for independent reading based on student choice
and interest to build stamina, confidence and motivation (may be more applicable across
the year or several units).
12. Use technology and media to deepen learning and draw attention to evidence and texts
as appropriate.
30
Dimension Rating and Descriptive Scales To
Synthesize Judgment
Rating Scale for Dimensions I–IV:
3: Meets most to all of the criteria in the dimension
2: Meets many of the criteria in the dimension
1: Meets some of the criteria in the dimension
0: Does not meet the criteria in the dimension
Descriptors for Dimensions I–IV:
3: Exemplifies CCSS Quality — meets the standard described by criteria in the dimension,
as explained in criterion-based observations
2: Approaching CCSS Quality — meets many criteria but will benefit from revision in
others, as suggested in criterion-based observations
1: Developing toward CCSS Quality — needs significant revision, as suggested in
criterion-based observations
0: Not representing CCSS Quality — does not address the criteria in the dimension
31
EXAMPLE:
Step 3. Apply Criteria in Dimension III: Instructional Supports
The lesson/unit is responsive to varied student learning needs:
 1. Cultivates student interest and engagement in reading, writing and speaking about
texts.
 2. Addresses instructional expectations and is easy to understand and use.
 3. Provides all students with multiple opportunities to engage with text of appropriate
complexity for the grade level; includes appropriate scaffolding so that students directly
experience the complexity of the text.
 4. Focuses on challenging sections of text(s) and engages students in a productive
struggle through discussion questions and other supports that build toward
independence.
 5. Integrates appropriate supports in reading, writing, listening and speaking for students
who are English language learners, have disabilities or read well below the grade level
text band.
 6. Provides extensions and/or more advanced text for students who read well above the
grade level text band.
32
EXAMPLE:
Step 3. Apply Criteria in Dimension III: Instructional Supports
A unit or longer lesson should:
 7. Include a progression of learning where concepts and/or skills advance and deepen
over time (may be more applicable across the year or several units).
 8. Gradually remove supports, requiring students to demonstrate their independent
capacities (may be more applicable across the year or several units).
 9. Provide for authentic learning, application of literacy skills, student-directed inquiry,
analysis, evaluation and/or reflection.
 10. Integrate targeted instruction in such areas as grammar and conventions, writing
strategies, discussion rules, and all aspects of foundational reading for grades 3–5.
 11. Indicate how students are accountable for independent reading based on student
choice and interest to build stamina, confidence and motivation (may be more applicable
across the year or several units).
 12. Uses technology and media to deepen learning and draw attention to evidence and
texts as appropriate.
33
EXAMPLE:
Step 3. Apply Criteria in Dimension III: Instructional Supports
Observations/Feedback and Rating
The unit is likely to cultivate student interest and engagement as it communicates a clear
purpose to students, uses engaging texts, and provides a mix of teacher modeling, pair/group
work and independent work that will support student success with the unit’s activities and
tasks.
The unit is exemplary in its commitment to having all students experience the text directly
and targeting difficult/critical sections. Graphic organizers are used strategically within the
instructional sequence to both support and assess students who may struggle with the
reading. The quality of the questions as well as the clear progression toward independent
discussion and writing are evident.
The unit would benefit from additional guidance about how to support students who are
learning English and students with disabilities, as well as students who are performing above
grade level. Consider including opportunities for students to make additional reading choices
of their own.
Rating: 2
Approaching CCSS Quality — meets many criteria but will benefit from revision in others, as
suggested in criterion-based observations
34
EXAMPLE:
Step 3. Apply Criteria in Dimension III: Instructional Supports
Compare Criterion-Based Checks, Observations/Feedback and Rating
• What is the pattern within our team in terms of the criteria we have checked?
• Do our observations and feedback reference the criteria and evidence (or lack of
evidence) in the instructional materials?
• Do our ratings correspond to the rating and descriptors on the rubric?
35
Criteria for Dimension IV:
Assessment
The lesson/unit regularly assesses whether students are mastering standards-based content
and skills:
1. Elicits direct, observable evidence of the degree to which a student can independently
demonstrate the major targeted grade-level CCSS standards with appropriately complex
text(s).
2. Assesses student proficiency using methods that are unbiased and accessible to all
students.
3. Includes aligned rubrics or assessment guidelines that provide sufficient guidance for
interpreting student performance.
A unit or longer lesson should:
4. Use varied modes of assessment, including a range of pre-, formative, summative and
self-assessment measures.
36
Dimension Rating and Descriptive Scales To
Synthesize Judgment
Rating Scale for Dimensions I–IV:
3: Meets most to all of the criteria in the dimension
2: Meets many of the criteria in the dimension
1: Meets some of the criteria in the dimension
0: Does not meet the criteria in the dimension
Descriptors for Dimensions I–IV:
3: Exemplifies CCSS Quality — meets the standard described by criteria in the dimension,
as explained in criterion-based observations
2: Approaching CCSS Quality — meets many criteria but will benefit from revision in
others, as suggested in criterion-based observations
1: Developing toward CCSS Quality — needs significant revision, as suggested in
criterion-based observations
0: Not representing CCSS Quality — does not address the criteria in the dimension
37
EXAMPLE:
Step 3. Apply Criteria in Dimension IV: Assessment
The lesson/unit regularly assesses whether students are mastering standards-based content
and skills:
 1. Elicits direct, observable evidence of the degree to which a student can independently
demonstrate the major targeted grade-level CCSS standards with appropriately complex
text(s).
 2. Assesses student proficiency using methods that are unbiased and accessible to all
students.
 3. Includes aligned rubrics or assessment guidelines that provide sufficient guidance for
interpreting student performance.
A unit or longer lesson should:
 4. Use varied modes of assessment, including a range of pre-, formative, summative and
self-assessment measures.
38
EXAMPLE:
Step 3. Apply Criteria in Dimension IV: Assessment
Observations/Feedback and Rating
The unit activities elicit observable evidence of students’ abilities to cite textual evidence
while they are reading, speaking and writing through the Making EBCs worksheet and the
final written claim that students produce.
The unit includes several tools to guide the interpretation of student performance including
the Making EBCs Checklist, the Text Discussion Checklist and an Evidence-Based Writing
Checklist. The notes labeled “Assessment Opportunities” provide additional guidance.
The unit assesses students using both writing and speaking.
Rating: 3
Exemplifies CCSS Quality — meets the standard described by criteria in the dimension, as
explained in criterion-based observations
39
EXAMPLE:
Step 3. Apply Criteria in Dimension IV: Assessment
Compare Criterion-Based Checks, Observations and Feedback
• What is the pattern within our team in terms of the criteria we have checked?
• Do our observations and feedback reference the criteria and evidence (or lack of
evidence) in the instructional materials?
• Does our feedback include suggested improvement(s)?
40
Overall Rating and Summary Comments
Overall Rating for the Lesson/Unit:
E: Exemplar –
Aligned and meets most to all of the criteria in dimensions II, III, IV (total 11 – 12)
E/I: Exemplar if Improved –
Aligned and needs some improvement in one or more dimensions (total 8 – 10)
R: Revision Needed –
Aligned partially and needs significant revision in one or more dimensions (total 3 – 7)
N: Not Ready to Review –
Not aligned and does not meet criteria (total 0 – 2)
41
EXAMPLE:
Step 4. Apply an Overall Rating and Provide Summary Comments
E/I: Exemplar if improved
This unit is well aligned, but the instructional supports need improvement. The
worksheets and questions are particularly well designed. However, it is not clear
how the students learning English or students with disabilities might
successfully engage in the activities. The developers might also consider ways
to streamline the instructional notes for ease of use.
42
EXAMPLE:
Step 5. Compare Summary Comments and Determine Next Steps
Guiding questions to synthesize criterion-based observations and suggestions:
• How do the observations and suggestions for improvement compare?
• Are the observations and suggestions criterion-based?
• Does this example serve as a model of CCSS instruction? What are its strengths?
Areas for improvement?
43
Reflection on Session Goals
• Did we use the EQuIP criteria to frame and explain evaluation of evidence found
in instructional materials?
• Did we develop a common understanding of EQuIP criteria among reviewers?
• Are there any criteria or evidence about which reviewers disagree?
• Did we develop reviewers’ abilities to use EQuIP criteria, rating
scales/categorizations and rating descriptors to accurately rate instructional
materials?
• To what degree were there differences among reviewers when checking criteria,
assigning dimension ratings and assigning overall ratings? What do you think
caused these differences?
44
Achieve
www.achieve.org
1400 16th Street, NW / Suite 510
Washington, DC 20036
45
Appendix:
Slides for reviewing a lesson
46
How To Use
The slides in this PowerPoint are currently set up to review the unit “Making EvidenceBased Claims.” To prepare for a review of the lesson “Part 1: Understanding EBCs – ‘Out of
Kilter’” use the following slides that reference the lesson in the place of previous slides that
reference the unit.
47
EXAMPLE:
Common Lesson for Review — ELA/Literacy
Grade 8 — “Ain’t I a Woman”
This state is working with vendors to create full instructional units and, ultimately,
comprehensive, year-long curricular materials aligned to the CCSS. This state uses the
CCSS, the Tri-State Rubrics and the Publishers’ Criteria to guide development. Teachers
will be able to use these materials for classroom instruction or model their own planning
after these examples.
Note: This lesson was submitted to EQuIP for review in October 2012. The lesson has been
revised by the developer in response to feedback from the EQuIP quality review process.
See the revised lesson/unit on the Engage NY site: www.engageny.org/resource/makingevidence-based-claims-units-ccss-ela-literacy-grades-6-12.
48
EXAMPLE:
Step 1. Review Materials
• Record the grade and title of the lesson/unit on the Quality Review Rubric PDF
– Grade 8, “Ain’t I a Woman”
• Scan to see what the lesson/unit contains and how it is organized
–
–
–
–
–
Overview
A lesson with instructional notes and assessment opportunities
Checklists and rubrics
Student handouts
Model handouts
• Read key materials related to instruction, assessment and teacher guidance
− Unit outline (p. 1-7)
− Alignment (p. 8)
− Instructional notes (p. 9–12)
− Assessment (p. 12)
− Text- “Ain’t I a Woman”
− Student handouts (Forming EBCs, EBC Checklist, Making EBCs)
49
EXAMPLE:
Step 1. Review Materials
• Study and measure the text(s) that serves as the centerpiece for the lesson/unit,
analyzing text complexity, quality, scope and relationship to instruction
– “Ain’t I a Woman” measures 770L
– The text contains complex ideas, figurative language and a distinct point of view
50
EXAMPLE:
Step 2. Apply Criteria in Dimension I: Alignment
The lesson/unit aligns with the letter and spirit of the CCSS:
 1. Targets a set of grade-level ELA/literacy standards.
 2. Includes a clear and explicit purpose for instruction.
 3. Selects text(s) that measure within the grade-level text complexity band and are of
sufficient quality and scope for the stated purpose (i.e., present vocabulary, syntax, text
structures, levels of meaning/purpose and other qualitative characteristics similar to
CCSS grade-level exemplars in Appendices A and B).
A unit or longer lesson should:
 4. Integrate reading, writing, speaking and listening so that students apply and synthesize
advancing literacy skills.
 5. (Grades 3–5) Build students’ content knowledge and their understanding of reading
and writing in social studies, the arts, science or technical subjects through the coherent
selection of texts.
*NOTE: Reviewers should be able to refer to evidence to support pattern of
checkboxes selected.
51
EXAMPLE:
Step 2. Apply Criteria in Dimension I: Alignment
Observations/Feedback and Rating
This lesson clearly targets one standard (RI8.1) and includes supporting standards (RI8.2,
RI8.6, SL8.1). The activities throughout the lesson correspond to this list of targeted and
supporting standards.
The lesson communicates a clear purpose for instruction.
The text measure of 770L is in the 4–5 grade-level text complexity band. However, the text
clearly is of sufficient quality and scope for the purpose of the unit — making claims based on
textual evidence. The text is also of cultural and historical significance.
Rating: 3
Exemplifies CCSS Quality — meets the standard described by criteria in the dimension as
explained by criterion-based observations
52
EXAMPLE:
Step 3. Apply Criteria in Dimension II: Key Shifts in the CCSS
The lesson/unit addresses key shifts in the CCSS:
 1. Reading Text Closely: Makes reading text(s) closely, examining textual evidence and
discerning deep meaning a central focus of instruction.
 2. Text-Based Evidence: Facilitates rich and rigorous evidence-based discussions and
writing about common texts through a sequence of specific, thought-provoking and textdependent questions (including, when applicable, questions about illustrations, charts,
diagrams, audio/video and media).
 3. Writing from Sources: Routinely expects that students draw evidence from texts to
produce clear and coherent writing that informs, explains or makes an argument in
various written forms (notes, summaries, short responses or formal essays).
 4. Academic Vocabulary: Focuses on building students’ academic vocabulary in context
throughout instruction.
53
EXAMPLE:
Step 3. Apply Criteria in Dimension II: Key Shifts in the CCSS
Observations/Feedback and Rating
The lesson is structured throughout to support students in reading text closely. The questions
and activities make text the focus of instruction and develop a clear lesson progression that
develops skills and causes students to search for textual evidence.
The text-dependent questions are well designed, and the instructional notes set a standard
for the richness and rigor that teachers should expect from student responses and dialogue.
There are opportunities for students to discuss and write about common texts in response to
these questions.
The lesson expects student to write from sources and draw evidence from the text. The
"Forming Evidence-Based Claims" graphic organizer is very thorough and specifically guides
students through the process of making a claim based on those ideas from the text.
There is little guidance about ways to develop academic vocabulary.
Rating: 2
Approaching CCSS Quality — meets many criteria but will benefit from revision in others, as
suggested in criterion-based observations
54
EXAMPLE:
Step 3. Apply Criteria in Dimension III: Instructional Supports
The lesson/unit is responsive to varied student learning needs:
 1. Cultivates student interest and engagement in reading, writing and speaking about
texts.
 2. Addresses instructional expectations and is easy to understand and use.
 3. Provides all students with multiple opportunities to engage with text of appropriate
complexity for the grade level; includes appropriate scaffolding so that students directly
experience the complexity of the text.
 4. Focuses on challenging sections of text(s) and engages students in a productive
struggle through discussion questions and other supports that build toward
independence.
 5. Integrates appropriate supports in reading, writing, listening and speaking for students
who are English language learners, have disabilities or read well below the grade level
text band.
 6. Provides extensions and/or more advanced text for students who read well above the
grade level text band.
55
EXAMPLE:
Step 3. Apply Criteria in Dimension III: Instructional Supports
Observations/Feedback and Rating
The lesson is likely to cultivate student interest and engagement as it communicates a clear
purpose to students, uses an engaging text, and provides a mix of teacher modeling,
pair/group work and independent work that will support student success with the lesson’s
activities.
The lesson is exemplary in its commitment to having all students experience the text directly
and targeting difficult/critical sections of text. Graphic organizers are used strategically
within the instructional sequence to both support and assess students who may struggle with
the reading. The quality of the discussion questions as well as the clear progression toward
independent discussion and writing are present in the unit.
While “Ain’t I a Woman” is a text that will support students reading well below the grade
level, supports for English language learners, students with disabilities or students reading
above grade level are not addressed.
Rating: 2
Approaching CCSS Quality — meets many criteria but will benefit from revision in others, as
suggested in criterion-based observations
56
EXAMPLE:
Step 3. Apply Criteria in Dimension IV: Assessment
The lesson/unit regularly assesses whether students are mastering standards-based content
and skills:
 1. Elicits direct, observable evidence of the degree to which a student can independently
demonstrate the major targeted grade-level CCSS standards with appropriately complex
text(s).
 2. Assesses student proficiency using methods that are unbiased and accessible to all
students.
 3. Includes aligned rubrics or assessment guidelines that provide sufficient guidance for
interpreting student performance.
A unit or longer lesson should:
 4. Use varied modes of assessment, including a range of pre-, formative, summative and
self-assessment measures
57
EXAMPLE:
Step 3. Apply Criteria in Dimension IV: Assessment
Observations/Feedback and Rating
The lesson activities elicit observable evidence of students’ abilities to cite textual evidence
while they are reading, speaking and writing through the Forming EBCs worksheet and the
final written claim that students produce.
The lesson assessments use both speaking and writing to assess students, but
accommodations for students learning English or with disabilities have not been identified.
Developers should consider additional guidance regarding accessibility during assessment.
The lesson includes several tools to guide the interpretation of student performance including
the Forming EBCs Checklist, the Text-Centered Discussion Checklist. The notes labeled
“Assessment Opportunities” provide additional guidance.
Rating: 2
Exemplifies CCSS Quality — meets the standard described by criteria in the dimension, as
explained in criterion-based observations
58
EXAMPLE:
Step 4. Apply an Overall Rating and Provide Summary Comments
E/I: Exemplar if improved
This lesson is a strong example of CCSS alignment and quality. This lesson is
well aligned, but the instructional supports need improvement. The
worksheets and questions are particularly well designed. However, it is not
clear how the students learning English or students with disabilities might
successfully engage in the activities. The developers might also consider
ways to streamline the instructional notes for ease of use.
59
60