Lighting the Way
The Georgia Formative Item Bank Sheds
Light on Student Performance Expectations
and Achievement
A Presentation by
Dr. Dawn H. Souter, Project Manager
Georgia Department of Education
Assessment and Accountability
1
Major reviews of research on the
effects of formative assessment
indicate that it might be one of
the more powerful weapons in a
teacher‘s arsenal.”
(Robert Marzano, 2006)
2
Purpose of Georgia’s Assessment Initiatives
• To provide assessment resources that reflect the
rigor of Georgia’s state-mandated content
standards
• To balance the use of formative and summative
assessments in the classroom
• To promote student learning
• To sustain implementation of Georgia’s rigorous
content standards
3
Inside the Formative Assessment Toolbox
• Development of a three-prong toolkit to support teachers and
leaders in promoting student learning
– An expansive bank of formative instructional assessment
items/tasks based on CCGPS in ELA and Mathematics as a
teacher resource - Phase I Release Fall 2012; Phase II release
Fall 2013
– A set of benchmark assessments in ELA, mathematics – Initial
Pilot Fall 2013
– An assessment literacy professional learning opportunity that
focuses on implementation of research-based formative
instructional practices (FIP) – Initial Pilot January/February
2013 with statewide launch 2013-2014
4
Formative Assessment Initiatives
Bringing a Balanced Assessment Focus to the Classroom
Phase I items
released into
OAS fall 2012;
Phase II items
to be released
in fall 2013
Formative
Item Bank
Assessment
Literacy
Professional
Learning
Benchmark
Assessments
Pilot in winter
2013; Statewide
launch in
summer 2013
Phase I item pilot in
fall 2013; Phase II pilot
in winter 2014
5
Using Formative Assessment
• Conducted during instruction (lesson, unit, etc.)
• Identifies student strengths and weaknesses
• Helps teacher determine next steps
– Review
– Differentiation
– Continuation
• Offers teachers immediate feedback to provide students with
detailed feedback
• Assessment for the purpose of improving achievement
• LOW stakes – no student performance reports generated
6
Formative Instructional Practices—
Formative Assessment in Action
Summative
Assessment
• Re-Design
• Teach
Formative
Assessment
• Re-Design
• Teach
Formative
Assessment/
Diagnostic
State-Mandated
Content
Standards
Formative
Assessment
• Design
• Teach
Formative
Assessment
• Re-Design
• Teach
Re-Design might involve changing activities, instructional techniques, assessment
methods or content, and/or differentiation based upon student needs.
7
The Georgia Formative Item
Bank
8
Purpose of the Formative Item
Bank
The purpose of the Formative Item Bank is to
provide items and tasks used to assess
students’ knowledge while they are learning
the state-mandated standards. The items will
provide an opportunity for students to show
what they know and show teachers what
students do and do not understand.
9
The Georgia Formative Item Bank
• Bank of over 1600+ classroom assessment items aligned with
the state’s content standards in ELA and Mathematics
• Created for exclusive use in Georgia classrooms
• Piloted with Georgia students
• Reviewed by Georgia educators
• Housed in the Georgia Online Assessment System (OAS)
• Preponderance of items at DOK 3 and 4
• Item, rubric and scored student sample papers provided
• Available to ALL Georgia Teachers!
– 700+ items piloted in spring 2012; currently in OAS
– 900+ new items piloted in spring 2013; available in OAS fall of 2013
10
Formative Item Bank Use
The Georgia Formative Item Bank can be used in order
to:
• Prepare students for the more rigorous expectations
of the state-mandated standards.
– To show these expectations, students must engage with a
variety of item formats beyond multiple-choice
• Provide students practice with open- and
constructed-response items
• Provide educators insight and access to evaluating
open-ended response items
11
The Nature of Georgia’s Formative Item
Bank
• Items created for Georgia educators as an
instructional resource to be used formatively during
instruction
– Provide information about student performance
throughout the academic year to inform instruction and
interventions
– Reteach, remediate, move forward, enrich
– Low stakes; grading discouraged
• Support Georgia educators foster learning with an informative
tool that keeps students, parents, administrators and
educators themselves informed of students’ current position
on the pathway to proficiency.
12
The Nature of Georgia’s Formative
Item Bank (continued)
• Aligned with state-mandated content
standards in English Language Arts (ELA) and
Mathematics, grades 3 – HS
• Various formats, but primarily constructed response, in order to measure the full
expectations of what students need to know
and be able to do to be on the trajectory of
exiting high school college- and career-ready
13
Item Content
• Georgia’s Content Standards
– Mathematics: Grades 3 – 8; high school Coordinate
Algebra, Analytic Geometry and Advanced Algebra
– English/Language Arts (including Reading): Grades 3 –
8; high school 9th and 10th grade literature and
American Literature
• Items aligned to multiple standards
– One primary standard
– One or more secondary standards
• Alignment verified by Georgia educators
14
Valuable Features of Formative
Items – Mathematics
• Items include intentional focus on assessing processes
used by students as well as the required content
• Items applied in real-world context
• Writing requirements, such as explanations and
reasoning
• Student responses on constructed-response
items/tasks
− make student knowledge and skills transparent to teachers
− illuminate student misconceptions
15
Item Formats
• Multiple Choice
• Primarily Constructed-response
– Extended Response
– Scaffolded
• Constructed-response items require students to
provide explanations/rationales, provide
evidence, and/or to show work
• Preponderance of items at DOK 3 and 4
• Provide teachers with evidence of true student
understanding of content and process
16
Multiple Choice Items
• Have four answer options
• Distractors (incorrect answers) should be
believable and represent common conceptual
and/or application errors
• Distractor rationales should assist teachers to
identify specific student misconceptions to
inform instruction
17
17
Extended Response Items
• May address multiple standards, multiple
domains, and/or multiple areas of the
curriculum
• May allow for multiple correct responses
and/or varying methods of arriving at a
correct answer
• Scored through use of a rubric and associated
student exemplars
18
Example of Extended Response Item
Math—Advanced Algebra
STEM: If the stem is to contain art, complete the art page provided. In the stimulus
Advanced
Algebra,
A.REI.2; A.REI.4; A.APR.6, A.REI.1; DOK 3
box below
place the artStandards
where applicable
Sreeja and Brandon solved the equation shown in different ways.
x 2  3x  2
 2x
x 1
Part A
Before solving the equation, what solution could Sreeja and Brandon identify as
extraneous? Explain your reasoning.
Part B
x 2  3x  2
2x

.
x 1
1
Demonstrate how Sreeja used the proportion to solve the equation. In each
step, explain the properties she used to determine the solution.
Sreeja solved the equation by creating the proportion
Part C
Brandon solved the equation by simplifying the left side of the equation first.
Demonstrate how Brandon simplified the expression on the left and then solved
the equation. In each step, explain the properties he used to determine the
solution.
Be sure to complete ALL parts of the task.
Write your answer and show your work on the paper provided.
Do NOT type your answer in the text box below.
19
Scaffolded Items
• Include a sequence of items or tasks
• Designed to demonstrate deeper
understanding
• May be multi-standard and multi-domain
• May guide a student to mapping out a
response to a more extended task
• Scored through use of a rubric and associated
student exemplars
20
Example of Scaffolded Item
Math—Analytic Geometry
Analystic STEM:
Geometry
Standards
G.SRT.6;
G.SRT.5;
DOK 4
If the stem is to
contain art, complete
the art page provided.
In the stimulus
box below place the art where applicable
In the figures shown,
KLM is similar to
PQR, with measurements as shown.
Part A
Write an equation to determine tan  L. Solve for the tangent. Explain your
answer.
Part B
Determine m L and m Q. Round your answers to the nearest tenth of a
degree. Show your work and explain your answer.
Part C
Use the Pythagorean theorem and corresponding sides of similar triangles to
determine the lengths of KL, PR, and PQ. Show your work.
Part D
Prove the measures of KL, PR, and PQ you found in Part C are correct by
using the trigonometric ratios relating to  L and  Q. Show your work or
explain your answer.
Be sure to complete ALL parts of the task.
Write your answer and show your work on the paper provided.
Do NOT type your answer in the text box below.
21
Rubrics
• Holistic
• 5-point scale (0 – 4)
– 4: Thoroughly Demonstrated
– 3: Clearly Demonstrated
– 2: Basically Demonstrated
– 1: Minimally Demonstrated
– 0: Incorrect or Irrelevant
22
Rubric
Score
4
Designation
Thoroughly
Demonstrated
3
Clearly
Demonstrated
2
Basically
Demonstrated
1
Minimally
Demonstrated
0
Incorrect or
irrelevant
Rubric
Description
The student successfully completes all elements of the
item by demonstrating knowledge and application of
the relation between the trigonometric function of an
angle and the side lengths of the triangle (G.SRT.6),
the use of trigonometric ratios and the Pythagorean
Theorem to solve right triangle problems (G.SRT.8),
and the similarity criteria for triangles (G.SRT.5). All
four parts are completed correctly with sufficient
explanations or work shown.
The student demonstrates clear understanding of the
standards listed above by completing all four parts, but
makes one or two minor calculation errors or omissions
Or
The student completes all four parts, but the
explanation or work shown in one part is insufficient
Or
The student correctly completes three of the four parts
with sufficient explanations or work shown.
The student demonstrates basic understanding of the
standards listed above by completing two of the four
parts with at least one sufficient explanation or work
shown.
The student demonstrates minimal understanding of
the standards listed above by completing one of the
four parts with or without sufficient explanation or work
shown.
The response is incorrect or irrelevant to the skill or
concept being measured.
23
Exemplar Papers
• Prototype answer – the “ideal” response
• Set of responses from actual Georgia students,
collected during item pilots
• Samples scored by trained raters using rubric
• Papers allow teachers to review and compare their
own students’ work to the sample responses for
each score point
– Helps standardize expectations of the standards
• Score point and annotations provided for each
sample item response
Note: The pilot was conducted using standard administration procedures in order to ensure that results
were comparable across the state. When items/tasks are used during instruction, these administration
rules do not have to apply and student results may vary; thus, teachers may want to modify the rubrics
and even raise expectations. Rubrics and exemplars should remain focused on high expectations.
24
Exemplar Paper--Excerpt
Exemplar
Part A
Step 2: f(1)  0, so x  1 is a factor, not x  1.
Step 3: (x  1) 6 x 2  x  12  6 x 3  5x 2  11x  12 , not 6 x 3  7 x 2  11x  12 .
Step 4: (3x  4)(2x  3)  6 x 2  17 x  12 , not 6 x 2  x  12 .


Part B
x  1 is a factor since f(1)  0. Student may use synthetic division or long
division. Work should be shown.
6x 2  x
 12
x  1 6x 3  7x 2  11x  12
6x 2  x  12  (2x  3)(3x  4), so
6x 3  7x 2  11x  12  (2x  3)(3x  4)(x  1).
Set each factor equal to 0 to determine the zeros/intercepts.
3
2
4
3x  4  0  3x  4  x 
3
2x  3  0  2x  3  x 
x  1  0  x  1
Part C
25
6 > 0, so as x  , f (x)   and as x  , f (x)  
Formative Item Bank
Pilot I Item Examples and
Results
Mathematics
26
Grade 6 Mathematics Item
Item from FIB Phase I
27
Grade 6 Mathematics
Exemplar and Sample Student Response
28
Grade 6 Mathematics Sample
Student Responses
29
Grade 6 Mathematics Item
•
•
•
•
Performance Data
Item Type: Extended Response
DOK: 3
Number of Students in Pilot: 87
Response Data
– 4’s:
– 3’s:
– 2’s:
– 1’s:
– 0’s:
0 for 0%
5 for 5.75%
34 for 39.08%
39 for 44.83%
9 for 10.34%
30
Grade 3 Mathematics Item
31
Grade 3 Mathematics Item
Performance Data
•
•
•
•
Item Type: Scaffolded
DOK: 3
Number of Students in Pilot: 44
Response Data
– 4’s:
– 3’s:
– 2’s:
– 1’s:
– 0’s:
5 for 11.36%
3 for 6.82%
18 for 40.19%
15 for 34.09%
3 for 6.82%
32
Overall Mathematics Pilot I (Spring 2012)
Summary Data
Grade
3
4
5
6
7
8
9-10
11-12
Number of students and percent falling into each score point
Total student
N/ %
0
1
2
3
4
771
667
373
81
36
1928
40.00%
34.60%
19.30%
4.20%
1.90%
100%
795
800
360
87
58
2100
37.90%
38.10%
17.10%
4.10%
2.80%
100%
548
513
252
124
44
1481
37.00%
34.60%
17.00%
8.40%
3.00%
100%
927
768
269
65
14
2043
45.40%
37.60%
13.20%
3.20%
0.70%
100%
896
632
243
62
11
1844
48.60%
34.30%
13.20%
3.40%
0.60%
100%
984
791
314
100
51
2240
43.90%
35.30%
14.00%
4.50%
2.30%
100%
798
697
186
45
27
1753
45.50%
39.80%
10.60%
2.60%
1.50%
100%
690
602
178
63
9
1542
44.70%
39.00%
11.50%
4.10%
0.60%
100%
33
Overall Mathematics Pilot II (Spring 2013)
Summary Data
Number and Percent of Students Achieving Each Score
Point
Grade
0
1
2
3
4
Total Student
N/%
3
1378
1152
539
121
47
3237
42.57%
35.59%
16.65%
3.74%
1.45%
100%
1323
1264
325
83
25
3020
43.81%
41.85%
10.76%
2.75%
0.83%
100%
1351
1049
391
64
15
2870
47.07%
36.55%
13.62%
2.23%
0.52%
100%
1579
1171
370
135
53
3308
47.73%
35.40%
11.19%
4.08%
1.60%
100%
1602
856
219
72
36
2785
57.52%
30.74%
7.86%
2.59%
1.29%
100%
1529
1049
619
217
88
3502
43.66%
29.95%
17.68%
6.20%
2.51%
100%
2570
1435
299
59
23
4386
58.60%
32.72%
6.82%
1.35%
0.52%
100%
4
5
6
7
8
9 - 12
Key Findings from Phase I Pilot
(Spring 2012)
• Overall performance shortfalls
–
–
–
–
–
–
Many students lacked organization and neatness
Don’t seem to understand what question was requiring
Don’t follow directions well
Didn’t answer all parts of questions
Don’t follow through with “units” in Mathematics answers
Don’t have keyboarding skills – future of testing is online
and there are ELA standards (W6) for keyboarding/word
processing in grades 4, 5, and 6.
35
Formative Item Bank
Phase II Examples
Mathematics
36
Grade 6
Mathematics
Item
Item from FIB Phase II
37
Coordinate Algebra Mathematics Task
38
Advanced Algebra Mathematics
Task
STEM: If the stem is to contain art, complete the art page provided. In the stimulus
box below place the art where applicable
Jocelyn is given 2 polynomial functions. They are
f(x)  6 x 3  7x 2  11x  12 and g(x)  2x 4  2x 3  6 x  8. Jocelyn needs
to find zeros of the function f(x). The work Jocelyn did is shown in the four
steps below.
Step 1: f(1)  6(1)3  7(1)2  11(1)  12  0  10  0
Step 2: f(1)  6(1) 3  7(1) 2  11(1)  12  0  0  0, so x  1 is a
factor
Step 3: f(x)  (x  1)(6x 2  x  12)
Step 4: f(x)  (x  1)(3x  4)(2x  3)
Part A
Identify any steps in which Jocelyn made an error. Explain your answer.
Part B
Find the zeros of the function f(x)  6 x 3  7x 2  11x  12. Show your work.
Part C
Sketch a graph of f(x). Include all intercepts. Show your work or explain your
answer.
Part D
Find the difference when f(x) is subtracted from g(x).
Be sure to complete ALL parts of the task.
Write your answer and show your work on the paper provided.
Do NOT type your answer in the text box below.
39
Advanced Algebra Task
STEM: If the stem is to contain art, complete the art page provided. In the stimulus
box below place the art where applicable
Jocelyn is given 2 polynomial functions. They are
f(x)  6 x 3  7x 2  11x  12 and g(x)  2x 4  2x 3  6 x  8. Jocelyn needs
to find zeros of the function f(x). The work Jocelyn did is shown in the four
steps below.
Step 1: f(1)  6(1)3  7(1)2  11(1)  12  0  10  0
Step 2: f(1)  6(1) 3  7(1) 2  11(1)  12  0  0  0, so x  1 is a
factor
Step 3: f(x)  (x  1)(6x 2  x  12)
Step 4: f(x)  (x  1)(3x  4)(2x  3)
Part A
Identify any steps in which Jocelyn made an error. Explain your answer.
Part B
Find the zeros of the function f(x)  6 x 3  7x 2  11x  12. Show your work.
Part C
Sketch a graph of f(x). Include all intercepts. Show your work or explain your
answer.
Part D
Find the difference when f(x) is subtracted from g(x).
Be sure to complete ALL parts of the task.
Write your answer and show your work on the paper provided.
Do NOT type your answer in the text box below.
40
FIB Items in OAS
• 700+ Phase I items already in Georgia Online
Assessment System (OAS)
• 900+ Phase II items will be loaded into OAS in
fall of 2013
• Same OAS log-in as used in the past
– If you need an OAS log-in access code,
• Contact your School Administrator for OAS log-in access
code
• School Administrators should contact their system test
coordinator for assistance if needed
41
Where do you Find the Items?
www.georgiaoas.org
42
Searching in the OAS
You create
test name
and ID that
are
meaningful
to you.
Naming Idea:
“Formative” and
Domain Name, such
as literary
comprehension
43
Searching in the OAS
44
Searching the OAS for Formative Items
Example Search
The name you created
in previous steps
Drop down to select subject and grade
level
Drop down to select domain and
standard (or select all)
All of the formative items are in Level 2 of the OAS which means all teachers have access.
45
Searching the OAS for Formative Items
Search Steps
1. Create Test (Test Name and Test Identifier, submit)
2. Item Level (Select Level 2, which provides access to
all Georgia teachers)
3. Subject (Select Language Arts or Mathematics)
4. Grade Level (Select grade 3 – 8)
5. Domain ( Select Formative)
6. Domain (Select Appropriate Domain or ALL, i.e.
Information and Media Literacy)
7. Standard (Select Standard or ALL)
8. Show Items
46
Finding the Rubrics and Sample
Student Papers
Once you create a test in the previous steps:
• Assign the test to a class
• Assign the test to a student in the class
• Take the test as if you were a student
• Click on “Score the Test”
• At that point, you will be able to see the
rubrics and student work samples
(Screen shots are unavailable at this time due to annual repairs being made to
the OAS.)
47
Formative Item Bank Information
On-line
For more
information
about the
Formative Item
Bank Project
www.georgiaoas.org
48
Formative Item Bank Information On-line
http://www.gadoe.org/Curriculum-Instruction-and-Assessment/Assessment/Pages/OAS-Resources.aspx
Includes:
• About the Formative Item
Bank (document)
• About the Formative Item
Bank (presentation)
• Student Checklist for ELA
• Students Checklist for
Mathematics
• Link to the OAS
• Link to Georgia Standards.org
49
Upcoming Benchmark Assessments
50
Purpose of the Benchmark Assessments
The purpose of the benchmark assessments is to
provide a tool for teachers to use throughout
the year in support of classroom instruction and
frequent or continual evaluation of student
proficiency, keeping students, parents,
administrators and educators informed of
students’ progress on the pathway to
proficiency.
51
Interim Benchmark Assessment
Item and Benchmark Development
• Benchmark Assessments will be created for:
– ELA: Grades 1, 2, 3, 4, 5, 6, 7, 8, 9, 10 and 11
– Mathematics: Grades 1, 2, 3, 4, 5, 6, 7, 8, Coordinate
Algebra, Analytic Geometry and Advanced Algebra
– Social Studies: U.S. History
– Science: Biology
• Additional items that will be housed in an item bank
will be created for:
– Science: Grades 3-8
– Social Studies: Grades 3-8
52
Benchmark Assessment
Production Schedule
Phase I
Fall 2013 pilot
Available Winter 2014
• ELA
– Grades 1, 2, 3, 6, 7, 8,
and 10
• Mathematics
– Grades 1, 2, 3, and
Coordinate Algebra
• U.S. History
Phase II
Winter 2014 pilot
Available Fall 2014
• ELA
– Grades 4, 5, 9, and 11
• Mathematics
– Grades 4, 5, 6, 7, 8,
Analytic Geometry, and
Advanced Algebra
• Biology
53
Comparison to Formative Item
Bank
•
•
•
•
•
Formative Item Bank
ELA and Mathematics
Grades 3 – High School
Item Bank only
Housed in OAS Level 2 (for
teacher-use)
Variety of item types
– Mostly constructed response
Benchmark Assessments
• ELA, Mathematics, Science and
Social Studies
• Grades 1 – High School
• Item Bank and one benchmark
test form per grade/subject
• Housed in OAS Level 3 (for
district-use)
• Variety of item types
– Mostly multiple-choice and
short-answer
54
On the Horizon for Georgia’s
Formative Assessment Initiatives
• Phase II Formative Item Bank loaded into
OAS—Fall 2013
• Phase I and Phase II Interim Benchmark
Assessments loaded into OAS, projected
Spring and Fall 2014
• Formative Instructional Practices (FIP)
professional learning, will be available
summer 2013 and implementation in school
systems TBD by each school system
55
“Quality assessment is a system of
assessing what students know and
are able to do in a manner that
garners accurate information from
students for the purpose of
improving learning.” (Rick
Stiggins, 2008)
56
Questions
57
Georgia Department of Education
Assessment and Accountability
Dr. Melissa Fincher
Associate Superintendent
Assessment and Accountability
404.651.9405
mfincher@doe.k12.ga.us
Dr. Melodee Davis
Director
Assessment Research and Development
404.657.0312
medavis@doe.k12.ga.us
Michael Huneke
Assessment Specialist, OAS Manager
404.232.1208
mhuneke@doe.k12.ga.us
Dr. Dawn Souter
Project Manager, RT3
404.463.6667
dsouter@doe.k12.ga.us
Dr. Jan Reyes
Assessment Specialist, RT3
Interim Benchmark Assessments
404.463.6665
jreyes@doe.k12.ga.us
Kelli Harris-Wright, Ed.S.
Assessment Specialist, RT3
Assessment Literacy Professional
Learning
404.463.5047
kharris-wright@doe.k12.ga.us
58