Martha Thurlow and Laurene Christensen
National Center on Educational Outcomes
CEC Preconvention Workshop #4
April 21, 2010
 Martha
Thurlow
 Laurene Christensen
 Introduce yourself –
 Name
 State
 One more thing about you
2




Review five-step process for monitoring
accommodations decision making and
implementation
Identify strategies for improving the use of
accommodations in instruction and
assessment
Share tools and techniques for checking on
decision making and use
Discuss how to address validity issues related
to accommodations
3
Accommodations – Changes in materials or
procedures that enable students to
meaningfully access instruction and
assessment. Assessment accommodations do
not change the construct that is being
measured.
Accommodations mediate the effects of a
student’s disability and do not reduce learning
expectations.
4
Modifications – Changes in materials or
procedures that enable students to access
instruction and assessment. Assessment
modifications do change the construct that is
being measured.
Modifications create challenges for
assessment validityAccommodations mediate
the effects of a student’s disability and do not
reduce learning expectations.
5
Another way to talk about this ------Accommodations remove construct irrelevant
variance – that is, variability that may be
systematic (e.g., the effect of the disability), but
that is not what the test is intended to measure.
Modifications may also remove construct relevant
variance – that is, variability that may be
systematic (e.g., the effect of the disability) – but in
this case, it is also related to what the test is
intended to measure.
6
Accommodations are intended to
produce valid measures of what a
student knows and is able to do. To
determine whether each is valid requires
relating the accommodation to the
standards and content that is being
assessed.
7
Standard accommodations ?
Nonstandard accommodations ?
Conditional accommodations ?
Adaptations ?
Other terms ?
Share your terminology –
and what might be confusing
about the terms.
8
Binder with agenda, powerpoints,
resources, and exercises
 Accommodations Monitoring Tool,
with examples, checklists, additional
resources, and glossary
 Accommodations Manual, with tools
 Hints and Tips
 Accommodations Video
 Flash Drive with materials

9





Analyses of accommodation policies (19922009) and effects (1999-2008)
Accommodations for computer-based tests
(2002)
Accommodations manuals projects (2005, 2009)
Principled Approach to Accountability
Assessments
Peer review project
◦ Hints and Tips Tool
◦ Technical Report on Peer Review of Accommodations
10
1. Selection of accommodations
2. Linking of instructional and assessment
accommodations
3. Monitoring accommodations
4. Accommodations use allows for valid
and meaningful scores
Big Issue that Emerged:
Monitoring accommodations
11







Developed in collaboration with CCSSO
Reviewed extensively by ASES SCASS group
Based on a review of publicly available
materials
5 Step process
Each step includes examples, a checklist,
additional resources
Appendices include additional examples from
states
Extensive glossary at the end
12
Federal Laws that Pertain to Monitoring
Accommodations
◦ IDEA
◦ ESEA
What is monitoring?
13

IDEA (Individuals with Disabilities
Education Act)
◦ Requires students with disabilities who need
accommodations to have them
◦ Document accommodations on an IEP
◦ Monitor the educational results for students
◦ Ensure States follow through with the
requirements of IDEA
Nothing specific about monitoring
accommodations.
14

ESEA (Elementary and Secondary
Education Act)
◦ Requires accommodations as necessary in
order to measure student achievement
◦ Requires accommodations use to result in a
valid score
Monitoring is identified in the standards
and assessment peer review materials.
15
So, what is meant by:
“monitoring accommodations”?
16
Monitoring accommodations is a
process of evaluating policies,
procedures, and practices in order to
ensure that a state’s assessment
system is inclusive of students with
disabilities and English language
learners.
17


The State has analyzed the use of
specific accommodations for different
groups of students with disabilities and
has provided training to support sound
decisions by IEP teams.
The State routinely monitors the extent
to which test accommodations are
consistent with those provided during
instruction.
But, it is about more than compliance . . .
18

Systematic attention to the provision of
accommodations can ensure that
students are able to show what they
know and can do
19





Training for decision-making teams
Evaluating IEP team decisions about
accommodations
Tracking accommodations students use
for instruction and assessment
Tracking whether the use of
accommodations results in a valid score
for the student
Reporting the use of accommodations
20
21
Break!
22




Keeping track of training
Keeping track of IEP decisions on
accommodations
Are accommodations used for
instruction consistent with those
used for assessments?
Keeping track of requests for
unique accommodations
23
Questions to ask:
 What kinds of training on
accommodations are offered by the state
or district?
 Who attends the trainings?
 How are materials made available to
participants and others who might need
them?
 Are materials available online?
24
IEP Team Making
Decisions about
Accommodations
25
Your ideas?
 Summarize on Flip Chart

26


Are there model IEP forms for
schools to use?
Does the IEP form reference
current state policy in some
way so that teams are aware of
the consequences of their
decisions?
27


Do IEP forms indicate instructional and
assessment accommodations separately?
Does the electronic data system (if one exists)
allow for matching instructional and
assessment accommodations?
28
Documentation of accommodations on
IEPs?
 Connection between instructional and
assessment accommodations?
 Ease of access to accommodations
information?

29
30
1. Do decision-making teams know how to make
special requests for accommodations?

Do they know who to contact? Do they know
what information should be provided to help the
state make a determination?
2. Is there a clear process in place so that teams
know how and when decisions on special
requests will be made?
3. How are special requests tracked?
31
32
33

What we have done so far:




Background
Step 1 – Know rules and regulations
Step 2 – Document decisions about
accommodations
What is still to come:
 Step
3 – Document use of accommodations
 Step 4 – Review accommodations decisions and
use
 Step 5 – Evaluate and report on
accommodations
34



Documenting accommodations on
student demographic sheets
Documenting inappropriate use of
accommodations on test day
Documenting that students received their
IEP accommodations on test day
35




What information is asked for?
How are accommodations listed?
Are invalidating accommodations
(modifications) noted on the bubble sheet?
What about “Other” accommodations?
36
Information about the Student
37
Information on accommodations
38
Accommodations – Provided or Used?
39
Bubble Sheets:
 Strengths
 Needed
Improvements
40

Testing irregularity forms

Testing certification forms
 How are these forms used?
 How are they reviewed? What is done with the
information?
41
42
43




Direct observation on test day
Record reviews
On-site visits on a day other than test
day
Interviews with students, teachers, and
administrators about the use of
accommodations
44



Is there an established process?
How are schools chosen?
Are visits announced or unannounced?
◦ Are there requirements to prepare in advance?
Example: Arkansas
45

How will records be reviewed?

How will records be selected?

How will information on accommodations be
tracked?
46
Monitoring Questions from South Dakota
1. Are the accommodations/modifications appropriate for the skill
area affected by the disability (no oral testing for math disability)?
Yes
No
If no, example:
2. Are the accommodations identified in the IEP for state and districtwide assessment provided in their instructional program? (Do they
match?)
Yes
No
If no, example:
3. Were the accommodations identified in the IEP for state and districtwide assessment “USED” during assessment administration? (compare
the coding on the assessment data sheet with the assessment
accommodations listed in
the IEP)
Yes
No
If no, example:
4. If the student is identified as taking an alternative assessment, does
he or she meet the criterion and has it been documented on the IEP?
Yes
No
If no, example:
47



Can be for compliance, or for technical
assistance and professional development
May serve as a needs assessment
May or may not include interviews with
stakeholders, such as administrators,
teachers, and students
48
Utah
Example
49
South Carolina
Example
1.
What accommodations do you use for instruction?
How do they help you learn?
2. What accommodations do you use for assessments?
How do they help you do your best?
3. Do you have any questions about the
accommodations you use for instruction or
assessments?
4. How do your teachers make sure you have the
accommodations you need on test day?
5. Is there anything else about the accommodations
you use that you want to share?
51
1.
How do you ensure that students receive the necessary
accommodations as indicated on the IEP?
2.
How do IEP teams make decisions about what
assessment a student will be given (i.e., what data are
used to make the determination)?
3.
How do IEP teams make decisions about what
assessment accommodations a student needs? How do
you ensure that accommodations happen?
4.
How are families/parents involved in assessment and
accommodations selection?
5.
How is the provision of accommodations monitored by
the school?
52
1.
How do you ensure that students receive the necessary
accommodations as indicated on the IEP?
2.
How do IEP teams make decisions about what assessment a
student will be given (i.e., what data are used to make the
determination)?
3.
How do IEP teams make decisions about what assessment
accommodations a student needs? How do you ensure that
accommodations happen?
4.
How are families/parents involved in assessment and
accommodations selection?
5.
How is the provision of accommodations monitored by the
school?
6.
As an administrator, how do you ensure that decision- making
teams receive appropriate training on accommodations?
53
Arkansas Case Study:
Discussion
54
55
Break!
56

Analyze accommodations

Report on accommodations

Revise accommodations policies
57
Alternatives to a formal research study:
1. Document process of collecting information
on accommodations
2. Look at existing research on
accommodations
3. Conduct extant data analyses
4. Triangulation: formal lit review, expert
judgment, and empirical evidence
58
59
Hawai’i
Example
Discrepancy Analysis
60
Delaware
Example
Accommodations Report
61
Resource: NCEO Dataviewer
http://data.nceo.info
62
63
64
Next Steps!
65
Contact Information
Martha Thurlow: THURL001@umn.edu
Laurene Christensen: chri1010@umn.edu
National Center on Educational Outcomes
(NCEO):
www.nceo.info
66