Fall 2014 Test
Coordinator
Training
End of Course
Exams (EoCs)
1
Purpose of an EoC Exam
• Statewide curriculum alignment.
• Determining and measuring teacher and school
leader effectiveness—Educator Effectiveness System
(EES) (6.60.8 NMAC).
– The evaluation system must include measures of student
achievement growth (50 percent):
• SBA/PARCC—35%
• Department approved assessments—15%
• Providing an Alternative Demonstration of
Competency (ADC) in the academic content standards
required for graduation (6.19.7.7 NMAC).
Developing an EoC Exam
• Step 1: Identify the construct, domains, and
indicators to be measured – standards based
assessment.
– The knowledge, skills and abilities proficient students
should know and be able to do at the end of a course or
grade.
Developing an EoC Exam
• Step 2: Develop a Table of Specifications.
– Table of Specifications becomes the concrete
assessment plan for the EoC exam.
– What type of assessment is most appropriate for each
domain?
– What is the cognitive difficulty or cognitive demand for
each domain?
– How many items are necessary to comprehensively
assess the knowledge, skills and abilities for each domain
of the assessment?
Developing an EoC Exam
• Step 3: Develop the Assessment
– Groups of teachers from across the state.
– Training on item writing.
– Collectively create items.
– Produce test form.
– Create and verify answer key.
– Produce rubrics and administration guide for all
constructed response and performance items.
Developing an EoC Exam
• Step 4: Content/Bias Review
– Groups of teachers from across the state.
• New and Previous
– Training
– Review blueprints
– Review test items for content and bias
– Verify logic of test form
– Verify answer key
Developing an EoC Exam
• Step 5: Give the test and submit data.
– Tests will be available online on Blackboard beginning
spring 2015
– (Yay!)
– Pilot fall 2014 (select districts/schools)
– No cost to districts/schools
– More information forthcoming
Data
Submission
STARS
SOAP
STARS REPORTS
EES
Item Analysis
ADC
Standard
Setting
Revision
Submission Full
Approval
Why is Submitting EoC Data
Important?
• Student Level Data is Used for
– Standard Setting (passing scores for new
tests)
– Item Review
– Student Records for passing an exam
– Alternative Demonstration of Competency
for Graduation
– Educator Effectiveness System
– Provides larger sample size
Data Submission
Two Required Submissions:
1)
SOAP Item Data
– Only for Select EoCs
– Used for setting passing scores
– Minimum 50 RANDOM student scores per test
2)
STARS Overall Score
– For ALL EoCs
– For ALL Students
EoC Reports in STARS
STARS Reports
• EOC Submission Validation Report
– Student level validation errors for tests.
– Report used by districts to validate 40, 80, 120
and End of Year EOC submissions.
– Becomes a formal process for Fall 2014
STARS EoC Report Elements
•
•
•
•
•
•
•
•
•
•
•
•
District Name
District Code
Location Name
Location ID
Student ID
Student First Name
Student Last Name
Subject
Subtest Identifiers
Grade From
Grade To
Student Grade Level
•
•
•
•
•
•
•
•
•
•
•
•
Max Score
Raw Score
Max Tests
Subject Count
Date Test Active
Date Test Inactive
Test Date
Exists Error
Date Error
Score Error
Grade Error
Max Test
Data Quality is Essential!
• Data needs to be accurate
• Follow instructions given by PED
– Data Submission Guide
– STARS Manual
– Not sure—Ask!
• Errors create delay in results
• STARS EoC data submissions tied to
40/80/120/EOY day reporting
Developing an EoC Exam
• Step 6: Data Analysis
– P-values
– IRT analysis
– Possible elimination of items
– Flag items for further review
Developing an EoC Exam
• Step 7: Standard Setting
– Groups of teachers
• New and Previous
– Set passing scores
• Bookmark Method
• Body of Work Method
Developing an EoC Exam
• Step 8: Content Review #2
– Groups of teachers
• New and Previous
– Review flagged items
– Consider district/school/DTC/STC feedback
Developing an EoC Exam
• Step 9: Re-version and re-administer
– Any change  new version
– Minor changes, passing score remains
– Major changes, new passing scores must be set
End of EoCs
EoC Procedures
• Current: paper tests
• Upcoming: online hosting on Blackboard
•
•
•
•
December: Blackboard pilot with select
districts
Spring: Widespread Blackboard
administration
• Paper administration will be permitted.
• Data will be automatically be submitted to PED!
Summer 2014 EoCs
Summer 2014 EoCs
EoC Administration Windows
Window
Students
Dates
Fall
Students in 1semester courses
and HS retakes
2 consecutive weeks during last 3
weeks of fall semester
Winter
HS retakes
1 week during first 3 weeks of
spring semester
Spring
All students
2 consecutive weeks during last 3
weeks of spring semester
Summer
HS retakes and
students in
summer school
6/22 – 6/26/15 or last week of
summer school course
EoC Test Structure
• Most Tests: Multiple Choice
• (request from AAAC)
• Elective Courses: Performance Components
• HS: 90 minutes
• MS and ES: 60 minutes
EoCs: High Stakes Tests
• Treat as Final Exams
• Security Procedures are Essential!
• Directions for Administration (DFA):
administration procedures, teacher access, use
of calculators, use of scratch paper, make-ups,
accommodations, materials chain of custody,
scoring requirements, administration script
EoCs: High Stakes Tests
• Testing Irregularities: submit to PED
• DFA administration procedures will change
with implementation of CBT.
• Security will be easier to maintain.
EoCs: Student Preparation
• Teachers:
• Access Blueprints (on PED website)
• PBA: Access administration guides (on PED
website)
• DFA (on PED website)
• NAEP Questions Tool (on NAEP website)
EoC Questions?
EoC Data
Judy Harmon
judith.harmon@state.nm.us
EoC Procedures
Joslyn Overby
joslyn.overby@state.nm.us