MAP Under the
Hood
A Comparison of Measures of
Academic Progress & STAR
NWEA Wisconsin
Partner Meetings,
Spring, 2014
NWEA & Your school
THANK YOU….for being a GREAT
Partner
• John Woodin, NWEA Account Executive
– MN educator for over 30 years, with NWEA for 5
years
• Staff of 3 Account Managers for STATE:
Linda Andres, Eric Merchant and Lynn
Lamers
• Our Mission: Partnering to help ALL kids
learn
• NWEA is the only adaptive test that…
– Adapts across all grades on a single scale (PK through
12)
– Based on 38+ years of solid research
– Has the most stable-predictive scale in the industry
– Uses norms that are truly “national”
– Is content neutral
– Has the largest item bank of any assessment program
– Is a non-profit, mission-driven organization
Setting the Standard in
Adaptive Testing for 38 years
Buying a New Car…..
Would you choose a new car by only looking at
the shiny new outside???
You Have to Look
Under the Hood…..
…..not all assessments are equal “UNDER
THE HOOD”
NWEA – 38 Years of
Assessment Experience
• MAP Under the Hood
– Scale & Items
– Norms
– Alignment
– Time on test
– College/Career Readiness
• Mission Driven Organization: Research
& Support
• Link to Instruction: Content Neutral
• What’s new coming down the pike?
Setting the Standard for
Adaptive Testing
• RIT Scale: stability and predictability
– Virtually no shift (change) in the RIT Scale
for 20 years
– Scale is stable and scores mean the same
thing over time
– Ex: a student with a 200 RIT is at the
same instructional level today as 20
years ago
– Question: How stable is the scale the
assessment is using?
Setting the Standard for
Adaptive Testing
• Item Depth & Breadth
– NWEA utilizes over 90 thousand items
(breadth of coverage)
– New Item Types to assess deeper levels of
DOK (depth of coverage). We can provide
samples of new items
– NWEA uses 7 to 10 items per strand;
shorter tests use fewer items per strand
– Question: What type of items and how
many items does the assessment use?
Setting the Standard for
Adaptive Testing
• Norming Process
– Kingsbury Center at NWEA re-norms MAP
test every 3 years*
– Norms for both growth & status are
nationally representative
– Our Norm Studies now include 8+ million
students and 4.5 billion test events
– Some test makers use a “quartile” or
“quintile” regression model
(misrepresents growth at extremes)
Setting the Standard for
Adaptive Testing
• Test Alignment
– Common Core alignment is critical
– NWEA has hand aligned and field tested
each of our test items to the C.C.S.S.
– Provides consistent-stable growth measure
as schools transition to C.C.S.S.
– Depth of assessment allows educators to
make confident instructional decisions
– Question: “To what degree is the
assessment aligned to C.C.S.S., and
predictive of proficiency?”
Setting the Standard for
Adaptive Testing
• Why does a MAP test take longer for
students to complete?
– average time is 45 to 50 minutes
– Assessing the depth of knowledge – higher
order skills (not just lower cognition)
– Providing accurate information based on
adequate item selection per strand
– Gives you MORE information on broader
range and depth of skills
– Fewer items means higher Standard Error
– LESS IS NOT ALWAYS MORE….especially in
assessment
Setting the Standard for
Adaptive Testing
• Question: How does the assessment
predict College/Career Readiness?
– Kingsbury Center researchers completed a
linking study with ACT in 2010
– Results indicate that MAP is highly
predictive of how students will score on
ACT series (Explore; Plan; ACT)
– Educators, students & families can set
student growth targets based on CCR
benchmarks
College/Career Readiness is a Systems Issue
MAP as an
ACT Predictor
ACT
Composite
State/Local College
24-26
Major “Big 10”
University
28-29
Ivy League
University
31-32
MAP as an
ACT Predictor
Sandra is a 7th
grade student in
Waving Grass, WI.
Sandra’s spring
7th grade RIT is
237.
Sandra’s goal is to
go to UW Madison.
What
conversations
would you
begin with
her?
What Type of Assessment
Organization Is It?
• NWEA…is a non-profit, mission
driven organization
• By educators…for educators
• Not beholden to stockholders, but to
STAKEHOLDERS (Partners)
• Put in place an outstanding
system of support:
– Technical, Partner Accounts,
Professional Development,
Researchers
Linking to Instructional
Content
• NWEA is curriculum content neutral
• We are not selling proprietary
curriculum with our assessment
• We DO partner with many different
curriculum vendors to link RIT scores
to curriculum
• Question: Is the assessment
company selling a test? Or a
curriculum program?
Linking MAP to Digital
Resources
• Improves relevance of content
by matching it to students
performance on the MAP
assessments
Study
Island
Khan
• Helps educators answer the
“what next question”
E2020
Plato
• Strengthens MAP’s links to the
classroom
Compass
HMH
What Improvements are
Coming from NWEA?
• DesCartes Redesign
• Full Progress Monitoring capability
DesCartes
Update
Next Generation DesCartes
The vision for a future DesCartes addresses the specific limitations of
the current offering, including the following:
• Teacher and student friendly content that clearly describes a
learnable skill or understanding
• Standards language crucial to teachers is not currently present in
DesCartes
• Improved organizational structure beyond simply listing all LC’s in
a sub-goals RIT range
• Interactive and customizable reporting capabilities & features
support instructional planning
Future DesCartes/PGID
Learning Ladders POC
View Learning
Statements by
State Standards
Automatically
generated classbased ladders for
instructional
groupings
validation from NCRTI
• MAP and MPG can serve as
Universal Screener for RTI
programming.
• We have the highest ratings for
classification accuracy
http://www.rti4success.org/
NCRTI – Screening Review
MAP as Universal
Screener: Accuracy
• Screeners should not over-identify or
under-identify which kids are at risk
• This is “classification accuracy”
• If Area Under the Curve (AUC) ≥ .85,
measure has excellent classification
accuracy
• MAP’s classification accuracy is
consistently above .88 (AUC)
• Question: can a shorter test provide
adequate accuracy for universal
screening?
NWEA is Creating a Companion
“Progress Monitoring” Tool
• Skills based diagnostic/intervention system;
tied to MAP testing
• Identify skill deficits and recommended
intervention strategies
• Content neutral – but can be used with other
intervention systems
• Tentative release date: July 2014
Review the Questions
• How stable is the scale the assessment is using?
• How are norms created, and how often updated?
• Is the assessment FULLY adaptive across a pre-K
through 12 scale?
• What type of items are used, and how many items
support the assessment? By strand?
• To what degree is the assessment aligned to the
C.C.S.S.? Your state test? How predictive is it?
• What type of organization is the assessment
company? What is their support system? How will
they support you?
• What is the assessment company selling:
assessments or curriculum?
More Questions
• Is the organization beholden to stockholders or
stakeholders?
• What level of support and research undergird
the assessment?
• Does it clearly link to college/career readiness
benchmarks?
• What information will we sacrifice by giving
students a significantly shorter large-scale
assessment?
• What norming methods
does
a test provider use?
• How often do they re-norm?
Its All About the Kids!!!
We hope you will decide to continue to
partner with us to help all kids learn!!!