PPT Template Presentation:
This PTT includes all slides to
FACILITATE a data analysis session of
PVAAS data for multiple elementary
teams. This was used with 7 teams
from 7 elementary schools in ONE
school district.
The Power of
PROGRESS
&
ACHIEVEMENT
Pennsylvania Value-Added Assessment System (PVAAS)
<XXXX> School District
PDE Data Tools
PSSA Data
Interactive
by eMetric
Interactive tool for analyzing PSSA
data by district, school, grade,
subgroup and student – Reporting
category information is also available
PVAAS
Value-added tool that reports growth
of cohorts of students and projections
for individual students for performance
on future PSSAs
PA AYP
Public site that reports all AYP related
data on district, school, grade and
subgroup - Reporting category
information is also available
4Sight
Member
Center
Reporting site for benchmark data by
grade, class, subgroup and student –
intended to be formative and used
often throughout the school year
Stair-Step Expectations
Grade Three
Grade 3 Test
Grade Four
Grade Five
Grade 4 Test
Grade 5 Test
In a perfect world students would start school at about the
same academic level. Proficiency levels could simply be
achieved by students gaining age appropriate knowledge
and skills each year.
Differentiated Reality
Grade Three
Grade Four
Grade 3 Test
Grade Five
Grade 4 Test
Grade 5 Test
Students begin school at different places, and they often
progress at different rates. Yet all students are expected
to achieve the same level of academic proficiency each
year.
The Need for Progress Measures
Grade Three
Grade Four
Grade 3 Test
Grade Five
Grade 4 Test
Grade 5 Test
To measure school effectiveness in this “differentiated”
world, we need to pay attention not only to proficiency
levels but also to how much progress students make in a
given year.
PVAAS IS:
• Analysis of existing assessment data
– Longitudinal PSSA scores in available
content areas
• Math, Reading, Science, Writing
– Provides reliable estimates
• A measure of a school’s effectiveness
• Critical Relationship:
Growth Data vs. Achievement Data
PVAAS:
•
Is not another test
•
Is not a comparison of single test
scores
•
Critical concept:
Does not provide a measure of
growth for an individual student
The TWO Types of
PVAAS Information
Looking
Forward/Planning…
Looking Back/Evaluation…
PVAAS Projection Reports
For Individual Students
and Cohorts of Students
Value-added Growth Reports For
Cohorts of Students
Today
NO individual measure of student growth.
PVAAS Reporting
(Math, Reading, Writing, Science)


Value-added growth reports for grades 4-8 & 11
Projections to Basic, Proficient, Advanced
 Grade last tested - 3  4 or 5 (future PSSAs)
 Grade last tested - 4  5 or 6 (future PSSAs)
 Grade last tested - 5  6 or 7 (future PSSAs)
 Grade last tested - 6  7 or 8 (future PSSAs)
 Grade last tested - 7  8 (future PSSAs)
 Grade last tested - 8  11 (future PSSAs)
Student Projections
Wouldn’t it be great
to know the likelihood
that a student will
be proficient on a
future PSSA?
Student Projection Methodology



A separate longitudinal modeling process
focused on the performance scores of
individual students.
The model uses all of the available data
(both math and reading) to project future
PSSA performances in Math and Reading.
Projections can be calculated for an
individual student or groups of students.
How do I know that the projection
is accurate?

The projection is precise, and is created using a model
that has been reviewed and approved by four different
peer review panels and the GAO (US Government
Accountability Office).

The most recent 2008 growth model proposal to USDOE
includes information regarding the statistical model and
projection reliability study.
(http://www.ed.gov/admins/lead/account/growthmodel/pa/index.html)

Recent studies have confirmed that the PVAAS
projections (even as far as 3 years into the future) are
more reliable at looking at the future performance of a
student than the most recent PSSA score.
PVAAS Student Projection Report
PVAAS Projections for
Groups of Students
Student Search
Identified Students
Meeting Search Requirements
Data Packets: Student Level
Timing
Source
Report
Initial Meeting –
Looking Back
PSSA: eMetric/GROW
Individual Student Report
Initial Meeting –
Today
4Sight
Baseline
Initial Meeting –
Looking Forward
PVAAS
PVAAS Projection
Periodic Meetings
(based on subject area and
grade level)
Examples:
DIBELS
4Sight
GRADE
GMADE
Core Phonics
Local Formative
Assessment
PVAAS Projections with Other Data:
Elementary Scenario/Reading
Student NOW in 5th Grade
PSSA
Grade 4 Proficiency
Range: 1255-1468
1210
4Sight
Baseline
Grade 5 Proficiency
Range: 1275-1496
1200
PVAAS
Projections
DIBELS
ORF
To Basic:
To Proficient:
To Advanced:
Grade 5 – Low Risk
104+ at beginning of year
124+ at end of year
93.6%
64.8%
5.5%
89
Some
Risk
PVAAS Projections with Other Data:
High School Scenario/Math
Student NOW in 9th Grade
PSSA
Grade 8 Proficiency Range
1284-1445
Grade 9 Proficiency Range
4Sight
1304-1508
Baseline
To Basic:
PVAAS
To Proficient:
Projections
(11th Grade)
Grade 8
Final
Grades
To Advanced:
Final Exam: 91%
Course Grade: 87%
1450
1507
98.7%
89.9%
45.7%
What Does This Mean to You?


The district could provide you each with
the probability of students in your
classrooms (current grades 4-9)
Grade level/course specific teams can
collaborate to make sure adequate
supports are in place for all students to
reach success.
How could we use this
information?

For students with each level of probability
of proficiency, what do we need to keep
doing to keep at this level? What do we
need to do to increase the level of
probability?
• 70-100% likelihood of proficiency

– what is our action plan?
• 40-70% likelihood of proficiency

– what is our action plan? Goal date for proficiency?
• 0-40% likelihood of proficiency

– what is our action plan? Goal date for proficiency?
Value-Added
Wouldn’t it be great
to know if we made one year’s
growth with last year’s 4th, 5th,
6th, 7th, and 8th graders?
11th graders?
Value-Added Measure of
Cohort Growth



Comparison to a State Growth Standard
The Growth Standard specifies the
minimal designated academic gain from
grade to grade for a cohort of students.
The use of a Growth Standard creates the
possibility that ALL schools can
demonstrate appropriate growth.
PVAAS Value-Added
Growth Descriptors – Grade 4 – 8
Math and Reading
The PA Growth Standard specifies the minimal acceptable
academic gain from grade to grade for a cohort of students. The
Growth Standard is met when the students in a school or district on
average maintain their relative position when compared to the 2006
statewide distribution of student scores (2006 is the base year).
Estimated mean NCE gain equal or greater than zero. There is
evidence that the school has met or exceeded the Growth
Standard. All schools can achieve this rating.
Estimated mean NCE gain less than zero by one standard error or
less. There is slight evidence that the school has not met the
Growth Standard.
Estimated mean NCE gain less than zero by between one and two
standard errors. There is moderate evidence that the school has
not met the Growth Standard.
Estimated mean NCE gain more than two standard errors less than
zero. There is significant evidence that the school has not met the
Growth Standard.
PVAAS School Report
Grades 4 through 8
Has every grade in the school met or
exceeded a year’s worth of growth?
Writing, Science & Grade 11
Reading/Math Growth Descriptors
Progress is defined by the district or school
effect, which is a measure of the difference
between observed performance and predicted
performance.
Progress is significantly above the progress of
the average district in the state.
Progress is not detectably different from the
progress of the average district in the state.
Progress is significantly below the progress of
the average district in the state.
PVAAS School Report
Grades 11- Math and Reading
Grades 4, 8 & 11 – Science
Grades 5, 8 & 11 – Writing
Performance Diagnostic Report
Has every predicted PSSA performance level (Below
Basic, Basic, Proficient, Advanced) in each grade met or
exceeded a year’s worth of growth?
Growth Standard Methodology
What the whiskers tell us…
Exceeded Growth Standard;
More than One Year’s Growth
Green
Met the Growth Standard;
Made One Year’s Growth
Yellow
Growth
Standard
(One Year’s
Growth)
Below Growth Standard;
Less than One Year’s Growth
Rose
Predictive Methodology
What the whiskers tell us…
ABOVE Expectations -Observed Score Was Greater
Than Predicted Score
NDD – No Detectable Difference
Between Observed Score and
Predicted Score
Observed –
Predicted
(One Year’s
Growth)
BELOW Expectations -Observed Score Was Less
Than Predicted Score
Patterns on Performance
Diagnostic Reports
A
Patterns of Growth
C
B
9/13/10
What Does This Mean to You?

The district could provide each grade
level/course specific team with their
growth reports
• Grades 4-8, and 11
• Subgroup reporting
How could we use this
information?





As a grade level/course specific team,
discuss why and how you may have
gotten the growth results on the report?
Which students made a year’s worth of
growth?
Which did not?
Why? Why not?
What is our team’s plan of action as a
grade level/course specific team?
Where do we go from here?
How do we plan to use our PVAAS data this next month?
This school year?
•
Value-added reports
•
Diagnostic reports
•
Student projections
How do we plan to share our PVAAS data?
•
With other Administrators?
•
With Teachers?
•
With Parents?
•
With Others?
District Goal: PVAAS will be used by teachers and
administrators for annual school evaluation and
planning, grade/subject level evaluation and
planning, and for student level planning.
Data Packet: Annual School-Level Analysis
Question
Tool
Report
1) School Met AYP?
PAAYP
Performance Chart
2) Every Grade Met AYP?
eMetric
3-Year Portrait
3) Every Grade Met Growth Standard?
PVAAS
School Value-added Report
4) Every Grade 3 Year Trend Change in
Proficiency?
eMetric
3-Year Portrait
5) Every Predicted Performance Level
Met Growth Standard?
PVAAS
Performance Diagnostic Report
6) Every Subgroup Met AYP?
eMetric
3-Year Portrait
7) Achievement Gap Narrower?
eMetric
3-Year Portrait
8) Every Grade Projected To Meet AYP
Target?
PVAAS
Grade Projection Summary
9) School and Subgroups Met Participation
Targets?
PAAYP
Data Table
10) Every Subgroup Met Growth
Standard?
PVAAS
Performance Diagnostic Report
11) Subgroups Performed Similarly in
Reporting Categories?
eMetric
3-Year Portrait
Data Packets: Grade Level
Timing
Source
Report
Initial Meeting –
Looking Back
eMetric
eMetric
PVAAS
PVAAS
eMetric/PVAAS
Performance Levels
Reporting Categories
Value Added Report
Performance Diagnostic Report
PSSA/ PVAAS Scatterplots
Initial Meeting–
Looking Forward
PVAAS
4Sight Baseline
4Sight Baseline
Projection Summary Report
Performance Levels
Subscale Reports
Initial Meeting–
Subgroup Comparisons
Same as above
Same as above
Periodic Meetings
4Sight
4Sight
Local Formative
Assessment
Performance Levels
Subscale Reports
Data Packets: Student Level
Timing
Source
Report
Initial Meeting –
Looking Back
PSSA: eMetric/GROW
Individual Student Report
Initial Meeting –
Today
4Sight
Baseline
Initial Meeting –
Looking Forward
PVAAS
PVAAS Projection
Periodic Meetings
(based on subject area and
grade level)
Examples:
DIBELS
4Sight
GRADE
GMADE
Core Phonics
Local Formative
Assessment
Questions and Feedback
Questions:
PVAAS Materials or
Statewide Implementation
pdepvaas@iu13.org
717-606-1911
PVAAS Report Web Site
https://pvaas.sas.com
www.pde.state.pa.us
333 Market Street
Harrisburg, PA 17126