PROGRESS
&
ACHIEVEMENT
Pennsylvania Value-Added Assessment System
Pennsylvania Department of Education
Session Overview
History of PVAAS
Pennsylvania’s Intent for PVAAS
PVAAS Methodology and Reports
Timelines for PVAAS Implementation
Data Needed for PVAAS Analyses
Costs
District Preparation for PVAAS
Supports
Pennsylvania Department of Education
A Focus on
Standards
Pennsylvania Department of Education
A Standards-based System
Goal of high achievement, college readiness
Standards aligned
Assessments aligned
Safety nets aligned
Curriculum aligned
Instruction aligned
Instructional materials aligned
Pennsylvania Department of Education
Data Driven Decision Making
Tools Provided to Districts
Leading for Learning!
Getting Ready for the PSSA
Getting Results!
Data Tools
Data Reports
PA GROW
Context Tools
Quality Reviews
Tutoring Evaluations
PVAAS
Pennsylvania Department of Education
School Structures for Data-Informed Decision Making
District-Level Support
(Budgetary Support, Professional Development, Resources and Time)
Demographic/Perceptual/Process Data
Student Learning Data
School Level
•School Demographics
•Discipline Data
•Attendance Data
•Mobility Rate
School Level
Annual Building-wide Planning Process
Focus: All Students
Who: School-wide Team
How: Data Retreat, School Planning Process
PSSA & PVAAS
Standardized Assessments
District End-of-Year Tests
Final Benchmark Test
Grade/Course Level
Grade/Course Level
•Class Demographics
•Class Engagement Data
•Satisfaction Data
•Attendance Data
Classroom Level
Qualitative Data
•Student Historical Information
•Student Medical Information
•Student Learning Information
© Lancaster-Lebanon IU13
Periodic Grade Level
Planning Process
Focus: Groups of Students
Who: Teacher Teams
How: Regular 1-2 hour meetings
Student Planning
Process
Focus: Classroom
of Students
Who: Teacher
Initial: PSSA/PVAAS/final tests
– class/subgroup levels
Cyclical:
•Benchmark Data - grade level
•District quarterly assessments
•Common Classroom Data
•Classroom Summaries
Classroom Level
Initial: PSSA/PVAAS/final tests
– student level
Cyclical:
•Benchmark Data – Student Level
Continuous
•Individual Classroom Assessments
•Progress Monitoring
History of PVAAS
2002: Recommended by Pennsylvania
League of Urban Schools (PLUS) to
State Board of Education
2002: State Board of Education passed
resolution for “value-added approach
across the Commonwealth”
2002 (SY04-05): 3 Pilot Phases
100 Districts, including the School District of
Philadelphia and Pittsburgh City School
District
Pennsylvania Department of Education
History of PVAAS
Fall ‘04 PDE Statewide Workgroup
Led by PA Secretary of Education, Dr. Zahorchak
50+ Members
Districts
Governor’s Office
Higher Education
Statewide Associations including, but not limited to:
PSEA, PFT, PASA, PSBA, PAIU, PAESSP, PASCD
Legislators
Business and Industry
SY05-06 Statewide Implementation - 501 Districts
1st Reporting in Fall 06
Pennsylvania Department of Education
PA’s Intent for PVAAS
Another Tool for Continuous Improvement
Measure of Growth to Add to
Achievement/Status Measures
Not to be Used in Isolation
No Teacher Level Analyses
Pennsylvania Department of Education
PA’s Intent for PVAAS
Integration of PVAAS into existing PDE
initiatives:
School Improvement Planning
Distinguished Educator Training
Governor’s Institutes
EAP
4Sight
Progress Monitoring
Content Initiatives: Literacy and Mathematics
Response to Intervention
PIL
PIMS (PA Information Management System)
Pennsylvania Department of Education
What is PVAAS?
Pennsylvania Department of Education
Student Learning
Measures
Status Measures
PSSA
% proficient or not proficient
Other local assessment data
Improvement Measures
Safe Harbor
Other local assessment data
NEW MEASURE: Progress/Growth Measure
PVAAS
Pennsylvania Department of Education
Value Added Analysis
Provides a measure of what “effect” schooling has
in a single year
Formally described: Mixed model multivariate
longitudinal analyses
– The statistical methodology has been reviewed by
nationally renowned statisticians since the 1980’s
– The reporting has been used by educators for more than
10 years
Pennsylvania Department of Education
PVAAS IS
NOT:
Another test
Being used for teacher level
accountability in Pennsylvania
Pennsylvania Department of Education
PVAAS IS:
A statistical analysis of existing assessment data
A precise measurement of student progress/growth
over time
Progress data to add to achievement data as
schools are analyzing their work with students and
making improvement plans
Pennsylvania Department of Education
How is PVAAS analysis
done?
1. We collect all of the individual student data that is
available for a minimum of two years.
2. All of these data are used to develop individual
student profiles.
Year 1
Year 2
Year 3
PSSA
PSSA
PSSA (7th)
5th
6th
Predicted
Actual
Reading
1280
1300
1290
??
Math
1330
1305
1320
??
Pennsylvania Department of Education
The PVAAS Methodology – Part 1
Prediction Model 1
Student A
Testing Scores
Prediction Model 2
Prediction Model 3
Pool of
Testing Histories
Prediction Model 4
Prediction Model 5
Prediction Model 6
Computer
The PVAAS Methodology – Part 2
Student A
Testing Scores
Mean of Student Scores
for a particular school
Prediction Model 1
Student D Predicted Score
Prediction Model 2
Student E Predicted Score
Prediction Model 3
Student B Predicted Score
Prediction Model 4
Student F Predicted Score
Prediction Model 5
Student C Predicted Score
Prediction Model 6
Student A Predicted Score
Mean of Predicted Scores
for a particular school
School Effect  Mean of Student Scores – Mean of Predicted Scores
PVAAS Ratings of School Effect
Remember:
School Effect  Mean of Student Scores – Mean of Predicted Scores
So, if the students perform as expected, School Effect  0.
The ratings are in terms of the expected value of 0.
Not Detectably
Below
Above
Different
-2 SE’s
0
+2 SE’s
Standard Error (SE) is a measure of variation in the prediction
Achievement and Progress
Pennsylvania Department of Education
Types of PVAAS Reporting
District Level
By School
School Level
Diagnostic Reports
By Quintiles
By Performance
Levels
By Grade
Reading and Math
Student Reports
Longitudinal Reports
Diagnostic Subgroup
Reports
Custom
Reporting/Querying
“Projections” on Future
PSSAs
Pennsylvania Department of Education
District Summary Report
School Report
Below
Average
Achievement
Above
Expected
Progress
Student Projections
Pennsylvania Department of Education
Search for students
by subgroup or achievement
Identified students meeting search requirements
How Can PVAAS
Help Me?
Pennsylvania Department of Education
Key Learning from
PVAAS
Value-added information will tell you:
How much progress students have made in
each subject area and grade level
How much progress students at different
prior achievement levels have made.
How students’ progress in one curricular
area or program compares to their progress
in others
Pennsylvania Department of Education
PSSA and PVAAS – The Power of 2!
PSSA tells us where a student is.
PVAAS tells us where that student is headed.
Location and direction are critical partners in the
quest for having all students achieve proficiency
The data from both achievement and progress
measures gives schools valuable information to
make data informed decisions.
Without data all we have is an opinion!
Pennsylvania Department of Education
PA Proposal to USDOE
PA submitted proposal to USDOE to use
“projections” as means to recalculate
proficiency/AYP; did not include valueadded reporting as part of proposal
PA is waiting to hear from USDOE
Pennsylvania Department of Education
Timelines for Implementation
Timeline Released in 2002
SY01-02 and SY02-03: Pilot Phase I
SY03-04: Pilot Phase II
SY04-05: Pilot Phase III
SY05-06: Statewide Implementation
 All 501 Districts Receive PVAAS Reporting in Fall ‘06
 Pilot districts get comprehensive PVAAS reporting
 401 Districts Receive Limited Reporting
 Grades 4 & 6
SY06-07: Comprehensive Reporting for All Districts
Pennsylvania Department of Education
Data Needed for PVAAS
PSSA reading and math will be the only data
needed for statewide implementation
Only common assessment for all 501 districts
No data preparation is needed
No purchase of any data is needed
All PSSA data is sent direct from DRC to SAS,
Inc.
Pennsylvania Department of Education
Costs Covered by PDE
Costs for PSSA Data Submission
Costs for PVAAS Analyses
Costs for PVAAS Web-Based Reporting
Costs for Communication Materials
Costs for Professional Development Materials
Costs for Professional Development Sessions
Available through SY06-07
Regional sessions
Videoconferences
Webinars
Pennsylvania Department of Education
How Can Districts Prepare?
Plan to Use PVAAS Data as part of Continuous
Improvement Planning
District Level
Building Level
Grade Level
Content level
Subgroup Level
Student Level
Pennsylvania Department of Education
How Can Districts Prepare?
Professional Development on PVAAS
District Administrative Team
Building Level Administrators
Teacher Leaders
Classroom Teachers
Other Key Constituents
Communications to Board and Community
Pennsylvania Department of Education
Supports
Communications Kits
Posted on PDE Website
Professional Development Materials
Professional Development Supports
Staff Trained:
PVAAS Statewide Core Team/PATTAN
PDE Staff
IU Staff
Distinguished Educators
PVAAS Statewide Work Group Members
Pennsylvania Department of Education
For Information, Resources or
Training
PVAAS Core Team:
pdepvaas@iu13.org
717-560-4611
Pennsylvania Department of Education
Pennsylvania Department of Education