PAIU CC Meeting
PVAAS Update
November 2014
Kristen Lewald, Ed.D.
Statewide PVAAS Director for PDE
Lancaster-Lebanon IU13
Topics
 Guide for Reflection on Teacher Specific
Reporting
 Demo Access Accounts
 Common Questions from Admins and
Teachers
 Feedback from Teachers
 Feedback from 499
Superintendents/Designee
 Help Menu System
 PVAAS Reporting/Website Updates
NEW: Guide for Reflection
PVAAS Teacher Specific Reporting
Guide for Reflection:
Process for Review and
Planning
Steps to Review
Reports and
Develop Plan of
Action
Guide for Review and Planning
Two Sets of Guiding Questions:
 Organized Based on Domains
1. Planning and Preparation
2. Classroom Environment
3. Instruction
Organized Based on CIAO!
1. Curriculum
2. Instruction
3. Assessment
4. Organization
Demo Account Access
PVAAS Teacher Specific Reporting
Demo Account Access
Share with LEAs
Common Questions from
Admins and Teachers
PVAAS Teacher Specific Reporting
Why No Report for a Teacher
Who Rostered?
PVAAS Teacher Specific Reporting
Common Questions
Students Included in
Reporting
To actually be included in PVAAS Teacher
Specific reporting, a student MUST:
• Have a PSSA or Keystone score from the most
recent year
• NOT be a foreign exchange student
• NOT be a first-year ELL student
• NOT be Proficient or Advanced on a PRIOR
Keystone exam
• NOT be claimed at less than 10% instructional
responsibility
• NOT have tested with the PASA (alternate
assessment)
Minimum N Counts
 Must have at least 11 students
 Must have an “Active N Count” of at least 6
students/6.0 FTE Students
Why No Individual Student
Growth Measure?
PVAAS Teacher Specific Reporting
Common Questions
Why No Individual Student
Growth Measure?
Student History Report
 Growth is about a GROUP
of students NOT an
individual student.
 Error would be too large
for an individual student to
know if they made growth
or not.
This report should
NOT be used or
interpreted as
growth for an
individual student!
Why No Predicted Score NOW?
PVAAS Teacher Specific Reporting
Common Questions
Why No Predicted Score
Available NOW for SY14-15?
 A student’s predicted score is based on the student’s own
testing history and on the average performance of
students in the same cohort statewide who have a similar
testing history.
 In other words, for each assessment we analyze with the
predictive model, we look at the past testing histories of
students across the state who took that assessment in
the most recent year.
 Because the predicted score is based on the performance
of other students statewide who took the assessment in
the same year, it’s not possible to generate the predicted
score prior to the assessment.
 What would the score tell you that a PVAAS projection/%
of likelihood does not?
Why Different?
School Value-Added vs.
Teacher Value-Added
Reporting
PVAAS Teacher Specific Reporting
Common Questions
Why Different?
School Value-Added vs. Teacher
Value-Added Reporting
2 Primary Reasons:
 PVAAS School Reporting uses full academic
year/Oct 1 as a requirement for including
students
 PVAAS Teacher Value-added reporting uses %
Student + Teacher Enrollment and % Shared
Instruction to determine the weighting of each
student on each teacher’s PVAAS teacher
Specific Reporting
Why Different?
Composite v. Teacher ValueAdded
PVAAS Teacher Specific Reporting
Common Questions
What is the Composite?
 A combined growth measure across a teacher’s PVAAS
reported subjects/grades/courses within SY13-14.
 For SY13-14, we only have one year of reporting so
the Composite Score is a combined growth measure
across a teacher’s PVAAS reported
subjects/grades/courses for SY13-14 only.
 In future years, the Composite Score will represent a
combined growth measure across a teacher’s PVAAS
reported subjects/grade/courses across years, up to
and including 3 consecutive school years.
 1 year Composite
 2 Year Composite
 3 Year Composite/3 Year Rolling Average
1 Pager in
Development
Teacher Value Added Summary
The composite is a combined measure of all tested
subjects, grades, and Keystone content areas in which a
teacher received a Value Added report.
Composite: How Calculated
 To calculate the composite, a simple average is
taken of all of the teacher's individual index values
for up to three years.
 Then the average is multiplied by the square root
of the number of individual index values that went
into the average.
 This step is a necessary step that accounts for the
fact that more data was used to generate the
average than was used to generate each individual
index, which affects the standard error for the
composite.
Why Different?
Composite v. Teacher Value-Added
The growth color indicators are categories
or ranges of indicators of growth. We
cannot “average” categories. In other
words, green + red ≠ yellow; likewise
green + dark blue ≠ light blue.
For PVAAS, the assumption is that the
achievement level of the teacher's group of
students is maintained (green) UNLESS
there is enough evidence in the assessment
data to say otherwise.
Why Different?
Composite v. Teacher Value-Added
 It is all about the amount of evidence in the student
assessment results – the assumption is that the achievement
level of the teacher's group of students is maintained (green)
UNLESS there is enough evidence in the assessment data to
say otherwise.
 The more data available, the more evidence we have to
see if the group of students exceeded the growth standard
OR not.
 There is more evidence when all data was combined for
a composite score. With data from multiple subjects (or
multiple years when that’s available) included in the
Composite, there’s more evidence/more data.
 The composite is not really an “average,” but rather an
accumulation of the evidence/data towards meeting,
exceeding, or falling short of the standard for PA
Academic Growth.
Why Different?
Teacher Value-Added v.
Diagnostic Reporting
PVAAS Teacher Specific Reporting
Common Questions
Why Different?
Teacher Value-Added vs. Diagnostic
Reporting
 Diagnostic Reports do NOT factor
in the percent of Instructional
Responsibility
 Students are weighted equally
 Reflect the growth of a group of
students who may have had more
than one teacher with instructional
responsibility in that specific
subject/grade or course
Methodology Questions
PVAAS Teacher Specific Reporting
Common Questions
More Info About Methodology
 PVAAS Methodologies, Measuring Growth &
Projecting Performance – PVAAS login page,
under Professional Development
 Virtual Learning Modules – eLearning Link
Value Added Reports – Math/Reading
Value Added Reports – Science/Keystones
Teacher Specific Reporting
 EVAAS Statistical Models in PA – PDE PVAAS
site, Methodology Page
Full Day TOT Sessions for Admins
PVAAS Teacher Specific Reporting
Participation
450 LEAs
Over 1700 Participants
Misunderstanding
PVAAS 3 Year Rolling Average
3 Consecutive Years of PVAAS
Teacher Reporting
Any state assessed
subject/grade/course
Does not need to be in the same
subject/grade/course
Half-Day Teacher Sessions at IUs
PVAAS Teacher Specific Reporting
Half-Day Sessions with Teachers
– We Need Your Support!
 Before session
 Setup of session – registration, handouts, technology
 Make sure all participants can get online
 Make sure all participants log onto PVAAS site - check username and
password and PPID before training starts
 Download session PPT
 Welcome Participants to your IU/Orient to Facility
 During Session
 Rotate around room during online work
 Gather questions if not comfortable answering
 Engage with participants
 Ideas for Follow-up IU services
 After Session
 Gathering and Packing Up Materials
 Copies of Session Feedback Forms
 Email Final Registrant List
Feedback from Teachers
PVAAS Teacher Specific Reporting
What will you tell your
colleagues about PVAAS?
Emails to
499 Superintendents/Designees
PVAAS Teacher Specific Reporting
Asked 4 Questions
How is it going overall?
Were most of the reports in sync with
what principals expected?
Any/many surprises for the principals?
Any questions or information needed
by you or your team?
“Best” Teacher
 Qualities such as: students engaged in teacher’s classroom,
using technology with students, positive feedback from
students and/or parents, meets with students before/after
school, goes to a student’s activities and sporting events,
contributes to the school community, contributes to the
district community, etc.
 Could a teacher do all of these things listed above, but students
may not be academically growing? is that possible?
 Why?
 Could a teacher do all of these things listed above AND their
students ARE academically growing? is that possible?
 Why?
NEW:
PVAAS Help Menu System
PVAAS Teacher Specific Reporting
New PVAAS Help Menu
System System
Updates to PVAAS
Keystone Reporting
PVAAS District/School Reporting
Updates to Both
PVAAS Reporting Sites:
PVAAS Restricted/Password
Protected Site: Thursday, Nov. 20th
Updates to Keystone Reporting
PVAAS Public Site: Thursday, Dec. 4th
Updates:
PVAAS Password Protected Site
• Keystones district/school value-added reports
• Keystones district/school performance diagnostic
reports
• Keystones district/school quintile diagnostic
reports
• PVAAS SPP/AGI reports
• PVAAS DPP/AGI reports
• Changes to students included on drill down in
above reports based on new business rules for
Keystone reporting
Questions? Feedback?
pdepvaas@iu13.org