Changing
Changing
Teacher Student
Practice
Summer
Institutes
2013
Outcomes
How to use Professional
Development to Ensure
Improved Student
Outcomes
This presentation was adapted from Learning Forward, 2012
Our Outcomes
• Define evaluation in relationship to professional
learning;
• Examine the process of teacher change and its
impact on student learning; and
• Acquire strategies, tools, and resources to
assist in evaluating professional learning Building PD Capacity Toolkit (link)
Our Essential Questions
• How can evaluating professional learning leverage
school, school system, and state improvement effort?
• How will I align professional learning objectives to
measurable short, medium and long-term results for
educators and students?
• How will I collaborate with others to construct a
framework that outlines a detailed plan for evaluation?
• How do I incorporate evaluation into my work and
normative practice?
“Norms”
• Listen as an Ally
• Value Differences
http://ecx.images-amazon.com/images/I/51%2BU-UWbxmL._SY300_.jpg
• Maintain Professionalism
• Participate Actively
The Standards for Professional
Learning
Professional learning that increases
educator effectiveness and results for
all students uses a variety of sources
and types of student, educator, and
system data to plan, assess, and
evaluate professional learning.
-Standards for Professional Learning, 2011
http://www.ncpublicschools.org/docs/effectivenessmodel/ncees/standards/prof-learn-quick-ref.pdf
Link to Student
Results
1. Standardsbased
professional
learning
2. Changes in
educator
knowledge, skills,
and dispositions
4. Changes in
student
results
3. Changes in
educator practice
DuFour Questions
• What do we want students to learn?
• How will we know if they have learned it?
• What will we do if they have?
• What will we do if they haven’t?
Apply this to Professional Learning:
Full implementation vs. Full Participation
Full Implementation - Evaluation
A systematic, purposeful process of studying,
reviewing, and analyzing data gathered from
multiple sources in order to make informed
decisions about a program. – Killion, 2008
The systematic investigation of merit and worth.
– Thomas Guskey, 2000
Which best describes your experience?
A
B
Externally driven and designed
Internally driven and designed
Summative evaluations only
Planning, formative, and summative
evaluation
Event-based
Program-based
Looking for answers/solutions
form others
Discovering or creating solutions and
alternatives with others
Feared
Embraced
Filed/shelved
Used
Done as an afterthought
Planned from the beginning
Documentation
Evaluation
Process-focused
Results-driven
Presentation of results
Reflective dialogue
Shifting Perspectives
From
To
Externally driven and designed
Internally driven and designed
Summative evaluations only
Planning, formative, and summative evaluation
Event-based
Program-based
Looking for answers/solutions form
others
Discovering or creating solutions and
alternatives with others
Feared
Embraced
Filed/shelved
Used
Done as an afterthought
Planned from the beginning
Documentation
Evaluation
Process-focused
Results-driven
Presentation of results
Reflective dialogue
Your Evaluation Process
Write 4-5 sentences that describe the evaluation
process/steps you currently use as a leader
responsible for assisting others with evaluating
professional learning.
Tech Tool Idea:
Penzu – online writing journal
Group Think – Table talk, share out
or Padlet
1. What aspect of
evaluating
professional learning
do you find
essential?
2. How do we know
that the professional
learning is making its
way to the
classroom?
How and Why of Evaluation
Good evaluations are the product of thoughtful
planning, the ability to ask good questions, and a
basic understanding about how to find valid
answers. In many ways they are simply the
refinement of everyday thinking. Good evaluations
provide information that is sound, meaningful, and
sufficiently reliable to use in making thoughtful and
responsible decisions about professional
development processes and effects (Guskey &
Sparks, 1991).
Guskey’s Evaluation Framework
Professional
Development
Quality
Content
Context
Process
Level 1:
Reaction to
Professional
Development
Level 2:
Learning
Level 4:
Changed
Instruction
Level 3:
Organizational Support
(practices, policies, resources)
Adapted from Guskey, 2000, pp. 79–81
16
Level 5:
Student
Outcomes
Lead Box Evaluations
Superman X-ray
Input
?
Output
A simplistic approach to professional learning evaluation that fail to amplify
the underlying theory and operation of the professional learning program.
Glass Box Evaluations
Actions
Results
A comprehensive approach to professional learning evaluation that
illuminates how professional learning program components interact to
produce results.
Lead Box
Professional
Learning
Action
?
Student
Achievement
Results
Focus on outputs
rather than what
occurs in the
program or what is
presumed to be
causing those
outcomes and why
Lead Box
Curriculum
Development
Professional
Learning
Nonacademic
factors
?
Student
Achievement
Results
Focus on inputs
and fail to shed
light on HOW a
program’s
activities and
resources interact
to produce results.
Glass Box
Focus on what
occurs and how it
occurs within the
program.
Professional
Learning
Actions
Student
Achievement
Results
Glass Box
Coaching/
Follow-up
Professional
Learning
Instructional
Resources
Implementation
Monitoring
Student
Assessment
Student
Achievement
Results
Focus on
illuminating factors
contributing to
transformation
process.
Lead Box vs. Glass Box
What is the difference?
Thorough planning
facilitates sound
evaluation.
4 Key Components to Evaluating PD
Systematic How rigorous is the process?
(Established
Is it conducted in accordance with standards and
process)
guidelines?
Standards Does it have merit and/or worth?
(Predetermined
Does it meet predetermined standards of success?
criteria)
Audience
Who will use the evaluation?
For whom is the evaluation being done?
Intended
Uses
How will the evaluation be used?
What decisions will be made as a result of the
evaluation?
8 Smooth Evaluation Steps
Planning
Conducting
Reporting
1. Determine
‘what’ to
evaluate
4. Collect Data
7. Disseminate
and Use
Findings
2. Formulate
Evaluation
Questions
3. Construct
Evaluation
Framework
5. Organize,
Analyze, &
Display Data
6. Interpret Data
8. Evaluate the
Evaluation
3 Types of Evaluation
1. Planning – before program design to provide
information on conditions or needs to address
2. Formative – during implementation to provide
information on whether the program is working
as designed
3. Summative – after completion to provide
information on outcomes or overall impact
Tiers and Benchmarks
1. Multiple settings
2. Data sources – affective, quantity,
performance data
3. Initial vs. embedded
4. 5 year plan for data collection
5. Universal Design for Learning (UDL)
Reflection
• Review your 4-5 sentences about evaluation
• Look at the Padlet responses
• ‘Steal’ ideas to take back to your district
• Reflect on what you can do differently as
result of your new knowledge
Change
When you come out of the storm, you won’t
be the same person who walked in. That’s
what the storm is all about!
― Haruki Murakami, Kafka on the Shore
Change is Learning
Concerns-Based Adoption Model : Developed by
Bill Rutherford, Gene Hall, Shirley Hord, and Susan
Loucks-Horsley
4 Components:
1. Stage of Concern – 7 stages of responses
2. Levels of Use – eight ranges of intervention use
3. Innovation Configuration – described actions
4. ‘Change’ facilitators – leaders of learning
Change Learning Exchange
• Distribute numbered cards (#1-4) at your table.
• Read and the corresponding article on change.
• Prepare a two-minute talk about your article. Use any of
the following to prepare.
– Why your focus area is important.
– Implications of the change process.
– Ways to facilitate your area of change.
– Challenges you anticipate when helping others
understand this area of change.
Change Spotlight
•
Find a partner that read a different article.
•
Take two minutes each, discuss your article.
•
Focus on any of the following:
– Why your focus area is important.
– Implications of the change process.
– Ways to facilitate your area of change.
– Challenges you anticipate when helping others
understand this area of change.
•
Listen for the timer to repeat the process.
Note to Self
What new insights did
you gain as a result of
your reading and
discussion with others?
Tech Tool Idea:
Penzu – online writing
journal
Summary & Wrap-up
Set Standards for Acceptable Performance
• Specify how good is good enough
• Specify “success” in advance
• Provide a benchmark/baseline for
comparison before and after professional
learning
Teachers participate in collaborative
learning experiences.
Teachers implement new learning in
their instruction.
Student performance increases.
Evaluation Assumptions
• The staff development program is data-driven, researchbased, and well-defined.
• The school, district, or regional agency has the capacity,
including fiscal and human capital, to implement both the
program and evaluation with fidelity to their designs.
• Key stakeholders in the school, district, or agency intend to
use the evaluation results to make decisions about the
program.
Toolbox
Debrief
• How are the ideas presented today
CONNECTED to what you already knew?
• What new ideas did you get that EXTENDED
or pushed your thinking in new directions?
• What is still CHALLENGING or confusing for
you to get your mind around? What questions
or wonderings do you now have?
FFT Google Doc