Deliberate Practice
Technical Assistance Day
1.877.411.7114
Copyright Learning Sciences International.
Copyright Learning Sciences International. Reproduction rights provided to Florida DOE.
Reproduction rights provided to Florida DOE.
1.877.411.7114
Goals
Participants will:
• Have a better understanding of Deliberate Practice
• Understand what the context is for DP
• Understand how DP fits into the framework
• Understand how to plan for DP
• Understand how to operationalize DP using the deliberate
practice plan
• Understand how the DP score contributes to the
instructional practice score
1.877.411.7114
Copyright Learning Sciences International.
Reproduction rights provided to Florida DOE.
2
Teacher Evaluation Metrics
Focus: Instructional Practice
Student
Growth
Instructional
Practice
Instructional Practice is
where classroom
observations and
feedback are provided to
teachers throughout the
year.
1.877.411.7114
Copyright Learning Sciences International.
Reproduction rights provided to Florida DOE.
3
Review: Last Summer’s Scoring and
Weighting Workshop - FLDOE Recommendation
Year 1
Instructional
Practice
50%
Year 2
Status
Score
100%
Instructional
Practice
50%
Status
Score
60%
Deliberate
Practice
Score 40%
1.877.411.7114
Copyright Learning Sciences International.
Reproduction rights provided to Florida DOE.
4
Why do we need Deliberate Practice?
1.877.411.7114
Copyright Learning Sciences International.
Reproduction rights provided to Florida DOE.
5
WHAT IF?
What if every teacher engaged in a self assessment of
their classroom practice?
What if the Individual Professional Development Plan
was aligned to the teacher evaluation system to help
teachers measurably improve the Instructional Practice
score?
Copyright Learning Sciences International. Reproduction rights provided to Florida DOE.
1.877.411.7114
6
IPDP statute 1012.98(4)(b)5
• 3(b) Each school district shall develop a professional development
system as specified in subsection (3)…. The professional
development system must:
• […]
• 5. Require each school principal to establish and maintain an
individual professional development plan for each instructional
employee assigned to the school as a seamless component to the
school improvement plans developed pursuant to s. 1001.42(18).
The individual professional development plan must:
– a. Be related to specific performance data for the students to
whom the teacher is assigned.
– b. Define the inservice objectives and specific measurable
improvements expected in student performance as a result of the
inservice activity.
– c. Include an evaluation component that determines the
effectiveness of the professional development plan.
Copyright Learning Sciences International. Reproduction rights provided to Florida DOE.
1.877.411.7114
7
Table Exercise
Instead of creating a new thing, how do we take
advantage of the IPDP to improve it into a high
leverage process that:
1. Empowers teachers?
2. Generates measurable improvements in
teacher practice?
3. Helps teachers raise their Instructional Practice
Score?
1.877.411.7114
Copyright Learning Sciences International.
Reproduction rights provided to Florida DOE.
8
The Deliberate Practice Plan may be used as
an option for districts in place of their current
IPDP if:
• Districts follow the FLDOE-approved model as
presented
• At least one of the Deliberate Practice growth
targets is developed with consideration of
appropriate student growth data
1.877.411.7114
Copyright Learning Sciences International.
Reproduction rights provided to Florida DOE.
9
What is Deliberate Practice?
1.877.411.7114
Copyright Learning Sciences International.
Reproduction rights provided to Florida DOE.
10
Deliberate Practice
Deliberate practice is a way for teachers to grow their
expertise through a series of planned activities, reflection,
and collaboration.
Involved in the series is a protocol:
• setting personal goals,
• focused practice,
• focused feedback,
• observing and discussing teaching, and
• monitoring your progress
1.877.411.7114
Copyright Learning Sciences International.
Reproduction rights provided to Florida DOE.
11
Making the Connections
Deliberate
Practice
Teacher
Behavior
Student
Achievement
1.877.411.7114
Copyright Learning Sciences International.
Reproduction rights provided to Florida DOE.
12
When these strategies are used, here is the
typical effect on raising student achievement (percentile gain corrected) :
Note Taking
17%
Practice
14%
Setting Goals/Objectives
25%
Student Discussion/Chunking 17%
Summarizing
19%
Tracking Student Progress and
Using Scoring Scales
34%
Building Vocabulary
20%
Effort and Recognition
14%
Graphic Organizers
13%
Homework
15%
Identifying Similarities and
Differences
20%
Interactive Games
20%
Nonlinguistic Representations 17%
1.877.411.7114
Copyright Learning Sciences International.
Reproduction rights provided to Florida DOE.
13
Deliberate Practice Involves:
Focused
Feedback
Focused
Practice
1.877.411.7114
Copyright Learning Sciences International.
Reproduction rights provided to Florida DOE.
14
Remember the Goal
An expectation that all teachers can increase
their expertise from year to year which produces
gains in student achievement from year to year
with a powerful cumulative effect
1.877.411.7114
Copyright Learning Sciences International.
Reproduction rights provided to Florida DOE.
15
Fundamentals of
Deliberate Practice
Deliberate Practice begins with empowering teachers!
• Teachers should take the lead in identifying their
personal growth goals, and collaborate with their
principals on their goals.
− Principal must approve the IPDP
• Deliberate Practice should be structured, resourced
and monitored by the principal.
1.877.411.7114
Copyright Learning Sciences International.
Reproduction rights provided to Florida DOE.
16
Steps for Developing
a Deliberate Practice Plan
1. Self
Assessment
2. Identify
Focus
Strategies
3. Create
Measurable
Goals
4. Actions in
the Classroom
5. Reflection
6. Tracking Progress
and Celebrating
Success
1.877.411.7114
Copyright Learning Sciences International.
Reproduction rights provided to Florida DOE.
17
Linking Performance Scales to
Teacher Reflection and Deliberate Practice
Autonomous
Phase
Shaping
Phase
Cognitive
Phase
1.877.411.7114
Copyright Learning Sciences International.
Reproduction rights provided to Florida DOE.
18
Rating Scale for Domain Elements
(State Model Teacher Evaluation Scale)
4
Formative
Ratings
Used for
Each
Domain
Element
3
Autonomous Phase
Innovating
Applying
2
1
Shaping Phase
Developing
Beginning
0
Cognitive
Phase
Not Using
1.877.411.7114
Copyright Learning Sciences International.
Reproduction rights provided to Florida DOE.
19
Rating Scale for Domain Elements
(Danielson Framework Teacher Evaluation Scale)
Distinguished
Formative
Ratings
Used for
Each
Domain
Component/
Element
Proficient
Autonomous Phase
Basic
Unsatisfactory
Shaping
Phase
Cognitive
Phase
1.877.411.7114
Copyright Learning Sciences International.
Reproduction rights provided to Florida DOE.
20
Linking Performance Scales to
Teacher Reflection and Deliberate Practice
Highly
Effective
Formative
Ratings
Used for
Each
Domain
Element
Effective
Autonomous Phase
Needs
Improvement/
Developing
Unsatisfactory
Shaping
Phase
Cognitive
Phase
1.877.411.7114
Copyright Learning Sciences International.
Reproduction rights provided to Florida DOE.
21
1.877.411.7114
Copyright Learning Sciences International.
Reproduction rights provided to Florida DOE.
22
Self-Assessment Example
1.877.411.7114
Copyright Learning Sciences International.
Reproduction rights provided to Florida DOE.
23
1.877.411.7114
Copyright Learning Sciences International.
Reproduction rights provided to Florida DOE.
24
Identify Focus Strategies Example
1.877.411.7114
Copyright Learning Sciences International.
Reproduction rights provided to Florida DOE.
25
1.877.411.7114
Copyright Learning Sciences International.
Reproduction rights provided to Florida DOE.
26
Write Measurable Goals Example
1.877.411.7114
Copyright Learning Sciences International.
Reproduction rights provided to Florida DOE.
27
1.877.411.7114
Copyright Learning Sciences International.
Reproduction rights provided to Florida DOE.
28
Action Steps Example
1.877.411.7114
Copyright Learning Sciences International.
Reproduction rights provided to Florida DOE.
29
1.877.411.7114
Copyright Learning Sciences International.
Reproduction rights provided to Florida DOE.
30
Reflection Example
1.877.411.7114
Copyright Learning Sciences International.
Reproduction rights provided to Florida DOE.
31
1.877.411.7114
Copyright Learning Sciences International.
Reproduction rights provided to Florida DOE.
32
Tracking Progress Example
1.877.411.7114
Copyright Learning Sciences International.
Reproduction rights provided to Florida DOE.
33
Appendix Tools
• Recommendations for Developing Goals and
Resources
• Goal #1 – Progress Check-In and Reflection Log
• Goal #2 – Progress Check-In and Reflection Log
• Goal #3 – Progress Check-In and Reflection Log
• Video Self-Observation Guide (Optional)
• Peer Observation Guide (Optional)
• Instructional Rounds Guide (Optional)
• Glossary of Terms
1.877.411.7114
Copyright Learning Sciences International.
Reproduction rights provided to Florida DOE.
34
Recapping the Benefits
of Deliberate Practice
• Empowers teachers to take responsibility for their own
professional growth and development
• Generates sources of evidences for other domains
(planning/preparing, reflection/professionalism, etc.)
• As teachers increase their effectiveness with target
strategies, it tends to lift their overall teaching
1.877.411.7114
Copyright Learning Sciences International.
Reproduction rights provided to Florida DOE.
35
5 Minute Pause
• What makes sense to me?
• What questions do I have?
1.877.411.7114
Copyright Learning Sciences International.
Reproduction rights provided to Florida DOE.
36
Question:
How is focused
practice different
from the common
conception of
practice?
1.877.411.7114
Copyright Learning Sciences International.
Reproduction rights provided to Florida DOE.
37
Planning Timeline 2012-13 Example
New Teachers
AugustSeptember
September
OctoberDecember
January
Teachers
complete the
self
assessment
Craft and
approve
deliberate
practice plan
Implement
plan, monitor,
make
adjustments
Review
growth as part
of the
summative
evaluation
conference
1.877.411.7114
Copyright Learning Sciences International.
Reproduction rights provided to Florida DOE.
38
Planning Timeline 2012-13
Category I
Teachers
and
Category II
August
September
October
March/April
May-June
Teachers
complete the
self
assessment
as part of the
summative
evaluation
process
Craft and
approve
professional
deliberate
practice plan
Implement
plan, monitor,
make
adjustments
Ensure
observations to
establish end
growth data
point for focus
strategies
Review
growth as part
of the
summative
evaluation
conference
Consider Teacher Review Teams that would work with teacher teams to review and
provide feedback regarding the plan before approval by the principal
1.877.411.7114
Copyright Learning Sciences International.
Reproduction rights provided to Florida DOE.
39
Planning Forward: Brainstorm
• What needs to take place this year for this
process to be put in place for the next
school year?
1.877.411.7114
Copyright Learning Sciences International.
Reproduction rights provided to Florida DOE.
40
District Table Activity
Develop a deliberate practice implementation plan:
• Communication and buy-in from what stakeholders?
• Impact on current teacher evaluation policies and
procedures?
• What training and resource needs are there to
implement?
• What expectations need to be set with central office,
principals, and teachers?
• What timeline milestones need to be determined?
1.877.411.7114
Copyright Learning Sciences International.
Reproduction rights provided to Florida DOE.
41
Deliberate Practice
Scoring and Calculation
Copyright Learning Sciences International. Reproduction rights provided to Florida DOE
1.877.411.7114
Review: Last Summer’s Scoring and
Weighting Workshop - FLDOE Recommendation
Year 1
Instructional
Practice
50%
Year 2
Status
Score
100%
Instructional
Practice
50%
Status
Score
60%
Deliberate
Practice
Score 40%
1.877.411.7114
Copyright Learning Sciences International.
Reproduction rights provided to Florida DOE.
43
5 Steps to Calculate the
Deliberate Practice Score
Step 1
District or teacher identifies target strategies following
district policies
Step 2
Obtain at least 2 data points for each target strategy
Step 3
For each element, determine the number of growth levels
from the first data point to the last data point
Step 4
Use the average growth level and the final data points
against the scale to obtain the Deliberate Practice Score for
each target element
Step 5
Average the Deliberate Practice Scores for each target
element for the final element score.
1.877.411.7114
Copyright Learning Sciences International.
Reproduction rights provided to Florida DOE.
Example Step 1
District or teacher identifies target element(s) following district policies
Teacher identifies the following targets:
– Domain1
• #1 Providing clear learning goals and scales (rubrics)
• #2 Tracking student progress
• #3 Celebrating success
1.877.411.7114
Copyright Learning Sciences International.
Reproduction rights provided to Florida DOE.
45
Example Step 2
Obtain at least 2 data points for each element
October
Data Point 1
March
Data Point 2
Domain 1:
#1 Providing clear learning goals and scales
(rubrics)
0
3
Domain 1:
#2 Tracking student progress
2
3
Domain 1:
#3 Celebrating success
1
3
Note: Scores are based on the Marzano scale: Innovating (4), Applying (3), Developing (2), Beginning (1) and Not Using (0)
1.877.411.7114
Copyright Learning Sciences International.
Reproduction rights provided to Florida DOE.
46
Example Step 3
For each element, determine the number of growth levels from
the first data point to the last data point using the Marzano Scale
October
Data Point
1
March
Data Point
2
Growth
Levels
Domain 1:
#1 Providing clear learning goals and
scales (rubrics)
0
3
3
Domain 1:
#2 Tracking student progress
2
3
1
Domain 1:
#3 Celebrating success
1
3
2
Note: Scores are based on the Marzano scale: Innovating (4), Applying (3), Developing (2), Beginning (1) and Not Using (0)
1.877.411.7114
Copyright Learning Sciences International.
Reproduction rights provided to Florida DOE.
47
Example Step 4
Use the scale to obtain the Deliberate Practice Score for each target element
Highly Effective
(4)
Effective
(3)
Needs Improvement
or Developing
(2)
Unsatisfactory
(1)
Grows 3 levels
Grows 2 levels
Grows 1 level
Achieves no growth
Or grows to Level 4
Or grows to Level 3
Or grows to Level 2
Or scores at Level 1
Note: The district may decide which rule takes precedence
1.877.411.7114
Copyright Learning Sciences International.
Reproduction rights provided to Florida DOE.
48
Example Step 5
Average the growth levels across target elements
Levels
Grown
Final Level
Deliberate
Practice
Score
Domain 1:
#1 Providing clear learning goals and
scales (rubrics)
3
3
4*
Domain 1:
#2 Tracking student progress
1
3
3
Domain 1:
#3 Celebrating success
2
3
3
Final Deliberate Practice Score:
3.33
* This score may be vary depending on the precedence of the scoring rules. This scenario uses the highest scoring rule.
1.877.411.7114
Copyright Learning Sciences International.
Reproduction rights provided to Florida DOE.
49
Deliberate Practice Score
Highly Effective
Effective
Needs Improvement
Or Developing
Unsatisfactory
3.5 - 4
2.5 - 3.4
1.5 - 2.4
1 - 1.4
1.877.411.7114
Copyright Learning Sciences International.
Reproduction rights provided to Florida DOE.
Issue: How to Handle No Growth
Two Options:
• Score as “Unsatisfactory” – 1 Point
• Score as “Unsatisfactory,” but allow for one target
element with no growth to be “forgiven” if multiple
elements were targeted
1.877.411.7114
Copyright Learning Sciences International.
Reproduction rights provided to Florida DOE.
No Growth Sample Scenarios
Deliberate Practice Score
Start
3, 3, 3
3, 3, 3
3, 3, 3
2, 2, 2
2, 2, 2
2, 2, 2
2, 2, 2
3, 3
3, 3
2, 2
2, 2
2, 2
3
2
Finish
3, 3, 3
3, 3, 4
3, 4, 4
2, 2, 2
2, 2, 3
2, 3, 3
2, 3, 4
3, 3
3, 4
2, 2
2, 3
2, 4
3
2
No Growth = Unsatisfactory
1 (Unsatisfactory)
2 (Needs Improvement)
3 (Effective)
1 (Unsatisfactory)
1.6 (Needs Improvement)
2.3 (Needs Improvement)
2.7 (Effective)
1 (Unsatisfactory)
2.5 (Effective)
1 (Unsatisfactory)
2 (Needs Improvement)
2.5 (Effective)
1 (Unsatisfactory)
1 (Unsatisfactory)
Forgive One Element
1 (Unsatisfactory)
2.5 (Effective)
4 (Highly Effective)
1 (Unsatisfactory)
2 (Needs Improvement)
3 (Effective)
3.5 (Highly Effective)
1 (Unsatisfactory)
4 (Highly Effective)
1 (Unsatisfactory)
3 (Effective)
4 (Highly Effective)
1 (Unsatisfactory)
1 (Unsatisfactory)
Copyright Learning Sciences International. Reproduction rights provided to Florida DOE.
Highly Effective
Effective
Needs
Improvement
Unsatisfactory
1.877.411.7114
52
Two Methods for Calculating
Instructional Practice When Including Deliberate Practice
Additive:
Inclusive:
Status Score
+ Addition of Deliberate Practice
= Final Instructional Practice Score
Status Score
+ Deliberate Practice
= Final Instructional Practice Score
Deliberate
Practice
Deliberate
Practice
Status Score
Final Instructional Practice Score
Copyright Learning Sciences International. Reproduction rights provided to Florida DOE.
Status Score
Final Instructional Practice Score
1.877.411.7114
53
Additive Method
Considerations:
• Deliberate Practice is added to the
Instructional Practice Score
• Performance in Deliberate Practice
only adds to the Instructional Practice
score; it is not punitive
• Top scores are possible without growth
in Deliberate Practice
• High reward for high growth
• More teachers will rank higher on
the performance scale
Deliberate
Practice
Status Score
Final Instructional Practice Score
1.877.411.7114
Copyright Learning Sciences International.
Reproduction rights provided to Florida DOE.
54
Inclusive Method
Considerations:
• Deliberate Practice is a percentage of
the Instructional Practice Score
• Performance in Deliberate Practice can
be punitive if the teacher shows no
growth.
• Top scores are not possible without
growth in Deliberate Practice
• Reward for high growth and high status
score
• More teachers will rank lower on the
performance scale
Deliberate
Practice
Status Score
Final Instructional Practice Score
1.877.411.7114
Copyright Learning Sciences International.
Reproduction rights provided to Florida DOE.
55
Sample Scenarios
Typical Teacher
Deliberate Practice
Status Score
Additive Score:
Inclusive Score (40%, 60%):
Difference: .74 points
2.5
3.1
3.60 (Highly Effective)
2.86 (Effective)
The Additive method puts the teacher in the Highly Effective
range rather than the middle of the Effective range.
1.877.411.7114
Copyright Learning Sciences International.
Reproduction rights provided to Florida DOE.
District Table Group Activity
What are the benefits and drawbacks of the additive and
inclusive model for teachers and for students?
Additive
Inclusive
1.877.411.7114
Copyright Learning Sciences International.
Reproduction rights provided to Florida DOE.
57
FLDOE Recommendation
2012/13
2013/14
Additive
Inclusive
2012/13
2013/14
2014/15
Pilot
Additive
Inclusive
OR
1.877.411.7114
Copyright Learning Sciences International.
Reproduction rights provided to Florida DOE.
58
Next Steps
• Implementing the
Monitoring Plan
• Monitoring for results and
making adjustments
• Identifying and sharing
best practices
• Celebrating successes
1.877.411.7114
Copyright Learning Sciences International.
Reproduction rights provided to Florida DOE.
59
Remember to Complete the Survey!
THANK YOU!
1.877.411.7114
Copyright Learning Sciences International.
Reproduction rights provided to Florida DOE.
60