USING RESEARCH TO IMPROVE
TEACHING AND LEARNING
Israel Ministry of Education, Culture and Sports
Department of the Centers for
Professional Development of Teachers
November 28, 2005
USING RESEARCH TO IMPROVE
TEACHING AND LEARNING
Edward Crowe
Teachers for a New Era
Carnegie Corporation of New York
Academy for Educational Development
National Commission on Teaching and America’s Future
Hunter Foundation of Scotland
Scottish National Executive
High Quality Teacher Preparation
Why does it matter?
Essential precondition for these outcomes:
 Student readiness for college success
 College admissions
 Student retention in colleges and universities
 Graduation rate outcomes
Compelling evidence that teachers matter…
How teachers make a difference
Value-added assessment
Explaining differences in student learning
 Differences across districts—5%
 Across schools within districts—30%
 Teachers within schools within districts—65%
www.sas.com/govedu/edu/research.html
How have we responded?
Revised accreditation standards and processes
Licensure and program approval policies
Teacher testing
“Highly qualified” teachers
Alternate routes into the classroom
Evidence that what we’re doing is working?
Beginning Teacher Attrition in the United States:
A Serious Problem
50%
40%
30%
20%
10%
14%
24%
33%
40%
46%
1st Year
2nd Year
3rd Year
4th Year
5th Year
0%
International Study:
Why are teachers a focus?
 Research shows that teaching quality is a key influence
on student learning
 Teachers are the largest item in school budgets:
-- The teaching workforce is large (2.5% of total labour force)
-- Teachers’ compensation: 63% of spending on schools
 Teachers’ roles are changing
 Efforts to improve schools will fail if there are serious
shortfalls in teacher supply and quality
8
Teaching Quality
Why does it matter?
 Efforts to improve schools will fail if there are serious shortfalls
in teacher supply and quality (OECD)
 Demonstrable links between teacher experience and student
learning (Sanders, New Teacher Center)
 Impossible to achieve key student learning outcomes without
effective teachers:
 Student readiness for college success
 Adequate and diverse supply of skilled talent in key
employment fields (National Research Council, National
Science Foundation, Thomas Friedman)
Evidence that what we’re doing is working?
Teacher Turnover in the United States
A Revolving Door
2,376,677
69%
Not In Transition
539,778
16%
Leaving
534,861
15%
Incoming
Total Teaching Force, 1999-2000: 3,451,316
DEVELOPING A RESEARCH AGENDA
Professionally
Rewarding Career
Paths
Hiring Incentives
Professional Development
Performance Pay
Board Certification
Schools
Professional
Teacher
Preparation
Professional
Teaching
Conditions
Professional
Teaching
Conditions
Professionally
Rewarding
Career Paths
Schools
Small Schools
Leadership
and
Mentoring
Professional
Teacher Preparation
Professionally
Rewarding
Career Paths
Schools
Professional
Teaching
Conditions
Careful Selection, Academic Preparation,
Clinical Practice & Induction Residencies,
Technology, and Assessment of Teachers & Programs
Accreditation, Licensing & Certification
Professional
Teacher Preparation
Professionally
Rewarding
Career
Advancement
Conditions
for
Professional
Teaching
Practice
High-Performing
Schools
Professional
Teacher
Preparation
Support from
Replace Solo
Induction to
Teaching with
Accomplished
Learning
Teaching, with
Communities
Professional
st
Compensation 21 Century
Learning
Preparation, Induction,
Assessment
NO DREAM DENIED:
A Pledge to America’s Children
Organize schools for success
Invest in quality teacher preparation
Create professionally rewarding careers for teachers
www.nctaf.org
Some Key Research Questions
Teacher Preparation and Professional Development
Students Preparing to be Teachers
What students bring to the program (student characteristics
predictive of success)
o Demographics
o Attitudes and values
o Previous education and work experience
Preparing New Teachers
(also applies to professional development programs)
The “treatment” (the program) and its effects on
o Candidate knowledge
o Teaching skills and teaching performance
Program Outcomes
Research on important outcomes
o Teaching performance
o Persistence in the profession
o P-12 student achievement
Studying Teacher Education:
Report of the AERA Panel
on Research and Teacher Education
www.aera.net/publications/?id=793
Improving the quality of education research
Applying Multiple Social Science Research Methods
to Educational Problems
American Psychological Association
National Research Council
American Educational Research Association
National Science Foundation
www7.nationalacademies.org/cfe/Multiple_Methods_Workshop_Agenda.html
Learning from Attempts to Improve Schooling:
The Contribution of Methodological Diversity
Stephen W. Raudenbush
University of Chicago
 Focus on evaluating the impact of interventions designed to improve
teaching and learning
 Randomized experiments are the gold standard for assessing these
effects
Learning from Attempts to Improve Schooling:
The Contribution of Methodological Diversity
Success depends on a well-integrated, methodologically diverse
research effort:
 Precisely define viable instructional aims and outcome variables
 Identify, refine, and test promising new interventions
 Clarify the instructional needs of children in various settings
 Learn whether interventions work, how they work and for whom
 CONCLUSION: Diverse methods can be combined, and a healthy
scientific community should collaborate to achieve these aims
Learning from Attempts to Improve Schooling:
The Contribution of Methodological Diversity
Recommendations
“Research using a variety of methods is essential:
o To define the student outcomes we seek to change and to build and
validate assessments of those outcomes;
o To support novel thinking about how best to intervene, to support
preliminary studies of those interventions; and
o To enable educators to test the feasibility of implementing those
interventions in ordinary school settings”
--Raudenbush, 2004
National and International
Research Efforts
On Teaching and Learning
Teachers for a New Era
Carnegie Corporation of New York
Annenberg and Ford Foundations
Academy for Educational Development
RAND
www.teachersforanewera.org
Teachers for a New Era
3 design principles
Decisions based on evidence
Engagement with the arts and sciences
Teaching as academically taught clinical profession
Teachers for a New Era
Key design principle: Decisions based on evidence
 Assessments linking program preparation to teaching
performance
 Assessments of teacher impact on student learning
Teachers for a New Era
11 colleges and universities
 4 pupil assessment pilot studies in 2004-05
 7 pupil assessment pilot studies in 2005-06
 Continuation of first 4 studies in 2005-06
Scottish Teachers for a New Era
www.abdn.ac.uk/cass/bulletin/school.shtml#b
Improving Teacher Supply and Effectiveness
Phillip McKenzie and Paulo Santiago
Directorate for Education
Meeting of OECD Education Ministers
Raisin
34
www.www.oecd.org/edu/teacherpolicy g the Quality of
Learning for All
Dublin, 18-19 March 2004
Campbell Collaboration
• The Campbell Collaboration is an international effort whose
mission is to prepare, maintain, and make accessible
systematic reviews of studies on the effects of interventions
• Funders: Smith Richardson, Robert Wood Johnson,
Rockefeller and Knight Foundations; governments of
Denmark, Norway, Finland
www.campbellcollaboration.org
Campbell Collaboration
Objectives
• Use standards for quality of evidence that are transparent and
can be critiqued
• Focus on randomized field trials first and on high quality
nonrandomized field trials second
Campbell Collaboration
Systematic reviews of research evidence
Randomized trials completed and reviewed:
 Teacher professional development programs
 Evaluation of Reading First Program
www.campbellcollaboration.org/Bellagio/papers
Ohio Teacher Quality Partnership
Project Focus
• Teacher candidates, impact of program design, program quality outcomes,
P-12 student learning
• Funding support from Ohio state education and higher education agencies,
Knowledge Works Foundation, Carnegie Corporation, Gund Foundation
(Ohio), Proctor & Gamble, Joyce Foundation, U.S. Congress
www.teacherqualitypartnership.org
Teacher Preparation Accountability System
Value-Added Teacher Preparation Program Assessment Model
Louisiana Board of Regents
Board of Elementary and Secondary Education
http://asa.regents.state.la.us/TE/value_added_model
Teacher Preparation:
Does the Pathway Make a Difference?
Project Focus
 Which teachers are most effective in improving student
outcomes?
 What characterizes their preparation, pathways into teaching
and qualifications to teach?
 What is the cost effectiveness of various pathways into
teaching?
www.teacherpolicyresearch.org
Assessments of Teaching Performance
 CLASS--Classroom Assessment Scoring System [Robert
Pianta, University of Virginia]
http://convention.allacademic.com/aera2004/
view_presenter.html?part_id1=905783
 SOLO--Structure of Observed Learning Outcomes
www.learningandteaching.info/learning/solo.htm
 Reliability and Validity of Instruments
Connecting research and practice
Developing the missing links
Research on Teaching and Learning
Value-added studies
Some current efforts:
Boston Public Schools
Dallas Independent School District
Denver Public Schools
Milwaukee Public Schools
RAND study funded by Carnegie Corporation
New York State:
All 4th grade teachers, 1994-2002
Project Focus:
Impact of assessment and accountability
Boyd, Lankford, Loeb, and Wyckoff (2005)
www.teacherpolicyresearch.org
Impact of assessment-based accountability
on teacher turnover
 Since NY State use of fourth grade testing, the turnover rate of fourth grade
teachers has decreased relative to teachers in other grades.
 Decrease is consistent across urban, suburban and rural areas and across
schools with differing levels of student performance.
 Little evidence that high-ability teachers are more likely to leave fourth
grade
 Drop in turnover less for more experienced teachers than for recent hires,
especially in suburban districts.
 New 4th grade teachers in lowest performing schools more likely to have
attended highly competitive undergraduate institutions.
Public Education Fund-Chattanooga
 Highly effective teaching
 High school reform
 Teacher development--reading and urban education
 Instructional leadership development
www.pefchattanooga.org/www/docs/3/new_society/
www.pefchattanooga.org/www/docs/5/effective_teaching/
Evidence Based Education-UK




Create evidence (teaching and learning)
Disseminate evidence (to teachers, schools, policy makers)
Promote a culture of evidence
Work against ‘non-evidence-based’ policy and practice
University of Durham
Curriculum, Evaluation and Management Centre
www.cemcentre.org/ebeuk/default.asp
Battelle for Kids
Improve teaching and learning
Project SOAR





Analyze, share and use data
78 Ohio school districts
718 schools
200,000 students (20% of all grades 3-8 in Ohio)
SAS Institute
www.battelleforkids.com/b4k/rt/about/our_work
SAS Institute
Schooling effectiveness
 Schooling effectiveness “technology”
 Education value-added assessment system (EVAAS)
 Bill Sanders, June Rivers
 Papers, resources, other links
www.sas.com/govedu/edu/research.html
The biggest challenges in research
on teacher education and professional development?
Keeping the focus on outcomes
“In the end the final test of a medical school
is its outcome in the matter of clinicians.”
Abraham Flexner (1910)
Designing and using metrics for success
Set goals
Establish indicators
Measure progress
Students
Teachers
Preparation programs
Professional development initiatives
Schools
Policy
After the Test:
Closing the Achievement Gaps With Data
Bay Area School Reform Collaborative
Compared improving and failing schools on achievement gap




School culture and leaders supportive of intensive focus on assessment
Definition of equity includes student achievement
Teachers learn about and use assessment tools regularly
Successful schools function as learning communities
www.ncrel.org/gap/studies/basrc.htm
Value-added assessment studies
Teacher Effects
 Estimate effects of teachers on student learning gains or
achievement growth
 Use of complex models to account for nonrandom pupil
assignments, variability in non-school inputs, background
variables, student attributes
www.rand.org/pubs/monographs/2004/RAND_MG158.sum.pdf
Overview of value-added assessment
Student learning gains the “gold standard” for evaluating
teaching quality and preparation program effectiveness
But…. many challenges as research projects move forward with
value-added assessment
Some value-added challenges
Access to relevant data
Complexity of methods
Research capacity of teacher education programs
Commitment to assessment and using evidence
P-12 test quality
Opening the VAM “black box”
Interpreting the results
The Flexner Report
“Society reaps at this moment but a small fraction of the
advantage which current knowledge has the power to confer.”
Abraham Flexner (1910)
[Bulletin No. 4, Carnegie Foundation for the Advancement of Teaching]
www.carnegiefoundation.org
Key Flexner Recommendations
Medical education in the U.S. and Canada
Raise entry standards for students
Close 80% of medical schools
Strengthen state oversight
Faculty should engage in research
Practitioners should be scientists
Full-time clinical faculty
Medical school-hospital clinical ties
The future of university teacher preparation?
USING RESEARCH TO IMPROVE
TEACHING AND LEARNING
Edward Crowe
Teachers for a New Era
Carnegie Corporation of New York
Academy for Educational Development
National Commission on Teaching and America’s Future
Hunter Foundation of Scotland
Scottish National Executive