Teacher Research Programs:
An Effective Form of Professional Development to Increase
Student Achievement & Benefit the Economy
2008
AGU Fall Meeting
December 16, 2008
Jay Dubner
Columbia University
Practice What You Teach
Columbia University’s Summer Research Program for Science
Teachers was established in 1990 by Dr. Samuel Silverstein to
contribute to the improvement of science achievement of students
by providing New York metropolitan area middle and high school
teachers with experiences in the practice of science. Teachers
become members of research teams for 2 consecutive summers.
1990-2008
• 241 middle and high
school science
teachers
• 88% public schools
• 55% women
• 46% minorities
7
3
Program Background
- Program participation open to New York metropolitan
area public, private & parochial science teachers
- Program receives 5 to 7 applications for each
available Fellowship
- 12 new participants are accepted each summer, joined
by 12 returning participants from previous summer
(peer coaches)
Program Summary
Two Summer Commitment
- 8 Weeks immersion in each of two consecutive summers
- 4 days per week in research lab
- 1 day per week in pedagogical meeting
Weekly summer seminar series serves several purposes:
Broaden Science Knowledge: Columbia’s faculty or a speaker from outside
the University gives a talk on a topic of scientific interest.
Develop a Professional Learning Community: Meetings provide protected
time for teachers’ energy to be directed towards solving persistent and common
problems encountered by students. Teachers lead hands-on demonstrations, engage
in discussions of common classroom problems, and exchange ideas on solutions
that work.
Peer Coaching: Second-year participants provide guidance and
encouragement to first-year participants
Program Performance: Meetings enable the members of the Advisory
Committee to become acquainted with the teachers personally, and to learn about
their professional needs and concerns. SRP culminates with a three-day End-ofSummer Symposium at which time the program’s Advisory Committee is able to
assess each teacher's grasp of the principles on which his/her work is based.
Program ‘Perks’
• Stipend - $6,000 each summer
• Classroom Enhancement Funds - $1,000 following each of the two
summers
• Research Group Funds – $1,000 in each of the two summers
• Travel to Professional Conference – Science education and/or
scientific society conferences in each of the school years following
participation in the program
• International Program – Following completion of program, 2 teachers
are selected to participate in an exchange of science teachers with
Singapore’s Ministry of Education and 2 teachers are selected to
participate in a 3rd summer (winter?) of research in Australia
School Year Classroom Support
Teachers are afforded the opportunity to have the graduate student with whom they
worked alongside in the lab, visit their classrooms one day a month.
Evaluation
A unique aspect of Columbia University’s Summer Research
Program is its emphasis on evaluation. Evaluation has
informed and guided the Program’s evolution from its
inception, and enabled it to respond to the changing needs of
teachers
Student Outcomes Studies
1.
Columbia’s Summer Research Program
(1993-present)
– Collected data from the NYC public high
schools, the program’s largest cohort, with the
assistance of the school science department
chairs and the NYC Dept. of Ed.’s Division of
Assessment & Accountability
2.
NSF Multi-site Study (1998-2002)
– 8 science teacher research programs
– Arkansas, California, Georgia, Idaho, New York,
Oregon, Texas, & Washington State
Instruments Administered
NSF Multi-site Study
1999-2000
•
•
•
•
•
Pre-program survey*
Post-program survey*
Mentor survey*
Student Attitudinal Survey*
Student Cognitive Tests
– Biology & Chemistry
*Surveys available at www.SweptStudy.org
(Revised versions of program and mentor surveys available at
www.RETNetwork.org)
Comparison of Study and Control Science Teachers (1)
Study
Teachers
Comparison
Teachers
Undergraduate or graduate major
or minor in biology, chemistry
physics, earth science or math
94%
87%
Advanced degree
63%
62.5%
Hours course credit beyond
highest degree
34 hours
54 hours
Years of teaching experience
7.5
13.2
Years of science teaching
experience
6.5
11.8
9.7 hours
9.25 hours
Assigned textbook covered in
science course
60%
59%
Used textbook publisher’s tests
rarely or never
68%
68%
Used textbook publisher’s tests
sometimes
18%
26%
Used textbook publisher’s tests
frequently
14%
6%
Hours/week spent outside of class
preparing for teaching
(1)
Data for 32 Study teachers and 32 Control teachers
Self-reported Changes in Attitudes and Classroom Practices
Study teachers
Comparison Teachers
Preteaching
survey
Postteaching
survey
Change
Preteaching
survey
Postteaching
survey
Traditional Goals & Objectives
247
256
7
254
254
0
Inquiry Goals & Objectives
244
250
6
258
232
-26*
Traditional Teaching Methods
248
250
2
260
249
-11*
Traditional Student Activities
245
251
6
258
256
-2
Inquiry Student Activities
247
257
10
249
235
-14*
Teacher Efficacy
240
255
15
247
252
5
Scale
Number of teachers =
58
Change
58
* Difference in two change scores is significant at the p<0.05 level
• The items that constitute each scale were identified through a confirmatory factor analysis and the scale scores are
standardized factor scores with a mean of 250 and a standard error of 50. Information about item wording and factor
loadings can be found at www.sweptstudy.org.
• These comparisons are based on 58 teachers and comparison teachers who completed all of the items comprising all of
the scales. Five teachers failed to complete all of the items in one or more of the scales.
Student Assessments of Classroom Practices
Study
Comparison
Number of
Students
Mean
Number of
Students
Mean
Reflected on course material by writing in a
notebook.2
2,817
2.55
1,469
2.39*
Used primary sources such as journals.1
2,817
1.93
1,469
1.8*
Listened to guest speakers or went on field trips.1
2,817
1.66
1,469
1.54*
Explored career opportunities in math or
technology.1
2,817
1.86
1,469
1.69*
Teacher has encouraged me to think about math/
science career.2
2,849
2.84
1,481
2.64*
•
1
Difference between Study and Comparison teachers’ students was statistically significant at the p<0.05 level
Measured on a 5 point scale where 1 = never, 2 = 1-2 times a month, 3 = 1-2 times a week, 4 = almost every class, 5 = every class in response
to the question “Approximately how often did you engage in the following learning activities in the mathematics/science class that you
are currently taking?”
2 Measured on a 5 point scale from strongly disagree (1) to strongly agree (5)
Changes in Student Achievement in Science
Study Teachers’ Students
Comparison Teachers’ Students
Subject
Pre-course
Postcourse
Change
Pre-course
Postcourse
Change
Second year
Biology
267
292
25*
263
284
21*
Chemistry
282
310
28*
275
295
20*
* Difference in two change scores is significant at the .05 level
The test scores were derived through IRT methods and converted into a scale score with a mean of 250 and a standard error of 50.
The student achievement scores are derived from tests constructed by Westat, and the Westat tests were built on sets of items that were used
previously in the National Assessment of Educational Progress (NAEP), the Third International Mathematics and Science Study (TIMSS), and
the Longitudinal Study of American Youth (LSAY).
2008 Multi-Program Surveys
(5 programs participated)
www.RETNetwork.org
Summer Research Program
1993 – 2008
•
•
•
•
•
Pre-program survey
Post-program survey
Mentor survey
Spring implementation survey
Student participation in Science
Clubs and Intel-type competitions
• Scores on NYS standardized science
exams (Regents)
2008 Teacher Survey Data
•
100% reported developing new or revised content to lessons and/or labs*
*341 standards-based lesson plans are on the program’s website:
www.ScienceTeacherProgram.org
•
91% reported increasing hands-on activities in their classrooms and/or new
laboratory exercises in response to their experiences at Columbia
•
78% reported including lessons on science careers and related job requirements
with their students
•
74% reported introducing new technologies in their classroom instruction (e.g.;
chromatography, pipetting, PowerPoint)
•
61% reported reading scientific journals more frequently
•
52% reported increased requirements for formal written reports and/or oral
presentation requirements
•
48% reported assuming new leadership roles/responsibilities in their
school/district/region
• Columbia’s Summer Research Program collects quantitative
student data from NYC public high schools
- Study Group -- Students in science classes of program
teachers
- Control Group -- Students in science classes of nonparticipating teachers from the same schools
Intel Science Talent Competition
0.80%
0.72%
0.59%
Percent
0.60%
0.44%
0.40%
0.40%
Teachers
0.20%
Control Group
0.00%
Prior
After 2nd Year
Year in Program
18 Complete Data Sets
NYS Science Regents Exam Pass Rate
32 Complete Datasets
60%
51.4%
57.0% *
53.5%
48.5%
50%
* ~27% gain
48.5%
45.0%
40%
Percentage
Passing
30%
Teachers
20%
Control
10%
0%
Prior
After 1 year
Year of Program Participation
P<.01
SRP Students: 5,701
Comparison Students: 28,426
After 2 years
NYS Science Regents Exams Pass Rate
70%
60.6% *
60%
Percentage Passing
51.4%
53.5%
50%
40%
48.5%
57.0%
48.5%
50.1%
* ~33% gain
45.0%
30%
Teachers
20%
Control
10%
0%
Prior
After 1 year
After 2 years
Year of Program Participation
SRP Students - 6,915
Comparison Students - 34,544
32 Complete Data Sets
1st Year Out
(20 datasets)
NYC Economic Impact
• Regents Diploma – Must pass 5 Regents exams1
(includes 1 Science Exam)
• NYC spends $12,930 per public school student2
• Student takes 5 courses/year = $2,586 per course
• 20 more students passing Regents over 2 years
• 20 students X $2,586 = $51,720 each teacher/year
1
2
New York State Education Department, New York State Total Public Report Card, 2005
The New York Sun, New York Outpacing Other States in School Spending, April 4, 2006
National Economic
Impacts
• High School dropouts earn $392,000 less in lifetime
earnings than a high school graduate thereby paying less
taxes1. At a tax rate of 20%, loss of $78,400 in local and
federal revenues.
• High School graduates are less likely to commit crimes.
Increasing HS completion of males by just 1% would
reduce criminal justice system costs by $1.4 billion per year
• High School graduates receive higher salaries thereby not
needing food stamps, housing assistance, etc. Estimated
annual savings are $8 billion per year
1
Fogg, N., Harrington, P., & Khatiwada, I., The Longterm Labor Market
Consequences of Dropping out of High School in Pennsylvania, Center for
Market Labor Studies, Northeastern University, October 2007
Publishing Our Findings
• “Scientific Work Experience Programs for Teachers: A multisite evaluation of impact on teachers and on student
achievement”
– Status: Very final draft form. Expected to be submitted in the next few
weeks for peer review: Educational Evaluation and Policy Analysis
• “High School Science Teacher Participation in Columbia
University’s Summer Research Program Improves Student
Interest and Achievement in Science”
– Status: Readying for final draft. To be submitted for publication in the
next 2 months. Publication to be determined.
Current Funding Sources
•
•
•
•
•
•
•
Ambrose Monell Foundation
Bay and Paul Foundations
Hebrew Technical Institute
Mellam Family Foundation
National Institutes of Health – SEPA Program
National Science Foundation
New York Times Company Foundation
Columbia University’s
Summer Research Program for
Science Teachers
www.ScienceTeacherProgram.org