NSF 0653270
An Integrative Science Success, Teaching and Retention Program for STEM
Education
PI:
Co-PIs:
Benjamin C. Flores bflores@utep.edu
James Becvar jbecvar@utep.edu
Helmut Knaust hknaust@utep.edu
Jorge Lopez jorgelopez@ute.edu
Josefina Tinajero Tinajero@utep.edu
The University of Texas at El Paso
Preamble
The University of Texas at El Paso (UTEP) is largely a commuter institution with
a student body that exceeds 21,000 students. Most of our students work while
attending school. The majority of our students are first-generation college
students living in a city with an estimated 2008 median household income of
$37,600. Around 80% of our students are Hispanic, with the "other" label
representing international students, primarily Mexican nationals that cross the
border daily to attend our institution.
One of the major challenges our
institution faces is student retention and success. Over the years, UTEP has
implemented a number of strategies to increase the number of students that
pursue and earn degrees in STEM disciplines. The main activity supported by
the current STEP project, peer led team learning (PLTL) falls in this category.
A team of five professors with distinguished trajectories have assembled
together to make PLTL the student success strategy of choice in several
gateway courses
1. What are the successes you have experienced?
At least 6 faculty members in the Chemistry Department are deeply involved in
PLTL and are clearly aware of the benefits of these PLTL activities.
In
addition, the Chair and the Assistant Chair are significantly aware of the value
of PLTL activities. The effect on attitude of the students that work as peer
leaders has been extremely positive. They realize that they can be successful
and they can make a difference in the academic performance of the students
they lead. Peer Leaders definitely gain stature and confidence of their worth
by the experience this program provides. The philosophy of PLTL is now
ingrained in the Chemistry Department.
In the Physics Department, two professors have taken charge of the delivery of
the two gateway courses, the corresponding PLTL workshops, and the
supervision of their peer leaders. They report that there is an increased
overall student satisfaction based on their own surveys and official university
evaluations. The professors are also very please with the improvements in
grade distributions (higher mean with respect to historical data), excellent
student rapport of peer leaders, efficient class organization with expanded
course syllabi, increased use of pod casts by students, and increased hours of
exposure of material to students through workshop time.
2. How many students have been impacted by each of the activities in your
project? What is the nature of the impact? What are the demographics of the
participants?
PLTL workshops have been implemented in second semester general chemistry,
organic chemistry for majors and non-majors, pre-calculus, and two physics
courses. The tables below show the number of students enrolled in each
course and corresponding PLTL workshop by term, by ethnicity. The
overwhelming majority of students are Hispanic. A small number of students
are African American or Native American. The demographics coincide with the
region’s population.
Pre-Calculus
Study Participants
Female
Fall 2008 (n=788)
Spring 2009 (n=622)
Fall 2009 (n=768)
344
263
294
Male
Asian
444
359
474
14
8
16
Asian
African
American
18
17
12
Hispanic
White
630
507
635
59
34
55
African
American
5
2
7
Hispanic
White
141
144
171
17
11
13
African
American
1
2
1
Hispanic
White
137
154
89
12
12
8
African
American
0
3
0
Hispanic
White
34
56
31
7
8
2
Native
American
2
1
2
Other
Native
American
0
0
2
Other
Native
American
1
0
0
Other
Native
American
0
0
0
Other
65
55
48
Physics 2420
Study Participants
Female
Male
Fall 2008 (n=196)
Spring 2009 (n=197)
Fall 2009 (n=234)
42
49
58
154
148
176
Study Participants
Female
Male
Fall 2008 (n=191)
Spring 2009 (n=212)
Fall 2009 (n=117)
48
51
28
143
161
89
Female
Male
23
43
25
21
32
14
2
2
4
31
38
37
Physics 2421
Asian
5
4
1
35
40
18
Chemistry 3321
Study Participants
Fall 2007 (n=44)
Fall 2008 (n=75)
Fall 2009 (n=39)
Asian
0
2
2
3
6
4
Chemistry 3324
Study Participants
Female
Male
Fall 2007 (n=183)
Spring 2008 (n=136)
Fall 2008 (n=168)
Spring 2009 (n=107)
Fall 2009 (n=157)
103
75
98
76
86
80
61
70
31
71
Study Participants
Female
Male
Fall 2007 (n=185)
Spring 2008 (n=214)
Fall 2008 (n=160)
Spring 2009 (n=220)
Fall 2009 (n=168)
94
125
95
115
88
91
89
65
105
80
Asian
2
0
5
3
6
African
American
3
6
3
3
2
Hispanic
White
147
113
141
84
121
22
13
14
7
17
African
American
9
4
2
3
4
Hispanic
White
145
164
128
178
137
16
20
9
21
12
Native
American
1
1
1
0
2
Other
Native
American
1
1
0
1
0
Other
8
3
4
10
9
Chemistry 1306
Asian
1
6
2
6
4
This grant has also given us the opportunity not only to identify talent but more
importantly to develop it by focusing on PLTL activities. These activities speak
to the importance of giving STEM majors the opportunity to take on leadership
roles. To date, 125 peer leaders have led PLTL workshops, receiving training
and financial support, through this grant. 97% of them have either graduated
or are still attending the university, in sharp contrast to the 49% six-year
retention/graduation rate seen university-wide.
3. Have the project activities been integrated within your (department,
school, college and/or university)? If so, what is the nature of the integration?
The primary activity of this project has been the implementation of Peer Led
Team Learning (PLTL) workshops in five gateway courses (two courses are in
Chemistry, two courses are in Physics, and one course is in Mathematics). As of
fall 2008, all workshops were seamlessly integrated to the degree plans. That
means that all STEM students required to take these courses were also required
to take the workshops concurrently. Faculty adjusted their lecture time, and
allowed for extra time to coordinate workshop activities with peer leaders.
Students adjusted their schedules to include two-hour workshops.
4. Has this project had any impact beyond the intended project goals, for
example on other students, faculty, departments, or institutions?
We have been promoting PLTL as a best practice in STEM education at several
institutions in the US. There is a strong interest by UT Pan American and UT
Brownsville and the New York City College of Technology to work with us on a
future dissemination phase of the project. The PI and two of the co-PIs have
visited universities in Chile to promote team-based learning and peer led
workshops as a means of increasing student performance in STEM courses. The
13
19
19
11
11
ideas shared thus far have been extremely well received by the Chilean
Ministry of Education and several universities.
5. If you have partner institutions, what are you achieving through this
partnership? Has the role of your partners changed from what was specified in
the original proposal?
Formally there are no partner institutions. Informally we have begun working
with El Paso Community College to implement peer led team learning in
Chemistry and Calculus courses.
6. Have you had any positive surprises, including unexpected benefits from this
grant?
There are now many ‘champions’ of PLTL in the Chemistry department. Our
expectation was that the young generation of untenured and newly tenured
faculty would be most likely to participate and would be most in favor of the
student-centered approach that PLTL provides. We were pleasantly surprised
that established, senior faculty members would get behind and readily adopt
this approach. The chair of the department clearly sees why this PLTL
approach is the future of instruction. In fact, he wants to find ways to expand
the program and offer all chemistry majors an opportunity to serve as a peer
leaders as part of their undergraduate experience.
In the department of Physics, there is now an increased interest from majors
who have taken the gateway course sequence to become peer leaders. The
degree of enthusiasm from more seasoned peer leaders has helped in making
the start-of-semester training seminars very engaging and productive. Also,
the chair of the department has joined in the effort and she decided to teach a
section of one of the gateway courses.
7. What challenges have you faced when implementing your STEP grant?
The main challenge has been related to the change in administration in the
College of Science at the dean and department chair levels. Since the start of
the STEP project in fall 2007, the deanship has changed once.
Similarly,
chairmanships in the departments of Physics and Mathematics have also seen
changes. Two of the co-PIs were chairs at the time and have now returned to
the faculty ranks. This change in guard has impacted the project as changes in
college and departmental priorities were rolled out and the support for the
project took time to rebuild. At this point the decision to directly involve the
Provost is seen as a major development that will help in stabilizing the course
of the project.
Another major challenge has been related to developing an interdisciplinary
programmatic approach that crosses disciplinary boundaries but respects
cultural differences of three rather distinct departments. For instance, the
department of Chemistry has almost a decade of experience in employing peer
leaders in freshman chemistry courses, which not only served to substantially
improve student passing rates but also served to promote Chemistry as a
major.
Peer led team learning enjoys substantial support by the
administration and faculty alike. In contrast, the department of Physics has
had more limited experience in peer led team learning as it has encouraged
undergraduate research as the principal enriching experience for its majors.
Consequently, the idea of serving as peer leader is still being promoted among
the administration, faculty and students. In the department of Mathematics,
peer leading has been applied for years in the context of supplemental
instruction. However, math faculty and their peer leaders have had to adjust
to the notion of team learning. With time, these challenges will be overcome.
8. Describe any significant changes or deviations from the planned grant
activities that you have made or would like to make. Explain why these
changes were necessary or seem to be necessary. Provide an explanation
about how these changes have already impacted or are expected to impact the
project.
There have been three major deviations from the planned grant activities.
These involve the following:



Professional skills seminar for sophomore STEM students.
Summer academy for peer leaders.
Project assessment and coordination.
One of the original activities involved the addition of a professional skills
seminar for sophomore students in all STEM degree plans. At the onset of the
grant, the Texas State Legislature mandated that all state institutions decrease
the number of credit hours in all baccalaureate programs from 128 to 120
credit hours. In response, five STEM departments streamlined their degree
plans to reduce requirements. The prevailing strategy was to take off courses
outside the discipline that were not part of the university core. Consequently
the idea of implementing the sophomore seminar could not get a solid footing
in the College of Science. Although this outcome was beyond the control of the
STEP management team, on the positive side, we are confident that the
reduction in credit hours will aid in increasing graduation rates and reducing
the time to graduation time, which is always an issue with our largely nontraditional student population.
Another original activity involved the development of a summer academy for
STEP peer leaders. The academy envisioned creating a cohort of STEM peer
leaders taking secondary education courses, first and foremost, to enhance
their pedagogical skills. However, after two years of trial and unsuccessful
recruitment, enrollment in the secondary education summer courses was
negligible. The feedback received from peer leaders was that they did not
perceive how the summer academy was directly linked to advancing their
degree plans, except for those already intending to minor in secondary
education in preparation for a career as science and mathematics high school
teachers. Since the primary goal of the academy was to tool the peer leaders
with effective teaching skills, we decided on implementing pre- and postsemester seminars for peer leaders with the aim of training them in the
essentials of team-based learning and reflecting on their experience in heading
PLTL workshops.
A positive alteration in the implementation of the project has been the hiring
of a peer leader coordinator and a director of evaluation. The peer leader
coordinator assists in the recruiting, selection, and appointment of students as
peer leaders, in making arrangements for training seminars, and in conducting
student and peer leader surveys.
The director of evaluation handles the
evaluation of the project, conducting longitudinal passing-rate and retention
studies of students and peer leaders, creating and analyzing participant
surveys, and participating in discussions about the progress of the project with
the internal and external advisory boards.
9. What intermediate measures and metrics are you using to monitor the
progress of your project? [For example, say the primary goal of your project is
to improve graduation rates and one of the strategies you proposed to
implement was changing the way that you teach calculus courses. An
intermediate measure of success for your project might be improvements in
calculus pass rates because improvements in pass rates would likely translate
into improved graduation rates by the end of the grant period.]
For assessment purposes, we are looking both at "hard numbers" and perception
of participants. The number of students enrolled in the five PLTL courses,
course pass rates, and opinions of students, peer leaders and faculty are
tracked by semester. Peer leaders are monitored for cumulative grade point
average, retention, and graduation. We are also looking to see if there is an
increased interest in teaching mathematics and science at the secondary level
for peer leaders. Several peer leaders and faculty have been interviewed and
PLTL students and peer leaders are surveyed at the end of each term. Two
additional intermediate measures will be assessed during the final two years of
the grant: success in follow-up course for Pre-Calculus (Calculus) and yearly
retention and/or graduation for all five courses.
10. Do these intermediate measures indicate that you are on track to meet
your project goals? Include data that support your response.
A longitudinal study was conducted for the five PLTL courses. The quantity of
students enrolled in the classes has fluctuated, but appears to have dropped
off in several of the courses. Further research is needed to find out if this is
due to changed degree plans or if there is some other cause. The five selected
courses are introductory courses required for engineers and science majors,
along with selected majors in several other colleges. University enrollment has
continued to see a modest increase but the UT System has implemented a
requirement to reduce the number of credit hours for bachelor's degrees which
could be affecting the quantity of students enrolling in the courses. Grades
have also decreased in Physics, but in interviewing several peer leaders and
faculty, this could be a result from a change in instructor and exam structure.
Interestingly, several experienced peer leaders felt that some students were
not reading the text and those that were not passing were the same students
not attending workshop or coming prepared.
Opinion data obtained through surveys of students and peer leaders, and
interviews with peer leaders and faculty are extremely positive regarding the
use of PLTL workshops. A recent survey of students in the five STEP
PLTL courses was administered at the end of the fall 2009 term that
addressed whether 1) students felt the mandatory two-hour weekly small-group
workshops were a better format to increase content understanding, 2) the
workshop peer leaders were knowledgeable and capable of presenting the
material, and 3) the workshop materials and amount of time spent in
workshops were meeting stated objectives.
The objectives include
reinforcing concepts initially presented in lecture through both individual and
group work, including hands-on activities. Given a scale from 1 to 5 (strongly
disagree to strongly agree), a mean 4.611 or 93% (n=644 of 693) students felt
the workshops were closely related to the material taught in the lectures. 84%
felt the workshop activities better prepared them for tests while 88% believed
workshop participation would improve their grade. While higher grades
(increased passing rates) across disciplines do not mimic those found in the
general chemistry class used as the base PLTL model for this current
experiment, it is premature to comment as to the long-term effects the PLTL
workshop will have on course success (grade), retention, and graduation.
Even with increasing contact hours by one hour, 81% of the students said “I
would recommend courses with workshops to other students.” A mean 4.4839
felt the workshop peer leader was well prepared and 93% (mean 4.5578) said
their peer leader was knowledgeable in the subject area. 85% agreed that the
workshops “are a big help in learning to solve problems.” A 5 point Likert scale
(1-materials do not meet this objective at all to 5-materials are excellent and
meeting this objective) received the following mean and percentage regarding
workshop materials presented:
Workshop materials are:
Well connected with the lecture
Challenging
Developed to review fundamentals
Useful for group work
Helpful for further study
Useful for reinforcing concepts
Mean
% receiving 4 or 5
4.3979
88%
4.0627
4.2718
4.1616
4.3125
4.3884
77%
84%
80%
85%
88%
Comments about the peer leader/workshop structure varied from not liking
workshops and peer leaders at all to “I really enjoy the workshops because it
gives us (and other students as well) a chance to fully grasp concepts presented
in previous lectures. Being in a smaller group with a peer leader also makes it
easier to ask questions.”
The faculty perspective is represented by comments from a veteran professor
in chemistry regarding peer leaders. "I don't dictate to them: 'You will go over
problems 17, 39, and 95 at the end of the chapter. Instead I tell them that they
need to make sure that their students understand the concept of limiting
reagents or the concept of enthalpy change, and I let them decide how they're
going to instruct their students. It puts them a bit ill at ease," he
acknowledged, "but that's part of the confidence building. That's a very strong
philosophical component in this program." Faculty involved in the project think
there has been a positive psychological effect on the leaders as they perceive
that they can be successful and can make a difference to the students they
lead. As one stated, “peer leaders gain stature and confidence of their worth
by the experience this program provides.”
125 peer leaders have participated since fall 2007. Since STEP was
implemented in the spring 2008, 117 peer leaders have participated. Of these
31 percent have graduated and 66 percent are still pursuing degrees, for a
combined 97 percent retention/graduation rate. This is in sharp contrast with
the overall undergraduate student population six-year retention/graduation
rate which stands at a combined 49 percent.
Overall, in discussing the PLTL model with students, peer leaders, and faculty,
all participants feel the model is a great improvement on the traditional large
lecture format. Students relate better with other students and feel they are
learning the material better. Only over the next few years will we be able to
tell if the model will positively impact the numbers of STEM graduates desired.
11. What actions have you taken to address sustainability of this project
beyond the grant period?
A key action has been to involve the Associate Dean of Science, the Chair of
Mathematics, and the Chair of Physics as members of the Internal Advisory
Board. Together with the PI and co-PIs, the IAB has worked on identifying the
key indicators of success and analyzing the data that can serve as evidence for
getting faculty buy-in and sustaining the effort.
Another key action has been to promote peer leading as an enriching activity
for upper division undergraduates and an opportunity for them to work on
campus. This not only allows us to maintain a healthy pool of qualified
applicants but also increases the retention of STEM students, the majority of
whom are self-supporting and must commute to campus on a daily basis.
12. What is the relationship between the grant activities and the internal
advisory board? How has the board been involved in assessing progress,
addressing challenges or facilitating change, if needed, and addressing
sustainability?
The Internal Advisory Board (IAB) met for the first time on May 14, 2009. The
committee discussed the following issues and action items:






A “project memory” should be developed to ensure that the original
objectives of the project drive all activities.
o Deviations from the original objectives should be identified and
assessed for their efficacy.
o The importance of comparison of the project’s results with
national data should be established.
Differences in PLTL approach among disciplines should be identified to
ensure optimum application of the strategy in each discipline.
Critical variables and assessment tools should be identified to measure
success.
A strategy to engage more professors (in Physics as an example) and gain
their commitment to the program should be developed.
The budget should be checked and decisions of paying peer leaders a
salary versus a stipend in the upcoming semester should be made.
Advertisements should announce the peer leader position openings and
include criteria for minimum GPA and course success.
The IAB will meet again in February 2010 to hear progress made on these action
items and to begin the discussion on the long-term sustainability of the
project.
13. What is the relationship between the grant activities and the external
advisory board? How has the board been involved in assessing progress,
addressing challenges or facilitating change, if needed, and addressing
sustainability?
The External Advisory Board met for the first time on April 17, 2009. The
primary objective of the meeting was to assess the progress of the project.
The EAB listened to presentations by each one of the co-PIs and made
recommendation on the next steps that should be taken. The EAB also
performed a SWOT analysis which was included in the annual report to NSF.
Based on the feedback of this analysis, we took on the challenge of better
documenting our work, monitoring the outcomes of the grant activities as
measured through official university evaluations, supervising peer leader hiring
standards and training, and effecting minor modification of the course
structure and laboratories. The EAB is scheduled to meet again February 5,
2010.
The chair of the EAB made an additional visit to our campus on October 1,
2009. He met with the PI and co-PIs to strategize ways of disseminating the
preliminary results of the project and to seek the participation of industrial
stakeholders that could provide insight on facilitating change in the context of
the project.
14. What have you learned from this project and its implementation?
We had hoped the adoption of PLTL in multiple STEM courses would have been
‘more automatic’ but we realized that academics from other disciplines need
to sense ‘I am making an original contribution’ in order to fully adopt any new
didactic strategy. Fortunately, PLTL offers the opportunity for adaptations
with varied twists; thus allowing adoption of the basic idea to take many
creative directions.
We were pleased to see that many of our instructors learned about the value of
enhanced communication between instructors and students as facilitated by
peer leaders and the advantages of increasing the time the students are
exposed to the material.
What advice would you offer to new STEP PIs?
In general we would advise new PIs to carefully study what has been done in
previous STEP programs and gauge learning curves. Also we would recommend
that they anticipate adapting and modifying initial plans in the first year of
operation, based on their results.
More specifically, we would advise STEM colleagues nationwide to consider
implementing PLTL in key gatekeeper courses on their campus. There is
quantitative and qualitative evidence that this strategy works extremely well
for students who need to strengthen their study skills.
15. Finally, please provide an overall summary of how the state of your
project relates to your original targets and to the other goals of the project.
Include the number of students majoring in fields affected by your project
versus the projections.
We had anticipated that by the end of the STEP project, the number of
undergraduate Hispanic students earning degrees in STEM programs would
double with respect to the baseline of AY 2000-2001, which corresponds to an
increase of about 28 BS degrees awarded per year. This target has already
been met due in part to the large inertia of the system, which has benefited
from past programs such as the Model Institutions for Excellence and the Louis
Stokes Alliance for Minority Participation.
Academic
Year
1999-00
2000-01
2001-02
2002-03
2003-04
2004-05
2005-06
2006-07
2007-08
2008-09
Total BS Degrees
Awarded
280
293
290
339
340
433
477
501
544
NA
BS Degrees
Awarded to
Hispanics
181
205
189
230
226
291
323
341
416
NA
Percentage
Awarded to
Hispanics
64.6%
70.0%
65.2%
67.8%
66.5%
67.2%
67.7%
68.1%
76.5%
NA
However, what we really want to see is an enhancement in the quality of
education in the first two years of STEM programs at UTEP and a shift in the
teaching/learning strategies to ensure that growth in degree production is
maintained. This is our overarching goal.