COMPARITIVE COHORT ANALYSIS FOR THE STEM SCHOLARS PROGRAM
Final Report
9/18/15
SUBMITTED BY:
Dr. Keith Thompson, Associate Professor
Civil and Environmental Engineering Department
&
Dr. Michael Black, Assistant Professor
Math Department
Overview
This AAF project was funded to supplement summative analysis of the STEM Scholars Program within the
College of EMS. The STEM Scholars Program was a five-year NSF sponsored S-STEM project which ended
in Spring 2015. The program provided scholarships for 69 new freshman and transfers pursuing majors
offered by the College of EMS between Fall 2010 and Spring 2015. NSF S-STEM projects also required
participation in support programming to enhance the scholarship recipients’ academic and career skills
and was specifically targeted to recruit underrepresented groups into STEM majors at UW-Platteville. As
part of the final work for the grant, summative assessment of the program was conducted throughout the
summer of 2015. Summative results of interest include STEM Scholars’ academic success and participation
in practices known to foster retention and enhance career preparation.
Furthermore, there is strong general interest in methods of assessment that can be used to assess
programs such as STEM Scholars because there are other similar programs that currently exist. Hence, a
broader goal of this AAF project is to enhance the institutional capacity of the College of EMS Office of
Student Success Programs for program assessment. The Director of EMS Student Success Programs,
Tammy Salmon-Stephens (who is also a Co-PI of the STEM Scholars project) was involved in this study.
Assessment Methods
The key assessment method utilized in this project was a statistical comparison between the students in
the STEM Scholars Program and similar cohorts of students within the College of EMS who did not
participate in the STEM Scholars program. The purpose of this assessment was to determine the degree
to which the STEM Scholars program enhanced academic performance of the participants in comparison
to reasonably similar students who were not in the program. This has been an attempt to determine valueadded by the program.
The Office of Institutional Research (now the Office of Institutional Effectiveness and Assessment) was
contacted to compile data for comparison cohorts. To create comparison cohorts, the following traits
were desired:

Each comparison cohort would exclusively contain students enrolled in STEM majors (considered
to be any major offered by the College of EMS for this study). This was also true of the STEM
Scholars.
1





Each comparison cohort would contain students with entry level academic indicators matching
those required to apply to the STEM Scholars program (HS Graduating Percentile  75% or ACT 
22; HS GPA  3.0 could not be set as a criteria because data on HS GPA is not stored within the
university’s student information system and was therefore inaccessible for this study).
Each comparison cohort would contain underrepresented/majority student ratio and a
male/female ratio matching the ratios found in each cohort of STEM Scholars.
Comparison cohorts would be enrolled in EMS majors over the same time period of the STEM
Scholars Program, so five cohorts were created in which the admit terms were Fall 2010 through
Fall 2014 respectively (matching the five cohorts of Scholars recruited for the STEM Scholars
Program).
Nobody in the comparison cohorts would have been a STEM Scholar at any time in their academic
history.
Beyond the constraints listed above, each comparison cohort was to be sufficiently large for
statistical analysis with as many students chosen randomly as possible from the available pool of
EMS majors (the students least likely to be randomly chosen were underrepresented minorities,
who were more likely to be the entire population of such students matching the criteria listed
above).
Data requested for each comparison cohort included: EMPLID; New Freshman/Transfer status; gender;
primary ethnicity group; first generation status; high school percentile; ACT Score; and, for each semester
possible from their admit term to the most recently completed term (Spring 2015), records of each
student’s academic program, units taken, units passed, term GPA, cumulative GPA, and academic standing
at the end of each term. Similar data were also collected for the STEM Scholars. Because of the sensitive
nature of this data and in accordance with the IRB approval for this project, no raw data or samples of
analysis are provided as appendices to this report.
A secondary assessment method attempted under this AAF project was to compare participation in High
Impact Practices (HIPs) between STEM Scholars and students from the comparison cohort. This data is not
routinely measured at UW-Platteville, so the researchers could not simply ask the institutional researcher
to gather this data. Originally, the researchers on this grant proposed to randomly survey students from
each comparison cohort on their participation in internships, co-ops, undergraduate research, and other
HIP’s in order to gather the data needed. However, in spring 2015 and mid-way through this project, the
College of EMS implemented an exit survey on HIP participation to the all students graduating from the
College of EMS in that spring. This survey will become an institutional practice within EMS each semester
and it will ultimately be more efficient if these survey results become the benchmark for assessment of
HIP participation rather than implementing a separate survey for each special program or other desired
assessment need. Thus, the original plan to create a survey to collect comparison data was abandoned
and the College of EMS survey results were used for comparison instead. This meant that the comparison
data was not as good a match for statistical comparison, however, in terms of streamlining practice for
institutionalization, the survey results from graduating seniors will be more cost-effective until the College
or the University implements more rigorous data tracking of participation in HIPs.
Data provided by the College of EMS HIP participation survey included (but was not limited to) the number
of times a graduating student had: completed a co-op; completed a summer internship; completed a
semester research project; and completed a study abroad (international experience).
2
Analysis and Results – Academic Success
STEM Scholar and comparative cohort data was available for five admit terms and are summarized in
Table 1. The values shown are the number of students in each cohort at the beginning of the admit term.
The total number of STEM Scholars for which any data is available is 69. This number excludes two
students who were offered scholarships, but left the university prior to receiving any money. In each
successive semester, the number of students in a cohort shrinks as students voluntarily leave EMS majors
for non-STEM majors; voluntarily leave UW-Platteville for other universities; are dismissed from their
academic program or the university; are dismissed from the STEM Scholars program (for STEM Scholar
cohorts only); or graduate from the university.
For analysis purposes, the cohort data was reorganized to reflect the student’s semester at the university
(1st, 2nd, 3rd, 4th, etc.). Students admitted in Fall 2010 and Fall 2011 provided at least 8 semesters of data
for analysis; students admitted in Fall 2012 provided 6 semesters of data; students admitted in Fall 2013
provided 4 semesters of data; and students admitted in Fall 2014 provided 2 semesters of data. Analysis
on the basis of semesters spent in the College of EMS provided more reliable statistical analysis for the
earlier semesters because it allowed all five STEM Scholar cohorts to be combined, increasing the amount
of data in the statistical pool. However, the size of each semester-based data set diminishes over time
due to natural sources of attrition and the limitations of the cohort data imposed by the admit term and
the number of available semesters for analysis. Table 2 summarizes the number of students available for
semester analysis in the College of EMS.
Using the data available for STEM Scholars and Comparison Students, the following items were compared:
average credits taken each semester, average credits passed each semester, average term GPA, and
average cumulative GPA. Figure 1 plots the averages for these four results over time. For the most part,
particularly for earlier semesters, STEM Scholars out-perform the comparison cohorts for each measure,
but these trends diminish over time. The diminishment may be influenced by the diminishing pool of data
available for each semester, so this result should be reassessed at a later time when later semesters of
data are available to the given cohorts.
Table 1 – Cohort Size Summary
Admit Term
of Cohort
Initial Size of STEM
Scholars Cohort
Fall 2010
Fall 2011
Fall 2012
Fall 2013
Fall 2014
10
15
17
13
14
Initial Size of
Comparison
Cohort
329
419
434
461
489
Number of
Semesters of
Available Data
10
8
6
4
2
Semesters
Contributed to in
Analysis
st
1 through 8th
1st through 8th
1st through 6th
1st through 4th
1st and 2nd
Table 2 – Available Data per Semester of Attendance at UW-Platteville
3
Semester of Comparison
Available STEM Scholar Data
1st Semester
2nd Semester
3rd Semester
4th Semester
5th Semester
6th Semester
7th Semester
8th Semester
69
68
65
48
46
35
32
18
Available Comparison Cohort
Data
2,132
1,979
1,797
1,290
1,219
833
776
472
Analysis is unreliable beyond the 8th semester due to diminishing size of the cumulative STEM Scholar group.
Credits Passed
16.00
15.00
15.00
14.00
14.00
13.00
13.00
12.00
Comparison
11.00
STEM Scholars
Credit hours
Credit hours
Credits Taken
16.00
10.00
12.00
Comparison
11.00
STEM Scholars
10.00
9.00
9.00
8.00
8.00
1
2
3
4
5
6
7
8
1
2
3
Semester
Term GPA
5
6
7
8
Cumulative GPA
3.50
3.50
3.40
3.40
3.30
3.30
3.20
3.20
3.10
3.00
Comparison
2.90
STEM Scholars
GPA
3.10
GPA
4
Semester
3.00
2.80
2.80
2.70
2.70
2.60
2.60
2.50
Comparison
2.90
STEM Scholars
2.50
1
2
3
4
5
Semester
6
7
8
1
2
3
4
5
6
7
8
Semester
Figure 1 – Four academic indicators plotted over time for STEM Scholars and the comparison cohorts
The data plotted in Figure 1 were analyzed statistically. T-test analysis was performed to measure
significance of the difference between the STEM Scholars and the comparison cohorts. P-Values (given as
proportions) from this analysis are presented in Table 3. The minimum criteria for significance was set at
the 5% level. For all values in which a significant difference was observed, the difference corresponded to
a positive gap between the STEM Scholars and the comparison cohort (i.e. the STEM Scholars significantly
out-performed the comparison cohort). The results from Figure 1 and Table 3 indicate the following:
4




STEM Scholars took on average 1 semester credit more than the comparison cohort students in
three of their first four semesters in college.
STEM Scholars passed on average 1 to 1.5 more credits than comparison cohort students in all of
their first four semesters in college.
STEM Scholars earned on average 0.3 to 0.45 more Term GPA points than comparison cohort
students in all of their first four semesters of college.
STEM Scholars consistently sustained on average 0.3 more Cumulative GPA points than
comparison cohort students in all of the eight semesters examined under this analysis.
Table 3 – Results (p-Values) of T-Tests of Difference between STEM Scholars and Comparison Cohorts
Semester
1st
2nd
3rd
4th
5th
6th
7th
8th
Credits Taken
0.0011**
0.0760
0.0038**
0.0127*
0.3779
0.4673
0.0692
0.4208
Credits Passed
0.0001***
0.0126*
0.0089*
0.0129*
0.1929
0.4527
0.0340*
0.4494
Term GPA
0.0000***
0.0000***
0.0178*
0.0091*
0.0648
0.0774
0.0114*
0.4353
Cumulative GPA
0.0000***
0.0000***
0.0001***
0.0001***
0.0002***
0.0002***
0.0015**
0.0026**
* Significant at a 5% level.
** Significant at a 0.5% level.
*** Significant at a 0.005% level.
Another aspect that was examined was retention in EMS majors and retention in good standing at the
university between the two groups. Retention in good standing indicates that students stayed at the
university and maintained good academic standing (cumulative GPA  2.0 and term GPA  1.0). Within
the group of students who do not retain in good standing are students who undergo academic dismissal
or who voluntarily leave the university with or without good academic standing. Tables 4 and 5
summarize numerical results from these analyses and Figure 2 presents plots of the data. The data show
that STEM Scholars retained in EMS majors and in academic good standing within the university at
higher rates than students in the comparison cohorts for their first six semesters of college. For
retention in good standing, the difference between STEM Scholars and the Comparison Cohort was 11%
higher in the first three semesters and maintained at around 8-9% up through semester 7. Retention in
STEM majors (those offered by the College of EMS) improved steadily until the 7th and 8th semesters.
Sharp reversals in both sets of data in the 7th and 8th semesters may be a result of the limited data
available for these semesters.
5
Table 4 – Retention in EMS Majors
Semester
1st
2nd
3rd
4th
5th
6th
7th
8th
N
69
69
55
55
42
42
25
25
STEM Scholars
n
69
61
44
40
28
27
13
12
n/N
1.000
0.884
0.800
0.727
0.667
0.643
0.520
0.480
Comparison Cohorts
N
n
n/N
2020
2020
1.000
2020
1744
0.863
1533
1115
0.727
1533
996
0.650
1106
658
0.595
1106
619
0.560
699
369
0.528
699
353
0.505
Difference
0.021
0.073
0.078
0.072
0.083
-0.008
-0.025
N – Number of students of the given group listed in EMS in the 1st semester and who could have taken classes in
the current semester. The number of students who could have taken courses in the current semester
decreases due to the shrinking numbers of semesters for which data can be analyzed for students in
cohorts for later admit years.
n – Number of students listed in the college of EMS in the current semester.
The “Difference” is the calculation of (n/N)STEM Scholars – (n/N)Comparison Cohorts.
Table 5 – Retention at UW-Platteville in Good Academic Standing
Semester
1st
2nd
3rd
4th
5th
6th
7th
8th
N
69
69
55
55
42
42
25
25
STEM Scholars
n
67
66
48
45
32
30
18
14
n/N
0.971
0.957
0.873
0.818
0.762
0.714
0.720
0.560
Comparison Cohorts
N
n
n/N
2132
1827
0.857
2132
1756
0.824
1643
1244
0.757
1643
1186
0.722
1182
794
0.672
1182
750
0.635
748
449
0.600
748
443
0.592
Difference
0.114
0.133
0.116
0.096
0.090
0.080
0.120
-0.032
N – Number of students of the given group in good standing at the start of the 1st semester and who could have
taken classes in the current semester. The number of students who could have taken courses in the current
semester decreases due to the shrinking numbers of semesters for which data can be analyzed for students
in cohorts for later admit years.
n – Number of students listed in good standing at the end of the current semester.
The “Difference” is the calculation of (n/N)STEM Scholars – (n/N)Comparison Cohorts.
6
Retention of Good Standing
1.000
0.900
0.900
0.800
0.800
0.700
Comparison
STEM Scholars
Proportion
Proportion
Retention in EMS
1.000
0.700
STEM Scholars
0.600
0.600
0.500
0.500
0.400
Comparison
0.400
1
2
3
4
5
6
7
8
1
Semester
2
3
4
5
6
7
8
Semester
Figure 2 – Retention in EMS and in good standing by semester for STEM Scholars and the comparison
cohorts
Further analysis of STEM Scholar retention examined why students left the program. This analysis could
be performed for the STEM Scholars due to the researchers’ direct knowledge of that pool, but could
not be performed for the comparison cohort as reasons for leaving are not tracked in the student
information system. Table 6 summarizes why students left the STEM Scholars Program. Scholars that
withdrew from the university typically did so to transfer to another university. Scholars who left to
change major, change universities, or who become dismissed did so predominantly within their first 4
semesters. Thus for the 23 Scholars who are still in the program with 4, 6, or 8 semesters completed at
the university, it is likely that they will successfully graduate with a STEM major and maintain the
academic requirements (Cumulative GPA  3.0) required from the program until that point.
Table 6 – Reasons STEM Scholars Left the Program
Semester at
which
Status
Applies
1st
2nd
3rd
4th
5th
6th
7th
8th
9th
10th
All Students
Status
Graduated
0
0
0
2
0
2
1
2
1
2
10
Changed
Major to nonSTEM within
Univ.
1
4
0
2
1
0
0
0
0
0
8
Voluntary
Withdrawal
from Univ.
Dismissed
from STEM
Scholars
Still in the
STEM Scholars
Program
3
2
0
0
0
0
0
0
0
0
5
0
7
1
2
1
0
0
0
0
0
11
0
12d
0
9c
0
10b
0
4a
0
0
35
a – 2011 Admit Term; b – 2012 Admit Term; c – 2013 Admit Term; d - 2014 Admit Term
7
Specific Results by Race/Ethnicity, Gender, and First Generation Status:
Value Added - Race/Ethnicity
To examine value added specifically for non-white STEM Scholars, the data available from the STEM
Scholars includes only 16 students (23% of the total) spread out over all five admit term cohorts.
Because the pool of data is small to begin with and shrinks for each successive semester, results beyond
the 1st and 2nd semester should be treated cautiously. The data that is available indicates a gap in valueadded between white and non-white students. Within the first two semesters, the STEM Scholars
Program correlated with improvements in credits taken, credits passed, term GPA, and cumulative GPA
for both whites and non-whites; however, these improvements were less for non-white students. Data
plotted in Figure 3 illustrates this. The data plotted for white STEM Scholars comes from comparison
with only white comparison cohort students, while the data plotted for non-white STEM Scholars comes
from comparison with only non-white comparison cohort students. Each groups of STEM Scholars shows
improvement over their respective comparison group, but the improvements were less for non-whites.
Figure 3 – Gaps between white and non-white STEM Scholars in value-added by the STEM Scholars
Program for four academic indicators
Value Added - Gender
To examine value added specifically for female STEM Scholars, the data available from the STEM Scholars
includes 31 students (45% of the total). This is better than the data available based on race and ethnic
differences, but should still be treated cautiously beyond the 4th semester (where the number of available
data for female STEM Scholars begins to dip below 20 individuals). As with race and ethnicity, academic
8
indicators for female STEM Scholars were compared only against females in the comparison cohort and
data for male STEM Scholars were compared only against males. The resulting differences are plotted in
Figure 4. The plots show that female STEM Scholars took on higher credit loads each semester for the first
four semesters than their female peers who were not in the STEM Scholars, whereas the male STEM
Scholars tended to take less than their peers. Aside from this result, the remaining indicators show
inconsistent results, perhaps suggesting that there was no significant difference in the value added by the
STEM Scholars Program between males and females.
Figure 4 – Gaps between male and female STEM Scholars in value-added by the STEM Scholars Program
for four academic indicators
Value Added - First Generation Status
There were 31 first generation students in the STEM Scholars Program (45% of the total). As with the prior
two demographic specific analyses, results beyond the first four semesters should be treated cautiously.
Once again, value added for first generation students and non-first generation students relative to each
group’s peers were determined for the four academic indicators used previously. These results are plotted
in Figure 5 with gaps between first generation and non-first generation students emphasized.
The data indicate some gaps in value-added between first generation and non-first generation students
and these gaps favor first generation students in three of the four indicators during the first two
semesters. In credits passed, term GPA earned, and cumulative GPA, first generation students outperform their comparison group more than non-first generation students out-perform theirs. There could
be a number of explanations for these results: ordinary first generation students may be underperforming, so that there is more room for value to be added by special programs such as STEM Scholars;
the programming provided for the STEM Scholars may have been attuned to the needs of first generation
9
students; non-first generation students may have been more resistant to the types of assistance provided
by the programming. Determining causality will require further analysis. Beyond the first two semesters,
the data becomes erratic for three of the indicators, with cumulative GPA more stable, but showing the
two groups converging in value-added performance.
Figure 5 – Gaps between first generation and non-first generation STEM Scholars in value-added by the
STEM Scholars Program for four academic indicators
Analysis and Results – Participation in High Impact Practices (HIP’s)
Participation in high impact practices such as (but not limited to) co-ops, internships, study abroad, and
undergraduate research were highly encouraged within the STEM Scholars Program. To assess the degree
to which the STEM Scholars participated in HIPs more than ordinary EMS students, HIP participation was
inventoried for all STEM Scholars. To compare this against norms within the College of EMS, data gathered
from a survey conducted on EMS seniors graduating in Spring 2015 was used. The comparison group is
not similar to the STEM Scholars. While the STEM Scholars contains students at all levels of academic
progress (freshmen, sophomore, etc.), the comparison group contains only graduating students who have
on average been at the university longer and had more time to participate in HIPs. Furthermore, unlike
the previous analysis of academic indicators using the comparison cohorts, there were no controls for
similar demographic make-up between the STEM Scholars and the comparative group of Spring 2015
graduates. Finally, there are discrepancies in how data was collected from the graduates and how data
was collected for the STEM Scholars. Data from the graduates came from a self-reported survey in which
some of the respondents may have misinterpreted questions. Data from the STEM Scholars comes from
information gathered over their time in the program under direct observation and is generally more
10
reliable. Taking these factors into account, the analysis presented here of HIP participation must be
interpreted with an understanding of its limitations.
The graduate HIP survey tracked the total number of HIP’s that graduates participated in at least once.
The full list of HIP’s in the survey included the following 12 HIP’s:












Was a tutor
Participated in Engineering EXPO (a UW-Platteville K-12 outreach event)
Participated in K-12 outreach (other than EXPO)
Combined in
unique HIP count
Participated in a PACCE project (a service learning initiative)
Attended a professional conference
Was part of a Living and Learning Community (LLC)
Was a key member of a student organization project
Participated in undergraduate research
Completed a co-op
Completed an internship
Completed an international experience (study abroad)
Participated in formal mentoring
Analogous participation by STEM Scholars was determined using the closest matches that could be
determined.
For this analysis, only students who retained at UW-Platteville in the College of EMS were considered,
leaving a population of 49 STEM Scholars (of these 3 had retained in the College of EMS at UW-Platteville,
but not in the STEM Scholars Program). Of these 49, only 5 have graduated so far, which would be the
appropriate subgroup to compare with Spring 2015 graduates, but this number is too small for reliable
statistical comparison. Lastly, among the 49 in the STEM Scholars data group, the number of semesters
spent at UW-Platteville ranges from 2 to 10, averaging 5 semesters for the 49 Scholars (2.5 years).
The number of graduating seniors who answered the HIP survey in Spring 2015 was 216. This group of
students would include some transfers with only 2 to 3 years of time at UW-Platteville, but would contain
mostly students that spent between 4 to 6 years to get their degree here. Considering these expected
ranges, we can reasonably guess that the average number of semesters for the graduating group is likely
to be between 8 and 10. Even if we say that 8 semesters is a fair guess, this is still 60% more time at the
university than the pool of STEM Scholars has averaged.
Table 7 presents data on HIP participation for STEM Scholars and graduates from EMS majors in 2016.
Among the HIP’s included, four show higher participation by the STEM Scholars than the graduates:
mentoring, international experience, student organization projects, and volunteer/outreach/tutoring.
Mentoring was provided to each STEM Scholar as part of the program’s support for the Scholars and this
insured a high participation value. Service was a requirement for the STEM Scholars Program and Scholars
often accomplished this through tutoring or outreach activities, so this explains the higher value seen in
this area. Students were also required to join student organizations which increased the likelihood that
they would become involved in some major project of that organization.
Also notable in the data is the participation in (summer) internships. STEM Scholars and graduates have
participated in these in roughly the same proportion, yet the STEM Scholars group has had on average
11
only two summers available for this HIP, while the graduates have had three or four. Furthermore, the
most common time for internships (as well as co-ops) is towards a student’s later time at the university,
so it would be expected that graduates would be weighted towards better performance in this area
disproportionately to their total time at the university because the later time is more opportune for this
HIP, yet the STEM Scholars compared relatively better against graduates than might be expected based
on these expectations.
Table 7 – Participation in High Impact Practices
Number who
Participated
at Least Once
Proportion
who
Participated
Number who
Participated
at Least Once
Proportion
who
Participated
Difference in
Proportion
Between STEM
Scholars and
EMS Graduates
Internship
33
0.673
149
0.690
-0.017
Co-op
10
0.204
58
0.269
-0.065
3
0.061
36
0.167
-0.106
8
0.163
18
0.083
0.080
18
0.367
89
0.412
-0.045
30
0.612
90
0.417
0.195
18
0.367
67
0.310
0.057
49
1.000
27
0.125
0.875
STEM Scholars
(N = 49)
Type of HIP
Undergraduate
Research
International
Experience
Attended a
Professional
Conference*
Volunteer,
Outreach, or
Tutoring
Was a Key
Member of a
Student
Organization
Project
Was in a formal
Mentoring
Program
Spring 2015 EMS Graduates
(N = 216)
* This is a combination of several HIP survey items all related to service. Ideally, this should be voluntarily
chosen types of service. Participation in UW-Platteville PACCE projects was omitted, because many
engineering capstone course projects take the form of PACCE projects, but these projects are not voluntarily
chosen by the students who are required to take the capstone courses.
Figure 6 shows a histogram of unique HIPs participated in at least once by students in the STEM Scholars
Program and by Spring 2015 EMS Graduates. STEM Scholars averaged 3.65 unique HIPs (with a standard
deviation of 1.99) and Spring 2015 EMS Graduates averaged 3.50 unique HIPs (with a standard deviation
of 1.77). T-test analysis to determine if the distributions are significantly different provided a p-value of
0.628 indicating that there was no significant difference in unique HIP participation between the STEM
Scholars and the Spring 2015 EMS Graduates.
12
Number of HIPs by STEM-Scholars vs Graduates
0.250
Frequency
0.200
0.150
Graduates
0.100
STEM
0.050
0
1
2
3
4
5
6
7
8
9
Number of HIPs
Figure 6 - Frequency rates for the number of unique types of High Impact Practices that Spring 2015 EMS
graduates and STEM Scholars participated in at least once
Summary and Future Work
The analysis to date has demonstrated several academic indicators where the STEM Scholars Program
correlates to enhanced academic performance. These include credits taken, credits passed, and term GPA
earned in the first two years of college and cumulative GPA through at least four years of college.
Additionally, STEM Scholars retained in STEM majors and at the university better than non-STEM Scholars
for at least the first three years of college. Within the first year of college, the academic value added by
the program favored first generation and female students particularly, however the value added to
underrepresented minorities (non-whites) lagged behind that of majority (white) students.
A keystone of the STEM Scholars Program was to encourage participation in High Impact Practices (HIP’s).
Some HIP’s showed higher STEM Scholar participation after just 5 semesters of average time at UWPlatteville than graduating EMS students with a likely 8 semesters of average time at UW-Platteville. Much
of this was a result of support services provided by the program or because of requirements imposed by
the program. Still, for some critical HIP’s such as internships, co-ops, and undergraduate research, the
data is more ambiguous and suffers from the limitations of the comparison pool.
The analyses performed for this assessment grant show much potential to assess other student programs
at UW-Platteville. There are some essential requirements to facilitate this sort of work. Any program for
which assessment using a comparison cohort is desired, should incorporate a flag in the student success
system to indicate students who participated in that program. This greatly facilitates the collection of
data. Secondly, check list forms should be created to easily identify the characteristics necessary for the
comparison cohort and the types of data essential for the analysis. This assessment project suffered from
unnecessary delays due to omissions in the data pulls requiring additional request to get the supplemental
13
data. Systematizing the process of the data request by using a form or checklist to prompt decisions
regarding data collection could mitigate these types of oversights. Lastly, cooperative working
relationships between the special program teams and statisticians are required to accomplish the
statistical analysis.
Analysis of High Impact Practice participation was greatly helped by the institutionalization of a survey
process by the College of EMS, however the college should be encouraged to continue refining this
process and adding more detail to the data collected. Currently, there is no way to correlate the survey
results to demographic characteristics or to know when students participate in various HIP’s during their
academic path. Improvements in the process of collecting HIP data are necessary as these are becoming
a key feature for many special programs and assessment cannot be performed without knowing the
baseline values for the university.
Further assessment of the STEM Scholars should occur in the future. This should happen once the last
cohort of Scholars has had time to accumulate more time at the university, perhaps in three or four years
time so that each admit term cohort has at least 8 semesters of data. With more accumulated data,
additional methods of statistical analysis should be examined including longitudinal analysis of each admit
term cohort which might reveal important changes in performance as various features of the STEM
Scholar Program were adjusted and another analysis of specific demographic groups including
ethnic/racial, gender, and first generation status.
14