STUDENT SUCCESS:
SIMPLE QUESTIONS,
COMPLEX ANSWERS
Karen Rothstein
Research Analyst I
Andrew Fuenmayor
Research Analyst I
THE SIMPLE QUESTION…
Will students be more successful if
they take this course
(e.g., Counseling, Study Skills,
College Readiness)?
THE SIMPLE ANSWER
Following Term Success Rate
70%
Students who took
Study Skills
60%
Students who did not take
Study Skills
THE LESS SIMPLE ANSWER
Following Term Success Rate
80%
50%
Students who successfully
completed Study Skills
Students who did not
successfully complete
Study Skills
THE COMPLEX ANSWER
 “Well that depends on several factors, one of which
is the course in question. In addition, we need to
examine other variables…”
 In other words…
Student Success:
More than meets the eye!
PURPOSE
 To encourage more complex and complete
interpretations of student success
 Issues of self-selection
 Comparison group
 To include student-level variables
 Student characteristics that could influence success
 Two applications
 Specific target course vs. comparison course
 Student support services
TARGET COURSE VS.
COMPARISON COURSE
BEFORE YOU START
 Target course
 Course of interest
 “TC” in syntax
 Comparison course
 How do you select comparison course?
 “CC” in syntax
 Time frame
 Proximal effect of target course
 Distal effect of target course




Uses first-time students
Uses first attempt at target course and comparison course
Success = A, B, C, P, CR, IP
No success = D, F, W, NP, NC, IF
CREATE DATA FILE
 Create a long MIS data file
 Create a first-time student (FTS) file
 SB15 = 1
 See “Creates first-time student file” syntax
 Merge long MIS and FTS file
 See “Creates data file for analysis” syntax
NOW FOR THE FUN PART!
 STEP I: Cleans data file
 Data earlier than 2000 will need a different term
variable
 Change target course (line 61)
 Change comparison course (line 63)
 STEP 2: Creates 3 cohorts
 Target course first
 Comparison course first
 Both target and comparison course in same semester
 Change target course (lines 19, 21, 24, 26)
 Change comparison course (lines 20, 22, 25, 27)
NOW FOR THE FUN PART!
 STEP 3: Identifies terms
 Concurrent
 Following
 Subsequent
 STEP 4: Creates variables to calculate success rates
 Concurrent
 Following
 Subsequent
 STEP 5: Generates success rate output
 Change destination for output as Excel file (line 208)
DOES TAKING THE TARGET COURSE
LEAD TO SUCCESS?
Success Rate in Term
Following Target Course
100.0%
80.0%
60.0%
60.2%
61.5%
Enrolled in target course
Did not enroll in
target course
40.0%
20.0%
0.0%
MAYBE IT’S SUCCEEDING IN THE
TARGET COURSE?
Success Rate in Term
Following Target Course
100.0%
80.0%
66.1%
60.0%
41.2%
40.0%
20.0%
0.0%
Successful in target course
Unsuccessful in target
course
MAYBE IT’S MORE COMPLICATED…
Success Rate in Term Following Target Course
100.0%
90.0%
80.0%
70.0%
60.0%
50.0%
40.0%
30.0%
20.0%
10.0%
0.0%
71.8%
65.7%
40.3%
Successful in target
course
Unsuccessful in
target course
46.4%
Successful in
Unsuccessful in
comparison course comparison course
GET CRAZY WITH THE CHEEZ WHIZ!
 Move from success rates to regression
 Use high school data to include prior success rates
 What else can we examine?
 Gender
 Ethnicity
 Age
 Education Plan
 Concurrent GPA (excluding target and comparison
courses)
 Other student-level variables?
STUDENT
SUPPORT
SERVICES
AGAIN, THE SIMPLE QUESTION
Will students be more successful
if they use this support service
(e.g., tutoring, workshops,
counseling appointments,
review sessions, etc.)?
DOES THE SUPPORT SERVICE
LEAD TO SUCCESS?
Success Rate in Target Course
100.0%
80.0%
71.4%
60.0%
40.0%
28.0%
20.0%
0.0%
Participated in
support service
Did not participate in
support service
THE COMPLEX ANSWER
 Possible independent variables
 Participation vs. no participation
 Fulfill required hours vs. failure to fulfill required hours
 Total number of hours of support service
 Type of support service
 Prior exposure to support service
 Prior success rate
 Prior number of units
 Dependent variable
 Success in a specific course (tied to the student support
service)
 Regression
EFFECT OF STUDENT SUPPORT SERVICE
ON COURSE SUCCESS
Variable
Standardized
Student Success Rate
1.698
18221.96
.000
5.46
Course with student
support services
-.922
542.66
.000
.398
.636
1489.53
.000
1.889
-.035
333.05
.000
.965
Number of Hours spent
in support services
(Number of Hours spent
in support services )2
(to capture non-linear
effects)
B
Wald Stat.
Sig.
Exp(B)
NOW WHAT?
IMPROVING DIGESTION OF DATA
 Carefully and thoughtfully explain why simple answers
are inadequate
 Highlight need to consider variables other than the
course in and of itself
 Present data in a user-friendly way
INSTEAD OF THIS…
Variable
Standardized
Student Success Rate
1.698
18221.96
.000
5.46
Course with student
support services
-.922
542.66
.000
.398
.636
1489.53
.000
1.889
-.035
333.05
.000
.965
Number of Hours spent
in support services
(Number of Hours spent
in support services )2
(to capture non-linear
effects)
B
Wald Stat.
Sig.
Exp(B)
TRY THIS…
100%
Course Success Rate
90%
80%
70%
4 hours
3 hours
2 hours
1 hours
0 hours
60%
50%
40%
30%
20%
10%
0%
Students’ Prior Success Rate
BE PREPARED FOR QUESTIONS LIKE…
 “Why aren’t you doing it the same way as last time?”
 “We need to look at the right subset of students.”
 “There’s more than just numbers.”
 “So what you are saying is that what I do doesn’t
matter.”
WHEN IT COMES DOWN TO IT…
 The reason these interventions and courses exist is
to help students succeed
 If the research suggests this is not the case, how can
we make the intervention more effective?
 Where has it worked? Why?
THANKS FOR LISTENING!
 Please feel free to contact me to discuss.
 Suggestions for improving syntax always welcome!
 Karen Rothstein: krothstein@lbcc.edu
 Andrew Fuenmayor : afuenmayor@lbcc.edu