Tracking Pathways to
Success
Identifying Learning Success Factors Across
Course Delivery Formats
Peter Usinger, State Assessment Meeting 2013
High W/F
As high as
50% attrition
of FTIC
cohort
45% of
Withdrawal
Reasons are
Course
Related
Student
Attributes
Financially
Limited
Nontraditional
Academically
Underprepared
Study
Context
Reviewing
High F/W
Engagement in
College Prep
F2F vs. Online
Instruction
Nursing
Orientation
SLS Course
Support
Student Success in College Environment
Q1
• Delivery format based differences between selfdirected behaviors and learning strategies.
Q2
• Similarities and Differences across subject
domains/disciplines and their sensitivity to
course delivery formats.
Q3
• Relationships among MSLQ constructs, and with
student characteristics, course characteristics,
and delivery formats.
Q4
• “Predictive” model to explain the relationships
for developing support mechanisms and change
curricular design for greater success/completion
Research Questions
Student
Characteristics
Demographic variables
Previous e-learning experience
Self-Directed
Learning
Habits
Metacognitive skills
Motivation
Self-discipline
Autonomy
Self-regulated behaviors
(Online) Course Completion
136 Courses Mixed withdrawal/failure rates
• Humanities, English/Letters, Mathematics and Statistics,
Sciences, Social Sciences, Nursing; Hybrids and Workforce
Dev. excluded from analysis due to low frequencies.
Delivery
1,179 Course sections (total)
• Online (216 Sections)
• Face to Face (931 Sections)
Population
>20,000 Students
• Invited via email to complete MSLQ online (15 min.)
Sample drawn from…
2,200 Participants:11% Response Rate
Discipline
F2F
Online
Humanities
43
29
Letters
314
84
Math
957
82
Nursing
136
33
Sciences
207
56
Social Sciences
85
64
Sample by Academic Area
Motivated Strategies for Learning
Questionnaire (MSLQ)
A. Motivational Constructs
1. Intrinsic Goal Orientation
2. Extrinsic Goal Orientation
3. Task Value
4. Control of Learning Beliefs
5. Self-Efficacy for Learning & Performance
6. Test Anxiety
B. Learning Strategies Constructs
1. Rehearsal
2. Elaboration
3. Organization
4. Critical Thinking
5. Metacognitive Self-Regulation
6. Time/Study Environmental Management
7. Effort Regulation
8. Peer Learning
9. Help Seeking
Total Items in Questionnaire
Original MSLQ
Applied in Study
Items
α
Items
α
4
4
6
4
8
5
0.74
0.62
0.90
0.68
0.93
0.80
4
2
2
3
8
4
0.75
0.63
0.90
0.76
0.96
0.75
4
6
4
5
12
8
4
3
4
81
0.69
0.75
0.64
0.80
0.79
0.76
0.69
0.76
0.52
2
2
2
5
10
7
4
3
3
61
0.69
0.58
0.54
0.80
0.84
0.82
0.71
0.77
0.72
MSLQ Scales and Reliability
Pearson Correlation Coefficients (N=2,200)
…between Grades Passing Online+
…and Motivational Constructs
1. Intrinsic Goal Orientation
0.19***
0.16***
0.04
2. Extrinsic Goal Orientation
0.03
0.08***
-0.04
3. Task Value
0.24***
0.19***
0.04*
4. Control of Learning Beliefs
0.30***
0.25***
0.10***
5. Self-Efficacy for Learning & Performance
0.46***
0.37***
0.07**
6. Test Anxiety
-0.23***
-0.15***
Note 1: *p < .05. **p < .01. ***p < .001.
-0.01
Note 2: +Method: F2F=0, Online=1.
Motivational Correlations
Pearson Correlation Coefficients (N=2,200)
(where *p < .05. **p < .01. ***p < .001.)
…between Grades Passing
Online
…and Learning Strategies Constructs
1. Rehearsal
0.00
0.02
-0.06**
2. Elaboration
0.09***
0.07**
0.02
3. Organization
0.04
0.03
-0.06*
4. Critical Thinking
0.04
0.01
0.06*
5. Metacognitive Self-Regulation
0.04
0.04*
0.00
6. Time/Study Environmental Management
0.19***
0.13***
0.07**
7. Effort Regulation
0.27***
0.21***
0.06*
8. Peer Learning
-0.04
-0.02
-0.19***
9. Help Seeking
-0.04
0.00
-0.32***
Learning Strategies
Due to the very minor statistical differences in the
results for grades vs. course success, and the fact
that a significantly higher proportion of C and D
students across most discipline areas are
subsequently failing, we’ve created a Student
Performance aggregate variable that combines
student’s Course Performance into three levels:
• Level 0: Failure or Withdrawal from Course
• Level 1: Course Grade Equals C or D
• Level 2: Course Grade Equals A or B
This Performance variable is used in the student
success analysis following.
Data Aggregation Rules
MSLQ Correlation w/
Course Performance
Online
Face to Face
Mean
Correlation
Mean
Correlation
Intrinsic Goal Or.
5.11
0.18***
4.99
0.19***
Extrinsic Goal Or.
5.82
0.01
5.94
0.06*
Task Value
4.99
0.23***
4.79
0.23***
Control Beliefs
5.83
0.31***
5.49
0.28***
Self-Efficacy
5.54
0.43***
5.20
0.44***
Test Anxiety
4.42
-0.21***
4.47
-0.21***
Rehearsal
4.79
-0.04
5.04
0.03
Elaboration
5.28
0.12*
5.20
0.06*
Organization
5.00
0.04
5.21
0.03
Critical Thinking
4.33
0.07
4.13
0.01
Metacog. S-Reg.
4.86
0.03
4.84
0.04
Time/Study Mgmt.
5.48
0.22***
5.26
0.15***
Effort Regulation
5.86
0.25***
5.69
0.24***
Peer Learning
2.58
-0.07
3.41
0.00
Help Seeking
3.10
-0.02
4.52
0.01
N=
366
1,751
Comparison by Delivery Type
MSLQ Correlation w/
Course Performance
Developmental
College Level
Mean
R Coeff.
Mean
R Coeff.
Intrinsic Goal Or.
4.97
0.21***
5.03
0.17***
Extrinsic Goal Or.
6.04
0.07
5.84
0.05*
Task Value
4.59
0.24***
4.94
0.21***
Control Beliefs
5.48
0.31***
5.57
0.28***
Self-Efficacy
5.15
0.43***
5.32
0.44***
Test Anxiety
4.56
-0.23***
4.40
-0.19***
Rehearsal
5.13
0.07
4.92
0.00
Elaboration
5.14
0.06
5.26
0.09***
Organization
5.33
0.05
5.10
0.04
Critical Thinking
4.13
-0.02
4.21
0.04
Metacog. S-Reg.
4.94
0.10*
4.80
0.03
Time/Study Mgmt.
5.42
0.23***
5.24
0.17***
Effort Regulation
5.79
0.27***
5.69
0.25***
Peer Learning
3.33
0.00
3.22
-0.04
Help Seeking
4.45
0.00
4.19
-0.01
N=
551
962
Comparison by Course Level
Important Note by Author:
Please note that the following slide has been corrected since it contained a
transfer error due to a truncated spreadsheet field. Affected is one of the
interesting findings with regard to the role of test anxiety in the course
performance of Nursing students. The original presentation slides showed a
positive correlation between Test Anxiety and Course Performance for the
whole Nursing group in this sample. However, this is only true for the 2nd
year cohort of the RN program involved. Pre-admission, 1st year, and
BSN students show patterns similar to other academic areas.
This is an extremely valuable research result since this correlation also
goes along with a positive correlation between Test Anxiety and Rehearsal
Strategies and Peer Learning. This indicates that our 2nd year Nursing
students were not negatively overwhelmed with anxiety, but had learned to
offset their anxiety levels successfully by rehearsing the subject matter at
hand, and working with their peers to master the course content.
We will follow-up on these findings with an appropriate discipline-specific
pathway model in the next phase of this longitudinal research project.
Pearson Correlation with
Course Performance
Humanities
(N=75)
Letters
(N=416)
Math
(N=1043)
Nursing
(N=174)
Sciences
(N=276)
Soc. Sci.
(N=156)
0.14
-0.01
0.25
0.30
0.32
-0.15
0.07
0.11
0.10
0.16
0.29
-0.04
0.22
0.06
0.21
0.31
0.46
-0.27
0.22
0.06
0.27
0.48
0.55
-0.20
-0.04
0.00
-0.03
0.15
0.34
-0.16
0.24
0.12
0.32
0.27
0.33
-0.09
-0.06
0.23
-0.05
0.09
0.02
0.20
0.19
-0.12
-0.15
0.03
0.11
0.10
0.00
0.03
0.25
0.22
-0.11
0.03
0.01
0.04
0.02
-0.02
0.07
0.15
0.27
0.02
0.02
0.06
0.13
0.02
0.07
0.11
0.28
0.39
0.02
-0.03
-0.08
-0.04
-0.04
-0.13
-0.12
0.09
0.05
-0.17
-0.14
0.12
0.12
0.11
0.00
0.11
0.31
0.32
0.02
0.08
Motivational Constructs
1. Intrinsic Goal Orientation
2. Extrinsic Goal Orientation
3. Task Value
4. Control of Learning Beliefs
5. Self-Efficacy Learn. & Perf.
6. Test Anxiety
Learning Strategies
1. Rehearsal
2. Elaboration
3. Organization
4. Critical Thinking
5. Metacognitive Self-Reg.
6. Time/Study Management
7. Effort Regulation
8. Peer Learning
9. Help Seeking
Correlations by Academic Area
Example: Math Differences
Motivational Constructs
Intrinsic Goal Orientation
Failure/Withdrawal
Passing Grade
Difference
58.3%
66.8%
8.5%
Extrinsic Goal Orientation
79.6%
84.0%
4.5%
Task Value
48.1%
59.7%
11.6%
Control Beliefs
63.4%
77.9%
14.5%
Self-Efficacy
50.0%
73.8%
23.7%
Test Anxiety
67.6%
Learning Strategies Constructs
Rehearsal
65.1%
Elaboration
65.9%
55.6%
-11.9%
67.1%
2.0%
68.3%
2.4%
Organization
67.6%
70.1%
2.5%
Critical Thinking
50.0%
48.9%
-1.1%
Meta-Cognitive Self-Regulation
61.5%
64.8%
3.3%
Time/Study Management
67.1%
73.4%
6.3%
Effort Regulation
70.4%
81.2%
10.7%
Peer Learning
36.0%
37.3%
1.4%
Help Seeking
53.7%
56.1%
2.5%
Success Comparison in Math
Multivariate Regression
Analyses (Total Sample)
Fit of Model
Parameter Estimates
Intrinsic Goal Orientation
Extrinsic Goal Orientation
Task Value
Control of Learning Beliefs
Self-Efficacy for Learning
Test Anxiety
Rehearsal
Elaboration
Organization
Critical Thinking
Metacognitive Self-Regulation
Time/Study Management
Effort Regulation
Peer Learning
Help Seeking
Predicting Course Performance
F-Value
Pr > F
R-Square
42.1
<.0001
0.24
t-Value
Pr > |t|
St-B
-1.97
-2.52
0.51
-1.84
14.84
-1.46
1.27
-0.47
-1.09
-0.75
-1.84
2.26
2.95
-1.07
-0.42
0.05
0.01
0.61
0.07
<.0001
0.14
0.20
0.64
0.27
0.45
0.07
0.02
0.00
0.29
0.67
-0.06
-0.06
0.01
-0.06
0.52
-0.03
0.03
-0.01
-0.03
-0.02
-0.07
0.07
0.09
-0.03
-0.01
Multivariate Regression I
Online Courses
Face-to-Face Classes
Multivariate Regression
Analyses w/ Performance F-Value Pr > F R-Square F-Value Pr > F R-Square
Fit of Model
Parameter Estimates
Intrinsic Goal Orientation
Extrinsic Goal Orientation
Task Value
Control of Learning Beliefs
Self-Efficacy for Learning
Test Anxiety
Rehearsal
Elaboration
Organization
Critical Thinking
Metacognitive Self-Regulation
Time/Study Management
Effort Regulation
Peer Learning
Help Seeking
7.1
<.0001
0.22
33.2
<.0001
0.24
t-Value
Pr > |t|
St-B
t-Value
Pr > |t|
St-B
-0.35
-0.54
-0.25
0.57
5.36
-0.40
-0.77
-0.26
-0.18
1.54
-1.67
2.37
-0.05
-0.99
-0.04
0.73
0.59
0.80
0.57
<.0001
0.69
0.44
0.80
0.86
0.13
0.10
0.02
0.96
0.33
0.97
-0.03
-0.03
-0.02
0.04
0.44
-0.02
-0.05
-0.02
-0.01
0.11
-0.15
0.17
0.00
-0.07
0.00
-1.47
-1.66
0.27
-2.30
13.60
-1.83
2.14
-0.76
-1.17
-1.88
-1.12
0.85
2.92
-0.48
-0.14
0.14
0.10
0.78
0.02
<.0001
0.07
0.03
0.45
0.24
0.06
0.26
0.39
0.00
0.63
0.89
-0.05
-0.04
0.01
-0.08
0.54
-0.05
0.07
-0.02
-0.04
-0.06
-0.05
0.03
0.10
-0.01
0.00
Multivariate Regression II
Developmental
College Level
Multivariate Regression
Analyses w/ Performance F-Value Pr > F R-Square F-Value Pr > F R-Square
Fit of Model
Parameter Estimates
Intrinsic Goal Orientation
Extrinsic Goal Orientation
Task Value
Control of Learning Beliefs
Self-Efficacy for Learning
Test Anxiety
Rehearsal
Elaboration
Organization
Critical Thinking
Metacognitive Self-Regulation
Time/Study Management
Effort Regulation
Peer Learning
Help Seeking
14.0
<.0001
0.25
30.0
<.0001
0.24
t-Value
Pr > |t|
St-B
t-Value
Pr > |t|
St-B
-0.51
-2.20
0.34
-0.42
7.31
-1.90
1.64
-1.97
-0.36
-2.03
1.08
2.33
0.69
-0.33
-0.77
0.61
0.03
0.73
0.67
<.0001
0.06
0.10
0.05
0.72
0.04
0.28
0.02
0.49
0.75
0.44
-0.03
-0.09
0.02
-0.02
0.48
-0.08
0.08
-0.11
-0.02
-0.11
0.07
0.13
0.04
-0.02
-0.04
-2.07
-1.21
-0.27
-2.08
13.29
-0.72
0.74
-0.30
-0.39
0.14
-2.55
1.44
3.07
-1.04
0.47
0.04
0.23
0.78
0.04
<.0001
0.47
0.46
0.76
0.70
0.89
0.01
0.15
0.00
0.30
0.64
-0.07
-0.03
-0.01
-0.08
0.55
-0.02
0.02
-0.01
-0.01
0.00
-0.11
0.05
0.11
-0.03
0.02
Multivariate Regression III
• MSLQ scales: excellent reliability & context sensitivity
• Motivational factors clearly driving student success
• Learning Strategies (cognitive skills) show multi-faceted
instructional improvement (learning) opportunities
• Peer Learning/Help Seeking: Underrepresented within
cohorts, not an integrated/facilitated college activity
• Many underprepared students succeed largely via intrinsic
motivation and confidence/self-efficacy, combined with
hard work and study management
• The lack of exposure to meta-cognitive abilities and critical
thinking skills nurtures course environments in support of
compliant learners instead of self-directed learners!
Conclusions
• Online: Assess Quality Matters impact across delivery
formats and feed information back into process
• 1st Year/EWS: Compare demographic differences
associated with the various success patterns
• SLS: Evaluate impact with SLS courses when applying
the MSLQ as formative assessment tool early in
Student Success or Developmental Ed classes
• Overall: Conduct additional focus groups at college
sites; seek expansion of study to inform interventions
• FCS: Seek study replication across service areas
Study Impact / Next Steps
Thank You!
pusinger@polk.edu