Exploring student non-completion in higher
education using electronic footprint analysis
Dr John Buglear
Nottingham Business School
This work was supported by funding from the Staff and Educational Development
Association (SEDA)
‘The Origin of the Thesis’
• Retention matters but institutional retention data is unreliable
• Why students leave is related to when they leave
• Virtual learning environments are an intrinsic part of the modern
undergraduate experience
• From an academic management perspective tracking electronic
engagement is more robust than physical registers of attendance
• Electronic engagement data is an information resource capability for
developing retention strategies
23 March 2016
2
The study
Building on pilot research of business students (Buglear, 2009)
• Final electronic engagement of first year undergraduates leaving
their course in 2008/9 by type of leaver
• The final electronic engagement by each leaver, the last login
– The last visit to the university electronic environment as a registered user
• Why first years?
– Most students who leave prematurely do so in their first year
• Defining types of leaver
– Notifiers; the ‘decided’, those giving formal notification of their
departure, recorded as e.g. ‘Transferred to other institution’, ‘Gone into
employment’, ‘Other withdrawn’.
– Non-notifiers; the ‘drifters’, those giving no such notification, recorded
as e.g. ‘Written off after lapse of time’, ‘Dormant’. ‘Academic failure’ is
included in this category as the last logins preceded the examination
period.
23 March 2016
3
The case study
• Nottingham Trent University (NTU), UK
• Student population of approximately 25,000 in 2008/9
• In 2008/9 nine schools located on three campuses
23 March 2016
4
Results
• Total last logins to May 2009 = 435
• 217 last logins in the first half year (October to January)
• 228 last logins in the second half year (February to May)
• Notifiers = 257 (59.1%)
• Non-notifiers = 178 (40.9%)
23 March 2016
5
Last logins over time
80
70
Last logins
60
50
40
30
20
10
0
October
November
December
January
February
March
April
May
2008/9 academic year
23 March 2016
6
Last logins over time by notification
Yes = notification of departure
No = no notification of departure
80
Variable
Yes
No
70
Last logins
60
50
40
30
20
10
0
O
23 March 2016
r
be
o
ct
v
No
r
be
em
m
ce
e
D
r
be
r
ua
n
Ja
y
F
y
ar
u
r
eb
ar
M
ch
ril
Ap
ay
M
7
Last logins by school and notification
Total Yes = notified departures from the school
Total No = departures from the school not notified
Variable
Total Yes
Total No
90
80
Last logins
70
60
50
40
30
20
10
0
School
23 March 2016
AR
E
BE
AD
A
&
D
ts
Ar
&
H
n
s
es
tio
n
a
i
s
uc
Bu
Ed
w
La
ie
Sc
e
nc
c
So
Sc
8
First half year: 71/217 last logins were non-notifiers (32.7%)
Second half year: 108/218 were non-notifiers (49.5%)
Test for difference in proportions = 0, P-Value=0.000
Difference is significant
250
Variable
Yes
No
Last logins
200
150
100
50
0
Half year
23 March 2016
Oct-Jan
Feb-May
9
Animal, Rural and Environmental Sciences
Difference in proportions is not significant
(Fisher’s exact test P = 1.000)
14
Notify _A nimal,
Rural & Env ironm
Yes
No
12
Last login
10
8
6
4
2
0
23 March 2016
Oct-Jan
Feb-May
10
Architecture, Design and the Built Environment
Difference in proportions is significant at 5%
(Fisher’s exact test P = 0.016)
40
Notify _A rchitecture
Design and
Yes
No
Last logins
30
20
10
0
23 March 2016
Oct-Jan
Feb-May
11
Art and Design
Difference in proportions is significant at 10%
(Fisher’s exact test P = 0.098)
50
Notify _A rt
& Design
Yes
No
Last logins
40
30
20
10
0
23 March 2016
Oct-Jan
Feb-May
12
Arts and Humanities
Difference in proportions is not significant
(Fisher’s exact test P = 0.747)
25
Notify _Arts &
Humanities
Yes
No
Last logins
20
15
10
5
0
23 March 2016
Oct-Jan
Feb-May
13
Education
Difference in proportions is not significant
(Fisher’s exact test P = 0.384)
30
Notify _Education
Yes
No
25
Last logins
20
15
10
5
0
23 March 2016
Oct-Jan
Feb-May
14
Nottingham Business School
Difference in proportions is not significant
(Fisher’s exact test P = 0.286)
40
Notify _Nottingham
Business Scho
Yes
No
Last logins
30
20
10
0
23 March 2016
Oct-Jan
Feb-May
15
Nottingham Law School
Difference in proportions is not significant
(Fisher’s exact test P = 1.000)
30
Notify _Nottingham
Law School
Yes
No
25
Last logins
20
15
10
5
0
23 March 2016
Oct-Jan
Feb-May
16
Science and Technology
Difference in proportions is significant at 10%
(Fisher’s exact test P = 0.099)
Notify _Science
& Technology
Yes
No
30
25
Last logins
20
15
10
5
0
23 March 2016
Oct-Jan
Feb-May
17
Social Sciences
Difference in proportions is not significant
(Fisher’s exact test P = 0.282)
50
Notify _Social
Sciences
Yes
No
Last logins
40
30
20
10
0
23 March 2016
Oct-Jan
Feb-May
18
Discussion
• Financial aspect
– Approximately 180 first year students drifted out of NTU programmes in
2008/9.
– Consequent loss of tuition fee revenue ≈ £2m.
• Pedagogical aspects
– first-semester decisions to exit […] are most aptly characterised as driven
by external factors’ (Peel et al., 2004)
– ‘second semester [leavers] seemed more disillusioned and unhappy, […]
expressing feelings of loneliness, isolation, and lack of recognition’, feeling
that ‘lecturers were “never there” or “always regard failure with disdain” or
“never gave me the help I needed” ’ (Peel et al., 2004)
23 March 2016
19
Discussion
• The Fitzgibbon and Prior (2003) timeline model
– ‘Zone 1: enrolment, induction and the first two weeks of teaching’,
– ‘Zone 2: late enrolment, late induction and early weeks of
teaching’,
– ‘Zone 3: middle to end of teaching period, first/second
assessments’,
– ‘Zone 4: final assessment period, revision and examination or
assessment’
– Zone 3 is when ‘students who have poorly established […] study
habits, really come under pressure’ and ‘students […] receive
feedback from their first assignment [;] constructive feedback and
reassurance is […] crucial’
– Yet by this stage ‘staff assume students have settled […] but this
is frequently not the case [,] students are still seeking significant
levels of contact with their tutors for a whole range of issues’
23 March 2016
20
Discussion
Retention strategies
• The Beatty-Guenter four-stage retention strategies model
(1994)
– Sorting students ‘into meaningful subsets […] to create strata that can be
matched with appropriate targeted retention strategies’
– Supporting, ‘making it more likely that they will be able to maintain their
status as students’
– Connecting, ‘bonding between a student and the institution’
– Transforming ‘students from uncommitted to committed, from
uninvolved to involved, from passive to active, or from failure threatened
to achievement motivated’
• How did we do?
– Sorting – partially applied e.g. international students
– Supporting – Welcome weeks, induction
– Connecting and Transforming – assumed to be intrinsic
23 March 2016
21
Conclusions
• A significantly greater proportion of second half year leavers than
first half year leavers didn’t tell us they were going
• Considerable variation between schools
• The majority, 60% of last logins before the examination period were
by students who told us they were going, the ‘decided’
– The notification suggests some form of dialogue about their departure
• The remaining 40% were by the ‘drifters’.
– The lack of notification suggests an absence of dialogue about their
departure
• The extent of non-notified departure is the scope for pay-off from
Zone 3 Connecting and Transforming strategies
• Not the whole retention picture, but another perspective of it
23 March 2016
22
References
• Beatty-Guenter, P. (1994) Sorting, supporting, connecting, and
transforming: Retention strategies at community colleges. Community
College Journal of Research and Practice, 18, 113-129.
• Buglear, J. (2009) Logging in and dropping out: exploring student noncompletion in higher education using electronic footprint analysis. Journal of
Further and Higher Education, 33, 381-393
• Fitzgibbon, K and Prior, J. (2003) Student expectations and university
interventions – a timeline to aid undergraduate student retention [online].
BEST Conference: Creativity and Innovation in Academic Practice, Brighton,
9-11 April 2003.
• Peel, M., Powell, S., and Tracey, M. (2004) Student Perspectives on
Temporary and Permanent Exit from University: A Case Study from Monash
University. Journal of Higher Education Policy and Management 26 (2), 239249.
23 March 2016
23