Indicators of student progress as flags for early intervention at UNSW
Rita Kusevskis-Hayes
Colin M. Clark
Student Life and Learning
University of New South Wales, Sydney, Australia
Abstract
This paper describes current initiatives at the University of New South
Wales around Early Intervention. A large-scale effort was undertaken to
extract information on student progress from university databases. This
information included special consideration applications, changes in
academic standing, access to Learning Management Systems, repeat fails in
core courses, failure rates, cohort tracking with retention and attrition and a
history of suspension/exclusion/appeals. Overall, the number of times a
student accesses an LMS is considered significant in courses that make use
of such systems. Academic standing is a strong indicator of a need for
intervention, as are repeated applications for special consideration.
Introduction
The University of New South Wales Early Intervention and Retention Program aims to
improve retention and success rates through targeting students early in their university
experience. It was established in November 2011 as a collaborative initiative between
Student Life and Learning, the Faculty of Science and the Australian School of
Business (ASB). The Early Intervention Team, incorporating faculty-based Student
Support Officers (SSO) during 2012, aims to develop strategies to identify students at
risk of failing or withdrawing from university as early as possible. Appropriate
proactive and remedial interventions targeting identified cohorts of students are being
developed, piloted and evaluated. These are based on data gathered from application
forms and records managed by Student Central and UNSW’s Institutional Analysis and Reporting Organisation, including appeals against suspension or exclusion, special
consideration applications, withdrawal without penalty, program enrolment variations
and financial loans.
Improved quantitative information will allow students to benefit from early intervention
in the form of appropriate support provided in time to maximise the likelihood of
positive outcomes in terms of academic performance and early resolution/assistance
with personal issues. Experience at other universities suggests that retention rates
should improve (Edith Cowan University, 2012); university resources allocated to
dealing with nonperformance and/or difficult student issues will be more appropriately
targeted in a timely manner.
Administrative data has been used at other universities to identify students at risk of
failure, particularly in ‘equity’ groups (e.g. Rienks & Taylor, 2009). This is a more systematic approach to identifying students in need than current efforts, which at
UNSW tend to vary between faculties and rely on personal judgements of academic or
professional staff. Similarly, the aim of this project was not to rely principally on
‘predictive’ demographic factors such as ethnicity, membership of equity group or age,
because of concerns that such factors may lead to self-fulfilling prophecies (Guyll,
Madon, Prieto, & Scherr, 2010; Jussim & Harber, 2005) but on observed behaviour,
events and results.
Method
The project began with identifying from available data and reports when students were
experiencing difficulties with their studies. Analysis commenced in 2011 of special
consideration applications (i.e. for students requiring extensions or other adjustments
because of illness, injury or similar circumstances), withdrawals without failure,
appeals of suspensions/exclusions and discontinuation information. The analysis also
considered repeat fails of core courses and academic standing, which is classified as
follows:
 Good: passing the majority of courses;
 Referral: failure to pass half the number of course credits undertaken—referred
to academic advisor for subsequent semester;
 Probation: Repeated failure to pass half the number of course credits;
 Suspension: For one semester
 Exclusion: For two years
Students may appeal suspension or exclusion if they feel the penalty is unfair.
The students at each level were tracked and changes in academic standing noted.
During 2012, the project piloted the allocation of two student support officers to
collaborate closely with the Faculty of Science and the ASB. Through a Faculty-based
approach, the officers were able to obtain further information relating to student
progress and areas of Faculty or School concern regarding red flags for students at risk.
These include declaring a major, program transfer for struggling students, lack of
engagement with LMS in the first few weeks of a semester, and attendance.
Descriptions of programs relevant to early intervention and retention were detailed as a
result of a university-wide mapping exercise with a focus on those with potential for
better systemic use for early intervention of students at risk. These included a peerassisted support scheme, peer mentoring programs, use of faculty-based academic
advisors, Student Centre advisors, and educational development units offering targeted
support programs such as tutors for mathematics and statistics support). There is also a
PITSTOP program (economics and maths), where students can drop in for assistance.
Student-focused programs such as peer mentoring exists in various formats across the
university with potential for greater development within an early intervention
framework. Of special interest was student academic course information, enhancing the
students at risk profile at UNSW.
In addition, the number of times students at the ASB accessed a learning management
system was tracked. UNSW currently uses Blackboard, but is piloting Moodle as an
alternative. The trial identified students at risk based on their non-participation on
Blackboard or limited academic engagement. This project expanded the courses to
include compulsory first year core courses and pivotal first and second year courses
required for selected majors for the Bachelor of Commerce and Bachelor of Economics
degree programs. Discussions have been held with and feedback collected from
stakeholders with interest in this scheme. A total of 23 core courses for the Bachelor of
Commerce and Bachelor of Economics degrees and majors were selected.
Data were retrieved from reports generated by UNSW databases, and in the case of
learning management software from the program itself.
Results
Access to Learning Management Systems
The disparate use of the online LMSs Blackboard and Moodle highlight that early
intervention strategies based on these systems (e.g. monitoring the frequency of student
log-ins) will only be effective in faculties/schools where they are commonly used. Their
usefulness as a measure of student engagement depends somewhat on how they are
used, whether as a repository of reference material or as a means of course delivery
such as blended learning. Other methods and strategies have to be found to include
students in courses that do not use online LMSs.
In the ASB, of 6,155 students enrolled:
 1,260 students were identified as having not accessed their online Blackboard
course material since enrolment into the course. Emails were sent to these
students.
 1,160 students were identified as having limited access over the four weeks
having only accessed their online Blackboard course material on one or two
occasions since their enrolment into the course.
Following the email, an average of 13% of students in 23 courses dropped out, however
there was no noticeable increase in enquiries or requests for assistance from Student
Services. This is consistent with previously reported difficulty of communicating with
first year students (Lodge, 2010).
Special consideration
In Semester 1, 2010, 1528 undergraduate students submitted Special Consideration
applications (5.1% of all undergraduate students); these students submitted a total of
3,126 applications within one semester (an average of nearly two applications per
applicant). Of this group, 1,207 students from the four faculties of ASB, Engineering,
Science and the Faculty of Arts and Social Sciences (FASS) submitted 2,424
applications. The number of applications per student from these four faculties in
S1/2010 range from one application (659 students, or 55% of applicants) to 33
applications lodged by a single student (or 0.08% of applicants).
The four most common reasons cited by the students in appeal cases across UNSW are
academic (58%), personal (57%), and health reasons (48%). Mental health (25%)
reasons are more common than physical health problems (17%), in particular, cases of
depression, anxiety and stress are regularly described in appeal cases. Social and
employment-related reasons were each mentioned in 23% of the appeals, whereas
financial or accommodation issues are mentioned in fewer than 20%.
Suspensions, exclusions and appeals
In 2010, the number of suspended or excluded students was 488, which was 1.8% of the
total enrolled undergraduate students at UNSW. Of this group, 172 students (35%)
appealed against their suspension or exclusion. The four faculties Engineering, ASB,
Science and FASS had the largest number of suspensions, exclusions and appeals.
In terms of residency, international students accounted for 20% of total undergraduate
enrolment numbers in 2010. Slightly more—24% of the excluded/suspended students—
were international, and 35% of the appeals were lodged by internationals. While 52% of
the suspended/excluded international students decided to appeal, only 30% of domestic
students did so. Therefore, it seems that international students are much more likely to
appeal against their academic standing than domestic students.
On average, undergraduate students were suspended or excluded after five semesters of
enrolment, with 43% of suspensions/exclusions occurring within the first four
semesters. The analysis shows that a high number of the appealing students have
struggled since the beginning of their enrolment, but often did not seek or get help and
support. Of these students, fewer than half consulted an academic advisor, suggesting
that Early Intervention does not currently occur to an adequate extent.
Academic standing
The percentages of undergraduate students who did not achieve Good Standing in
S1/2012 vary between 3% (Law and Medicine) and 13% (FASS). The average across
all faculties is 10% (2,729 students). Comparison with the academic standing levels of
the previous semester indicates some trends. For example, the majority of students on
referral in S2/2011 improved their standing in the following semester (64%), whereas
the academic standing of more students who were on probation declined in the next
semester (52%). The standing of half of the students on suspension improved and that
of the other half declined (50% each), while only 38% who were on Provisional
Exclusion achieved improvement in the following semester with the majority remaining
at this level. In general, the data indicates a trend that the further down the students are
academic standing the more difficult it is to improve.
Discussion
The collection of data on student progress at UNSW has only recently begun, but
already shows some promise as a method of revealing students in difficulty before the
problems become unmanageable. Unlike predictors based on purely demographic
factors, these are valid on an individual basis because they rely on actual results,
observed behaviour and events. While all these data are available to faculty staff, there
is often little attempt to use it in a systematic manner to target students for assistance.
Current research at UNSW seeks to extend this approach and create a model by which
combinations of factors may be assessed in identifying students in need of intervention.
Key questions
 How might such information be used to best effect?
 How can professional staff achieve a consistent approach to intervention across
faculties?
 Who is responsible for intervening with students who are at risk of failure or
leave university?
References
Edith Cowan University Student Services Centre. (2012). Connect 4 Success Proposal:
A proactive student support approach.Retrieved from
http://intranet.ecu.edu.au/__data/assets/pdf_file/0019/310753/Connect-for-SuccessProposal.pdf
Guyll, M., Madon, S., Prieto, L., & Scherr, K. C. (2010). The potential roles of selffulfilling prophecies, stigma consciousness, and stereotype threat in linking Latino/a
ethnicity and educational outcomes. Journal of Social Issues, 66 (1), 113–130.
Jussim, L., & Harber, K. D. (2005). Teacher expectations and self-fulfilling prophecies:
knowns and unknowns, resolved and unresolved controversies. Personality and
Social Psychology Review. 9( 2), 131–155.
Lodge, J. (2010). Communicating with first year students; so many channels but is
anyone listening? A Practice Report. The International Journal of the First Year in
Higher Education, 1(1), 100–105.
Rienks, J., & Taylor, S. (2009). Attrition and academic performance of students
identified as at-risk using administrative data alone. Paper presented at the First
Year in Higher Education Conference 2009. Retrieved from
http://www.fyhe.com.au/past_papers/papers09/content/pdf/1B.pdf