Focus Groups - Brief Summary
The following details information gathered from the series of six focus groups
comprising a total of thirty psychology students in their first and second years of
study. The aim of these groups were to gather student views on the design,
implementation and usefulness of an Early Warning System to highlight students that
may be having difficult at university. The purpose of developing this system can be
summarised with the following quote:
"Often I find that the university staff, they'll always sort of realise
students have had problems when it's almost too late, if not already
too late, because things like attendance, handing in work, I don't think
that's something that's checked regularly. I don't think they go on
moodle and see, 'OK this person has definitely handed it in', I think
once they're marking it towards the end of the term, or when people
have already handed in and everything has moved on, that's when they
sort of realise and I think by then it's already too late."
Key Warning Indicators
All groups identified the following indicators as being important:




Not submitting work
Submitting work late
Attendance
(Falling) Grades
Attendance tended to be the first sign mentioned by most students. However,
subsequent questions suggest the non-submission or late submission of work are more
important indicators with regards to student difficulties. Students also consider their
grades to be an important in regards to how well they are doing, as such falling grades
or making no improvement in grades is a prime concern for many students.
Performance Indicators
When asked what kind of data apart from that listed above might be useful in
measuring student performance the following response were given:








Pattern of resource use
Timing of submission in relation to deadline
Frequency of email login
Frequency of moodle login
Click count in moodle
Number of resources accessed in moodle
Key resource access on moodle
Performance compared to student average
The virtual learning environment was consistently cited as a crucial resource for
students, especially those living off campus.
System Implementation
Students had varying views on how the system should operate and how interventions
for at risk students should be handled.
"I don't expect my tutor to come and follow me up because, for me
that's the whole point of university, like independent, you have to
do your own thing"
"I think that it would be good for some people, because some people,
its the transition from school to here, they're not used to having to do
that by themselves, so some people might need it and some people
might not"
The majority of groups agreed that interventions should be performed as soon as
possible. In all instances this was by the end of the first term, the average range
suggested this should be done around week 9 and the earliest instance was week 6.
However, in all cases it was apparent that intervention time was linked with
assignment submission deadlines. Attendance was not the prime consideration for
students in regards to reasons for intervention, week 6 was suggested as the earliest
stage at which this should be done in a non-automated capacity.
It is worth noting, as the students did, that for the majority of the course, no
assignments typically get set before week 5 of the course. As such, assignment
submission tends to be in week 8 or later. If non/late submission are the key indicators
with regards to being at risk and since these are not relevant until two thirds of the
way through the first term, any problems arising are likely to be sizeable and therefore
require more work to address.
System Automation
To avoid embarrassment and mixed messages the system should be automated as far
as possible, particularly in regards to attendance concerns. As such, students are
concerned the data presented is accurate. The operation of the system should be
transparent and must be seen to act in line with the specification given to students.
Although students have no wish to be treated like children, they would prefer concern
that was unnecessary to no concern when it was needed.
It is likely a three tiered approach to automation would be most suitable, the first
sends a mail to the student only, second send mail to student and a copy to the
lecturer, third sends a mail to the lecturer asking them to set up a meeting.
The system may also benefit from sending pre-emptive communications, for example,
reminding students they have an upcoming deadline. The system should also allow
students to view their own statistics and relevant information, at both the module and
course level.