Equitable Assessment for a Diverse Student Population
Pilot Study Report
2008-9
A study carried out by
Parveen Bhattia, Sabiha Bibi,a Susan Murphyb, Leena Sandhua, Sonia Sahotaa,
Elizabeth Thomasb, , Beverley Westwoodb and Simone Wilsona
in collaboration with
Samantha McGinty and Christine Hockings (Institute of Learning Enhancement).
a
School of Humanities, Languages and Social Sciences
b
School of Education
Background
This pilot study explored the ways in which first-year students from diverse
backgrounds within two subjects (Education and Social Science) experienced
assessment within a post-1992 university. A key aim of the study was to identify,
from the students’ own testimonies, positive and negative aspects of assessment that
might account for differences in performance and grade attainment. The key question
we aimed to address in this pilot was: in what ways does assessment allow the
expression of learning and knowledge that students from a range of social, cultural
and educational backgrounds bring?
In this report we briefly outline the literature that helped us explore the issues
around fairness in assessment in Higher Education (HE). We then describe the
qualitative research design we used to address our research question, before focusing
on the key findings. Finally we identify what further research and development is
needed to enhance fair assessment. This project, led by Professor Christine Hockings
with PhD student Samantha McGinty, was funded by the Institute of Learning and
Enhancement (ILE) at the University of Wolverhampton as the pilot study for a larger
project to be submitted for funding by the NTFS in 2008-2009.
1
Context
There has been a growing concern among UK academics and researchers in recent
years over the unexplained gap in the degree attainment of white British students and
their minority ethnic peers (Broecke and Nicholls 2007, Connor et al, 2004;
Richardson, 2008). The research suggests that minority ethnic students are less likely
to gain a first or upper second class degree than their white peers. Even when taking
into account the lower percentage of students likely to gain a first or upper second in
post-92 universities, the differences between minority ethnic and white students
remain. 54 per cent of white students in post-92 universities achieve a first or upper
second class degree in comparison to 35 per cent of minority ethnic students (Connor
et al, 2004, p.77).
Satisfaction with assessment and marking arrangements has also been reported
differently by students from different ethnic backgrounds. For example, the National
Student Survey for 2004-05 and 2005-06 (Surridge 2007) revealed that 74 per cent of
white students agreed with the statement ‘assessment and marking arrangements have
been fair’, compared to 66 per cent of black students, 64 per cent of Asian students
and 67 per cent of ‘others’ (in HEA/ECU, 2008, p.16). This may suggest that black
and minority ethnic students (BME) experience assessment in way that is different
from their white British peers. We were keen therefore to explore how the assessment
practices within one post -1992 might be seen from the point of view of a small but
diverse group of first year students. In indicating students' social background,
particularly in terms of ethnicity, we are aware that we run the risk of suggesting a
simplistic correlation between what students say and this dimension of
themselves. We take the view that people speak from a complexity of individual and
social experiences but given that research has identified trends according to social
factors, our exploration has included an engagement with these in order to identify
any noteworthy variation across social groups'.
Research design
We took a broadly interpretative approach to understanding the meanings and
behaviours that students attached to their experiences of assessment. Whilst much of
the literature focuses on the attainment of students from different ethnic backgrounds,
we wanted to compare the assessment experiences of students from a range of social
groups
2
Participants
We attempted to engage participants who, as far as possible, represented the diversity
of the student population in a number of first year core modules in sociology,
religious studies and education. Of the 91 student participants, 70 per cent were aged
between 18 and 20 and almost 80 per cent were female1. 56 per cent described
themselves as white, 27 per cent were of South Asian heritage and 12 per cent were
from African or Caribbean backgrounds. Just under half of the participants (48 per
cent) described their parents’ occupations as professional, managerial and technical or
skilled non manual which we grouped as middle class occupations. Just under a third
described their parents’ work in categories associated with working class occupations.
20 per cent of participants’ parents were not in work (see Appendix 1 for table of
participants’ demographic data).
Student Researchers
A key feature of the research design was the involvement of eight third year students
as co-researchers within the project team. We felt that student researchers would not
only make a unique contribution to the study (e.g. capturing the authentic student
voice, refining student-friendly research instruments), they would also gain credits as
part of their degree programmes for research skills and experience developed within
an authentic research environment (Jenkins et al. 2003). Although no male student
volunteered to join the project, the eight female students that did join came from a
range of ethnic backgrounds and age groups. All student-researchers had a
background in either education or sociology. The student-researchers contributed to
all aspects of the research process although some specific training and support was
needed. Initially they were given key readings to familiarise themselves with the
topic and main issues. With this body of knowledge and their own experiences of
assessment, they were then able to help make informed decisions about the questions
to ask the first year participants at our regular team meetings.
Impact survey
1
The high level of female participation is not reflective of the female student population in the
institution generally which in 2006-7 was reported as 58 percent (Data source: HESA and SITS,
November 2007). However in Education nationally, there is an over representation of female students.
This explains to some extent the high level of female participants.
3
With the help of the student researchers we designed a short list of questions that
would be asked of the first year students on the selected modules. With the consent of
the module tutors and their first year students, the student-researchers entered class
rooms at break times or at the end of lectures to conduct 5 minute interviews with up
to 10 willing participants each. The aim of this ‘impact survey’ (see appendix 2) was
to canvass students’ opinions and experiences on assessment, to capture demographic
data and make initial contact with students in the hope that they would agree to be
interviewed in-depth at a later convenient date to themselves. It also provided the
novice student-researchers an opportunity to practice their interviewing skills in an
authentic setting. The student-researchers recorded over 90 participants’ responses by
hand but subsequently entered them into an on line questionnaire that the senior
researchers had set up using the on line package ‘Surveyor’. This allowed all
researchers to get an overall picture of the demographics, additionally open-ended
responses could be attributed to each individual (but anonymised) participant –
allowing us to identify if particular groups of students were expressing similar issues.
Interviews
The student-researchers followed up the survey participants who had initially
expressed an interest in being interviewed further. Whilst we were keen to interview
an equal number of male and female students from a range of ethnic backgrounds, age
groups, we did not fully achieve this. Of the 24 students interviewed, 11 were over 21
years of age on entry to university and therefore considered ‘mature’ students. 12
described their ethnic background as white, 4 were of South Asian heritage and 5
were described as Black British. Only 6 were male (See table 2 Appendix 1). Being a
small scale study our intention was to explore in depth the students’ positive and
negative experiences of assessment, their approaches to doing assessments, the
different forms of support they drew upon, as well as their perceptions of fairness.
Despite the small numbers, we anticipated seeing some evidence of certain
experiences, behaviour and perceptions reported more often by some students than
others. However our intention was not to seek generalisation but to understand the
issues.
Student-researchers made contact via email and arranged mutually convenient times
to conduct the interviews. The interviews were semi-structured in nature with suitable
prompts to elicit more detailed responses (see Appendix 3). This had the advantage
4
of clear questions, whilst allowing the interviewee to highlight and discuss any
assessment issues that they felt were pertinent. These interviews varied in length to
between 20 minutes and 1 hour and were recorded using digital voice recorders. Each
interviewer conducted between 3 and 5 interviews and transcribed at least one
interview. All names were changed in the transcripts to aliases to ensure anonymity
and each transcript was sent to the interviewee for member-checking (Silverman,
2001:229). At this stage, time was critical for the student-researchers because they
needed to have all interviews transcribed so that they could analyse the whole data set
and write up their individual reports in time for their own module assessment
deadlines. To speed up the production of transcription all remaining interview sound
files were transcribed by a dedicated transcriber.
Findings and discussion
Our analysis of both the impact survey and individual interview transcripts revealed
no major variation in the ways students approached, felt about or experienced
assessment that could be attributable to race or gender. Overall students felt that
assessment was fair and equitable because ‘people get similar grades’ and ‘everyone
gets the same assignment.’ These comments correspond to an equity theory / social
justice model of fairness and, in particular, to what Lizzio et al (2007) call
‘comparative equity’ i.e. that rules and procedures are applied consistently and fairly.
Beneath these students’ comments lay their unquestioning faith in the objectivity and
fairness of the assessment system, yet ‘we maintain the myth to our students that there
is a correct judgment about the value of their work... and that we are capable of
making it’ (Bloxham 2007). Whilst some students did criticise some aspects of their
assessment, such as the language used, this was seen as ‘unhelpful’ rather than unfair.
Students were more inclined to attribute poor grades to some failing on their part,
rather than a failing of the assessment system.
In the next section we explore the personal circumstances, beliefs and behaviours that
some students felt affected their capacity to perform to their best or as well as others.
We then look at the factors within the assessment system that they felt were unhelpful.
In doing so we draw attention to cases in which either personal or institutional factors
appear to have affected assessment performance of some students more than others.
We illustrate our findings with extracts from the survey and interview data, providing
5
some demographic student information in order to indicate the diversity of students
expressing these ideas.
Personal circumstances and characteristics affecting assessment
Assessment Anxiety
Analysis of the 91 responses to the impact survey, revealed that the majority of first
year students experienced high levels of assessment anxiety. They feared not being
up to the higher standard that they felt would be expected of them at university both in
terms of quality and quantity (see also Christie et al. 2008). Typical comments
included:
I think the work is obviously going to be harder than at Sixth Form. I'm scared and
nervous. (Asian Indian Female)
I'm scared because it’s like 2500 words (white female)
Further analysis suggested that some of these concerns were linked to students’
confidence as academic writers and a lack of familiarity with what Lillis (2001)
describes as ‘essayist literacy’. Assessment anxiety was also felt among those who
did not get or seek tutor help (see later). Of the ethnic groups who reported general
feelings of anxiety about assessment, twelve came from white British backgrounds
and fifteen came from BME backgrounds. This represents 25% and 40% of the white
and BME students respectively in our impact survey sample. Of those who said they
felt ‘okay,’ ‘fine so far’ or ‘confident’ about the assessments, only 28% were from
BME backgrounds compared to 72% from white backgrounds. This may suggest that
white students are more confident and have fewer concerns about completing
assessments than BME students. The reasons for this are not clear from our data
although ,as we discuss later, there is appears to be a high level of interaction between
white students and academic / support staff that may impact on their understanding of
what is required.
Due to the low numbers of male participants in this pilot, it was not possible to make
comparisons between male and female students experiences and feelings about
assessment. However, there is some evidence in the literature that suggests that
female students experience greater test anxiety than their male peers. For example
6
Leathwood and O’Connell (2003) found that the female students in their study talked
about a lack of confidence and not being good enough for university. . Martin (1997)
also suggests that males are generally less anxious about examinations than their
female peers. Other studies suggest that students approach to learning is also a key
factor influencing assessment anxiety (see for example Ben- Shakhar & Sinai, 1991,
Birenbaum & Feldman, 1998).
Age matters
Age was also seen as a factor influencing students’ perception of who might have
more or less advantage in completing assignments and gaining the highest grades.
Most of the students in this study were new to higher education. Most were uncertain
as to what was expected of them in terms of their written work. Mature students,
however, believed that the younger, school and college leavers would be more
familiar with academic writing conventions than them because of their recent
educational experiences. They therefore perceived themselves to be disadvantaged by
their time out of formal education. These mature students explained:
…academic writing is new to us from such a long time ago …we’re, we’re working
backwards, we’re having to write essay type questions and I’m working back on how
to write it….. (Asian Indian, mature male)
I'm worried. I haven't been to school for a long time while others may have. However,
maybe everyone else is worried to. I'm worried I won't be able to do what is expected
of me. (Black African, mature female)
Mature students also believed that the younger students had better IT knowledge and
skills than they had, giving them a further advantage. One interviewee explained:
….using the electronic resources was something that was completely new to me... I’d
hardly used computers before...They’re [other students] all much younger than me
and their really good on the computers, you know, which I’m not, I’m having to learn.
(white, mature female)
7
Despite this perceived disadvantage, many of the mature students spoke of their
motivation, determination, hard work and commitment to their studies, qualities that
they believed would help them achieve high grades. They contrasted their attitude
and approach to study with those of their younger, apparently more carefree peers:
I think the younger person is playful, so when they do well they don’t even… think
it’s really something special. When they flunk, they’re not sorry. So, either way
they’re not bothered. Sometimes they do well, sometimes they don’t do well, but I
don’t think they’re bothered at all. (Black African, mature male)
When it came to mixed group assignments, however, their perception of younger
students as less motivated and more carefree was seen as a potential threat to the
overall performance of the group and to the grade they could collectively achieve.
Prior Subject Knowledge
The sense of feeling overwhelmed by the volume of new knowledge and abstract
ideas and new ways of coming to know them in the first year of university is common
among many students (Kember, 2001). Some of the students we interviewed had
recently studied the subject at school or college and were au fait with specialist
terminology and key concepts. Others entered as non-specialists perhaps with
qualifications in other subjects or with work experience in the field. Those who had
not studied the subject formally before, or who felt weak in the subject, felt they had
to work extra hard to ‘catch up’ with the rest of the class and to ‘keep up ’ with the
pace and quantity of learning. As a result of their perceived lack of subject
knowledge, some did not expect to achieve grades comparable with their A level
peers... one student articulated her feelings:
Erm, I think I’ll probably do a little… worse than the people that have done Sociology
at A Level because they have a, an understanding of, you know, Sociological issues
that we’re talking about…They have a bit more of an understanding. Erm, so I might
do a little bit worse than they will. Erm, but then again I could just….I have been
doing a lot of reading so I could easily catch up, so it’s kind of….you don’t know
what’s gonna happen so. (Black African, mature female)
8
So whilst entry requirements have allowed wider access to some degree subjects,
those who enter with different knowledge sets appeared to be disadvantaged if the
course and the assessment focus on building traditional knowledge that tends to be the
focus of A level syllabi.
A perceived disadvantage in one area can often be outweighed by advantage in
another. For example, lack of prior subject knowledge can be outweighed by a deep
interest in the subject and motivation to learn. Indeed, subject choice and interest in
the subject have been identified in the literature as key factors influencing student
achievement. Recent studies (Jacobs et al., 2007, Magadi et al., 2007, Scales and
Whitehead 2006) have indicated that students who choose a degree course to satisfy
some extrinsic demand (such as parental wishes (see Ahmad 2001, Davis et al. 200))
rather than to satisfy their own intrinsic interests, are unlikely to feel motivated to
study hard. This may be reflected in their grades with low intrinsic interest
correlating with low achievement. Whilst none of the students in our study spoke of
having chosen the wrong subject, both the Magadi et al (2007) and Scales and
Whitehead (2006) reports argue that South Asian students are often under parental
pressure to take professional subjects, such as medicine, law and accountancy, with
often disappointing results. This is something we shall explore in subsequent research.
Time
Lack of time to prepare for assessment was raised by a number of students. Some
students recognised that they needed to get organised and start preparing earlier for
assessment if they wanted to get better grades. However there were others whose time
to study was always limited by competing demands. Some students were working
several hours a week in order to finance their studies. This student felt unable to seek
help because she had prioritised her paid employment over study time (See also Bowl
2005):
I am employed I work at [a high street clothes shop] and also I work for Nursing as
well…. If I hadn’t have left it so late then I’ll say yeah I could go to my tutor…but I
think it’ll be a bit rude of me at such a late stage in the day, to go to her and say well
look I’m, I’m stuck...cos they offered us help on Tuesday but because I actually hadn’t
9
looked at it properly I couldn’t very well go up there and say… (Black Afro
Caribbean, mature female)
Students with caring responsibilities also found limited time available for assessment
preparation. This student was unwittingly excluded by her peer group because child
care responsibilities made it difficult for her to change her study patterns at short
notice:
We had to do a presentation… we’ll meet up the following week… and I was like
“great,”… I don’t have to worry about childcare…or anything then and then they
emailed me erm, on the Tuesday night to say they changed their mind they we’re
meeting up of the Wednesday at 3 o’ clock and I couldn’t make it. (White, mature
female).
A small number of disabled students also talked about how their disability meant they
needed more time than others to complete the work for the assessment. This point is
raised along with other issues regarding the assessment of disabled students by Grace
and Gravestock (2009) and McCarthy and Hurst (2001). For example whilst this
student felt the various support that she had accessed had been excellent:
… being a disabled student it is actually a lot harder for me and because of
having health conditions there could be times when I have to go to the hospital and I
miss out on hours of facilitation, which means I have to catch up at other times.
She went on to add,
So, even going through your work and proof reading is essential but, sometimes for
me when after I’ve just, say, spent three hours (which is more like what it would take
me because of my disability on an assignment and then I’m in pain) it’s more difficult
for me to read through and actually spot the mistakes because I’m tired or I’m in pain.
So that is when I’m unfairly disadvantaged. (white, female)
Whilst the University tries to ensure that students who have particular needs (e.g.
those with childcare commitments, limited finances, English as a second language,
10
disabilities or medical condition) are not disadvantaged by lack of time, support or
access, there was still a feeling that the assessment tasks were more onerous for them
than for ‘others’. Nevertheless students do not want special treatment, they don’t want
to feel ‘different’. They want mainstream assessment to cater for all their needs as far
as possible (Grace and Gravestock 2009)
So far then, students in this study were aware of a range of personal circumstance and
factors that could affect student performance and attainment. Overall students
internalised these problems. Nevertheless there were a number of criticisms of the
assessment system and institutional support for it which may have disadvantaged
some students. We discuss these in the next section
Criticisms of the assessment system
Academic support
A number of students raised concerns about the level of academic support they might
expect to receive at university, often in contrast to the high levels of tutor support they
received at school or college.
The teachers don't help you all the time like they do in Sixth Form. I don't even know
what I have to do for the work. (Asian Indian female)
Not sure what’s happening. Need more help. There’s a lot of reading (White Other
male)
At college there is an opportunity to discuss in detail on what is needed to obtain a
high grade so the support was there from the teachers. Also there was an opportunity
to resubmit if needed but at university that is not the case. (Black Afro Caribbean,
mature female)
Many students raised the issue of needing to speak to academic staff to fully
understand what is required for assignment tasks and to establish the marking
standards. This was also noted in the report on Ethnicity, Gender and Degree
Attainment ( HEA/ECU 2008:16) perhaps suggesting that the formal assessment
11
criteria and associated information are inadequate for helping students understand
what is required. (See later). We were keen to explore, therefore, the ways and the
extent to which different groups of students accessed staff help for assessments. Of
the 11 students in our impact survey who stated specifically that their assessment
anxiety was caused by a lack of support from academic staff, five were from BME
backgrounds (3 Asian female students, 1 Black Afro Caribbean student, 1 Black
African student). Four of the white students who expressed similar views were
female suggesting that feelings around anxiety may be linked to gender as well as
ethnicity. However, as previously mentioned a high number of female students did
participate in the survey (79 per cent) so we cannot draw any general conclusions
along gender lines from this skewed sample. However, given that there were 37 BME
students (just over 40 per cent of the sample) in the study, the number of comments
about assessment anxiety from these groups seemed to be significant. This finding is
consistent with other research, for example, Dyke (1998) who suggests that white
students tend to benefit from higher levels of interaction with academic staff who are
predominantly white and they do better than their minority ethnic peers as a
consequence. Variable support from personal tutors for BME students has also been
noted in other studies (e.g.Magadi et al. 2007).
Criteria
Openness and clarity are fundamental requirements of a fair and valid assessment
system (Sambell et al 1997) and students want clear briefs and clear assessment
criteria (see also Drew 2001). However as O’Donovan et al. (2008: 207) point out:
…whilst a focus on explicit articulation as a means of communicating standards can be considered the
current dominant logic of UK HE with widespread appeal, it is arguably only of limited use in today’s
more fragmented programmes which serve an increasingly diverse student population.
Students in our study said they wanted to know what their tutors expected from them
but whilst some felt their tutors had explained the assessment well, others experienced
difficulties in understanding the assessment criteria. These students had not been
engaged in negotiating, actively using and applying the standards on which their
work would be judged (see for example O’Donovan et al. 2008), nor did they
complain about the use of academic jargon, lack of clarity or ambiguous assessment
tasks and criteria. Rather they attributed their lack of understanding to ‘being a first
12
year’ and not yet being familiar with academic language. They felt it was their
responsibility to ask the lecturer if they were unsure:
Okay. I think that sometime the information that you get on the assessment is a bit
vague. Erm, it’s, there’s not, there’s not much instruction there. I think a lot of people,
even though they’ve talked together I think sometimes they err, they kind of…. we
don’t all sing from the same, you know, song sheet because not everyone is sure of
exactly what they’re meant to be doing. Erm, but you know then again I suppose you
could always go and ask them, ask the lecturer anyway. So, … it’s just some,
sometimes it, it takes a while to actually sit down and think what you actually, what
you’ve actually been asked to do. I think that’s the hardest. (White, mature male)
There was no suggestion from the students of criteria being classed, raced or gendered
yet what knowledge is assessed, how and for what purposes are not neutral, objective
or value-free. ‘Rather what counts as legitimate knowledge is the result of complex
power relations and struggles among identifiable class, race, gender, and religious
groups’ (Apple 2000) Assessment criteria will inevitably privilege the knowledge,
skills and experiences with which some and not ‘other’ students come (Anderson
2001). In her review of the assessment, standards and equity literature, Leathwood
(2005) draws upon a number of studies to illustrate the point that social and cultural
groups differ in the extent to which they share the values that underlie and promote
assessment. In a study of ethnic minority progression and achievement in two English
universities, she cites Dyke (1998) who identified issues relating to the type of
assessment, the assessment criteria as well as the curriculum and staff/student
interaction as ‘not operating in a way that helped equalize life chances but in ways
that helped reinforce stratification on ethnic/racial grounds’ (p.119 cited in
Leathwood, 2005). This work brings into question the apparent transparency and
objectivity of assessment criteria that students accept as fair. Fairness of criteria is
also brought into question by Bloxham (2007) who exposes the ‘fragile enterprise’ of
grading students against only ‘tacitly understood’ criteria and argues that it is time
that we ‘let students into the secret’.
Methods of assessment- choices and preferences
13
In their review of 35 documents relating to students perceptions of assessment in
which 36 empirical studies were discussed, Struyven et al (2002) concluded that when
conventional assessment (e.g multiple choice, exams, essays) and alternative
assessment methods (e.g. self, peer, portfolio and overall assessment) were compared,
students perceived alternative assessment as being more 'fair' than the traditional
'normal' assessment methods. Fairness in this review of the literature was perceived in
different ways with some students focusing on the extent to which the assessment
rewarded last minute cramming or consistent time and effort invested. Others
described fairness in terms of the authenticity of the task in the real world and validity
of the task as a means of long term learning. Unfairness was conceptualised in terms
of irrelevance of task and arbitrariness or ‘luck on the day’. Many of the students in
their study also felt powerless and unable to exercise any degree of control within the
context of the assessment of their own learning and again this was seen as ‘unfair’.
(Sambell et al 1997).
With this in mind we asked students to describe the sort of assessment they would
choose that would enable them to best demonstrate what they know, understand and
can do. The majority had not contemplated the possibility of being allowed to choose
the method by which they would be assessed for any one module. They seemed to be
more trusting of the traditional methods chosen for them than the students in Sambell
et al’s study (ibid), perhaps because they had little experience of alterative forms of
assessment, perhaps because they had been reasonably successful with the methods
chosen for them in the past. However, one student spoke of individual differences and
imagined a system in which students could choose how they would be assessed by
picking from a menu of assessments types:
Not everybody’s gonna find it easy to do an oral presentation. Not everybody’s gonna
find it easy to write a 2,000 word essay. Everyone’s different and it would just….I
know it’ll probably never happen but it would be better if there was several different
forms and then as long as it was accredited in the module people could….to the same
standard….people could pick which one they’d rather do. (White female)
Whilst choosing their preferred method of assessment may have seemed a fairer
system of assessment for this student, many others felt that all students should be
14
assessed in the same way using the methods selected by the teacher as this would be
fair for everyone. Again there was a sense that teachers know best. There was no
apparent pattern to these responses by gender, ethnicity or age.
Erm, well frankly I think me, personally, would choose the same as the uni as, do the
same…I don’t know whether I would look at things differently…and so I would go
along the same lines as the uni. (Black African, mature female)
I don’t think you can really get a much better way then what they’re doing now. I
think you know the assessment by coursework and stuff like that is… I think it’s quite
a fair, fair assessment the way that you know ‘cos they’re giving a chance for people
to go away collect the information, what they need to collect and put it into, put it
into, you know, written form. (White, mature male).
I would keep the same type of assessment. I’d probably just ask for more lectures on
that assessment, you know. (Black African female)
Formative assessment
Feedback on learning through opportunities for formative assessment featured high in
students’ perceptions of fair assessment. On the whole students in this study were
satisfied that their teachers had provided opportunities for them to get feedback on
their learning before the submission of their summative work. For example:
I think it’s useful that they’ve given it you back with feedback like so they’ve looked at
your way of working so they can help you improve it for the rest of it. Rather than
handing it all in like after Christmas and not knowing at all where you’re sort of lying.
So I think that helps a lot, yeah. (White female)
… that feedback helps me to go right well you know. Like if, for example, one of my
modules where we’ve got an essay and then we get feedback and then we can change
it erm, with their feedback. I can then go, right okay, well they don’t like me when I,
they don’t like it when I write this and do that so I’ll get rid of those and then I know
that for next time as well. (White male)
15
They also commented on the benefits of more than one summative assessment
throughout the module giving them a second or even third chance to improve their
overall grade.
Erm, … we split up into small parts instead of just one big assignment and you know,
so we’re encouraged to work together more regularly instead of just right at the end
(Asian, mature male)
Working in a collaborative and realistic environment rather than individualist and
competitive one also seemed to add to this students’ sense of the fairness of the
assessment:
…writing one essay, we have to work in groups and things and the, the, the research
and things it helps. It’s helping us with other subjects as well, the research studies for
this topic. (Asian, mature male)
But whilst the students described examples of good and fair assessment practice,
they frequently used contrasting experiences of poor or unfair assessment practice
where the feedback was negative, inadequate or received too late for them to act upon
(see for example Ramsden 2003, Varlander 2008 for a helpful discussion on feedback
to students). Students were realistic about what they could expect from tutors under
increasing pressure of time, so they appreciated good feedback when they got it.
What we have not been able to establish from this study is the ways in which tutors
allocated time for feedback across the diverse student population nor the extent to
which this feedback varied in quality. This will be explored in the main study.
Conclusion and further research
Throughout this project we have been careful not to draw general conclusions from a
small set of data. We have also tried to avoid focusing on a particular characteristic
of an individual (e.g. female or BME) as though this somehow explains the whole
story. We recognise that mature, black, working class women are more likely to be
time poor than young, white, middle class women, and that may be a reflection of
structural inequalities (see Bowl 2005). However, the fact that a women is black is not
16
in itself the reason for being time poor. We should not therefore assume that she is
disadvantaged because of her ethnicity as though it has some ‘master status’. Thus
we have tried to look at the whole person highlighting, where possible the range of
factors that affect individuals’ performance, rather than attributing their experience
and perceptions of assessment to any one or given combination of characteristics.
This study has explored some of the positive and negative experiences and
perceptions of assessment of a small number of first year students in two social
science subjects. Most of the negative experiences of assessment and criticisms of the
system correspond to those found in the assessment literature in HE. From this point
of view the study has revealed no new findings. However our aim was to explore the
ways in which diverse students experience assessment to see if there were differences
that might account for differences in attainment. From students’ testimonies we have
identified reasons why some students feel disadvantaged compared to other students,
however, negative experiences of assessment such as failure and anxiety, were
conceptualised as personal weaknesses and ‘individualised’ rather than as issues of
unfairness in the system attributed to students’ ethnicity, gender or social class. We
have not, therefore, been able to explain from these data why white students often
gain better grades and ultimately better degrees than BME students. We have,
however, identified areas to explore further that may be related to poorer attainment
such as variable take up of or availability of academic support for assessment and the
emphasis on assessing knowledge that builds on from A level syllabus rather than
knowledge gained in other ways. There is evidence that BME students are more
likely to take non A level qualifications (see Leslie et al 2002) than white students for
example. Work is already underway in two Schools within the University to explore
these specific issues.
The majority of students in the study believed assessment to be inherently fair.
Students who had had direct experience of unfairness and disadvantage at school or
college were more conscious of the issues of fairness in assessment and articulated
these clearly. However, since our participants were in their first year of study, their
experience of assessment at university was very limited. It was perhaps too early in
their course for them to be able to critically reflect on the issues we raised. In the
follow on study we plan to interview second and third year students and, where
17
possible, follow up some of the students from this pilot study to see how and if their
conceptions and experiences of assessment have developed or changed as they
progress.
A further recommendation from this study is that we widen the subjects under study
to include subjects where the variation in degree attainment by ethnicity and or gender
has been recorded. By comparing the assessment experiences of students whose social
and prior educational profiles are similar but whose ethnicity and module grades
different, we hope to be closer to understanding this variation.
Finally we believe that this study or ‘intervention’ has engaged student-researchers in
the development of their research skills:
I feel that being part of a real research team enabled us to develop our professional capabilities in the
area of research and has definitely enriched my university experience. (Liz Thomas - studentresearcher)
I found working as a student researcher on the ‘equitable assessment project’ to be rewarding and
beneficial. I was able to experience a ‘real’ process of fieldwork and research analysis in a social
setting. What’s more I was able to contribute back to the community where learning and diversity for
future generations of students can be pushed to reach greater potentials of achievement at the
University of Wolverhampton (Sabiha Bibi - student researcher)
I found the project interesting, in terms of understanding that not all students approach assessments in
the same way, which highlighted the need for different types of assessment. I also enjoyed working as
part of a research team, learning to examine information from multiple perspectives (Bev Westwood –
student researcher)
In addition we believe this project has also developed student-researchers’ and
participants’ ‘pedagogical intelligence’ (O’Donovan et al 2008) that allowed them:
… to construct meaning for themselves fosters negotiated meanings and thereby more effective
understanding. Through socialisation and practice, tacit knowledge is built up which the student is able
to internalise and gives shape to their understanding of standards and expectations. As a learner’s
knowledge grows, he or she can more readily participate in the joint enterprise of the community of
assessment practice, further developing understanding through absorption into that community through
involvement in both formal and informal processes (O’Donovan et al 2008: 215).
References
Ahmad, F (2001) Modern Traditions? British MuslimWomen and Academic
Achievement. Gender & Education, 13(2), 137 - 152.
18
Anderson, P. (2001) Betwixt and between: classifying identities in higher education.
in P. Anderson and J. Williams (eds), Identity and Difference in Higher
Education: Outsiders Within. Aldershot, Ashgate.
Apple, M. (2000) Offical knowledge, 2nd edition. London, Routledge
Ben- Shakar, G., & Sinai, Y. (1991). Gender differences in multiple- choice tests: the
role of differential guessing tendencies. Journal of Educational Measurement, 28, 2335.
Birenbaum, M., & Feldman, R. A. (1998). Relationships between learning patterns
and attitudes towards two assessment formats. Educational Research, 40 (1), 90-97.
Bloxham, S. (2007) A system that is wide of the mark. Times higher education
Supplement, 26th October. Available on line at www.thes.co.uk [accessed 8th April
2009]
Bowl, M. (2005) Valuing diversity in the social science curriculum’, LATISS –
Learning and Teaching in the Social Sciences 2 (2): 121–136,
Broecke, S. and Nicholls, T (2007) Ethnicity and degree attainment. Research report
RW92 Department for Education and Skills (DFES) Available on line at:
www.dfes.gov.uk/research/
Connor, H., Tyers, C., Modood, T. and Hillage, J. (2004), Why the Difference? A
Closer Look at Higher Education Minority Ethnic Students and Graduates, DfES
Research Report RR552. www.dfes.gov.uk/rsgateway/DB/RRP/u013556/
index.shtml [Accessed 26 February 2009]
Christie, H., Tett, L., Cree, V., Hounsell, J. and McCune, V. (2008)''A real
rollercoaster of confidence and emotions': learning to be a university student',Studies
in Higher Education,33:5,567 — 581
Davis, P., Pampaka, M., Williams, J., Hutcheson, G., Hernandez, P., Kleanthous, I,
Black, L.,Nicholson, S & Wake, G. (2007?) Aspirations, subject choice and drop out:
decision-making amongst AS Level mathematics students. Opening doors to
mathematically-demanding programmes in further and higher education (FHE): Working
Paper Series. Manchester University. [Accessed on 17th April 2009 at
http://www.lta.education.manchester.ac.uk/TLRP
Drew, S. (2001). Perceptions of what helps learn and develop in education. Teaching
in Higher Education, 6 (3), 309-331.
Dyke, R. (1998) Monitoring the progress and achievement of ethnic minority
students: a new methodology in Modood, T and Acland, T (Eds.) Race and higher
education. London, Policy Studies Institute pp 115-133.
Grace, S. and Gravestock, P. (2009) Inclusion and Diversity. Meeting the needs of all
students. Oxon, Routledge.
19
Higher Education Academy/ Equality Challenge Unit (2008) Ethnicity, Gender and
Degree Attainment Project, Final Report January 2008, HEA/ECU Research Report.
www.heacademy.ac.uk/assets/York/documents/ourwork/research/Ethnicity_Gender_
Attainment_report_Jan08.pdf [Accessed 4th March 2009]
Jacobs, S., Owen, J., Sergeant, P. and Schostak, J. (2007), Ethnicity and Gender
in Degree Attainment: an Extensive Survey of Views and Activities in English
HEIs, HEA/ECU Research Report. www.heacademy.ac.uk/projects/detail/
Ethnicity_Degree_Attainment_project [Accessed 26 February 2009]
Kember , D. (2001) Beliefs about knowledge and the process of teaching and learning
as a factor in adjusting to study in higher education, Studies in Higher Education, 26
(2): 205 - 221
Leathwood, C. (2005) Assessment policy and practice I higher education: purpose ,
standards and equity. Assessment and Evaluation in Higher Education, 30 (3) 307324.
Leathwood, C. and O’Connell, P. (2003) It’s a struggle: the constructions of the ‘new
student’ in higher education. Journal of Education Policy 18 (6): 597-615.
Leslie, D., Abbot, A. and Blackaby, D. (2002) Why are ethnic minority applicants less
likely to be accepted into higher education? Higher Education Quarterly, 56 (1) 6591.
Lillis, T (2001) Students writing. Access, regulation, desire. London, Routledge.
Lizzio, A., Wilson, K. and Hadaway, V. (2007) University students’ perceptions of a
fair learning environment: a social justice perspective. Assessment and Evaluation in
Higher Education 32 (2): 195-213..
Magadi, M., Beckhelling, J., Phung, V-H., Chzhen, Y., France, A. and Harvey, J.
(2007) Understanding the experiences of ethnic minority (and other) students at
Loughborough University, Preliminary Report, Centre for Research in Social Policy,
CRSPNO571, Loughborough: University of Loughborough
www.crsp.ac.uk/publications/experiences_of_ethnic_minority_students.htm [accessed
26th February 2009]
Martin, M. (1997) emotional and cognitive effect of exam proximity in female and
male students. Oxford Review of Education 23 (4): 479-486.
McCarthy, D. and Hurst, A. (2001) A briefing on assessing disabled students.
Assessment Series no 8, November, Learning and Teaching Support Network, (LTSN)
Generic Centre.
O'Donovan, B., Price, M. and Rust, C. (2008) 'Developing student understanding of
assessment standards: a nested hierarchy of approaches', Teaching in Higher
Education, 13 (2): 205 - 217
20
Ramsden, P. (2003) Learning to teach in higher education 2nd Ed. London,
RoutledgeFalmer
Richardson, J, T. E. (2008) The attainment of ethnic minority students in UK higher
education, Studies in Higher Education, 33(1), pp. 33-48
Sambell, K., McDowell, L., & Brown, S. (1997). 'But is it fair?': an exploratory study
of student perceptions of the consequential validity of assessment. Studies in
Educational Evaluation, 23 (4), 349-371.
Scales, J. & Whitehead, J. M. (2006) The Undergraduate experience among three
ethnic minority groups: summary report submitted to the joint committee on academic
performance, Cambridge University Reporter, Cambridge: University of Cambridge
www.admin.cam.ac.uk/reporter/2005-2006/weekly/6031/5.html [accessed 26th
February 2009]
Silverman, D. (2001) Interpreting qualitative data methods for analysing talk, text
and interaction. 2nd ed., London: SAGE Publications
Struyven, K Dochy, F and Janssens, S. (2002) Students' perceptions about assessment
in higher education: a review. Paper presented at the Joint Northumbria/ Earli SIG
Assessment and Evaluation Conference: Learning communities and assessment
cultures, University of Northumbria at Newcastle, August 28 - 30 2002.
Surridge, P (2007) The National Student Survey 2006 Report to HEFCE. University of
Bristol
Värlander, Sara (2008) The role of students' emotions in formal feedback situation,
Teaching in Higher Education, 13(2): 145 – 156
21
Appendix 1
Table 1 Survey Demographics
GENDER
Female
Male
Total
No. of participants
72
19
91
% of participants
79%
21%
100%
AGE
18-20
21-25
26-30
31-35
36-40
40+
Total
No. of participants
62
12
3
5
2
4
88
% of participants
70%
14%
3%
6%
2%
5%
100%
ETHNICITY
White British
No. of participants
47
% of participants
53%
White European
3
3%
White Other
1
1%
Mixed Other
3
3%
Asian or Asian British Indian
Asian or Asian British Pakistani
Asian or Asian British Other
Black or Black British Caribbean
Black or Black British African
Chinese
11
13%
11
13%
1
1%
4
5%
6
7%
1
1%
Total
88
100%
22
Parental Occupation
Professional
No. of participants
22
% of participants
25
Managerial & Technical
Occupations
Skilled non-manual
occupations
Skilled manual
occupations
Partly skilled occupations
12
14
8
9
16
18
7
8
Unskilled occupations
5
6
Not in employment/
unemployed
Total
17
20
87
100%
Table 2 Interview Demographics
Ethnicity & Gender
Number of participants
Total
White British Female
5
3+
1+
2
1+
8
1
1
1
1
1+
1
1+
1
2+
1
1+
3
1+
1
White British Male
White European Female
Asian or Asian British
Indian Female
Asian or Asian British
Indian Male
Asian or Asian British
Pakistani Female
Asian or Asian British
other Female
Asian or Asian British
other Male
Black or Black British
African Female
Black or Black British
African Male
Black or Black British
Afro Caribbean Female
Total
3
1
1
21
* + refers to a mature student (over 21 on date of entry to university).
23
Appendix 2
Initial Survey – Equitable assessment project
We are conducting a short survey about first year students’ experiences of
assessment. Can I have 5 minutes of your time? Don’t worry, it is anonymous.
Gender: (You shouldn’t need to ask this!)
Male
Female
What degree are you currently studying?
1. What lecture have you been in/ going to (module)?
2. Are you aware of the type of assessment for the module so far?
Can you tell me more about the assessment for this module? (Probe: what
methods? What choices do you have? How is it marked/ what are the assessment
criteria? How is the assessment weighted? When are the deadlines?)
(Repeat participant’s response to check they are happy with it)
3. How do feel about this assessment so far?
(Repeat participant’s response to check they are happy with it)
4. How do you feel about assessment generally now you’re at uni? (prompt: when
you see the assessment in the module guide – how do you feel? What is your gut
reaction?)
(repeat participant’s response to check they are happy with it)
24
5. In all of your educational experience so far what’s your best and worst
experience of assessment ever?
(Repeat participant’s response to check they are happy with it)
Thanks very much for that. Can I just ask you:
6. How old are you? …………….
18 - 20
21 - 25
26 - 30
31 - 35
36 - 40
41+
7. How would you describe your ethnic origin?
White
British
Irish
European
Other White background
Mixed
White & Black
Caribbean
White & Black African
White & Asian
Other mixed background
Asian or Asian British
Indian
Pakistani
Bangladeshi
Other Asian backgrounds
Black or Black British
Caribbean
African
Other Black background
Chinese
Other ethnic group (please
state)
8. What do your parents do for a living?..
Jot down the occupation in the space but tick which category the main breadwinner’s
occupation fits as appropriate.
25
......................
I. Professional
II. Managerial & Technical occupations
IIIN. Skilled non- manual occupation
IIIM. Skilled manual occupation
IV. Partly skilled occupation
V. Unskilled occupation
Self-Employed
Not in employment/ unemployed
9. Has anyone else in your family been to uni? (if yes, note if this is sibling or
parent(s)
Thanks very much! One last thing, we are giving memory sticks to students who
agree to be interviewed further about assessment. Would you be interested?
Okay I’ll need to contact you again then, so I will need some details. But don’t
worry we will keep all of this info confidential and anonymous.
Contact details: (Student number?)
Name:
Email:
Phone no:
Best days/ times for being interviewed?
I’ll be in touch very soon to arrange a time and place for us to meet. You’re
been a great help. Thanks
26
Appendix 3
Equitable Assessment Project -First Year Student Semi Structured Interview
We are interested in understanding how 1st year students feel about assessment. We are
particularly interested in hearing about your positive and negative experiences of assessment
so far since being at the university perhaps compared to your experiences of assessment at
school or college.
Demographic data:
Gender
Approx age
Ethnic origin
Parent occupation
Immediate family experience of HE
Entry qualification for this course
Mode of study (PT/FT)
No hours paid or unpaid employment
Medical condition/disability affecting study at university
May need to come back to any of the above during course of interview.
Module assessment
1. So maybe you could start by telling me about the module (name of module) and
about the assessment. (description of the assessed pieces of work from their
perspective, probe for anything they may have missed e.g. essay on what? How long?
Group work or individual? Etc etc.
2. Tell me about how you prepare for an assessment like this. Talk me through what
you do.
3. What sort of things can you put into your assignment? For example, do you include/
use things from your own experience (e.g. work, home, life) or your knowledge of
related subjects? Who / what do you turn to for help?
4. What do you like or dislike about the assessment? (Ask about other forms of
assessment that they like or dislike and probe why), (probe in to looking at how their
feelings and emotions influence their assignment.
5. If you could choose or design your own assessment for the module what would it look
like? What form would it take? What things would it assess? Why would this be
better for you?
6. How would your preferred method of assessment allow you to really show what you
can do and what you know? Explore
7. What grade do you think you’ll get for the work you are doing/ have done so far?
Why do you think that? (May be based on previous grades from school, maybe
because tutor has given clear marking criteria – probe but don’t lead)
27
Teachers
Thinking about the teachers/lecturers for a moment…
8. How has your teacher/lecturer prepared you for this assessment (in class work on the
assignment, revision sessions, past exam papers, talking about the marking criteria,
opportunity for formative feedback, opportunity to have a say in what and how they
are assessed)
9. What do you do if you aren’t sure about anything to do with the assignment?
10. What advice and support do you expect from your teachers/lecturers when you are
not sure about anything to do with the assessment? How do you get this sort of help?
11. Talk me though what happens when you get your work back. If you haven’t had a
piece of work back for this module, tell me about one that you’ve had work back for
recently. What did you do with the work? How did it feel? What does/did the teacher
do/say?
Differences
12. Thinking about the other students in your class, how do you think your expected
grade will compare with theirs? Why do you think that?
13. Are there any ways in which you feel that an assignment (or other form of
assessment) you’ve been given has been easier or harder for some students than
others?
14. Why do you think that? What do you think makes it more likely that you will do
better/worse than others?
15. Have you had any assessed work back? If yes –
16. How did your grade compare with what you expected? In what ways did it vary if at all?
Why do you think there was a difference between what you expected and what you
got? Has this happened before in other modules or at school or college? If yes
explore further – what were the circumstances (answers may include e.g. same type
of assessment, didn’t like the subject, teacher, etc etc. explore these further)
17. How did your grade for this module compare with other students’ grades? What is
different about your work and other students’ work? If nothing, what other
explanations could there be for the discrepancy?
18. Was the grade you were given and the feedback you received a fair reflection of your
work? If not, in what ways do you believe it to be unfair? Explore
28
29