Layering methods to analyse the
relationship between language
use and attainment among OU
undergraduate students
Maria Leedham, Elizabeth J. Erling, Lina Adinolfi
Contact: m.e.leedham@open.ac.uk
Faculty of Education and Language Studies
The Open University
This paper will
• report on a research project at the OU into the
relationship between use of academic language and
attainment among a small cohort of Health and Social
Care undergraduate students.
Layers 1 & 2 – MASUS and contextual research
• explore further research which ‘layers’ methods to
investigate the texts
• Layers 3 & 4 - intuitive identification of lexical chunks,
corpus linguistic extraction of chunks
Context: a Health and Social Care
course (HSC)
• K204: Working with
Children and Families
• “This course is designed
to meet the educational
and training needs of
those who work (or will
work) with children and
their families across social
care, childcare, health,
education, and leisure
settings.”
• Level 2, 60 points,
distance-learning course,
• 6 essays + 1 exam
• 26 students
• 72 HSC texts were
analysed
• 7 students interviewed
Layers 1 & 2:
THE ORIGINAL STUDY
Layer 1: MASUS
Measuring Academic Skills of University Students
Categories
A Use of source material
B Structure and development of
text
C Control of academic writing
style
D Grammatical correctness
E Qualities of presentation
Text analysis:
Two language specialists and
one subject specialist
Rating
4 Excellent/no problems/
accurate/very appropriate
3 Good/minor problems/mainly
accurate/largely appropriate
2 Only fair/some problems/often
inaccurate/often inappropriate
1 Poor/major
problems/inaccurate/inappropriate
1.0-2.4 = low score
(Bonanno and Jones 1997)
Layer 1: MASUS
Findings: Attainment and language
Course
DD100
K204
T175
100.00
Average mark on TMAs
80.00
60.00
40.00
20.00
1.50
2.00
2.50
3.00
Overall score on MASUS
3.50
4.00
Layer 1: MASUS
Findings: MASUS scores for cohort
(n=26) Assign. 1 (n=25)
A
Use of source material
B
Structure and development
C
D
Assign. 3 (n=24) Assign. 5
7
27%
5
20%
2
8%
15
58%
14
56%
15
63%
Academic writing style
8
31%
7
28%
4
17%
Grammatical correctness
4
15%
7
28%
6
25%
The number of students in sample with low language scores (1.0 – 2.4)
Layer 2: Contextual investigation
• Analysis of course materials  insight into important key
concepts and language of the field
• Tutorial visits  a contact point for researcher and
students
• Interviews with students  insights into students’ views
of their own writing
• Collaboration between language and discipline
specialists  ensured that both language and discipline
expertise were represented
Layers 3 & 4:
SEARCHING FOR LEXICAL
CHUNKS
Lexical chunks
• Conventionalised word combinations
• Contribute to ease of language production and reception
• Demonstrate membership of academic community
•
•
•
•
Layer 3: corpus linguistic research on chunks
E.g. work from Douglas Biber, Ken Hyland
Layer 4: intuitive identification of chunks
E.g. work from Alison Wray, Pauline Foster
Layer 3: Corpus linguistic
extraction of lexical chunks
•
•
•
•
•
•
•
28,000 words
15 HSC texts
Assignments 1, 3, 5
Used WordSmith Tools v.5
Extracted 3, 4, 5 word chunks
Compared no. of types of chunks over time
Ongoing… could extend to cover keyness
Layer 3: Corpus linguistic extraction of chunks
Assignment 1 : Why should we listen to
children?
• quality of life
(31, 4/4 texts)
• listening to children (17, 4)
• children and families (10, 4)
• right to be heard (6, 4)
Assignment 3: Child development theories
can have both a positive and a negative
influence on how practitioners respond
to children. Discuss.
• in the reader (27, 3/6 texts)
• negative influence on (7, 4)
• as well as (5, 4)
• I will discuss (5, 3)
Assignment 5: Critically discuss how
the quality of life of children living
away from home can be improved
by those working with children
and their families.
• living away from home (49, 4/5)
• quality of life (37, 5)
• it is important (8, 3)
• if we are to (4, 2)
Layer 3: Corpus linguistic extraction of chunks
Assignment 5
if we are to (4, 2)
= Example of students taking on the role of HSC
worker?
N Concordance
1
2
3
4
5
and we all need love and affection if we are to develop into functional human
need. Foster carers are volunteers and if we are to encourage others to become
by the local authorities a positive one. If we are to give children any quality of life we
These changes need to be made if we are to meet the five key outcomes of
greatly enhance a child’s quality of life if we are willing to do a great deal of hard
Layer 4: Intuitive identification of
lexical chunks
Advantage of using intuition
• Can identify single chunks
• Simulates the tutor’s experience
Difficulties with intuition
• Subjective
• Difficult to justify
• Problem with boundaries
• Partial solution is to adopy Wray
and Namba’s 11 criteria (2003)
“By my judgement…”
Two types of chunks focused
upon:
• Linkers
B Structure and development
of text
• Discipline-specific
C Control of academic writing
style
Layer 4: Intuitive identification of chunks
Case study A: Laura - Assignment 1: Extract
The United Convention of the Rights of the Child implements legislations internationally
when it comes to the delivery of children’s welfare due to all children have the right to
be heard, these are known as participation rights. Government with in countries then
have to implement legislations and policies to enforce these rights for children welfare.
In addition, children commissioners are appointed to overlook these legislations and
that children receive services that they are entitled to. The role of children
commissioners will be, explored further with in the essay.
Key:
Red = ill-formed or inappropriate chunks
Blue =well-formed and appropriate chunks
Layer 4: Intuitive identification of chunks
Case study A: Laura - Assignment 5: Extract
It is important for children to maintain their health when growing up so they can fully
develop to their potential. Children that come into the care system at times their health
as been neglected or is under development for their age. If children whose, health and
development are impaired or ones health and development are at risk children they
become a child in need (Law card S6 EW), it is the responsibility of local authorities to
ensure that appropriate assessments of children’s health are completed (a statutory
medical assessment). Foster carers or residential carers have a legal duty to ensure that
children’s general health is met i.e. GP registration if relocated to placement out of area,
dentist, optical, mental health, in working in conjunction with children’s social workers.
In addition, their are specialist health visitor (s) in which are attached to local authorities
and work in conjunction with Social Workers and carers to ensure that looked after
children medical needs are continually met. It is the Specialist Health Visitor to overlook
all looked after children cases and communicates with agencies where necessary in
regards to children’s health.
Layer 4: Intuitive identification of chunks
Case study B: Sally - Assignment 1: Extract
Adults tend to assume that they know best, when implementing polices and
practises concerning children, who are frequently viewed as vulnerable and
helpless, but are often capable of far more than they are given credit for, and
may have a different view to a good quality of life to that of an adult. In terms of
what constitutes a good quality of life everyone has the right to food, shelter,
health care and in terms of children access to a good education and protection
from abuse and harm. The social, political, cultural and economic aspects of the
society in which people live all have an impact on determining the quality of life
that children experience, quality of life if often determined by material
possessions but in reality it goes a lot deeper. How good life is depends on the
expectations of the individual and the culture in which they live. For children to
have a good quality of life they need to be helped to achieve their full potential,
consulted about things that matter to them and to be involved in plans and
decisions made about themselves and the environment in which they live.
• Layer 4: Intuitive identification of chunks
Case study B: Sally - Assignment 5: Extract
There is no doubt that children living away from home face more difficulties both
emotionally and in terms of achieving their full potential academically than those from
stable non-abusive backgrounds. All those who work with these children need to think
about what has happened to each child before they entered the care system, and how
that has impacted on their behaviour and coping ability. They also need to be aware of
what interventions are available and appropriate. Quality care can be promoted by
listening and responding to children’s needs, views and goals. Every person who works
with or has contact with that child can contribute to their well being.
Key:
Red = ill-formed or inappropriate chunks
Blue =well-formed and appropriate chunks
Layer 4: Intuitive identification of chunks
Findings
• There is a correlation between the grades given to
students and the intuitive assessment of the nature of
chunks they use in their writing (ie the number and
range of discipline-specific chunks used appropriately)
• Low achieving students appear to use fewer disciplinespecific chunks than higher achieving ones.
• There appears to be an improvement in the range and
number of chunks produced accurately by individual
learners over time, whether low achieving or higher
achieving.
Conclusions
• Students need more direct language development
• Investigation into the features of highly valued academic language
can help in developing language support
• MASUS – offers a way of gathering judgements on aspects of
texts from multiple raters, including a discipline-based perspective
• Contextual data – provides explanations for changes in students’
texts and differences in students’ abilities/desires to acquire and
use the ‘highly valued’ features of academic language
• Corpus linguistics – offers a way of quantifying the number and
range of chunks across texts & gives insights into disciplinary
differences
• Intuitive extraction of chunks – provides a language specialist’s
view of which chunks are most marked – and how this
markedness might affect student attainment
References
• BONNANO, H & JONES, J. (1997) Measuring the
Academic Skills of University Students: The MASUS
procedure. University of Sydney.
• ERLING, E. J. (2009) An investigation into the
relationship between the use of academic language and
attainment – with a focus on students from ethnic
minorities. Open University.
• WIKTORSSON, M. (2003) Learning idiomaticity: a
corpus-based study of idiomatic expressions in learners'
written production. Lund, Lund University
• WRAY, A. & NAMBA, K. (2003) Formulaic language in a
Japanese-English bilingual child: a practical approach to
data analysis. Japan Journal for Multilingualism and
Multiculturalism, 9, 24-51.
Acknowledgements
• We would like to thank Student Services at the Open
university, who funded the research project The
relationship between academic language use and
attainment – with a focus on students from ethnic
minorities in 2008-2009.
• We would also like the thank the researchers involved in
the project: Elizabeth Erling, David Hann, John Kearsey,
Christina Healey, Christine Buller, Kerry Bannister, Chris
Lee, Zoe Doye, Harish Mehra; and the project
supervisor, Jim Donohue.
Maria Leedham
Centre for Language and
Communication
Faculty of Education and
Language Studies
The Open University
Walton Hall
Milton Keynes MK7 6AA
m.e.leedham@open.ac.uk
Dr. Elizabeth J. Erling
Open English Language Teaching
Dept. of Languages
Faculty of Education &Language
Studies
The Open University
Walton Hall
Milton Keynes MK7 6AA
e.j.erling@open.ac.uk
Lina Adinolfi
Open English Language Teaching
Dept. of Languages
Faculty of Education & Language
Studies
The Open University
Walton Hall
Milton Keynes MK7 6AA
l.adinolfi@open.ac.uk