PR4101 – Research
and the
Professional
Emma Kirby
Hanham High
School, Bristol
June 2006
PR4101 Emma Kirby
1
Introduction ......................................................................................................................... 3
What is the nature of the research you have chosen? ..................................................... 3
What are the aims and purposes of the research? ........................................................... 3
In what context was the research conducted? ................................................................. 3
Is the research question clear? How valid or worthwhile do you think it is? ................ 4
What criteria could be used to evaluate the research? .................................................... 4
Assumptions........................................................................................................................ 5
What explicit assumptions has the researcher made about his/her epistemological and
ontological paradigm? ..................................................................................................... 5
What assumptions are implicit? ...................................................................................... 5
How do these affect the nature of the research? ............................................................. 6
Methodology ....................................................................................................................... 6
What methodological decisions have been made? Is a rationale offered for these
decisions? ........................................................................................................................ 6
Data Collection ................................................................................................................... 8
How was the data collected? ........................................................................................... 8
What issues are related to this? ....................................................................................... 8
How does the data collection relate to the aims of the research? ................................... 9
Are there any gaps in the data collection? .................................................................... 10
Data analysis ..................................................................................................................... 10
How was the data analysed? ......................................................................................... 10
How was it presented? .................................................................................................. 10
Are there any alternative analyses or explanations of the data? ................................... 10
Other issues ....................................................................................................................... 10
What ethical questions were addressed by the researcher (if any)?.............................. 10
What researcher effects were discussed and were potentially of concern? .................. 11
Conclusions ....................................................................................................................... 12
How would you evaluate the research overall? ............................................................ 12
How would you evaluate the way the research has been written up? ........................... 12
Has your critique and evaluation of your chosen article informed your understanding of
research and / or your own approach to research in the future – discuss what you have
learnt from engaging in this process. ............................................................................ 12
References ......................................................................................................................... 14
PR4101 Emma Kirby
2
Introduction
What is the nature of the research you have chosen?
A small-scale research project which took place in Bristol, England between
April 2002 and May 2003 although it purports to be of national and
international interest. The project focused on the transition of Modern
Foreign Language (MFL) learners from Key Stage 2 (KS2) to Key Stage 3
(KS3) from the perspectives of pupils, teachers, Heads of MFL departments
and head teachers. It was carried out by 3 colleagues at the Graduate School
of Education at the University of Bristol.
What are the aims and purposes of the research?
The aim of the research is to investigate some of the problems that arise
during the transition of pupils studying languages from KS2 to KS3 in 6
schools in Bristol. It also aims to explore and suggest some strategies to
overcome such problems. The purposes or objectives of the research are:
 To identify the issues that learners, teachers and managers in the focus
schools consider important in relation to transition.
 To research the impact the identified issues on children’s enjoyment
and achievement of language learning at KS3.
 To establish the strategies that schools have in place to minimise the
possible negative impacts that the identified issues.
In what context was the research conducted?
On the 25th March 1999, the Department for Education and Employment
launched an initiative to promote and develop the provision and quality of
MFL teaching and learning in primary schools. This was followed by the
publication of the National Languages Strategy on 18th December 2002,
which states that by September 2009 there should be “an entitlement to
language learning for pupils at KS2” (DfEE, 2002). This research was
carried out at a time when primary and secondary schools were beginning to
look at formalising the teaching that was happening in many primary
schools. Many secondary schools, including my own, were beginning to
become more concerned about the issues that surround the transition of
learners to KS3.
PR4101 Emma Kirby
3
Is the research question clear? How valid or worthwhile do you think it
is?
Cohen, Manion and Morrison (2000) explain that internal validity “seeks to
demonstrate that the explanation of a particular event, issue or set of data
which a piece of research provides can actually be sustained by the data”
(page 107).
Cohen, Manion and Morrison (2000) go on to describe external validity as
“the degree to which the results can be generalised to the wider population,
cases or situations”.
Ensuring validity is imperative at every stage of a piece of research if “the
researcher is to be able to have confidence in the elements of the research
plan, data acquisition, data processing, analysis, interpretation and its
ensuing judgement” (Cohen, Manion and Morrison, 2000, page 115)
What criteria could be used to evaluate the research?
As a teacher and a linguist I would use the following criteria to evaluate the
research:
 Is the research relevant to me and my situation?
 Do the recommendations come from the data collected?
 Are the recommendations realistic?
As a researcher with several years of experience:
 Does the research draw on recent data?
 Has the research been validated?
 Have the researchers used an appropriate methodology and
appropriate methods of data collection?
This is difficult as many of the findings of the study for example, enjoyment
of language learning, openness to other cultures and confidence are very
hard to evidence.
Needs moving
PR4101 Emma Kirby
4
Assumptions
What explicit assumptions has the researcher made about his/her
epistemological and ontological paradigm?
Cohen and Manion (1994, page 6) state that ontological assumptions
“concern the very nature or essence of the social phenomena being
investigated”. In this case the researcher’s ontological assumptions are
based on their values and opinions of the problems faced by teachers and
learners as they move from KS2 to KS3 i.e. the problems of continuity and
progression. As explained by Cohen and Manion (1994, page 6)
epistemological assumptions are concerned with the “nature, form,
acquisition and communication” of knowledge. In this case the assumptions
are based on ‘soft’ knowledge i.e. more subjective, spiritual knowledge
based on their personal experiences of being teachers/researchers.
What assumptions are implicit?
The research uses only one primary school yet makes suggestions for the
teaching of MFL at primary schools in general. In my opinion the scale of
the research is too narrow to make the wide-ranging assertions posited by
the authors, particularly as four of the five secondary schools studied were
selective independent schools. Any recommendations and changes in policy
adopted by the government would not necessarily be implemented by the
secondary schools in this case. As a case study the research is credible but
as a basis for national and international claims it is unreliable.
The researchers make a massive generalized claim when they state on page
36 that “one of the major disappointments in our (their) findings was the
virtually total lack of liaison between primary and secondary phases”. If
they are referring to the one primary and five secondary schools involved in
the research then they need to be more explicit as the report reads that they
are referring generally to schools, which is a hugely unsupported claim. In
my opinion, the researchers, who are university lecturers involved in ITE,
had pre-conceived ideas of the situation in schools before they began the
research. They then applied some things they had learnt during their
research and some issues raised by Burstall (1970), to schools in general.
On page 37, the report states that “neither the primary school coordinator nor
the secondary teacher had sufficient influence to be able to coordinate the
PR4101 Emma Kirby
5
curriculum between their two institutions”. The assumption here is that
neither teacher held an influential post within the school for example as part
of the senior management team, but the researchers again make assumptions
without backing up their claim sufficiently and explaining their comments.
Another implicit assumption is that there is a successful formula that can be
applied to the transition of learners between phases. The researchers on
page 37 state that no successful formula had yet been found. Given the fact
that language learning deals with young people and teachers who live and
work in many different contexts it would be narrow-minded to think that one
solution could be found to suit all situations.
On page 37 in the section titled ‘Organisational Issues’ another huge
assumption is made. The report says that “some of the uncertainties about
the curriculum and its focus stem from a need for training” but there is
nothing in the report write-up to support this claim, neither from the research
project itself or from other sources. In my opinion the internal and external
validity of the project is questionable, as the hypothesis does not seem to
have been validated by any other source.
The authors of this report make more generalized, unsubstantiated claims on
page 37 when they state that “such schemes do exist” and that “limitations
on secondary teachers’ availability were obvious”. They do not suggest
where such schemes might be found nor why the limitations on teacher
availability are obvious and to whom.
How do these affect the nature of the research?
These failings draw into question the credibility of the research project. It
becomes a project that makes hugely impacting, generalized claims based on
a very small-scale project.
Methodology
What methodological decisions have been made? Is a rationale offered
for these decisions?
The researchers involved in this project have used decided to use a case
study to explore possible problems with the transition of MFL learners from
KS2 to KS3 however, at no point during the write-up do they offer a
PR4101 Emma Kirby
6
rationale for doing so. Stake (1994) says that case studies can be qualitative
or quantitative; in this case it is qualitative.
The research is what Stake (1994) calls “an instrumental case study” as it
looks at one case, in this case a small number of schools closely linked
geographically, in order to “provide insight into an issue or refinement of a
theory”.
Bassey (1999) states that there are at least three categories of educational
case study:
 Theory-seeking and theory testing case study
 Storytelling and picture-drawing case study
 Evaluative case study
This project could be seen to fit into two of the above categories. It could be
a theory seeking case study, as it seems to look to establish a theory on how
transition from KS2 to KS3 should be managed. However, it contains
elements of an evaluative case study as it evaluates the current management
of transition between the phases. In my opinion, it sets out to be the former
but due to lack of rigor at times it ends up being the second.
Bassey (1999) quoting Stenhouse stays that there are four types of case
study:




Ethnographic
Evaluative
Educational
Action Research
Stenhouse, in Bassey (1999) says that “in evaluative case studies a single
case or collection of cases is studied in depth with the purpose of providing
educational actors or decision makers (administrators, teachers, parents,
pupils etc) with information that will help them to judge the merit and worth
of policies, programmes or institutions”. This backs up my claim that this
piece of research is evaluative which is valid in its own right but tries to be
more than it is in the way it makes generalised claims.
Cohen and Manion (1989) explain that case studies involve the “observation
of the characteristics of an individual unit – a child, a clique, a class, a
PR4101 Emma Kirby
7
school or a community” (page 124). They then continue by saying that after
such observations and investigation of the subject, case studies aim to
establish generalisations that can be applied to a wider audience. If this is to
be believed then it is acceptable that the researchers in this project draw
generalisations form their work. However, my main concerns are that the
sample is extremely small and not particularly representative. Nor did the
recommendations made in the concluding part of the report seem to come
from the body of research described in the report.
There are advantages and disadvantages to using a case study method of
enquiry. As described by Nisbet and Watt (1984), an advantage of a case
study approach is that the results are accessible to a wider readership
providing that the report is well written. In my opinion, this is one of the
failings of the piece of research I have chosen to critique in that at times the
language is poor and ambiguous. They also suggest that case studies are
useful in identifying patterns of behaviour that may not be identifiable by
other methods. I would agree with this point providing that the sample is
wide and representative.
Nisbet and Watt (1984) continue by explaining that the weaknesses of case
studies include that the results are not necessarily easily generalizable,
something which the researchers in this case do frequently. There are
examples of this on page 37 as I have explained above.
Data Collection
How was the data collected?
The data was collected using semi-structured interviews and discussion
groups in a school-based context. These were secondary schools in the UK
What issues are related to this?
Cohen, Manion and Morrison (2000) suggest some steps that can be taken
during the design and data gathering, analysis and reporting stages of a
research project to minimise the risk of compromising the validity. The ones
that could have been used in this case and could have therefore improved the
internal and external validity of this piece of research are:
PR4101 Emma Kirby
8
 Using an appropriate sample e.g. one which is representative, not too
small or too large
 Selecting appropriate foci to answer the research questions
 Reducing the Hawthorn Effect
 Avoiding making inferences and generalisations beyond the capability
of the data to support such statements
(Cohen, Manion and Morrison (2000)
Cohen, Manion and Morrison (2000) define triangulation as the “use of two
or more methods of data collection in the study of some aspect of human
behaviour” (page 112). On page 36 of the report, in the section titled
Methodology, the authors wrongly use the tool that is triangulation. The
article states “the research involved interviews and small focus group
discussions, designed to achieve triangulation by ensuring that each
interview covered the same range of questions, regardless of the age or
status of the interviewee”. Although the researchers used two different
methods for collecting data they used only one method with each group.
One could question as to whether they would have collected the same data if
they had used a second method with each group.
In the same section the researchers seem to have misunderstood the concept
of semi-structured interviews. On page 36, they say that “each interview
covered the same range of questions” but go on to say that “a semistructured approach was adopted for the interviews so as to allow
interviewees the maximum opportunity to express their views”. This is
contradictory and raises the question of internal validity of the report. Either
the interviews asked the same questions to everyone or they were semistructured which allows for diversion of the questions depending on the
responses of the interviewee. It is not possible to be both.
How does the data collection relate to the aims of the research?
It is important here to look at whether the researchers have taken steps when
choosing a method of data collection to select one that will allow then to
meet the aims and objectives of the project. For examples one of the aims of
the research was to measure children’s achievements, yet it is unclear how
semi-structured interviews and discussion groups can achieve that.
PR4101 Emma Kirby
9
Are there any gaps in the data collection?
I am not sure to what extent semi-structured interviews with one cohort
(staff) and focus group discussion with another (students) constitute a
comprehensive data collection and maybe using both methods with both
groups would have been fairer to the different communication styles of
individuals.
Data analysis
How was the data analysed?
There seems to be no explanation of the analysis of the data except that the
researchers mention that all interviews and discussions were taped and later
transcribed. I would have expected to see an explanation of how data was
coded and synthesised into evidence.
How was it presented?
The findings of the research are described under the following headings
‘Transition-links between primary and secondary schools’, ‘Organisational
Issues’ and ‘Effects on Pupils’ motivation and achievement between KS2
and KS3’.
Are there any alternative analyses or explanations of the data?
No, there are none offered.
Other issues
What ethical questions were addressed by the researcher (if any)?
There are many ethical issues raised in the research however none of them
seem to be addressed by the researchers. I have concerns about the following
ethical issues in relation to this piece of research:
 “Each interview covered the same range of questions regardless of
age or status of the interviewee” (Bolster, Balandier-Brown and ReaDickens, 2004, page 36). How can we be sure that the language of
the interview was accessible to the pupils?
PR4101 Emma Kirby
10
 The report does not make clear that permission was sought to record
and transcribe the interviews.
 It is not clear whether the findings were taken back to the
interviewees and checked that they were an accurate account of their
thoughts and opinions.
 This small-scale project leads the reader to make generalisations
about KS2 to KS3 transition issues in English school. However, the
recommendations are based on the experiences at 1 primary and 5
secondary schools. It may have been more representative to take the
experiences of pupils at 5 primary schools with varying amounts of
exposure to MFL at KS2 and see how they progress in 1 secondary
school.
What researcher effects were discussed and were potentially of
concern?
There are a number of potential researcher effects in this case but none were
discussed. I have identified the following possible effects:
 The researchers were unknown to the pupils and staff at the school,
how do we then know that the responses given to the interview
questions were honest and accurate and were not just what the
interviewees thought the researchers wanted to hear?
 The researchers say that it is a small-scale study but do not seem to
recognise the limitations of the study as they make massive
generalisations.
PR4101 Emma Kirby
11
Conclusions
How would you evaluate the research overall?
Positive Aspects of the Research
Negative Aspects of the Research
 Draws attention to the
problems faced by MFL
teachers and learners during
transition from KS2 to KS3
 The recommendations do not
grow from the body of
research
 There are too many
generalisations and
assumptions
 The report is poorly written
and lacks rigour
 The sample is too small and
therefore unrepresentative
How would you evaluate the way the research has been written up?
There are a few examples of the where the poor writing of the report leads to
confusion and therefore a lack of internal validity. For example, on page 36
in the “Findings” section, the researchers claim that there were “visits made
by the senior management team (SMT) of the secondary to the primary
schools”. This misuse of the plural “schools” causes confusion as in the
“Methodology” section it sets out that one primary and five secondary
schools were involved.
Has your critique and evaluation of your chosen article informed your
understanding of research and / or your own approach to research in
the future – discuss what you have learnt from engaging in this process.
The process of critiquing this piece of research has definitely impacted the
way in which I carry out research of my own in the future. Below, I have
summarised what I have learnt:
 I feel that I have a deeper understanding of the importance of selecting
an appropriate methodology and of justifying why I would use a
particular methodology over another.
PR4101 Emma Kirby
12
 It seems to me that the issue of validation, both internal and external,
is extremely important and the extent to which they happen can
seriously affect the credibility of a research project.
 I now understand more clearly that the clarity with which a piece of
research is written up and reported may lead to misinterpretation on
the part of the reader which can be catastrophic in terms of
recommendations that are made and to the reputations of the
researchers.
 I understand the importance of explaining the process used to analyse
the data.
 When selecting subjects to study I will ensure that they are wideranging and representative to ensure that any claims to knowledge are
validated and credible.
PR4101 Emma Kirby
13
References
Bassey, M., (1999), Case Study Research in Educational Settings,
Maidenhead: Open University Press
Cohen, L., and Manion, L., (1989) (3rd Edition), Research Methods in
Education, London: Routledge
Cohen, L., and Manion, L., (1994) (4th Edition), Research Methods in
Education, London: Routledge
Cohen, L., and Manion, L., & Morrison, K., (2000) (5th Edition), Research
Methods in Education, London: RoutledgeFalmer
DfES, (2002), Languages for All: Languages for Life – a strategy for
England, DfES publication
Nisbet, J., & Watt, J., (1984) Case Study, Chapter 5 in Bell, K., et al (eds),
Conducting Snall-Scale Investigations in Educational Management, London:
Harper & Row
Stake, R., (1994), Case Studies, Chapter 14 in Denzin, N., & Lincoln, Y.,
(1994), Handbook of Qualitative Research, London: Sage
PR4101 Emma Kirby
14