THINKING ABOUT RESEARCH
AND RESEARCH QUALITY IN
YOUR ACADEMIC WORK.
Presentation for the annual Oxford-Cambridge
Exchange
Pam Sammons and Linda Bakkum
WHAT IS RESEARCH?

Research is a disciplined attempt to address questions or solve problems
through the collection and analysis of primary data for the purpose of
description, explanation, generalization and prediction (Anderson 1998, p 6)

The nature of the subject matter determines what kind of research is valid
or relevant (Pring 2000, p 6)

Reasoning: deductive (Aristotle) formal steps of logic

inductive (Bacon) empirical evidence for verification

Inductive-deductive moving from observations to hypotheses then back to
implications (backwards & forwards).

Subjective belief must be checked against objective reality, research is selfcorrecting.
WHAT IS DISTINCTIVE ABOUT EDUCATIONAL
RESEARCH?
•
The distancing of theory from practice is associated with public and policy
scepticism about value of educational research
•
Need for clarity in defining key terms identified from your literature review
and as used in your study e.g. ‘good’ ‘effective’ c’ompetent’ teacher, what it
means to be an ‘educated’ person
•
Need to attend to the ‘logic of the discourse’ the rules implicit in the use of
particular words and those to which they are logically related
•
For Dewey “education concerned the development of the distinctively human
capacities of ‘knowing’ ‘understanding, ‘judging’; ‘behaving intelligently’
“(Pring 2000, p 12)
•
What that makes your study distinctive in relation to the field of education?
KEY FEATURES OF EDUCATIONAL RESEARCH
•
The attempt to make sense of the activities, policies and institutions which, through the
organisation of learning, help to transform the capacities of people to live a fuller, more
distinctively human life.
•
The distinctive focus of educational research must be upon the quality of learning and
thereby teaching
•
Much writing sets up a false dichotomy between different research traditions
•
Variety in approaches to educational research is desirable, depending on questions explored
and philosophical position
•
Is it the ‘real’ world that we observe – or one interpreted through my own personal &
subjective scheme of things?
•
What is the connection between language and the world language is used to describe? After
Pring (2000)
•
All links to notion of clarity in writing and argument &
demonstrating critical engagement with substantive, theoretical &
methodological literature
WHAT IS...?


A research design is “an integrated statement of and
justification for the technical decisions involved in
planning a research project” (Blaikie, “Designing
Social Research”, p. 15).
A research project is a temporary organisation that
is created with the purpose of carrying out systematic
and rigorous enquiry to address a particular problem
arising from a gap in knowledge (a theoretical puzzle,
a pragmatic need etc).
FEATURES OF QUALITY IN (EDUCATION)
RESEARCH
Rigour of research process
 Trustworthiness
 Reliability/ validity
 Usefulness – implications for research
methodology, for policy &/practice in education
 Originality
 Contribution to theory?

How can you demonstrate rigour in
these areas in your study?
WHAT IS PHILOSOPHY
RESEARCH?
OF EDUCATIONAL
A “second order activity” which explores



the beliefs about the nature of (social) reality or of a
phenomenon (including self and other – “what exists,
what it looks like, what units make it up and how
these units interact with each other”) - ontology
the beliefs about the nature of educational research
knowledge (and its relationships to other kinds of
knowledge) - epistemology
the beliefs about principles and values (including the
right, the good and the virtuous) in the practice of
educational research - axiology
(see D. Bridges, 2003, p. 15; N.Blaikie, 2000, p. 8)
WHAT IS/ARE YOUR...
...Ontological position?
...beliefs about epistemology?
•Positivist ?
•Constructivism
•Post-positivist?
•Pragmatic?
•Critical Theory
HOW DOES THIS AFFECT YOUR CHOICE OF:
• RESEARCH
• RESEARCH
AIMS & QUESTIONS?
DESIGN & METHODOLOGY?
Quantitative?
Qualitative?
Mixed Methods?
QUANTITATIVE VERSUS QUALITATIVE




Some researchers have argued that it may be
appropriate to think of Qualitative & Quantitative as
being on a continuum Gray and Densten (1998),
Tashakkori & Teddlie 2003
‘Qualitative and quantitative choices viewed as polar
opposites may be viewed as a ‘false dualism’ (Frazer
1995)
Can you clarify & justify your own view and
approach in your study?
How has your view evolved over the course of your
PhD research?
Pragmatism as the Foundation for MM Research
Pragmatism supports the use of both QUAL & QUAN
methods in the same study & rejects the either/or
incompatibility thesis
It considers the research questions to be more
important than either the method or paradigm that
underlies the method – the dictatorship of the RQ
Pragmatism
avoids the use of metaphysical concepts eg ‘truth’
‘reality’
Pragmatism presents a very practical & applied
philosophy
After Tashakkori & Teddlie 2003 p 20-21
RESEARCH QUESTIONS

The ‘big’ research question: one over-arching
question

The sub-questions which help to guide your enquiry
Characteristics of Good Research Questions
Clarity
 Empirical focus
 Accessible evidence
 Manageable
 Awareness of
assumptions
 Awareness of implicit
values

Awareness of political
implications
 Related to previous
research
 Significant
 Ethical
 Practical use (relevant)
 ‘fun’ (interesting to you)

source: Ingrid Lunt.
CLARITY
The question(s) in your study should be
answerable i.e can be illuminated or addressed
by your methodology (you are looking to find the
answer to a genuine question)
 The question should be intelligible to the reader
who may not be an ‘expert’ in your topic
(understandable)
 The questions should offer the prospect of
making an ‘original contribution to knowledge’ in
some way (methodologically theoretically
empirically etc)
 Are the terms clearly defined?
 Are the questions precise?

source: Ingrid Lunt.
EMPIRICAL FOCUS



Require that you generate data to answer question
Lead you to determine methods of enquiry and data
collection
NB it is usually most appropriate for methods to
follow questions; different types of questions will lead
to different approaches to research and methods of
data collection, but this is not always the case
source: Ingrid Lunt.


Reflect on your own Research aims/RQs:
How have they evolved over the course of your study
How far have they driven your choice of design &
methodology and the specific methods you are using?
SIGNIFICANT
Is there a clear rationale for the question?
 So what?
 Does this question matter?
 Why is it of interest and to whom?

source: Ingrid Lunt.
POSSIBLE AIMS & OBJECTIVES
Description: what does it look like (what, when,
where, who)?
 Explanation: why did it happen?
 Prediction: what is to be expected?
 Understanding: how is it grasped in human
experience?
 Interpretation: what does it mean?
 Prescription: how ought it be?
 Change and emancipation: how can it be
transformed for the better?
 Critique and disruption: what are the limitations
and hidden assumptions? How can these assumptions
be challenged/ interrupted?


Etc. (e.g., exploration, demonstration, classification)
AIMS AND CLAIMS
KINDS OF RESEARCH
QUESTION
EXAMPLES OF
RESEARCH
Explanatory
What is the relationship
between?
Survey, experiment
Explanatory
Descriptive
Prescriptive
What happens if . . . ?
Experiment, participatory
research, action research
Descriptive
Explanatory
‘What’ and ‘why’?
Mixed methods research
Explanatory
Descriptive
What happened in the past/
how to make sense of the
past?
Historical research
Understanding
Interpretative
How can we understand a
situation?
Ethnographic and
interpretive/
Case study
Critique
Emancipatory
How to disrupt convention
and empower participants?
Critical approaches
As by Alis Oancea.
SOME INFLUENCES ON SOCIAL RESEARCH
Values
•
These can affect choice of
research topic, formulation of
research questions, choice of
methods, choice of research
design and instruments,
ethics, sample & process of
data collection, interpretation
of data and findings,
conclusions, reporting and
dissemination
•
Need be self-reflective, and to
exhibit reflexivity about the
part played by the
researcher’s own values and
their potential influence on
research process and
outcomes
Practical Considerations
•
Existing knowledge base on
topic, is this a new topic of
interest? (generation or
testing of theory more
appropriate?), resources
available,
availability/interest of
participants
•
All social research is a
coming together of the ideal
and the feasible
As by Alis Oancea.
ROLE OF VALUES & OF RESEARCHER
•
The value determined nature of enquiry in anti- positivist
research such as Critical theory and Constructivism,
Advocacy and activism encouraged, researcher
transformative intellectual or passionate participant
•
What can be known is mediated by interaction between
investigator and subject of investigation
•
For constructivists there are multiple realities, that depend
on the individuals or groups holding constructions,
constructions may change/be altered and thus so can
‘realities’
•
Researcher and subject are interactively linked and
findings are created through hermeneutical and dialectical
techniques and are relative
•
Aims to critique & transform (critical theory) or to
understand & reconstruct, subject to continuous revisions.
HOW VALUES MAY INFLUENCE SOCIAL
RESEARCH
•
•
•
•
•
•
•
Choice of research area
Formulation of research questions
Choice of method
Formulation of research design & data collection
techniques
Implementation of data collection
Interpretation of data
Conclusions drawn
BODIES OF KNOWLEDGE
•
Theories, propositions and explanations
accumulated through enquiry, criticism,
argument and counter argument. What has
survived testing and criticism…public property.
Their credentials depend upon their being open
to public challenge and refutation.
•
Any body of knowledge can only be provisional
and is open to further challenge through
criticism.. The link between knowledge &
certainty is broken.
•
Disciplined, critical and reflective thinking is the
mark of educational research, at odds with
unquestioning ‘common sense’ beliefs.
POINTS TO ESTABLISH IN EXAMINING DIFFERENT
RESEARCH APPROACHES & IN CRITICAL READING OF
RESEARCH
•
Research assumptions - are they explicit?
•
Aims – explanation or understanding
•
The subjective-objective dimension
•
Role and definitions of theory
•
Doing research/reading research
•
Theoretical and empirical domains
•
Values and interpretation
•
•
Use of findings/ audience
Stages in the development of enquiry
ISSUES IN QUALITATIVE RESEARCH
Generalisability



Enriching
understanding and
generating theory
Fuzzy
Generalisations
Falsification

Using extreme
(most/least likely to
fit theory), atypical,
and critical cases
Validity

Often concerns:
honesty, credibility,
richness,
authenticity, depth,
scope, subjectivity,
strength of feeling,
capturing
uniqueness,
idiographic
statements, fidelity
to participants’
accounts
Reliability

Dependability,
consistency,
comprehensiveness,
‘checkability’, empathy,
uniqueness,
explanatory and
descriptive potential,
confirmability,
“neutrality”,
applicability,
transferability
As by Alis Oancea.
STRATEGIES FOR...
Generalisability




Careful, sometimes
strategic selection of
cases
Intense participation
and effort to develop
valid and rich
descriptions
Challenging theories,
conventional wisdom,
and prior assumptions
Letting the case “talk
back” – sensitivity to
diversity, uniqueness,
history and context
Reliability






Good preparation for
fieldwork
Piloting and peer and
participant debriefing
Justification of decisions (e.g.
transcription; recording; types
of questions; extent of
‘mapping’ and ‘summarising’
in case presentation etc.)
Awareness of transcriber
selectivity and other
limitations
Independent audits and audit
trails
Multiple coders
As by Alis Oancea.
STRATEGIES FOR VALIDITY
Prolonged engagement
in the field
 Persistent observation
 Rich and thick
description
 Leaving an audit trail
 Reflexive diaries
 Respondent validation
 Peer debriefing
 Checking for researcher
effects

Making
contrast/comparisons
 Ruling out spurious
relations
 Following up surprises
 Using extreme cases
 Assessing rival
explanations
 Triangulation
 Back translation

As by Alis Oancea.
SOME ISSUES IN QUANTITATIVE RESEARCH
Generalisability


Can findings be
generalised outside
the sample?
Validity

Importance of
sample


Concept of statistical
probability

Measurement
valididy, face validity,
concurrent validity
predictive validity,
construct validity,
convergent validity
Role of confidence
intervals
Reliability

Fundamentally
concerned with the
reliability of
measures.

Stability

Dependability

Replicability

Internal reliability

Inter-observer
consistency
STRATEGIES FOR...
Generalisability



Careful sample
selection.
Random selection
can be useful because
of known properties.
Be cautious with
making inferences.
Validity




Reliability
Appropriate
instrumentation,

Test – Retest

Chronbach Alpha
Appropriate treatment
of statistical data

Multiple coders

Careful sampling
At best strive to
minimize invalidity
and maximize validity

Consider the
consistency of your
observations.
Controllable,
predictable, consistent,
replicable.
(Cohen et al. 2007)
MIXED METHODS APPROACHES
Issues








All the same problems as
with Quant and Qual!
But also:
Design choice
Data synthesis
Can your data inform one
another?
Two separate studies?
Quant and Qual findings dont
match?
Skill and confidence in both
research approaches?
Should be more than the sum
of its parts.
Strategies




Careful design of each (Qual
and Quant) component.
Think about how your data
might be used to inform one
another.
Explore what the combined
set of findings indicate.
If not confident with a
particular method, hit the
books, ask for help!
A Dynamic Conceptual Model for MM research
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
QUAN
MIXED METHOD
QUAL
Sphere of Concepts (abstract operations) Purposes Questions
Deductive Qs
Objective purpose
Value neutral
Politically neutral
............................................
............................................
............................................
.............................................
Inductive Qs
Subjective purpose
Value informed
Transformative
Experiential sphere (concrete observations & operations) Data
Observation
Numerical data
Structured process
Statistical analysis
.............................................. Narrative data
.............................................. Emergent process
............................................... Content analysis
Sphere of Influence (abstract explanations & understandings)
Theories Explanations Inferences
Deductive logic
...............................................
Objective inference ...............................................
Value neutral
...............................................
Politically neutral
...............................................
Inductive logic
Subjective inference
Value involved
Transformative
after Tashakkori & Teddlie 2003
MM Designs characterised by


Multiple positions along each attribute traditionally assumed to
distinguish QUAN & QUAL eg they have both confirmatory and
exploratory research questions
They are near the end of one continuum on one attribute ( eg inductive
questions but near the other end of the continuum on another attribute
eg statistical analysis)
Multiple Method Designs (more than 1 method or more than 1 world view
A. Multi method designs ( more than 1 method but restricted to within
1 world view (eg Quan/Quan or Qual/Qual)
B. Mixed methods designs (use of QUAL & QUAN)
Mixed method research (occurs only in methods stage of a study)
 Mixed model research (can occur in all stages of a study )

after Tashakkori & Teddlie 2003
MM Designs characterised by
•
•
1.
A.
B.
Multiple positions along each attribute traditionally assumed to distinguish
QUAN & QUAL eg they have both confirmatory & exploratory research
questions
They are near the end of one continuum on one attribute ( eg inductive questions
but near the other end of the continuum on another attribute eg statistical
analysis )
Multiple Method Designs (more than 1 method or more than 1 world view)
Multi method designs ( more than 1 method but restricted to within 1 world view
eg Quan/Quan or Qual/Qual)
Mixed methods Designs (use of Quan & Qual methods/data collection/analysis
strategies)
1.
Mixed Method research (occurs in the methods stage of study only)
2.
Mixed Model research (can occur in all stages of a study)
–
Concurrent Mixed Method design one kind of question simultaneously
addressed by collecting & analysing QUAN & QUAL data then one type
inference made from both sources
–
Concurrent mixed Model 2 strands of research with both types of question,
both types of data & both types of analysis then both types of inferences are
pulled together to create meta-inferences at the end
after Tashakkori & Teddlie 2003
Purpose / Question
Purpose /
Question
Data Collection
Data Collection
Data Analysis
Data Analysis
Inference
Inference
Meta - Inference
Concurrent Mixed Model Design (Fig 26.6 p688)
Purpose / Question
Purpose /
Question
Data Collection
Data Collection
Data Analysis
Data Analysis
Inference
Inference
Meta - Inference
Sequential Mixed Model Design (Fig 26.8 p688)
Purpose / Question
Purpose /
Question
Data Collection
Data Collection
Data Analysis
Data Analysis
Inference
Inference
Meta - Inference
Fully Integrated Mixed Model Design (Fig 26.11)
 Your
research will be informed by your
readings.
 Critical
reading of the literature is a
major part of good research!
HOW TO READ RESEARCH ARTICLES CRITICALLY (AN
APPRECIATION OF STRENGTHS &
WEAKNESSES/LIMITATIONS)









Identify research aims/questions?
Identify nature/type of study (scholarly review,
empirical work, new or secondary analysis)
Identify ontological position, epistemological &
methodological assumptions
Is researcher’s value position explicit?
Identify location, date, sample, methods used
Examine use of theory, deductive? Inductive?
Are analysis methods clearly explained ?
Are conclusions appropriately supported by evidence ?
What are the implications for policy/practice?
.
FINAL COMMENTS







No study can be ‘perfect’
Research rigour is about clarity of research process
throughout
Justifying your choices, design, interpretations, conclusions
Persuasion of arguments re original contribution
Awareness of strengths & limitations
How your research fits into existing body of knowledge
Implications for policy practice, future directions for research

Your viva involves an oral ‘defence’ a justification
of the rigour of your research to probe your
understanding and ‘ownership’ of your study

It is helpful to practice thinking, talking about and
presenting your study with special attention to
demonstrating rigour