T
.
Presentation for the annual Oxford-Cambridge
Exchange
Pam Sammons and Linda Bakkum

W
?

Research is a disciplined attempt to address questions or solve problems through the collection and analysis of primary data for the purpose of description, explanation, generalization and prediction (Anderson 1998, p 6)

The nature of the subject matter determines what kind of research is valid or relevant (Pring 2000, p 6)

Reasoning: deductive (Aristotle) formal steps of logic

inductive (Bacon) empirical evidence for verification

Inductive-deductive moving from observations to hypotheses then back to implications (backwards & forwards).

Subjective belief must be checked against objective reality, research is selfcorrecting.

W HAT IS DISTINCTIVE ABOUT EDUCATIONAL
RESEARCH ?
•
The distancing of theory from practice is associated with public and policy scepticism about value of educational research
•
Need for clarity in defining key terms identified from your literature review and as used in your study e.g. ‘good’ ‘effective’ c’ompetent’ teacher, what it means to be an ‘educated’ person
•
Need to attend to the ‘logic of the discourse’ the rules implicit in the use of particular words and those to which they are logically related
•
For Dewey “education concerned the development of the distinctively human capacities of ‘knowing’ ‘understanding, ‘judging’; ‘behaving intelligently’
“(Pring 2000, p 12)
•
What that makes your study distinctive in relation to the field of education?

K EY F EATURES OF E DUCATIONAL R ESEARCH
•
The attempt to make sense of the activities, policies and institutions which, through the organisation of learning, help to transform the capacities of people to live a fuller, more distinctively human life.
•
The distinctive focus of educational research must be upon the quality of learning and thereby teaching
•
Much writing sets up a false dichotomy between different research traditions
•
Variety in approaches to educational research is desirable, depending on questions explored and philosophical position
•
Is it the ‘real’ world that we observe – or one interpreted through my own personal & subjective scheme of things?
•
What is the connection between language and the world language is used to describe? After
Pring (2000)
•
All links to notion of clarity in writing and argument & demonstrating critical engagement with substantive, theoretical & methodological literature

W HAT IS ...?

A research design is “an integrated statement of and justification for the technical decisions involved in planning a research project” (Blaikie, “Designing
Social Research”, p. 15).

A research project is a temporary organisation that is created with the purpose of carrying out systematic and rigorous enquiry to address a particular problem arising from a gap in knowledge (a theoretical puzzle, a pragmatic need etc).

F
(
)


Rigour of research process
Trustworthiness




Reliability/ validity
Usefulness – implications for research methodology, for policy &/practice in education
Originality
Contribution to theory?

W
?
A “second order activity” which explores


 the beliefs about the nature of (social) reality or of a phenomenon (including self and other – “what exists, what it looks like, what units make it up and how these units interact with each other”) - ontology the beliefs about the nature of educational research knowledge (and its relationships to other kinds of knowledge) - epistemology the beliefs about principles and values (including the right, the good and the virtuous) in the practice of educational research - axiology
(see D. Bridges, 2003, p. 15; N.Blaikie, 2000, p. 8)

W HAT IS / ARE YOUR ...
...Ontological position?
...beliefs about epistemology?
•Positivist ?
•Post-positivist?
•Constructivism
•Pragmatic?
•Critical Theory
H OW DOES THIS AFFECT YOUR C HOICE OF :
• RESEARCH AIMS & QUESTIONS ?
• RESEARCH DESIGN & METHODOLOGY ?
Quantitative?
Qualitative?
Mixed Methods?

Q
Q


Some researchers have argued that it may be appropriate to think of Qualitative & Quantitative as being on a continuum Gray and Densten (1998),
Tashakkori & Teddlie 2003
‘Qualitative and quantitative choices viewed as polar opposites may be viewed as a ‘false dualism’ (Frazer
1995)


Can you clarify & justify your own view and approach in your study?
How has your view evolved over the course of your
PhD research ?

Pragmatism as the Foundation for MM Research
Pragmatism supports the use of both QUAL & QUAN methods in the same study & rejects the either/or incompatibility thesis
It considers the research questions to be more important than either the method or paradigm that underlies the method – the dictatorship of the RQ
Pragmatism avoids the use of metaphysical concepts eg ‘truth’
‘reality’
Pragmatism presents a very practical & applied philosophy
After Tashakkori & Teddlie 2003 p 20-21

R ESEARCH Q UESTIONS

The ‘big’ research question: one over-arching question

The sub-questions which help to guide your enquiry






Clarity
Empirical focus
Accessible evidence
Manageable
Awareness of assumptions
Awareness of implicit values






Awareness of political implications
Related to previous research
Significant
Ethical
Practical use (relevant)
‘fun’ (interesting to you) source: Ingrid Lunt.

C LARITY





The question(s) in your study should be answerable i.e can be illuminated or addressed by your methodology (you are looking to find the answer to a genuine question)
The question should be intelligible to the reader who may not be an ‘expert’ in your topic
(understandable)
The questions should offer the prospect of making an ‘original contribution to knowledge’ in some way (methodologically theoretically empirically etc)
Are the terms clearly defined?
Are the questions precise? source: Ingrid Lunt.

E MPIRICAL FOCUS



Require that you generate data to answer question
Lead you to determine methods of enquiry and data collection
NB it is usually most appropriate for methods to follow questions; different types of questions will lead to different approaches to research and methods of data collection, but this is not always the case source: Ingrid Lunt.


Reflect on your own Research aims/RQs:
How have they evolved over the course of your study
How far have they driven your choice of design & methodology and the specific methods you are using?

S IGNIFICANT

Is there a clear rationale for the question?



So what?
Does this question matter?
Why is it of interest and to whom?
source: Ingrid Lunt.

P
&








Description: what does it look like (what, when, where, who)?
Explanation: why did it happen?
Prediction: what is to be expected?
Understanding: how is it grasped in human experience?
Interpretation: what does it mean?
Prescription: how ought it be?
Change and emancipation: how can it be transformed for the better?
Critique and disruption: what are the limitations and hidden assumptions? How can these assumptions be challenged/ interrupted?

Etc. (e.g., exploration, demonstration, classification)

AIMS AND CLAIMS
Explanatory
Explanatory
Descriptive
Prescriptive
Descriptive
Explanatory
Explanatory
Descriptive
Understanding
Interpretative
Critique
Emancipatory
KINDS OF RESEARCH
QUESTION
What is the relationship between?
What happens if . . . ?
‘What’ and ‘why’ ?
EXAMPLES OF
RESEARCH
Survey, experiment
Experiment, participatory research, action research
Mixed methods research
What happened in the past/ how to make sense of the past ?
How can we understand a situation ?
Historical research
Ethnographic and interpretive/
Case study
How to disrupt convention and empower participants ?
Critical approaches
As by Alis Oancea.

•
•
S OME I NFLUENCES ON S OCIAL R ESEARCH
Values
These can affect choice of research topic, formulation of research questions, choice of methods, choice of research design and instruments, ethics, sample & process of data collection, interpretation of data and findings, conclusions, reporting and dissemination
•
Practical Considerations
Existing knowledge base on topic, is this a new topic of interest? (generation or testing of theory more appropriate?), resources available, availability/interest of participants
Need be self-reflective, and to exhibit reflexivity about the part played by the researcher’s own values and their potential influence on research process and outcomes
•
All social research is a coming together of the ideal and the feasible
As by Alis Oancea .

•
•
•
•
•
R OLE OF V ALUES & OF R ESEARCHER
The value determined nature of enquiry in anti- positivist research such as Critical theory and Constructivism,
Advocacy and activism encouraged, researcher
transformative intellectual or passionate participant
What can be known is mediated by interaction between investigator and subject of investigation
For constructivists there are multiple realities, that depend on the individuals or groups holding constructions, constructions may change/be altered and thus so can
‘realities’
Researcher and subject are interactively linked and findings are created through hermeneutical and dialectical techniques and are relative
Aims to critique & transform (critical theory) or to understand & reconstruct, subject to continuous revisions.

•
•
•
•
•
•
•
H OW V ALUES M AY I NFLUENCE S OCIAL
R ESEARCH
Choice of research area
Formulation of research questions
Choice of method
Formulation of research design & data collection techniques
Implementation of data collection
Interpretation of data
Conclusions drawn

•
•
•
B ODIES OF K NOWLEDGE
Theories, propositions and explanations accumulated through enquiry, criticism, argument and counter argument. What has survived testing and criticism…public property.
Their credentials depend upon their being open to public challenge and refutation.
Any body of knowledge can only be provisional and is open to further challenge through criticism.. The link between knowledge & certainty is broken.
Disciplined, critical and reflective thinking is the mark of educational research, at odds with unquestioning ‘common sense’ beliefs.

P OINTS TO E STABLISH IN E XAMINING D IFFERENT
R ESEARCH A PPROACHES & IN C RITICAL R EADING OF
R ESEARCH
•
•
•
•
•
•
•
•
•
Research assumptions - are they explicit?
Aims – explanation or understanding
The subjective-objective dimension
Role and definitions of theory
Doing research/reading research
Theoretical and empirical domains
Values and interpretation
Use of findings/ audience
Stages in the development of enquiry

I SSUES IN Q UALITATIVE R ESEARCH
Generalisability Validity



Enriching understanding and generating theory
Fuzzy
Generalisations
Falsification

Using extreme
(most/least likely to fit theory), atypical, and critical cases

Often concerns: honesty, credibility, richness, authenticity, depth, scope, subjectivity, strength of feeling, capturing uniqueness, idiographic statements, fidelity to participants’ accounts
Reliability

Dependability, consistency, comprehensiveness,
‘checkability’, empathy, uniqueness, explanatory and descriptive potential, confirmability,
“neutrality”, applicability, transferability
As by Alis Oancea.

S TRATEGIES FOR ...
Generalisability




Careful, sometimes strategic selection of cases
Intense participation and effort to develop valid and rich descriptions
Challenging theories, conventional wisdom, and prior assumptions
Letting the case “talk back” – sensitivity to diversity, uniqueness, history and context
Reliability






Good preparation for fieldwork
Piloting and peer and participant debriefing
Justification of decisions (e.g. transcription; recording; types of questions; extent of
‘mapping’ and ‘summarising’ in case presentation etc.)
Awareness of transcriber selectivity and other limitations
Independent audits and audit trails
Multiple coders
As by Alis Oancea.

S








Prolonged engagement in the field
Persistent observation
Rich and thick description
Leaving an audit trail
Reflexive diaries
Respondent validation
Peer debriefing
Checking for researcher effects







Making contrast/comparisons
Ruling out spurious relations
Following up surprises
Using extreme cases
Assessing rival explanations
Triangulation
Back translation
As by Alis Oancea.

S OME ISSUES IN Q UANTITATIVE R ESEARCH
Generalisability Validity Reliability


Can findings be generalised outside the sample?
Importance of sample

Concept of statistical probability


Measurement valididy, face validity, concurrent validity predictive validity, construct validity, convergent validity
Role of confidence intervals







Fundamentally concerned with the reliability of measures.
Stability
Dependability
Replicability
Internal reliability
Inter-observer consistency

S TRATEGIES FOR ...
Generalisability Validity Reliability



Careful sample selection.
Random selection can be useful because of known properties.
Be cautious with making inferences.




Appropriate instrumentation,


Appropriate treatment of statistical data


Careful sampling
At best strive to minimize invalidity and maximize validity

Test – Retest
Chronbach Alpha
Multiple coders
Consider the consistency of your observations.
Controllable, predictable, consistent, replicable.
(Cohen et al. 2007)

M IXED M ETHODS A PPROACHES
Issues Strategies








All the same problems as with Quant and Qual!
But also:
Design choice
Data synthesis
Can your data inform one another?
Two separate studies?
Quant and Qual findings dont match?
Skill and confidence in both research approaches?
Should be more than the sum of its parts.

Careful design of each (Qual and Quant) component.


Think about how your data might be used to inform one another.
Explore what the combined set of findings indicate.

If not confident with a particular method, hit the books, ask for help!

•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
QUAN MIXED METHOD QUAL
Sphere of Concepts (abstract operations) Purposes Questions
Deductive Qs ............................................ Inductive Qs
Objective purpose ............................................ Subjective purpose
Value neutral ............................................ Value informed
Politically neutral ............................................. Transformative
Experiential sphere (concrete observations & operations) Data
Observation
Numerical data .............................................. Narrative data
Structured process .............................................. Emergent process
Statistical analysis ............................................... Content analysis
Sphere of Influence (abstract explanations & understandings)
Theories Explanations Inferences
Deductive logic ............................................... Inductive logic
Objective inference ............................................... Subjective inference
Value neutral ............................................... Value involved
Politically neutral ............................................... Transformative after Tashakkori & Teddlie 2003



Multiple positions along each attribute traditionally assumed to distinguish QUAN & QUAL eg they have both confirmatory and exploratory research questions
They are near the end of one continuum on one attribute ( eg inductive questions but near the other end of the continuum on another attribute eg statistical analysis)
Multiple Method Designs (more than 1 method or more than 1 world view
A. Multi method designs ( more than 1 method but restricted to within
1 world view (eg Quan/Quan or Qual/Qual)
B. Mixed methods designs (use of QUAL & QUAN)


Mixed method research (occurs only in methods stage of a study)
Mixed model research (can occur in all stages of a study ) after Tashakkori & Teddlie 2003

MM Designs characterised by
•
•
Multiple positions along each attribute traditionally assumed to distinguish
QUAN & QUAL eg they have both confirmatory & exploratory research questions
They are near the end of one continuum on one attribute ( eg inductive questions but near the other end of the continuum on another attribute eg statistical analysis )
1 . Multiple Method Designs (more than 1 method or more than 1 world view)
A.
B.
Multi method designs ( more than 1 method but restricted to within 1 world view eg Quan/Quan or Qual/Qual)
Mixed methods Designs (use of Quan & Qual methods/data collection/analysis strategies)
1.
Mixed Method research (occurs in the methods stage of study only)
2.
–
–
Mixed Model research (can occur in all stages of a study)
Concurrent Mixed Method design one kind of question simultaneously addressed by collecting & analysing QUAN & QUAL data then one type inference made from both sources
Concurrent mixed Model 2 strands of research with both types of question, both types of data & both types of analysis then both types of inferences are pulled together to create meta-inferences at the end after Tashakkori & Teddlie 2003

Purpose / Question
Data Collection
Data Analysis
Purpose /
Question
Data Collection
Data Analysis
Inference
Inference
Meta - Inference
Concurrent Mixed Model Design (Fig 26.6 p688)

Purpose / Question
Data Collection
Data Analysis
Purpose /
Question
Data Collection
Data Analysis
Inference
Inference
Meta - Inference
Sequential Mixed Model Design (Fig 26.8 p688)

Purpose / Question
Purpose /
Question
Data Collection
Data Collection
Data Analysis
Data Analysis
Inference
Inference
Meta - Inference
Fully Integrated Mixed Model Design (Fig 26.11)










H OW TO R EAD R ESEARCH A RTICLES C RITICALLY ( AN
APPRECIATION OF STRENGTHS &
WEAKNESSES / LIMITATIONS )



Identify research aims/questions?
Identify nature/type of study (scholarly review, empirical work, new or secondary analysis)
Identify ontological position, epistemological & methodological assumptions
Is researcher’s value position explicit?
Identify location, date, sample, methods used
Examine use of theory, deductive? Inductive?
Are analysis methods clearly explained ?
Are conclusions appropriately supported by evidence ?
What are the implications for policy/practice?
.

F INAL C OMMENTS







No study can be ‘perfect’
Research rigour is about clarity of research process throughout
Justifying your choices, design, interpretations, conclusions
Persuasion of arguments re original contribution
Awareness of strengths & limitations
How your research fits into existing body of knowledge
Implications for policy practice, future directions for research

Your viva involves an oral ‘defence’ a justification of the rigour of your research to probe your understanding and ‘ownership’ of your study

It is helpful to practice thinking, talking about and presenting your study with special attention to demonstrating rigour