Dr. Anne Adams
Institute of Educational Technology & eSTEeM

TOP TIPs
1) Review key issues – I will give initial input & frame of reference
2) Develop TIPs together
3) Pioritise these TIPs
• TO GIVE YOU: Points for further reference.
IET

3 FOCUS AREAS
Research Planning / Reflexivity
Conducting Research (methods, stages, Issues)
Analysing Findings (developing a narrative)
IET

Research Cycle
(Action Research)
What is my question /
Why am I doing this ?
REFLECT
PLAN
OBSERVE
ACT
IET

RESEARCH PLANNING /
REFLEXIVITY
IET

Research Reflection
• What you bring to research:
– Own values, experiences, interests, beliefs etc.
• What others bring to research:
– Their values, experiences, interests, beliefs etc.
• Why are you doing this research:
• How will the research impact on you & others
– How the research changes us, as people and as researchers.
– How the research will change others.
– Expectations of how the research will change others.
IET

WHAT ARE YOUR RESEARCH OBJECTIVES
• Improve your practice
• Improve others practice
• Change systems and procedures
• Change mind-sets / approaches
& beliefs
• Help understand something
IET

Would you be convinced to change your work practices by an 8 /10 cats prefer approach?
IET

OR
Would you need a convincing argument
/ story as to why you need to change?
IET

Have you ever changed your vote because of a voting poll –
OR would a convincing argument / story change your mind?
IET

CONDUCTING RESEARCH: methods, stages, issues
IET

VARIATIONS IN RESEARCH
• TRADITIONAL (reductionistic)
• TECHNICAL DEVELOPMENT
• ETHNOGRAPHIC / EXPLORATORY
IET

Traditional Reductionist
• Reduce – Break-down & define variables (concepts / things) to measure e.g. What is improvement
?
• Measure –
How to measure, When, With what tools & analysis.
Objective measures: Logs, critical incidents, experiments.
Subjective measures: interviews, focus groups,
Surveys.
• Sampling
- (e.g. types of students, times of year),
Biases (e.g. tutor teaching style).
IET

Technical Development
Developing virtual worlds & applications etc.
• Development – feed in from students & / or tutors
• Objective Evaluation of system – for usability / accuracy / speed of responses and students perceptions of the system – user trials, experiments
• Subjective Evaluation of the system
- posthoc interviews, focus groups, surveys.
IET

Exploratory Research
Initial
Question
Reflect
Plan
Observe
Act
Further
Question
Reflect
Plan
Observe
Act
IET

Exploratory Research
• Uncover new convincing findings
• Provides context of where to go forward
• Through sampling can highlight specific & generic issues.
IET

Interviews
 Structure (semi-structured/ structured)
 Style (expert / novice)
 Setting (natural, office)
 Recording the data (audio – quotes, written notes distract, phone interviews).
 Biasing – talking (<15%), your opinion (NO)
IET

Focus Groups
 6 or 7 participants
 Moderate to keep focus & obtain all perspectives
IET

Questionnaires (open ended)
 Design (quantitative / qualitative)
 Purpose (obtain background info & recruit)
 Open not closed to allow ‘participants’ accounts of experiences, feelings, observations
 E.G. :
 Give examples of using mobile devices for learning purposes
 Detail how you felt about those experiences
 Provide information about where you have seen others using mobile devices for learning.
IET

Critical Incidents
 Used in conjunction with other methods
 Used as a data collection & analysis approach
 Identify critical moment / factor
1.
Ask respondent to recall
2.
Observe, record interactions identify incidents
3.
Review documentation, logs identify incidents
IET

Observational Diaries
1) Interaction patterns
2) Attitudinal changes (e.g. becoming scared of the computer)
3) Behavioural changes
4) Sequencing (e.g. incremental changes experienced over time)
IET

Institute of Educational Technology & eSTEeM

Observing interactions with prototypes
– Think-aloud: talk though how they are working with something for their job
Institute of Educational Technology & eSTEeM

Participant Video
 Gaining in popularity as a research method
 The OU Participatory Video Group
 Give participants video’s they capture data on what is important with regard to your focus of research.
 Can be as focused or as open-ended as you want.
IET

• Q&A postcards
• Maps and how annotated
• Cameras
• Personal diaries
• Media diaries
• Photo-albums
• Voice activated recordings
• Visitors book
• Scrapbook, ‘post-it’ notes, pens, pencils, crayons
“… a rich and varied set of materials that … let us ground
[our designs / processes / policies] in the detailed
textures of the local cultures.” (Graver at el, 1999)
Institute of Educational Technology & eSTEeM

ANALYSIS: Developing a narrative
IET

QUANT / QUAL Comparison
Quantitative approaches
'Simple' numeric data
Measurement
Explanation
Prediction
Generalisable account
Representative population sample
Hypothesis-testing
Claims objectivity
Closed system
(experimental control)
Qualitative approaches
'Complex' rich data
Meaning
Understanding
Interpretation
Contextual account
Purposive/ representative perspective sample
Exploratory
Accepts subjectivity
Open system
(ecological validity)
IET

Qualitative Analysis Methods
•
Conversational / Discourse Analysis
• Thematic Analysis / Grounded Theory
• Content Analysis / Critical Incident Analysis o counting o imposing established frameworks
“Both qualitative and quantitative approaches share a common concern with theory as the goal of research”
(Henwood & Pidgeon, 1992 p.101)

DEVELOPING a NARRATIVE
Hans Rosling – TED talks etc.
Developing stories around quantitative data http://www.youtube.com/watch?v=jbkSRLYSojo

• Data in whatever form is :-
• Broken down, conceptualised, and put back together in new ways.
• Analysis Stages – 3 levels of coding :
 open,
 axial,
 selective (with process effects)
Institute of Educational Technology & eSTEeM
30

Levels of Research & Analysis
• In-depth throughout
• Iterative at different levels
• Quick and Dirty into in-depth
 1. quick eyeball test all data
 2. identify interesting points of focus
 3. initially map – hypothesise relationships
 4. review sections in-depth
 5. adapt and expand high-level
Institute of Educational Technology & eSTEeM
31

1.
Identify important points in the data!
2.
Ideas on possible relationships
3.
How related: sequences, cause & effect, time, subordinate and superior concepts?
Institute of Educational Technology & eSTEeM
32

ANALYSE DATA NOW!!!!
Institute of Educational Technology & eSTEeM
33

1.
Identify important points in the data!
2.
Ideas on possible relationships
3.
How related: sequences, cause & effect, time, subordinate and superior concepts?
Institute of Educational Technology & eSTEeM
34

1.
Possible Important data points
2.
Ideas on possible relationships
3.
How related: sequences, cause & effect, time, subordinate and superior concepts?
Institute of Educational Technology & eSTEeM
35

nd
Coding!!
Institute of Educational Technology & eSTEeM
36

1.
Concepts are identified.
2.
Concepts are grouped into categories
3.
Properties and dimensions of the category identified
Institute of Educational Technology & eSTEeM
37

Open coding: detailed
• Concepts are:Conceptual labels placed on discrete happenings, events, and other instances of phenomena
• Categories are:where concepts are classified and grouped together under a higher order – a more abstract concept called a category.
• Properties are:characteristics pertaining to a category
• Dimensions are:Location (values) of properties along a continuum
Institute of Educational Technology & eSTEeM
38

Open coding: example
“ When I want to have a personal conversation , I encrypt the message .
I think that makes the email private . Stops people from listening in ”
Institute of Educational Technology & eSTEeM
39

Open coding: analysis
• “ When I want to have a personal conversation
(private interaction), I encrypt the message
(security measure).
I think that makes the email private (Securing privacy). Stops people from listening in (Surveillance).”
• Concepts are:- private interaction, security measures, securing privacy, surveillance
• Categories are:- Interaction, privacy, security
Institute of Educational Technology & eSTEeM
40

nd
Take section of an important point in your data & start to code concepts!! If you want also review categories
Institute of Educational Technology & eSTEeM
41

Category
Class surveillance
Property Dimensio n
Being observed frequency
Dimensional
Range often ........never
scope intensity duration more ........less
high.........low
long .........short
Institute of Educational Technology & eSTEeM
42

• Discuss coding, accuracy, does it convey appropriate meaning, what other words could be used more appropriately. (inte-rater reliability)
• Make notes, adjust codes
Institute of Educational Technology & eSTEeM
43

• Decide continue OR Close
• GT would say continue until saturation point (only repetitive concepts, issues occurring)
• Summarise analysis with high-level story-line combining abstract relationships with detailed findings.
Institute of Educational Technology & eSTEeM
44