Dr. Anne Adams
eSTEeM

POINTS TO TAKE AWAY
• STAGES IN RESEARCH (over time)
• DIFFERENT APPROACHES Suit YOUR
Needs & Expertise
• Importance of RESEARCH QUALITY
• OU Research Administration (Data
Protection, SRPP, ETHICS)
• SUPPORT: you’re not alone eSTEeM

STAGES in RESEARCH
Research Reflection
Action Research: Approaches to
Research and Process.
Research Quality and methods eSTEeM

1 st Stage in Research:
RESEARCH REFLECTION eSTEeM

“Reflexivity requires an awareness of the researcher's contribution to the construction of meanings throughout the research process, and an acknowledgment of the impossibility of remaining 'outside of' one's subject matter while conducting research. Reflexivity then, urges us "to explore the ways in which a researcher's involvement with a particular study influences , acts upon and informs such research ." (Nightingale and Cromby,
1999, p. 228).

‘Personal reflexivity’
• Own values, experiences, interests, beliefs, wider aims in life and social identities have shaped the research.
• How the research may have affected and possibly changed us, as people and as researchers.
‘Epistemological reflexivity’
• Research question limiting what can be 'found?'
• Study design & analysis impacted on findings?
• How could this be done differently

Research Reflection
• What you bring to research:
– Own values, experiences, interests, beliefs etc.
• What others bring to research:
– Their values, experiences, interests, beliefs etc.
• Why are you doing this research:
• How will the research impact on you & others
– How the research changes us, as people and as researchers.
– How the research will change others.
– Expectations of how the research will change others.
eSTEeM

WHAT ARE YOUR RESEARCH OBJECTIVES
• Improve your practice
• Improve others practice
• Change systems and procedures
• Change mind-sets / approaches
& beliefs
• Help understand something eSTEeM

Would you be convinced to change your work practices by an 8 /10 cats prefer approach?
eSTEeM

OR
Would you need a convincing argument
/ story as to why you need to change?
eSTEeM

Have you ever changed your vote because of a voting poll –
OR would a convincing argument / story change your mind?
eSTEeM

2 nd Stage in Research:
APPROACHES & PROCESS eSTEeM

Research Cycle
(Action Research)
What is my question?
REFLECT
PLAN
OBSERVE
ACT eSTEeM

Action Research
Specific Issue / Problem (e.g
How does tutor contact time affect TMA scores)
What is my question?
Specific PROBLEM with a technical solution (e.g. computer
Visualisations increasing problem understanding)
Broad Issue (e.g. WHY do some people benefit from learning in collaboration more than than others?) eSTEeM

VARIATIONS IN RESEARCH
• TRADITIONAL
• TECHNICAL DEVELOPMENT
• ETHNOGRAPHIC / EXPLORATORY eSTEeM

Traditional T&L example
Will open diary access improve interactions
PLAN: Who, when, where diary access / measurement of improvement
REFLECT:
Did it make a difference
OBSERVE:
Collect data & analyse findings
ACT:
Implement diary access eSTEeM

Initial PROBLEM / ISSUE
• Students always email / call me wanting immediate responses when its not possible (e.g Away at a conference, at a funeral, asleep).
Then they are frustrated / upset by the delay in response.
eSTEeM

Traditional T&L example….
• MAYBE (hypothesis) – if they could see my diary they’d time their contacts so that I could respond right away and we’d both be happy?
eSTEeM

Reductionistic
• Reduce – Break-down & define variables (concepts / things) to measure e.g. What is improvement?
• Measure – How to measure, When,
With what tools & analysis.
• Sampling - (e.g. types of students, times of year), Biases (e.g. tutor teaching style). eSTEeM

Traditional Research
Question Methods….
• Objective measures: Logs (e.g. first class logs) with critical incidents, experiments.
• Subjective measures: interviews, focus groups, Surveys.
eSTEeM

WHAT OTHER APPROACH COULD
THERE BE TO THIS PROBLEM? eSTEeM

Technical Development
Developing a agent based application as a first stop for students that evaluates questions by key words as a) initial FAQ type problems gives the appropriate reply b) less urgent NON-FAQ gives student a time-frame for response c) Urgent Questions: sends these to a separate urgent email account & texts the lecturer of these emails.
eSTEeM

Technical Development example?
What is my question?
Plan (sys req, dev cycle) Who
REFLECT Develop & evaluate
Collect data / analyse eSTEeM

Technical Development
• Development – feed in from students & / or tutors
• Objective Evaluation of system – for usability / accuracy / speed of responses and students perceptions of the system – user trials, experiments
• Subjective Evaluation of the system - posthoc interviews, focus groups, surveys.
eSTEeM

WHAT OTHER APPROACH COULD
THERE BE TO THIS PROBLEM? eSTEeM

The big picture: Gathering all the data from OTHERS PERSPECTIVE ?
eSTEeM

Action Research &
Exploratory Research
What is my question?
PLAN
REFLECT ACT
OBSERVE eSTEeM

Action Research (2)
Initial
Question
Reflect
Plan
Observe
Act
Further
Question
Reflect
Plan
Observe
Act eSTEeM

FOCUS / contextual (1)?
Face to face at residential schools
Letters via the post
Formative assessments & feedback
Telephone conversations
Email communication between student
& tutor
Moodle forum tutor moderating student & tutor communication eSTEeM

FOCUS / Point of Inquiry ?
At the end of a specific course
What courses
Who (different student types)
Type of email: first class, personal
When – e.g. throughout, after
TMAs student & tutor email communication
F eSTEeM

FOCUS / Point of Inquiry ?
Who contacted &
Why (tutor, students, others)?
What good about student / tutor communications?
How do communications with tutor change over time?
What are the barriers to student
/ tutor communications?
Expected response times
& why?
How do these interactions differ from other forms ?
student & tutor email communication eSTEeM

Exploratory Research
• Uncover very new convincing findings
• Provides context of where to go forward
• Through sampling can highlight specific & generic issues.
eSTEeM

3 rd Stage in Research:
RESEARCH QUALITY and METHODS eSTEeM

Good Quality Research
eSTEeM

• NO RIGHT OR WRONG
BUT
• GOOD QUALITY RESEARCH
• MOST EFFECTIVE ROUTE
FOR YOU & RQ eSTEeM

Good Quality Research
eSTEeM

Experiment Scenario
Studies were done between 1924 and around 1933 at the Hawthorne works of the Western Electric Company in Chicago on the productivity of workers. Various conditions (pay, light levels, rest breaks, etc.) were manipulated but each change resulted on average over time in productivity rising, including eventually a return to the original conditions. This was true of each of the individual workers as well as of the group mean.
• WHAT WAS HAPPENING?
eSTEeM

In-depth Interview Scenario
In-depth Interviews were conducted to identify professional designers perceptions of what wearable computing was and what the role of human computer interaction was in relation to this. Participants where asked for informal consent to the interviews, given detailed information before the interview on what wearable computing and human computer interaction where and later de-briefed on what the information was going to be used for.
eSTEeM

Questionnaire Scenario
• Why do you think online assessment is wrong.
• Do you attend online tutorials on a regular basis?
• How do you rate tutor response times to urgent & non-urgent request?
• In online conferences which description best describes yourself (an enthusiast, occasional participant or lurker) eSTEeM

Personal / Auto-Ethnography
& Reflection
• Observation and analysis are applied to one’s own experiences.
• Challenge to view oneself objectively.
• Then interpret the identified experiences in the light of applicable theory.
• Valuable for novice ethnographers.
• Encourages critical introspection
• Allows technique practice before interacting with their informants eSTEeM

• Q&A postcards
• Maps
• Cameras
• Personal diaries
• Media diaries
• Photo-albums
• Voice activated dictaphones
• Visitors book
• Scrapbook, ‘post-it’ notes, pens, pencils, crayons
“… a rich and varied set of materials that … let us ground
[our designs / processes / policies] in the detailed
textures of the local cultures.” (Graver at el, 1999) eSTEeM

eSTEeM

eSTEeM

Observing interactions with prototypes
– Think-aloud: talk though how they are working with something for their job eSTEeM

Heuristics, Verbal Protocol /
Think Aloud
Use the application for a specified or unspecified set of actions. Think
Aloud while you do this – ‘ohhh I don’t know why it’s asking me to do that, where’s the next section’ eSTEeM

Jenny Lee Labs
• Objective Measures: Observation, eyetracking, biometrics, experiments. eSTEeM

eSTEeM

eSTEeM

OU Research ADMINISTRATION
• 1 st DATA PROTECTION (Legal
Requirement)
• 2 nd SRPP (OU specific governance on research access to students)
• 3 rd Ethics (HREC – all universities have ethics boards)
1 st step can be done NOW
2 nd & 3 rd require more information eSTEeM

OU Admin information required
• Information sheet
• Consent form
• Research tool (rubric of questions, final questionnaire, experimental design)
Summary of research with research questions
Participant selection process, ethical risks & countermeasures,
Awareness of other related research in OU & externally eSTEeM

ANY QUESTIONS eSTEeM