Nexus IBDP Psychology – Paper 3: Research methods
Assessment
Paper 3 is assessed in a somewhat different way than Papers 1 and 2. It is a good idea to
understand what you are aiming to achieve before we start exploring the detailed issues relevant
to qualitative research methods.
In paper 3 you will be provided with an example of research, referred to here as the ‘stimulus
material’. You will then be asked 3 questions, each worth a total of 10 marks. You should aim for a
minimum of 6/10 for each question; a total of 18 would usually be the equivalent of a 7 in the final
exam. Realistically, all Nexus students are capable of scoring consistently in the top band of
marks, (8+) allowing you to achieve a total score of 24. This would be fantastic as it takes some
pressure off Paper 2 which is far more difficult to control as it depends entirely on fluent essay
writing skills under timed conditions and is therefore a significant challenge for most students.
The level descriptors below, (taken directly from the Paper 3 rubric) will give you some
understanding of the characteristics of strong answers.
4 to 7 The question is partially answered. Knowledge and understanding is accurate but limited.
Either the command term is not effectively addressed or the response is not sufficiently explicit
in answering the question. The response makes limited use of the stimulus material.
8 to 10 The question is answered in a focused and effective manner and meets the demands of
the command term. The answer is supported by appropriate and accurate knowledge and
understanding of qualitative research methodology. The response demonstrates a critical
understanding of qualitative research methodology applied to the stimulus material.
TOP TIP
Notice the emphasis on ‘use of the stimulus material’ – it is a good idea to use
quotes taken directly from the stimulus material as evidence to support any points
you are trying to develop in your answer. Failure to do this well is a common mistake
which you should seek to avoid as this will make it impossible for you to access the
top band of marks.
TOP TIP
It is essential that your response ‘meets the demands of the command term’ –
make sure you understand clearly what each command term requires you to do in
Paper 3.
TOP TIP
All students should be able to demonstrate ‘appropriate and accurate knowledge .
. . of qualitative research’; this is largely a question of knowing what the technical
terminology means and is effort responsive; if you work hard, you will achieve this
standard. Demonstrating ‘a critical understanding of qualitative research’ is more
challenging; this implies explaining why certain issues are important, (rather than just
knowing that they are) and recognising the limits of that importance. To do this well
you will need to focus on the feedback from your teacher and trying hard to apply it to
your next attempt!
Qualitative Research Methods
KEY CONCEPT
Paper 3 is focused on Qualitative Research Methods, (i.e. not experiments) – it is
important to keep this focus clear in your minds whenever you are working on this
module.
You will already be familiar with much of the relevant content, but qualitative research
often has its’ own ‘vocabulary’ to discuss these issues. Students who work hard to
become confident with the vocabulary will quickly become successful in this module!
What is Qualitative Research?
Qualitative research usually takes place in the ‘real world’ rather than a laboratory, and allows
people to explain their thoughts, ideas and feelings in detail. Data is gathered in a natural setting,
followed by an attempt to interpret the behaviour which has been observed and/or the things
participants have said.
The qualitative research methods we will focus on are observations, interviews and case
studies. These will often involve face-to-face interactions between the researcher and the
participant. Generally the aim of qualitative research is to allow themes, categories or theories to
emerge from the data, rather than to focus narrowly on a quantitative measure of a pre-determined
behaviour.
Core Concepts
Paper 3 often requires students to comment explicitly on some core concepts
relevant to all qualitative research methods. These core concepts are addressed by
the learning outcomes listed below.
Distinguish between qualitative and quantitative data.
Explain strengths and limitations of a qualitative approach to research.
To what extent can findings be generalized, (transferability, dependability) from qualitative
studies?
Discuss ethical considerations in qualitative research.
Discuss sampling techniques appropriate to qualitative research (for example, purposive
sampling, snowball sampling).
Explain effects of participant expectations and researcher bias in qualitative research.
Explain the importance of credibility in qualitative research.
Explain the effect of triangulation on the credibility/trustworthiness of qualitative research.
Explain reflexivity in qualitative research.
A clear understanding of the core concepts is important as it is often possible to apply this
understanding to more specific discussions about a particular research method. For example, it
may be possible to evaluate the use of semi-structured interviews in terms of the extent to which
the findings can be generalised.
Establishing a clear understanding of these issues, why they are important and the limits of
their usefulness or application is our first goal!
TOP TIP
These core concepts can often be used to add substance to answers for other
learning outcomes. For example, if you were asked to “Evaluate the use of case
studies in research” you might refer to the relative credibility or transferability of the
data or mention the use of triangulation of results or the importance of reflexivity.
1. Qualitative Data v Quantitative Data
Learning Outcomes
Distinguish between qualitative and quantitative data.
TOP TIP
The command term ‘distinguish’ requires you to explain the differences, (not
similarities) between qualitative and quantitative research methods. A strong
response will explain the different types of data collected but will also explain the
different purposes of each type of method.
What is different about Qualitative Data?
Qualitative data are gathered through direct interaction with participants—for example, through
one-to-one or group interviews, or by observations in the field. This usually allows participants a
free range of possible behaviour or response – people can say what they like in an interview or do
what they like in an observation. Quantitative data is often measured in a less direct way, where
something is measured in pre-determined way with a limited range of responses. For example, a
measure of blood pressure or a scale where participants can only select a rating of 1-7 for a
predetermined list of items.
In contrast to quantitative data (where something is measured with a number) qualitative data
consists of non-numeric data - for example, from detailed transcripts of an interview or field notes
from an observation. As a result, qualitative data are usually very detailed and it is therefore
usually possible or even necessary for researchers to use one of several different methods, (such
as content analysis) to interpret the data they have collected rather than simply using a measure
of central tendency to compare data. Qualitative data is often described as “rich data”; they are
rich in their description of the behaviour being observed.
Why are qualitative and quantitative methods used?
This is a big question, (with clear links to the relative strengths and limitations of each method) but
answering it allows us to distinguish very clearly between the purpose of qualitative and
quantitative data as well as the nature of the data collected. Sensible psychologists don’t consider
qualitative and quantitative research methods as two competing teams but recognise that each
type of method can be useful depending on what we are hoping to achieve. This means we can
distinguish between the two types of data by explaining why each approach might be used.
Quantitative data allows clear comparisons between groups, (because it is easy to compare
‘scores’) so if you are interested in how groups of people differ, or how the same people differ
under different conditions, a quantitative approach may be appropriate. This might also allow us to
generalize the results to other groups of people under the same condition. This would be useful if
we wanted to know if the EE process was stress inducing – we could measure the stress levels
before and during the process and compare the two.
However, this approach often ignores important details about the subjective experience of the
individuals involved. A qualitative approach would produce more detailed information, perhaps
using multiple different methods to collect data (see triangulation) and often allows the
participants to explain in detail how they feel and what they think is important.
In a nutshell
Quantitative research is useful to find out what happens, qualitative research may
better suited to explain why it is happening.
EXAMPLE: We could use a simple scale (quantitative) to test the directional hypothesis that G12
students have a higher mean score for ‘stress’ than G11 students, but this might be a very limited
view of exactly how G12 students are feeling and may tell us nothing at all about what exactly is
making them feel that way and what could be done to improve the situation. A qualitative approach
might allow G12 students to identify the issues that matter to them as well as the way they feel
and what they would like to change.
Task:
Summarise the key differences between qualitative and quantitative data. You should
be able to summarise at least three but could summarise as many as five!
●
Qualitative data is where highly detailed information such as reasoning behind the decision
e.g. why choose to support Russia in the 2018 World Cup is it because of patriotism or
family ties. While Quantitative data provides numerical answers like time or ratings not
descriptive
● Quantitative data and Qualitative data are enriching each other
● Qualitative data is gained through semi-structured interviews, surveys and observations but
they are not measurable. While, Quantitative data are taken through measurements from
ordinal
data
and
therefore
can
be
measured.
● Qualitative data can been through transcripts or essay style results while Quantitative data
can be seen from ordinal data that would be gathered from the study.
● Qualitative data gives you free range of possible responses or answers and questions while
Quantitative data gives you a limited but completely relevant data in relation to the the
study’s aim.
2. Strengths and Limitations of Qualitative Research
Learning Outcomes
Explain strengths and limitations of a qualitative approach to research.
Evaluate semi-structured, focus group and narrative interviews.
Evaluate the use of case studies in research.
Evaluate participant, non-participant, naturalistic, overt and covert observations.
TOP TIP
The general question, (explain strengths and limitations of a qualitative approach to
research) is never asked, but you are likely to be asked to explain the relative
strengths and limitations of specific methods, (case studies, observations and
interviews). This will be covered later in this workbook when we have a detailed view
of each method!
Key Concept
Quantitative researchers, (i.e. psychologists who do experiments) would argue that
the ‘interpretation’ of qualitative data can be overly subjective. This is therefore a key
factor for good qualitative researchers to consider; many key concepts in qualitative
research are intended to address this concern, (reflexivity, credibility etc).
Key Concept
Qualitative data is often collected in a face-to-face context and can sometimes
address sensitive issues. It is very important that researchers consider the
implications of this in terms of the value of the data collected, (credibility,
transferability etc) and the welfare of the participant, (confidentiality, protection
from harm etc).
3. Generalising Findings of Qualitative Research
Learning Outcomes
To what extent can findings be generalized, (transferability, dependability) from qualitative
studies?
Distinguish between qualitative and quantitative data.
Discuss the extent to which findings can be generalized from a single case study.
Evaluate semi-structured, focus group and narrative interviews.
Evaluate the use of case studies in research.
Evaluate participant, non-participant, naturalistic, overt and covert observations.
Key Concept
Quantitative research is often very concerned with generalisability, usually
considered in terms of external validity, (population, ecological). Qualitative
research is not intended to be generalised.
Key Concept
Qualitative researchers differ from quantitative researchers in that they
presume that ‘generalising’ findings is primarily the responsibility of the person
who wishes to generalise the results rather than the person conducting the
original research. The person who wishes to "transfer" the results to a
different context is then responsible for making the judgment of how sensible it
might be to do so.
Most qualitative researchers will have little concern for the potential generalisation of their findings
as their focus is to understand behaviour in the current context – how do these participants
behave in this situation) rather than making predictions about what other people might do in other
situations. However, there are ways in which generalisation can be discussed.
a) Representational generalisation
Glossary
Representational generalisation The degree to which the results of qualitative
research can be generalized or transferred to other people.
KISS – should we expect similar results with some other, similar group of
people (or person)?
Representational generalisation might be possible if:
● The qualitative researcher has done a thorough job of describing the inclusion
criteria for participants (exactly what type of people were included in the sample) in
very great detail. This allows others to make a reasonable judgement about whether or
not they should expect the results to be generalised to participants on their own
research or clients / patients they are working with.
● Representational generalisation might be more reasonable if the sample is larger.
● The findings of other studies seem to support the same conclusion, (triangulation of
data), especially if a variety of methods was used but findings seem to converge on
the same conclusion.
In a nutshell
If the researcher provides enough information for me to feel confident that
the people I am working with are the same (or very similar) to those
included in the research I might expect the results to be generalised to my
participants. I will be even more confident that this is a sensible thing to do
if the initial findings were consistent for a larger group of people.
b) Transferability (or inferential generalisation)
Glossary
Transferability The degree to which one researcher feels confident that the
results of previous qualitative research can be generalized (or transferred) to
their context or settings.
KISS – should we expect similar results in some other, similar situation?
Transfer might be possible if:
● The qualitative researcher has done a thorough job of describing the research
context, (how, where and when the research was conducted) in very great detail. This
allows others to make a reasonable judgement about whether or not they should
expect the results to be generalised to their own context, (how, where and when their
own research will be conducted).
● The findings of the research are supported by other research in a similar context,
(triangulation of data). This would be even more powerful if the findings of data
collected in a variety of ways all suggest the same conclusions, (methodological
triangulation) and would increase confidence that results might be generalised.
In a nutshell
If I have enough information to feel confident that my context is the same
(or very similar) to the research context I might expect the results to be
transferable to my context. I might be even more confident about doing this
if other research using a variety of methods seem to converge on the
same conclusion.
TOP TIP
You could consider the relative potential for findings to be generalised as a strength
or a limitation of a specific research method. For example, if you are asked to
evaluate the use of a case study involving a single participant you might argue that
the potential for generalising the findings is low and this limits the value of the
research.
Task:
Read through the stimulus material about ‘teenage female Afghan refugees’ on the
next page and then work together to highlight any information which might be
relevant, i.e. factors which may make you more or less confident that it could be
sensible for the results to be generalised to another situation or population.
Task:
Read through the stimulus material about ‘teenage female Afghan refugees’ and then
answer the following question on lined paper:
To what extent can findings from this qualitative study be generalised?
The findings can be generaised to a large extent as the findings has a high degree of
transferability. Transferability refers to the idea of whether the results can be find in
other general situations. As seen in the stimulus, a lot of detail was given regarding
the charcteristics of the participants such as “fourteen teenage Afghan females who
had left Afghanistan within the last five years” as well as how the experiment was
conducted which was a “interview” and followed by “informal dicussion”.
Moreover, the researcher was also “an Afghan refugee” which would have made the
participants more confortable in interview and informal discussion. Additionally, “all
communication was conducted in Afghan” which is more likely to make the
participants to be even more comfortable speaking in their native tonuge which will
therefore allow the answers in the interview to be more in depth giving the
researchers a deeper understanding.
However, some critism the study can have is that the participants had moved out of
the country and therefore, no longer being in the country, the participants might look
at those memories more fondly as they are no longer in danger.
Before you start your answer, work together to highlight any information which might
be relevant, i.e. factors which may make you more or less confident that it could be
sensible for the results to be generalised to another situation or population.
TOP TIP
A strong response will use the line numbers to refer directly to statements in the
stimulus material in order to support your answer. Highlighting key phrases when you
first read through the material is therefore a useful strategy – you will get better at
highlighting the most useful information if you practice this skill!
Knowledge Needed?
Use this space to make a note of anything mentioned above which you don’t
understand.
Things you need to do if you are uncertain about something:
1. Find it – Can you use this workbook to find the answer for yourself?
2. Friends – If you can’t find an answer ask a friend; it will help them
consolidate their understanding by explaining it to you!
3. Feedback Surgery - ask your teacher or book a slot in the feedback
surgery
4. Ethical Considerations in Qualitative Research
Learning Outcomes
Discuss ethical considerations in qualitative research.
Distinguish between qualitative and quantitative data.
Evaluate semi-structured, focus group and narrative interviews.
Evaluate the use of case studies in research.
Evaluate participant, non-participant, naturalistic, overt and covert observations.
TOP TIP
It is very unlikely that you will be asked the general question, discuss ethical
considerations in qualitative research. It is much more likely that you will be asked to
discuss the ethical considerations of the specific method used in the stimulus
material, (case study, observation or interview). This will be covered later in this
workbook when we have a detailed view of each method!
5. Sampling Methods in Qualitative Research
Learning Outcomes
Discuss sampling techniques appropriate to qualitative research (for example, purposive
sampling, snowball sampling).
Discuss ethical considerations in qualitative research.
Distinguish between qualitative and quantitative data.
TOP TIP
Sampling methods can sometimes raise ethical concerns and often affect the extent
to which findings from research can be generalised. The sampling methods used in
qualitative research are also often significantly different to those used in quantitative
research.
Key Concept
Quantitative research is often very concerned with collecting a representative sample
as this increases the extent to which findings can be generalised to the target
population. However, because qualitative research is not usually intended to be
generalised having a representative sample is much less of a concern so
qualitative researcher often employ other sampling methods.
Review: Sampling methods commonly used in quantitative research:
Task:
Summarise how each of the sampling methods below would be conducted and note
one strength and limitation for each.
Random sample
Strength
Limitation
Volunteer sample
Strength
Limitation
Opportunity sample
Strength
Limitation
Stratified Sample
Strength
Limitation
Quota sample
Strength
Limitation
Systematic Sample
Strength
Limitation
Purposive Sampling
Glossary
A purposive sample is a deliberate process to select participants based on their
characteristics (or ‘salient features’) so as to meet the objectives of the
research.
KISS: Choosing participants who meet specific criteria defined by the
researcher.
Task:
Think about the process of purposive sampling. What strengths and limitations can
you add to the list below?
Key Concept
The number of participants used in qualitative studies is often small and may, in
some cases, be limited to a single individual. This is not normally considered to be
a limitation since the value of qualitative research lies in understanding the in-depth
experiences and feelings of individuals rather than generalising findings to others.
Strengths and Limitations of Purposive Sampling
+ Well selected participants will be more likely to be able to provide rich data of value to the
research.
+
+
+
+
- Sampling is not concerned with collecting a representative sample which can make it difficult to
generalise results to other populations.
-
-
-
-
Snowball Sampling
Glossary
Snowball Sample: Similar to a purposive sample initially, but once a relevant
participant is contacted they may be asked to reach out to people they know who
share the salient features (personal characteristics and/or experiences) of
interest to the researchers
KISS: asking your participants to contact anyone else who might meet the
criteria for inclusion in the research and inviting them to take part.
Task:
Think about the process of snowball sampling. What strengths and limitations can
you add to the list on the next page?
Strengths and Limitations of Snowball Sampling
+ This may make it possible to identify participants who meet unusual inclusion criteria necessary
for a specific research aim.
+
+
+
+
- Participants may share the purpose and expectations of the research when they approach their
friends and this may reduce the validity of the research due to social desirability bias and / or
demand characteristics.
-
-
-
-
Task:
Read through the stimulus material about ‘Romance in Western Cultures’ and then
work together to answer the following question in a Google Doc:
Evaluate the sampling technique used in this study.
TOP TIP
You must apply your answer to the stimulus material. To do this well, try to
highlight specific elements of the study which might mitigate strengths and limitations,
i.e. factors which might limit/increase the benefit of a strength or reduce/amplify the
extent of your concern about a limitation of the sampling technique used.
For example, “in this example the risk of experimenter bias due to a purposive
sample is unusually high because . . .{line 11-12}”
TOP TIP
Remember that the command term ‘evaluate’ means you MUST have at least one
well-developed strength and limitation, (balanced) but it is not necessary to have an
equal number of both.
Knowledge Needed?
Use this space to make a note of anything mentioned above which you don’t understand.
5. Participant Expectations and Researcher Bias
Learning Outcomes
Explain effects of participant expectations and researcher bias in qualitative research.
Explain reflexivity in qualitative research.
Participant ‘Expectations’
Key Concept
In this context it is possible to use many of the research methods issues with which
you are already familiar, (demand characteristics etc) as well as some other basic
Psychological concepts such as conformity but it is important to explicitly recognise
the element of ‘expectation’ in each case.
Task:
For each of the participant effects below, briefly state what is being ‘expected’ by the
participants.
● Reactivity – This is the most basic example of a participant expectation, referring to the
simple fact that knowing your behaviour is being studied can cause an individual to behave
in a different way. NB – This is similar to the Hawthorne Effect, but the IB no longer
want you to sue that term!
When reactivity occurs, participants expect . . .
That a certain behavior is being tested and would alter their behavior in that fashion in order
to fit with what they think the experimenters want them to instead of their own natural
behavior making the results inaccurate
● Demand Characteristics – When demand characteristics are present participants may feel
they know what the hypothesis of the research is and consequently change their behaviour,
(most commonly by trying to ‘help’ but sometimes deliberately seeking to deliver an
unexpected outcome). NB – It is not necessary for participants to correctly guess the
hypothesis of the research for this to be a problem!
When demand characteristics are present, participants expect . . .
● Social Desirability bias – People are unwilling to disclose anything about themselves
which they think might be judged negatively. This can result in participants providing
inaccurate information or withholding information or both!
When social desirability bias occurs, participants have an expectation that . . .
● Self-fulfilling prophecy – In some qualitative research methods participants may hear
about the experiences of others, (focus group interviews for example). If they identify
strongly with these other people, they may begin to suffer from similar complaints or
experience similar benefits.
When a self-fulfilling prophecy occurs, participants expect . . .
● Conformity – In some situations people are likely to conform towards a group norm; their
behaviour becomes increasingly similar, even though nobody has requested, suggested or
demanded that this happen. This is most commonly due to normative influence but can also
be due to informational influence.
When conformity occurs, participants expect . . .
Task:
Read through the stimulus material about ‘occupational therapy for terminally ill
patients’ and work together to highlight any aspects of the method which might
increase or decrease the influence of participant expectations.
Task:
Answer the following question on lined paper:
Explain one or more possible effects of participant expectations in this qualitative
study.
TOP TIP
Although it is possible to get all of the marks with a very detailed explanation of one
participant expectation this will be very difficult to do in practice and it may therefore
be sensible to explain at least two if you have the opportunity to do so.
Researcher Bias
Glossary
Researcher bias is the possibility that the researcher’s own beliefs could influence
the research. This could include the way the method is designed and implemented as
well as the way in which data is analysed and interpreted, (confirmation bias)
Key Concept
This is a broader concept than experimenter bias which refers more specifically to
actions taken by the experimenter during the procedure which might influence the
outcome.
The important skill here is to recognise features of the research method which might increase,
decrease or check the influence of researcher bias.
Task:
For each of the following strategies, explain how the technique would either increase,
decrease or check for researcher bias.
● Employ Research assistants and train them to collect the data.
This would increase / decrease / check for researcher bias because . . .
It would decrease researcher bias because the assistants may be trained on how to collect
data from the participants.
● Have more than one researcher collect and/or analyse the data and compare their
findings and conclusions.
This would increase / decrease / check for researcher bias because . . .
This would check for researcher bias because any significant differences could indicate that
a bias was present. This would allow the researchers to check if the results are genuine.
● Ask the participants whether they agree with the findings, (credibility).
This would increase / decrease / check for researcher bias because . . .
This would check for researcher bias because they can hear the participant’s response and
point of view to make sure they interpret it correctly.
Can also increase researching bias because participants can lie to make themselves look good.
● Make sure the researcher reflects on how their own biases may have influenced the
study as part of the discussion of their findings, (reflexivity).
This would increase / decrease / check for researcher bias because . . .
It would increase bias because you have the belief that you are not wrong
It would decrease bias because you have the belief that you have done something wrong
Continued on the next page!
● Compare the findings to those from similar research conducted by other
researchers, (confirmability).
This would increase / decrease / check for researcher bias because . . .
It would check for researcher bias as you may feel that the other researchers may have
bias results and you may feel that yours may be bias in a similar way
● Selecting a purposive sample from amongst the researcher’s own contacts.
This would increase / decrease / check for researcher bias because . . .
Would increase because they can chose participants consciousnessly or unconsciousnessly choosing
the group of participants that can favor their theory
● Offering to pay the researcher’s students to act as research assistants.
This would increase / decrease / check for researcher bias because . . .
Would increase as the student may alter the results to favor the teacher’s theory in order to gain
favoritism
● Focussing on a topic of intense personal interest for research purposes.
This would increase / decrease / check for researcher bias because . . .
Increase because they may have a strong passion for or against something and want to be
proven right
Task:
Read through the stimulus material below about ‘aggression in hockey players’ and
then work together to highlight any information which might be relevant, i.e. ways in
which the risk of researcher bias has been increased, decreased or checked.
Task:
Answer the following question:
To what extent could researcher bias have affected this qualitative study?
Researchers bias can affect this study as the Coach had to ask for consent before
conducting the experiment
The following study wanted to see if there are any trends in aggressive behaviour during hockey tournaments.
Participants were 79 male varsity hockey players. Coaches were approached and informed of the study. Once
permission was granted the teams were approached and asked for consent. The consent process occurred
roughly 3 weeks before the tournament, in an attempt to minimize any social desirability biases. On game day,
two cameras were placed on opposite sides of the rink. The cameras were placed at centre ice, in order to
provide the most detailed picture of the entire playing surface. Camera operators were instructed to capture as
many of the players as possible at any one time, while always maintaining a relatively clear picture (i.e., viewing
the players’ numbers). Three games were observed.
The competitive tapes were coded by two independent observers using an operationalized check-list, with a high
inter-rater reliability. The “intent to harm” was the defining characteristic of aggressive behaviour. Fourteen
behaviours were labelled as “aggression”, including: cross-checking, fighting, charging, head-butting, kneeing,
spearing, high sticking, and elbowing.
A total of 74 aggressive behaviours were coded from the three games under investigation. Of the 74 behaviours
coded by the two independent observers, only 14 received actual on-ice penalization. With respect to overall
performance, winning, losing, and tied teams committed relatively equal numbers of aggressive acts. Also, there
was no significant difference observed between players occupying different positions. Both offensive and
defensive players committed relatively the same number of aggressive acts.
Gee, C. J., & Sullivan, P. J. (2006). Using a direct observation approach to study aggressive behavior in ice hockey: Some preliminary
findings. Athletic Insight, 8.
Task:
Read through the stimulus material below about ‘the experiences of Congolese
refugees’ and then work together to highlight any information which might be
relevant, i.e. ways in which the risk of researcher bias has been increased,
decreased or checked.
Task:
Answer the following question:
To what extent could researcher bias have affected this qualitative study?
Africa is the site of many refugee camps, and non-governmental agencies are often responsible for providing
programs and services to improve refugees’ quality of life and health in temporary encampments. Pavlish (2007)
carried out a study of Congolese refugees living in displaced persons’ camps in Rwanda. The research
examined meaningful life experiences as narrated by women and men Congolese refugees residing in a refugee
camp in Rwanda.
Two Congolese community health workers recruited potential research participants. Recruiters were asked to
select potential participants from various categories, such as frequent visitors to the health clinic and
marginalized groups in the camp.
During the first interview, the researcher asked participants to describe memories and anecdotes about
significant events and people in participants’ past and present lives, as well as stories about their ordinary days.
The participants were allowed to tell their stories and anecdotes without interruption, unless clarification was
needed to assist with understanding.
The second interview was conducted within 3 days of the first interview. During the second interview, the
researcher first reviewed the topics participants described during their initial interview. The researcher then
asked the participant for additional informational anecdotes about those topics.
Written notes taken during the interviews were immediately transcribed into written research text. At the
completion of the data collection, a systematic inductive content analysis was conducted. Research text was
separated by gender.
The researcher found that women’s experiences revealed themes of leaving the good life behind, worrying about
their daughters, feeling ambivalent about marriage and lacking hope. Men’s experiences revealed themes of
leaving the good life behind, having no peace in the heart and fearing the future.
Pavlish, C, (2007). Narrative inquiry into life experiences of refugee women and men. International Nursing Review, 54, (1), 28-34.
6. Credibility and Triangulation
Learning Outcomes
Explain the importance of credibility in qualitative research.
Explain the effect of triangulation on the credibility/trustworthiness of qualitative research.
Explain effects of participant expectations and researcher bias in qualitative research.
Evaluate semi-structured, focus group and narrative interviews.
Evaluate the use of case studies in research.
Evaluate participant, non-participant, naturalistic, overt and covert observations.
Glossary
Credibility (or ‘trustworthiness’) involves checking that the participant agrees
that the findings and conclusions of qualitative research are credible’, i.e. they
think they are correct.
KISS: asking your participants if your findings and conclusions seem to
accurately reflect their experience, emotions, behaviour or thoughts.
Glossary
Triangulation occurs when we consider a combination of methods to study the
same phenomenon. This could include comparing results from different
observers in the same study, comparing findings from different methods applied
in the same study or comparing findings from the same method but collected at
different times, in different situation or with different participants. The
purpose of triangulation is often to identify convergence – findings common to
different researchers, methods or studies.
KISS: a variety of strategies to confirm your findings are accurate.
TOP TIP
The Importance of Credibility
It is very unlikely that you will be given the very general learning outcome, ‘explain
the importance of credibility in qualitative research’ but it remains an important
concept as it can allow you to evaluate specific examples of research as well as
being one of the key purposes of triangulation.
Task:
Carefully read through the information about credibility and triangulation and then
work together to complete the ‘knowledge needed’ task – note down any concepts or
ideas you are unsure of or would like some more explanation of.
Credibility is important because:
● The purpose of qualitative research is to accurately describe or understand the
thoughts, feelings or behaviour of each specific individual in rich detail. It is
therefore important to check the accuracy of the data and conclusions; the
participants are the only ones who can legitimately judge the credibility of the
results; is it a fair reflection of their feelings, thoughts, behaviours etc?.
● Establishing credibility can also be important to safeguard the ethical standard of
protecting participants from psychological harm. Participants could become distressed
if they feel data which misrepresents their thoughts, feelings and behaviour is used to
support conclusions with which they don’t agree and/or to justify policies to which they are
opposed.
● Credibility can also be important as it may increase the extent to which we feel confident
about the transferability of the findings. If my participants and context closely match those
described in the research and I feel positive that the data was credible, I am more likely to
feel confident that the findings can be transferred to my own context.
Triangulation
Triangulation is a strategy applied to cross check the findings and conclusions of a qualitative
research study. There are three main ways in which we could triangulate findings:
● Data triangulation – data is collected using the same method and by the same researcher,
but in a different context, (i.e. different times and/or in different situations and/or with
different people).
● Method triangulation – data is collected by the same researcher in the same context,
(time, situation, people) but using two or more different methods. For example, collecting
data in an interview, then through an overt observation and finally through a covert
observation of the same people and comparing the findings and conclusions.
● Researcher triangulation – data is collected using the same method in the same context
but involving different researchers. For example, three researchers use the same
behavioural checklist to conduct the same covert observation of the same people at the
same time and then compare their results. Researcher triangulation could also include the
use of a devil’s advocate strategy – another researcher is introduced after the findings
and conclusions have been established with the explicit intention of challenging them.
Key Concept
Data triangulation is not the same as transferability (although it may increase the
transferability). Data triangulation involves the same researcher and the same
method whereas transferability is about generalising findings to a different method
conducted by different researchers.
Triangulation is important because:
● All three methods of triangulation can improve the credibility of the findings and
conclusions. If triangulation has been conducted and the findings and conclusions
converge, (i.e. they are the same or very similar) we are more confident that the findings
and conclusions will be credible, (the participant will agree that they accurately reflect the
truth).
● All three methods of triangulation can improve the transferability of the findings and
conclusions. Where triangulation has been applied and findings and conclusions converge,
this may make me more confident that I can transfer the findings to my own research. More
specifically:
- if researcher triangulation has been applied, I am confident that the results can be
transferred even though I am a different researcher, (with different biases).
- if data triangulation has been applied, I am confident that the results can be transferred
even though I collecting data in a slightly different context.
- if method triangulation has been applied, I am confident that the results can be transferred
even though I am using a different method.
● Researcher triangulation is a good way to check for researcher bias. This should help to
increase the credibility of the findings and conclusions.
● A devil’s advocate strategy can help to increase the confirmability of findings – the
extent to which other researchers are able to confirm the results with their own research,
(this is very closely linked to transferability).
Knowledge Needed?
Use this space to make a note of anything mentioned in the task above which you don’t
understand.
Glossary
Confirmability: The degree to which results can be confirmed by others. This is
a broader concept than transferability – some research is not confirmable, (for
example, there is no way it could be confirmed because an individual in a case
study is unique) and so transferability is never really considered.
KISS: is it possible for other researchers to confirm your findings?
(transferability = is it sensible for other researchers to expect your
findings to be generalised to their research?)
Task:
Read through the stimulus material below about ‘expectant fathers’ and then work
together to highlight any information which might be relevant, i.e. elements of
triangulation which may act to increase the credibility of the findings and conclusions.
Task:
Answer the following question on lined paper:
Explain how triangulation could contribute to the credibility/trustworthiness of this
qualitative research study.
TOP TIP
For this question you could explain actions which have been taken, (triangulation)
which have increased the credibility, but you could also explain actions which
could have been taken which would improve the credibility even further.
Remember to refer to specific lines in the material as evidence in each PEECH!
Knowledge Needed?
Use this space to make a note of anything mentioned in the stimulus material above
which you don’t understand.
7. Reflexivity
Learning Outcomes
Explain reflexivity in qualitative research.
Explain effects of participant expectations and researcher bias in qualitative research.
Evaluate semi-structured, focus group and narrative interviews.
Evaluate the use of case studies in research.
Evaluate participant, non-participant, naturalistic, overt and covert observations.
Glossary
Reflexivity is when a researcher reflects on ways in which their own biases may
have influenced how the research was conducted (and therefore the findings)
and how the findings were interpreted.
KISS: asking yourself if your own opinions have shaped how things were
done or the conclusions drawn.
Key Concept
Reflexivity is not a weakness of research – it is a process which acknowledges a
possible bias in the research and which seeks to control this bias in order to increase
the value of the findings in terms of credibility, transferability and confirmability.
There are two types of reflexivity.
● Personal Reflexivity.
This is where the researcher reflects on how his or her own biases or personal experiences
may have influenced how they interpret their findings. This is important because it will help
to reduce researcher bias and therefore increase, transferability, credibility and
confirmability.
Personal Reflexivity can be applied by:
- Thinking about your own values, beliefs and emotions and how these things may
have influenced the research. For example, were some topics avoided in an interview
because the researcher felt uncomfortable discussing them?
- Thinking about how the data collection has affected the researcher. For example,
was the researcher developing an emotional link with one or more participants, or perhaps
becoming emotionally distressed by information disclosed by participants? Did these things
influence the data collection, findings or conclusions?
- Researcher triangulation as this should rule out any unwanted bias if findings converge
regardless of who collected the data.
- Keeping a field diary as a record of the researcher’s thoughts, feelings and decision
making throughout the research process.
- A devil’s advocate approach – sharing a field diary as well as the findings and
conclusions with a ‘critical friend’ could also be applied to against any bias. This strategy is
a more robust application of reflexivity than researcher triangulation because it requires the
external judge to be as critical as possible and challenge the bias of every decision, even if
they actually think you did a good job!
● Epistemological Reflexivity.
This is when the researcher reflects on how they carried out the study and whether their
choice of research method or procedure could have influenced the findings. This could also
improve the credibility of the research, (for example, has an interview been too personal,
causing the participant to withhold information and therefore reducing credibility). For
example, has the choice of a structured interview limited the ability of the research to fully
achieve its aim?
Epistemological Reflexivity can be applied by:
- method triangulation as this would rule out any unwanted bias if findings converge
regardless of method.
- asking other researchers and/or the participants if they think the methods used could
have influenced the outcome.
TOP TIP
For this question you could explain actions which have been taken to apply
reflexivity, but you could also explain actions which could have been taken in
order to apply reflexivity to a greater extent.
TOP TIP
It is usually a good idea to start reflexivity answers with a justification for applying
reflexivity in the stimulus material. For example, if features of the study may have
increased the risk of researcher bias, this would be a good reason to apply reflexivity!
Task:
Read through the stimulus material below about ‘focus groups for terminally ill
patients and then work together to highlight any information which might be relevant,
i.e. ways in which reflexivity was applied and reasons why it should be.
Task:
Answer the following question on lined paper:
Explain the use of reflexivity in this qualitative study.
Reflectivity is when the researchers evaluate their study to determine possible biases
that they could have made during their research and how it could it have influenced it.
Reflexivity is important in this study as it is a qualitative study. In a qualitative study,
data is collected through direct interactions with the participants. Therefore, the data
would be more informative and detailed. From the text, it mentions that the “ The
researcher participated in the group sessions” and that “The researcher recorded
down her own thoughts and feelings in a field diary to note any biases that may have
been present as she interacted with the patients”. These two quotes suggested that
throughout the entire experiment she was the only researcher that interacted and
recorded down the observations made. Therefore, reflexivity is useful in this situation
as having only her observations as the only information from the experiment, this
greatly increase the chance of bias occurring as the researcher have her schema or
beliefs which can affect how she perceives certain behaviours which can greatly vary
from other researchers.
Therefore, the use of reflexivity is important in this study as to deduce how biases
could have the affected the experiment through evaluating her “her own thoughts and
feelings in a field diary” as the text suggests that it could have “note any biases that
may have been present as she interacted with the patients” allowing the results to be
more accurate and reliable.
8. Interviews
Evaluating Different Types of Interview
Learning Outcomes
Evaluate semi-structured, focus group and narrative interviews.
It is essential that you can evaluate three different types of interview; semi-structured, group and
narrative.
Core Concepts
Evaluation of interviews is likely to include reference to one or more core concept
such as ethical issues, transferability, credibility, participant expectations and
researcher bias.
● Semi-structured interview
Glossary
A semi-structured interview involves a predetermined list of questions but with
a blend of closed and open questions as well as opportunities to deviate from the
predetermined list if necessary.
The core features of a semi-structured interview are:
1. The interviewer has prepared some questions before hand and will use these during every
interview.
2. The questions will include a blend of open and closed questions. Open questions prompt
the participant to talk about a pre-determined topic but give them the freedom to say
whatever they like. Open questions will typically ask the participant to describe or explain
their feelings, opinion or an experience in their own words. They might also be asked to
contrast their feelings or opinions about two different situations.
3. The interviewer will listen carefully to what the participant says and sometimes ask ‘extra’
questions to clarify or confirm what the participant meant or to follow up on something
interesting but unexpected that the participant has said.
4. The interview is conducted in an informal environment.
5. The interviewer will seek to establish a rapport with the participant.
Task:
Think about the core features of a semi-structured interview and then work together
to suggest likely strengths and limitations.
TOP TIP
It is possible to identify ‘likely’ strengths and limitations, but strong responses in an
exam will consider the strengths and limitations of the specific example provided –
some limitations may have been carefully controlled for example and this should be
acknowledged in your response!
Strengths
The inclusion of open questions makes it
possible for participants to explain their
personal experiences in rich detail and is
therefore likely to increase the credibility of
findings.
Being put in an informal environment, will
make the person more relaxed and open to
conversation therefore increasing the detail the
participant would give the researcher
The researcher can ask a wider range of
questions in order to clarify whether their
interpretation of the participant’s answer is
correct increasing the accuracy of the results
and reduce possible biases in their results.
Limitations
Open questions may allow some participants
to wander off topic and describe experiences
which are of limited relevance to the research,
making results difficult to analyse.
Hearing other participants answers can result
in the participants altering their answers in
order to make themselves sound better
therefore reducing the credibility of the
findings.
The participant might feel that they will have to
agree with the researcher’s interpretation and
therefore could lie to them by agreeing to it
making the results inaccurate unbeknownst to
the researcher.
● Focus Group Interviews
Glossary
A focus group interview is similar to a semi-structured interview, but involves a
dynamic discussion involving a group of participants who may introduce questions
of their own and/or comment on each other’s responses.
The core features of a group interview are:
1. A small group of participants engage in discussion led by a moderator (rather than an
interviewer). The role of the moderator is to steer the conversation and this is often
achieved with a discussion guide; a list of open and closed questions intended for use
during the discussion.
2. Participants can respond to the contributions of other participants as well as to questions or
observations made by the moderator.
3. The moderator should make sure that all participants have been encouraged to share their
views.
4. Participants can be either concerned, (have a specific interest in the topic) or naïve (have
no specific pre-existing interest in the topic). They could also share specific characteristics
or represent a mixture of characteristics, depending on the purpose of the interview.
5. The interviewer will seek to reassure participants of mutual confidentiality and nonjudgement.
Task:
Think about the core features of a group interview and then work together to suggest
likely strengths and limitations.
Strengths
The role of participants in generating new
questions or ideas may reduce the influence of
researcher bias and therefore increase the
credibility of the findings.
Limitations
Hearing about the experiences of others could
give rise to a self-fulfilling prophecy where
participants begin to expect to have similar
experiences following the interview.
Participants would be more comfortable in a
group interview as it will not feel as awkward
compared to a one on one interview with the
researcher and be more open and give more
detail into their answer
Hearing other participants answers can result
in the participants altering their answers in
order to make themselves sound better
therefore reducing the credibility of the
findings.
If the focus group share similar characteristics
to each other such as they are suffer from post
traumatic syndrome, this would result in them
being more open as they understand what the
other is going through
Participants may feel uncomfortable to share
their personal experience with strangers and
therefore may refuse to take part in the
discussion
● Narrative Interviews
Glossary
A narrative interview provides the participant with a general topic. The topic is
sometimes phrased as a question or a request to describe their feelings,
thoughts, opinions and experiences regarding a specific context.
The core features of a narrative interview are:
1. The interviewer provides one opening prompt and then allows the participant to ‘tell their
story’ in response to that prompt.
2. The interviewer should avoid interrupting once the participant has started to talk, unless the
participant asks for clarification or reassurance on a specific point. Interviewers could
sometimes interrupt to ask for clarification of something the participant has said.
Task:
Think about the core features of a narrative interview and then work together to
suggest likely strengths and limitations.
Strengths
The passive role of the interviewer should
reduce the possibility of researcher bias and
therefore increase the credibility of the
findings.
Through a narrative interview, researchers are
able to get more details from different
perspectives this would increase the variety of
the results that the researcher is able to work
with therefore decreasing the chances of the
researcher’s confirmation bias from occurring
and affecting the experiment
Limitations
Ethical issues such as privacy and
confidentiality can be compromised if the
participant discloses something unexpected
and of serious concern.
The researcher themselves can have a
confirmation bias about the characteristic of
the participants that is being studied on which
could affect the way that they interpret the
entire interview resulting in bias occurring
Task:
Read through the stimulus material about ‘romance in Western Culture’ and then
work together to highlight any information which might be relevant, i.e. strengths and
limitations of this specific interview.
Task:
Answer the following question on lined paper:
Evaluate the use of semi-structured interviews in this study.
A semi-structured interview is when the researcher ask participants a series of
questions in relation to the purpose of the experiment. The question will contain a blend
of open and closed questions using the opening question as a prompt to ask them to
describe their thoughts and feelings and using closed questions to confirm whether their
interpretation is correct. In this experiment that the stimulus talks about, the aim of the
study was “to investigate the extent to which heterosexual women’s perception of
romantic love matched what they had experienced in their own romantic love
relationships”. From the aim, we can deduce that they would need more detail from the
participants such as their experiences and opinions in order to make the results credible
which is something that can be achieved by conducting a semi-structured interview.
Therefore, because they need to gain more detail on things such as the participant’s
interpretation, using a semi-structured interview in this study would be very effective.
Planning and Conducting an Interview
Learning Outcomes
Discuss considerations involved before, during and after an interview (for example, sampling
method, data recording, traditional versus postmodern transcription, debriefing).
Core Concepts
Planning and conducting interviews well is likely to be related to one or more core
concepts such as ethical issues, credibility and researcher bias.
TOP TIP
In an exam you are most likely to be asked about considerations before OR during
OR after an interview. Data analysis should not be considered to be a consideration
after the interview – this is a separate question (see below).
Glossary
Traditional (or verbatim) transcription – writing down, (or otherwise recording)
everything the participant says.
Glossary
Post-modern transcription – in addition to verbatim transcription, also recording
things the participant does which might indicate what they are thinking, feeling etc.
For example, body language, sighing, laughing, facial expressions, periods of silence
etc.
Considerations BEFORE an Interview
● Choose which type of interview to use. Think carefully about the purpose of your
interview, including the topics you would like to discuss and the participants you are likely to
recruit. Consider which type of interview is most likely to meet the aim of your research
without raising any ethical concerns.
● Plan your sample. Consider the most effective way to collect a useful sample, (purposive,
snowball or something else). This will often include carefully selection inclusion criteria to
determine who should take part and who should not.
● Think about ethics. Think carefully about the ethical issues which could be raised by your
interview, (despite choosing the most ethical option) and plan controls to limit any risk to
participants.
● Plan a list of questions, a discussion guide or a prompt (for narrative interviews).
Think carefully about the questions you need to ask, a blend of open and closed questions,
the order in which they are asked, when participants may need time to explain themselves
more clearly and concepts you may need to clarify for your participants.
● Think about collecting data. Think carefully about how you will record what your
participants say. Consider how different methods of recording their thoughts might make
participants feel as well as how accurate your record might be. Some methods of recording
data might reduce your ability to listen carefully and respond appropriately to what your
participant is saying.
● Traditional v post modern transcription – Decide if you will use traditional (or verbatim)
transcription; this means recording everything the participant says. Alternatively, you could
use post-modern transcription; this also records non-verbal indications of how the
participant is feeling such as body language, sighs, laughter, facial expressions etc.
● Do some pre-search. Consider the topic and your participant’s likely responses and then
make sure you are well informed on the topic area. Participants can be reassured by an
interviewer who seems to be an expert.
● Plan the environment. Think carefully about where, when and how the interview will be
conducted to make sure participants feel comfortable and willing to share openly.
TOP TIP
In an exam you can consider things which have been done in the stimulus material
but could also discuss additional things which could/should have been done.
Considerations DURING an Interview
● Be ethical. Confirm that participants are giving fully informed consent. Make sure they
are still happy to continue with an interview on the planned topic, that they are happy with
how data is being recorded and what you intend to do with the results, (including
confidentiality). Remind them of the right to withdraw before you begin, and perhaps
again during the interview if they seem to be uncomfortable, angry or upset. This is an
important aspect of protection from harm.
● Be sensitive. You should pay attention to your participant. If they are displaying behaviour
or giving even more explicit signs that, they are uncomfortable (such as saying ‘I am very
uncomfortable with this discussion’!) you should at the very least acknowledge that fact,
change the direction of the discussion and remind them of the right to withdraw. In more
extreme circumstances you should stop the interview; protection from harm is essential.
● Be attentive. Don’t interrupt; wait until they have finished speaking if you need to ask for
clarification of something they have said. Interrupting would increase researcher bias and
decrease credibility. You should also use body language such as nodding, eye contact
and ‘verbal assent’ (‘I see’ or ‘ok . . . I understand’) to make it clear that you are listening!
This may make it more likely that they will be honest and therefore increases credibility.
● Be non-judgmental. It is important that you do not express opinions, (critical or otherwise)
about what the participant is saying. Doing so can constitute researcher bias at best and
reduce the credibility of your findings. In more extreme cases you may cause the participant
to feel angry, upset or over-confident in a belief which may be unhelpful in their day-to-day
life. This applies even if the participant asks for your opinion – simply steer the conversation
back to their opinions. You could for example ask them why they think your opinion would
matter or whose opinion they value most and why.
● Be flexible. If the participant says something interesting or unexpected, give them time to
explain what they mean. If the participant clearly feels strongly that they want to explain
something which is of marginal relevance to your aim, let them.
● Be grateful. Thank your participants for their time and help.
TOP TIP
In an exam you can consider things which have been done in the stimulus material
but could also discuss additional things which could/should have been done.
Considerations AFTER an Interview
● Be ethical. As part of a debrief you should remind participants of their right to withdraw
and clarify your intentions regarding their data, including the level of confidentiality you can
guarantee.
● Be reflexive. Summarise what you think you have learned from the interview and ask them
if they think it reflects what they said. This helps to establish the credibility of your findings.
You could also ask another researcher to act as a devil’s advocate and critically review the
way you conducted the interview to identify and control any signs of researcher bias. You
should also reflect on your own conduct and consider ways in which your own bias may
have shaped what the participant said.
TOP TIP
Establishing credibility by asking participants if they agree with your summary of the
interview could be regarded as either ‘during’ the interview or ‘after’ the interview.
Include it either type of answer!
Task:
Read through the stimulus material about ‘doctor’s beliefs’ and then work together to
highlight any information which might be relevant, i.e. things done before the
interview.
Task:
Answer the following question on lined paper:
Explain considerations involved before conducting the semi-structured interviews in
this study.
Analysing interview data: using inductive content analysis
Learning Outcomes
Explain how researchers use inductive content analysis (thematic analysis) on interview
transcripts.
Glossary
Inductive content analysis (or thematic analysis) is a process applied to
qualitative data to allow interpretation through the identification of broad
themes which emerge from the data.
KISS: summarising what was said into broad categories, then giving each
category a label.
How do researchers use inductive content analysis?
There are various ‘versions’ of inductive content analysis, (often referred to more simply as just
‘thematic’ analysis for reasons which will become clear!); it is not essential to understand these
more specific systems but if you are interested you could research ‘grounded theory’ and / or
interpretive phenomenological analysis.
Inductive content analysis is best thought of as a step by step process once data collection is
completed:
● Step 1: If you have an audio or video recording of the interview, you will need to transcribe
the interview. This means producing a written record of everything which was said.
● Step 2: Read through the transcript and make a note of any general themes which
emerge. For example, in a transcript about the school canteen you may feel that an
emerging theme is ‘the behaviour of younger students’.
● Step 3: Consider any themes which could be grouped together and identify these higher
order themes with a label. For example, you might decide there is a higher order them
about ‘behaviour’ or ‘younger students. You may also decide that ‘behaviour of younger
students’ is significant enough to be a higher order theme in its own right.
● Step 4: Conduct a systematic analysis of the transcript. This involves noting down every
specific quote which seems to support each theme. This can help you to judge the strength
of feeling beneath each theme; something which is mentioned often and in stronger terms
is perhaps of greater significance in the mind of the participant.
● Step 5: Construct a summary table displaying all of the themes and the specific
statements supporting them. Consider any connections between themes as well the
relative importance of each one. This should allow you to draw conclusions.
● Step 6: Share your analysis with other researchers to confirm your conclusions and identify
any possible confirmation bias, (personal reflexivity). This should help to increase the
credibility of your findings.
TOP TIP
Depending on the command term, (‘describe’, ‘explain’) you may simply need to
describe how this general process should be applied to the specific example.
Alternatively, (‘evaluate’) you could identify elements of the process which were
applied well, partially or which should have been applied but were not.
Task:
Read through the stimulus material about ‘doctor’s beliefs’ and then work together to
highlight any information which might be relevant, i.e. elements of inductive content
analysis which have been completed.
Task:
Answer the following question on lined paper:
Describe how the researchers in this study could use inductive content analysis
(thematic analysis) on the interview transcripts.
Glossary
Back translation – this is technique to check analysis of transcripts which have
been translated from one language to another. The interviewer translates from
the participant’s language in that of the researchers who will analyse the
transcript. This ‘translated’ transcript is then translated back into the original
language by someone else and presented to the participant who can confirm if it
reflects their thoughts, opinions and values.
KISS: a way to check researcher bias does not influence analysis of data
where translation is necessary
TOP TIP
If you were asked to discuss the use of inductive content analysis in a specific
example, you should illustrate how the process could be applied to the stimulus
material through reference to some of the themes which could emerge, linking them
to specific statements. You must discuss ways in which the whole process was (or
should have been) applied.
Task:
You could be asked to evaluate the use of inductive content analysis. Work together
and then use the space below to list at least two strengths and two limitations of this
process.
+
+
+
-
-
-
9. Observations
Learning Outcomes
Evaluate participant, non-participant, naturalistic, overt and covert observations.
Discuss considerations involved in setting up and carrying out an observation (for example,
audience effect, Hawthorne effect, disclosure).
Discuss how researchers analyse data obtained in observational research.
Observations can be conducted in a variety of ways, reflecting the researcher’s choice in terms of
three key characteristics of their observation:
● Covert or Overt?
Glossary
In a covert observation, participants are not aware they are being observed.
This is sometimes known as a discrete observation.
Glossary
In an overt observation, participants are aware that they are being observed.
For example, in a Drama lessons students are asked to improvise a sketch whilst the
teacher observes their performance.
● Naturalistic or lab?
Glossary
In a naturalistic observation participants are observed in their usual
environment.
NB – ‘usual environment’, not necessarily a ‘natural environment’ in the sense of
‘outdoors with nature’.
Glossary
In a lab observation participants have been invited into an artificial environment for
the purpose of the observation.
NB – This need not be a ‘science’ lab, just somewhere participants would not
normally be and which has been set up to facilitate the specific observation.
● Participant or non-participant?
Glossary
In a participant observation the observer is part of the group and/or activity
under observation. For example, a teacher observing students attempting a novel
thinking skills task remains part of the group whilst also observing what happens.
Glossary
In a non-participant observation the observer is apart from the group and does not
take part in the activity being observed.
NB – this does not necessarily mean the observation is covert!
For example, when teachers are being observed by other teachers, they know they
are being observed, but the observer does not take part in the lesson.
Task:
Work together to identify the likely strengths and weaknesses of different types of
observation. You should consider ethical issues as well as credibility,
triangulation, researcher bias, participant expectations and validity.
Type of
Observation
Covert
Overt
Naturalistic
Lab
Participant
Nonparticipant
Strengths
Limitations
Key Concept
Most observations can be defined in more than one way. For example, you might
conduct a naturalistic observation which is overt and in which you were an active
participant. Such a combination of observational techniques can alter the relative
significance of specific strengths and limitations. Make sure you address any
evaluation to the exact situation described in the stimulus material.
Task:
Read through the stimulus material about ‘adults in McDonalds’.
Working on lined paper, complete each of the following tasks:
● Identify exactly what type of observation has been used in this example.
● Explain at least two strengths and at least two possible weaknesses of this
observational method.
Michael Cheang studied a group of older adults at a fast food restaurant in Honolulu. Before he was
able to begin the study, he obtained permission from the manager under the conditions that he 1)
appear to be a paying customer, 2) not conduct any formal interviews or surveys on the premises, and
3) not interfere with the flow of business.
Cheang spent the first two weeks simply observing the behaviour of the regular customers. He
focused on a specific group of about 26 adults, one of the larger groupings that came there frequently
and was diverse with regard to gender and age. Cheang sat at a table and eventually began talking to
one of the group members. The participant introduced him to other people in the group. They were
friendly, but at first viewed him as an outsider. He showed up every Tuesday and Thursday, and
sometimes Friday, and soon became part of the group.
He observed the group in the setting for 2 or 3 days a week for 9 months. His notes were narrative in
quality as he retold the adults' stories. By the 5th month, several themes of behavior and experiences
emerged: sociability, play, and laughter. Findings suggest that: (a) older adults congregate at this fastfood restaurant to be with their buddies “to play”; (b) the group is fun for members and there are lots
of laughter; and (c) group membership in this restaurant provides structure, meaning, and
opportunities for these older adults to engage in personal expression.
In month 6 he shared his observations and the themes with group members, and interviewed them
with regard to their perceptions of the themes that were generated from his field notes.
Cheang, Michael (2002). Older Adults' Frequent Visits to a Fast-Food Restaurant: Nonobligatory Social Interaction and the
Significance of Play in a Third Place. Journal of Aging Studies, 16 (August), 303-21.
Learning Outcomes
Discuss considerations involved in setting up and carrying out an observation (for example,
audience effect, Hawthorne effect, disclosure).
Considerations BEFORE an Observation
● Conduct a Pilot Study. This is an important step as it allows you to generate a realistic
and useful behavioural checklist.
Glossary
Pilot study is a small scale preliminary study designed to assess the feasibility of
a project in terms of ethics, validity and other practical matters such as the
usefulness of specific tools.
KISS: try it out on a small scale first to see if it works.
● Create a behavioural checklist. This is an essential part of most observations. The
checklist should provide very specific definitions of behavioural events; things a participant
could which could be measured in terms of frequency. For example, how many times does
a student under observation raise their hand for attention. A checklist could also include
behavioural events; clearly defined states which a participant could enter for a measurable
duration. For example, the length of time a student waited for a teacher to notice their hand
in the air and respond. A behavioural checklist is intended to reduce researcher bias (and
therefore increase credibility) as it provides an objective method to record data during an
observation.
Glossary
A behavioural checklist is a list of behaviours you expect to observe and
measure during an observation. This can include behavioural events which can be
counted or behavioural states which can be measured in terms of duration.
● Trial the behavioural checklist. It is essential to trial any checklist to make sure the
definitions are specific enough. It can also be the case that behaviours included on the list
seem to be redundant whilst other important events or states are missing.
● Choose a data sampling method. This could be event sampling, (where we record the
frequency with which a series of events occur) time sampling (where we note down what is
happening at fixed time intervals) or point sampling (where we note down what each
individual is doing in succession). It is also important to think about exactly how data will be
recorded; much information can be lost and participant expectations can be increased if
observers are clearly busy writing things down!
Glossary
Emergent coding is an alternative way of recording data from an observation.
The observer takes more general notes, (rather than recording events and
state, sometimes referred to as a priori coding) and then applies an inductive
content analysis to the notes after the observation is completed.
● Check Inter-rater reliability. It is a good idea to use more than one observer to apply
triangulation of researchers. In an observation this means checking that each different
observer is using the checklist in the same way. If they have been watching the same
individual they should have recorded the same events and / or states. This can take a lot of
practise!
● Choose an observational method. It is important to think carefully about ethics, the
participants, the situation and the behaviour of interest to decide which type of observation
is most appropriate, (i.e. covert or overt? lab or naturalistic? Participant or non-participant?)
● Consider who, when and where. Think carefully about the best time, place and people on
which to conduct your observation. This can include consideration of sampling methods as
well as ethics and other logistic issues such as how an observer might remain undetected
in a covert, non-participant observation.
● Consider Ethics. Some observational methods raise important ethical concerns, most
obviously the issue of informed consent in covert observations. It is important to think about
the likely response of the participants when they find out they have been observed.
TOP TIP
In an exam you can consider things which have been done in the stimulus material
but could also discuss additional things which could/should have been done.
Considerations DURING an Observation
● Be consistent. If you have decided a non-participant observation is most appropriate is
important that you do not start to intervene in the activity unless a participant is at risk!
Be objective. Participant observations raise the risk of researcher bias which can then decrease
the credibility of the findings. It is important to be aware of how your actions may be
shaping the behaviour of others. For example, expressing a preference for a specific
behaviour may increase the frequency of that behaviour due to social desirability bias.
● Be efficient. Data sampling, (event, time, point or emergent coding) during an observation
can be difficult. It is important to practise this skill before you start your observations in
earnest.
● Be sensitive. In any kind of overt observation it is essential to remember that participants
have the right to withdraw and to act quickly if it seems clear that a participant is
becoming distressed or even uncomfortable with something during the observation.
● Be ethical. It is not acceptable to discuss what you have seen with others unless this is
something the participant has given consent to. Failure to adhere to this rule diminishes the
participant’s right to privacy and increases the risk of psychological harm.
Analysing Data from Observations
Learning Outcomes
Discuss how researchers analyse data obtained in observational research.
Data from observations is most typically analysed using the same process of inductive content
analysis described earlier.
Researcher bias is a particular concern for observations as what is recorded could be overly
subjective, in addition to the interpretation of that data. Triangulation by researcher, method
and data is very important to make sure the data is credible.
10. Case Studies
Learning Outcomes
Evaluate the use of case studies in research.
Glossary
A case study can be defined as an in-depth investigation of an individual or a
small group of individuals. It is usually longitudinal and idiographic.
Longitudinal research simply means it is conducted over a long time period.
Idiographic research is focussed on an individual or small group with the sole
intention of understanding and/or helping them in some way. There is no
intention to apply or transfer results to others.
Key Concept
In order to be sufficiently in-depth’ case studies are generally longitudinal and apply
triangulation of methods and data. For example, a case study could include an
observation, and several interviews as well as more quantitative measures.
TOP TIP
Using a case study as the stimulus material can allow examiners to ask you
questions about interviews and observations as well. This is a common strategy!
Task:
Consider the following strengths and limitations of case studies. For each statement,
work together to add an explanation; why are these strengths and limitations present
in a case study method.
+ We may be more confident that the findings will be credible to the participant because . . .
+ Case studies often allow research which would not be possible any other way because …
+ Case studies can highlight flaws in nomothetic theories, (ideas expected to apply to everyone)
because . . .
-
It is not possible to replicate a case study because . . .
-
It is difficult to generalise results from a case study because . . .
-
Case studies may be more vulnerable to researcher bias because . . .
Learning Outcome
Explain how a case study could be used to investigate a problem in an organization or group (for
example, a football team, a school, a family).
Considerations BEFORE a case study
● Decide WHO to study. This is sometimes an obvious decision when researchers are
interested in a specific individual who has come to their attention because they are unique
in some way. In other situations researchers may have an interest in a particular behaviour
or the consequences of a specific environment or experience. In these cases most
sampling methods will be purposive.
● Decide HOW to study. This can be a complex decision. It is important to consider the
purpose of the study; what do you want to find out? Which combination of methods is most
likely to meet this purpose? Which methods are most appropriate ethically? Many
individuals in case studies are unusually vulnerable in some way and this can mean that
greater attention than normal is given to ethical issues such as privacy and protection from
harm.
TOP TIP
If you are asked to do this in an exam you should explain specific elements of a case
study planned for the specific example in the stimulus material. This could mean
suggesting semi-structured interviews and an overt non-participant observation as
ways to triangulate data concerning a specific behaviour. Link everything to the
example!
Considerations DURING a case study
● Be objective. Case studies are very vulnerable to researcher bias as the researchers
often develop close bonds with the participants and their families over a long period
of time. This can then decrease the credibility of the findings and therefore reduce the
value to the participants (which should be the main purpose of a case study).
● Be ethical. Case studies are often used to explore unusual and sometimes distressing
circumstances. It is important to remain considerate of the welfare of the participants and
their families. The right to withdraw should always be reinforced and protection from harm
should remain a consideration at all times.
● Be reflexive. Good researchers will reflect regularly on their personal and epistemological
reflexivity. This could be achieved by keeping a field diary of the process and decision
making along the way and then asking other researchers to review this and comment on
any concerns.
Considerations AFTER a case study
● Remain ethical. Case studies can sometimes generate a lot of public interest but it is
essential to maintain the privacy of the participants. This has not always been the case in
the past, but some high profile participants remain anonymous, (Genie for example) whilst
others have chosen to sacrifice their anonymity in the hope of helping others understand
their situation (such as Clive Wearing).
Task:
Read through the stimulus material about ‘helping terminally ill patients’ and then
work together to highlight any information which might be relevant.
On lined paper, answer the following question:
Evaluate the use of the case study in this qualitative study (for example, the data
collection methods used).
Learning Outcome
Discuss the extent to which findings can be generalized from a single case study.
We already know that it can very difficult to generalise results from case studies because it is hard
to be sure that results found in a case study can predict the outcome of research with other
individuals, (and this is rarely if ever the intention of case study). This is because it is very unlikely
that any other combination of an individual and their circumstances will be exactly the same as the
individual and circumstances in the case study.
However, it is possible to transfer the findings of a case study. This remains very difficult to do
however and depends on very detailed description of the participants and circumstances of the
participant(s) in the case study to allow other researchers to decide if it is sensible to transfer the
findings to other people and situations.
TOP TIP
If you are asked to discuss the extent to which findings from a case study could be
generalised, it is important to explain why the specific case study may face problems
in this regard. Don’t just provide a general response. You could however usefully
comment on how specific elements of the case study might be more or less likely to
be transferred, (i.e. an observation or an interview).
Paper 3 Progress Checker!
Assess your self-efficacy using the table below. How confident do you feel that you could complete
each task well? If you are unsure that you could do so, what are you doing about it?
Explain strengths and limitations of a qualitative research
study.
Discuss the extent to which findings can be generalized
(transfer) from qualitative studies?
Discuss ethical considerations of a qualitative research
study.
Discuss sampling techniques appropriate to qualitative
research (for example, purposive sampling, snowball
sampling).
Explain effects of participant expectations and researcher
bias in a qualitative research study.
Explain the importance of credibility in a qualitative research
study.
Explain the effect of triangulation on the
credibility/trustworthiness in a qualitative research study.
Explain reflexivity in a qualitative research study.
Interviews
Evaluate semi-structured, focus group and narrative
interviews.
Discuss considerations involved before, during and after an
interview (for example, sampling method, data recording,
traditional versus postmodern transcription, debriefing).
Explain how researchers use inductive content analysis
(thematic analysis) on interview transcripts.
Observations
Evaluate participant, non-participant, naturalistic, overt and
covert observations.
Discuss considerations involved in setting up and carrying
out an observation (for example, audience effect, Hawthorne
effect, disclosure).
Discuss how researchers analyse data obtained in
observational research.
Case studies
Evaluate the use of case studies in research.
Explain how a case study could be used to investigate a
problem in an organization or group (for example, a football
team, a school, a family).
Discuss the extent to which findings can be generalized
from a single case study.