Qualitative Validity and Reliability File

Paper III Lecture notes
Depending on their philosophical perspectives, some qualitative researchers reject the
framework of validity that is commonly accepted in more quantitative research in the
social sciences. They reject the basic realist assumption that their is a reality external to
our perception of it. Consequently, it doesn't make sense to be concerned with the
"truth" or "falsity" of an observation with respect to an external reality (which is a primary
concern of validity). These qualitative researchers argue for different standards for
judging the quality of research.
For instance, Guba and Lincoln proposed four criteria for judging the soundness of
qualitative research and explicitly offered these as an alternative to more traditional
quantitatively-oriented criteria. They felt that their four criteria better reflected the
underlying assumptions involved in much qualitative research. Their proposed criteria
and the "analogous" quantitative criteria are listed in the table.
Traditional Criteria for Judging
Quantitative Research
Alternative Criteria for Judging
Qualitative Research
internal validity
external validity
The credibility criteria involves establishing that the results of qualitative research are
credible or believable from the perspective of the participant in the research. Since from
this perspective, the purpose of qualitative research is to describe or understand the
phenomena of interest from the participant's eyes, the participants are the only ones who
can legitimately judge the credibility of the results.
Transferability refers to the degree to which the results of qualitative research can be
generalized or transferred to other contexts or settings. From a qualitative perspective
transferability is primarily the responsibility of the one doing the generalizing. The
qualitative researcher can enhance transferability by doing a thorough job of describing
the research context and the assumptions that were central to the research. The person
who wishes to "transfer" the results to a different context is then responsible for making
the judgment of how sensible the transfer is.
The traditional quantitative view of reliability is based on the assumption of replicability or
repeatability. Essentially it is concerned with whether we would obtain the same results
if we could observe the same thing twice. But we can't actually measure the same thing
twice -- by definition if we are measuring twice, we are measuring two different things. In
order to estimate reliability, quantitative researchers construct various hypothetical
notions (e.g., true score theory) to try to get around this fact.
The idea of dependability, on the other hand, emphasizes the need for the researcher to
account for the ever-changing context within which research occurs. The research is
responsible for describing the changes that occur in the setting and how these changes
affected the way the research approached the study.
Qualitative research tends to assume that each researcher brings a unique perspective
to the study. Confirmability refers to the degree to which the results could be confirmed
or corroborated by others. There are a number of strategies for enhancing
confirmability. The researcher can document the procedures for checking and
rechecking the data throughout the study. Another researcher can take a "devil's
advocate" role with respect to the results, and this process can be documented. The
researcher can actively search for and describe and negative instances that contradict
prior observations. And, after he study, one can conduct a data audit that examines the
data collection and analysis procedures and makes judgements about the potential for
bias or distortion.
There has been considerable debate among methodologists about the value and
legitimacy of this alternative set of standards for judging qualitative research. On the
one hand, many quantitative researchers see the alternative criteria as just a relabeling
of the very successful quantitative criteria in order to accrue greater legitimacy for
qualitative research. They suggest that a correct reading of the quantitative criteria
would show that they are not limited to quantitative research alone and can be applied
equally well to qualitative data. They argue that the alternative criteria represent a
different philosophical perspective that is subjectivist rather than realist in nature. They
claim that research inherently assumes that there is some reality that is being observed
and can be observed with greater or less accuracy or validity. if you don't make this
assumption, they would contend, you simply are not engaged in research (although that
doesn't mean that what you are doing is not valuable or useful).
All qualitative data can be coded quantitatively.
Anything that is qualitative can be assigned meaningful numerical values. These values
can then be manipulated to help us achieve greater insight into the meaning of the data
and to help us examine specific hypotheses. Let's consider a simple example. Many
surveys have one or more short open-ended questions that ask the respondent to supply
text responses. The simplest example is probably the "Please add any additional
comments" question that is often tacked onto a short survey. The immediate responses
are text-based and qualitative. But we can always (and usually will) perform some type
of simple classification of the text responses. We might sort the responses into simple
categories, for instance. Often, we'll give each category a short label that represents the
theme in the response.
What we don't often recognize is that even the simple act of categorizing can be viewed
as a quantitative one as well. For instance, let's say that we develop five themes that
each respondent could express in their open-ended response. Assume that we have ten
respondents. We could easily set up a simple coding table like the one in the figure
below to represent the coding of the ten responses into the five themes.
Person Theme 1 Theme 2 Theme 3 Theme 4 Theme 5
This is a simple qualitative thematic coding analysis. But, we can represent exactly the
same information quantitatively as in the following table:
Person Theme 1 Theme 2 Theme 3 Theme 4 Theme 5 Totals
Notice that this is the exact same data. The first would probably be called a qualitative
coding while the second is clearly quantitative. The quantitative coding gives us
additional useful information and makes it possible to do analyses that we couldn't do
with the qualitative coding.
Now to the other side of the coin...
All quantitative data is based on qualitative judgment.
Numbers in and of themselves can't be interpreted without understanding the
assumptions which underlie them. Take, for example, a simple 1-to-5 rating variable:
Here, the respondent answered 2=Disagree. What does this mean? How do we
interpret the value "2" here? We can't really understand this quantitative value unless
we dig into some of the judgments and assumptions that underlie it:
Did the respondent understand the term "capital punishment"?
Did the respondent understand that a "2" means that they are disagreeing with the
Does the respondent have any idea about alternatives to capital punishment (otherwise
how can they judge what's "best")?
Did the respondent read carefully enough to determine that the statement was limited
only to convicted murderers (for instance, rapists were not included)?
Does the respondent care or were they just circling anything arbitrarily?
How was this question presented in the context of the survey (e.g., did the questions
immediately before this one bias the response in any way)?
Was the respondent mentally alert (especially if this is late in a long survey or the
respondent had other things going on earlier in the day)?
What was the setting for the survey (e.g., lighting, noise and other distractions)?
Was the survey anonymous? Was it confidential?
In the respondent's mind, is the difference between a "1" and a "2" the same as between
a "2" and a "3" (i.e., is this an interval scale?)?
The bottom line here is that quantitative and qualitative data are, at some level, virtually
inseparable. Neither exists in a vacuum or can be considered totally devoid of the
other. To ask which is "better" or more "valid" or has greater "verisimilitude" or whatever
ignores the intimate connection between them. To do good research we need to use
both the qualitative and the quantitative.
What is triangulation? Triangulation is the application and combination of several
research methodologies in the study of the same phenomenon. Triangulation was
originally used in social sciences and has now spread to psychology.
It can be employed in both quantitative(validation) and qualitative(inquiry)
It is a method-appropriate strategy of founding the credibility of qualitative
It becomes an alternative to " traditional criteria like reliability and validity"
It is the preferred line in the social sciences
Why use triangulation?
By combining multiple observers, theories, methods, and empirical materials,
sociologists can hope to overcome the weakness or intrinsic biases and the problems that
come from single method, single-observer, single-theory studies. Often the purpose of
triangulation in specific contexts is to obtain confirmation of findings through
convergence of different perspectives. The point at which the perspectives converge is
seen to represent reality.
Types of Triangulation:
There are four basic type of triangulation:
a. data triangulation, involving time, space, and persons
b. investigator triangulation, which consist of the use of multiple, rather than single
c. theory triangulation, which consists of using more than one theoretical scheme in
the interpretation of the phenomenon;
d. methodological triangulation, which involves using more than one method and
may consist of within-method or between-method strategies.
e. multiple triangulation, when the researcher combines in one investigation multiple
observers, theoretical perspectives, sources of data, and methodologies.
Summary of common research methods
Sample Surveys
Rapid Appraisal
Participant Observation
Case Studies
Participatory Learning and Action
Specialised methods
Key Features
Collect quantitative data
through questionnaires.
Usually a random sample and
a matched control group are
used to measure predetermined indicators before
and after the intervention
A range of tools and
techniques developed
originally as rapid rural
appraisal (RRA). Involves the
use of focus groups, semistructured interviews with key
informants, case studies,
participant observation and
secondary sources
Extended residence in a
programme/project community
by field researchers using
qualitative techniques and
mini-scale sample surveys
Detailed studies of a specific
unit ( a group, locality,
organisation) involving openended questioning and the
preparation of ‘histories’.
The preparation by
beneficiaries of a programme
of timelines, impact flow
charts, village and resource
maps, well being and wealth
ranking, seasonal diagrams,
problem ranking and
institutional assessments
through group processes
assisted by a facilitator.
E.g. Photographic records
and video.
Source: Herbert and Shepherd, 2001, adapted from Hulme (1997) and Montgomery et al (1996)
The case study method
Research method originated in clinical medicine (the case history, i.e. the patient’s personal history
(idiographic method)
Description of the symptoms, the diagnosis, the treatment and eventual outcome (descriptive method)
but also in newer research explanatory case studies
Uses the person’s own memories, the memories of friends and relatives, or records of various types
such as diaries, photographs etc.
Often combines interviews and observations.
In-depth investigation of experiences that allow to identify interactions and influences on
psychological processes
Opens up and explore aspects of human experience that can be investigated using other types of
research methods (qualitative study/inductive research)
The case study method often involves simply observing what happens to, or
reconstructing ‘the case history’ of a single participant or group of individuals (such as a
school class or a specific social group), i.e. the idiographic approach. Case studies allow a
researcher to investigate a topic in far more detail than might be possible if they were
trying to deal with a large number of research participants (nomothetic approach) with
the aim of ‘averaging’.
The case study is not itself a research method, but researchers select methods of data
collection and analysis that will generate material suitable for case studies such as
qualitative techniques (semi-structured interviews, participant observation, diaries),
personal notes (e.g. letters, photographs, notes) or official document (e.g. case notes,
clinical notes, appraisal reports). All the approaches mentioned here use preconceived
categories in the analysis and they are ideographic in their approach, i.e. they focus on the
individual case without reference to a comparison group.
Intrinsic versus extrinsic case studies
Intrinsic case studies represent nothing but themselves. The cases in intrinsic case studies
are chosen because they are interesting in their own right. The researchers want to know
about them in particular, rather than about a more general problem or phenomenon.
Extrinsic case studies constitute exemplars of a more general phenomenon. They are
selected to provide the researcher with an opportunity to study the phenomenon of
interest. The research question identifies a phenomenon (e.g. stress, bereavement, fame
etc) and the cases are selected in order to explore’ how the phenomenon exists within a
particular case’. In this design, individuals who are experiencing the phenomenon under
investigation are all suitable cases for analysis.
Data collection in case studies
Researchers gather information in different ways, so although interviewing (esp. semistructures interviews) is a widely used method it is not the only one used by case-study
researchers. Clinical neuropsychologists who are investigating someone who has suffered
a distinctive brain injury will use a number of specific tasks, designed to reveal if there
are neurological deficits occurring as a result of the injury. These may range from
specific memory tasks (e.g. the person is asked to listen to something like a news report
and then tell the researcher what it was about), to drawing tasks in order to detect if the
person shows any kind of visual neglect, to single-eye and handedness tasks, which might
indicate whether there are unusual differences in functioning between the two halves of
the participant’s brain.
Smith (1997) studied the experience of pregnancy undergone by four women. By seeing
each of the women at regular intervals throughout their pregnancy and afterwards, Smith
was able to explore detailed aspects of their experiences, and also to see how their
memories of the experience changed over time and in retrospect. The women’s
experiences were explored using a number of different techniques, including repertory
grids and diary methods, i.e. triangulation was used in order to take up different
perspectives on how pregnancy is perceived by different women over time.
Table 1. Types of case study
The study of one single individual, generally using several different research
The study of a single distinctive set of people, such as a family or small group
of friends.
The study of a particular place, and the way that it is used or regarded by
The study of a single organisation or company, and the way that people act
within it.
The study of a particular social or cultural event, and the interpretations of that
event by those participating in it.
Complex cases may be seen as social, cultural or psychological systems. In such cases it
is often helpful for the researcher to adopt a systems analysis approach to the study. This
involves identifying the four major dimensions to the system: elements, order, processes
and functions.
Table 2. Dimensions of system analysis
The separate parts which make up the system
Coherence between the elements, e.g. patterned interactions or mutual
Changes over time, or transactions or exchanges (both psychological and
The goals or outcomes of activity within the system
A case can also be viewed as a psychological field. The concept of psychological field
was first introduced by Lewin in 1952, and it was a way of expressing the complexity of
social experience by organising into different dimensions such as e.g. psychological
dimension, spatial dimension, cultural dimension, historical dimension and social
dimension. There are other possibilities of the psychological field, and by using
triangulation the psychologist may be able to understand more fully what is going on,
because the collection of different kinds of data and analysing them to see to what extent
they may converge to influence the experience or the behaviour under investigation.
Table 3. Psychological field analysis
Psychological dimension
Spatial dimension
Cultural dimension
Historical dimension
Aspects of individual experience and identity.
Places or locations within which a particular event or experience is set
(home, pre-natal clinic, school, the psychological laboratory. Orne
(1962) showed how this can affect people’s understanding of what is
going on, e.g. demand characteristics).
Symbols and rituals involved in the event.
Previous or related events influence on the situation or how it is
Social dimension
Relationships, lifestyles and social networks.
The main characteristics of the case study
1. A descriptive study
a. (I.e. the data collected constitute descriptions of psychological processes and
events, and of the contexts in which they occurred (qualitative data).
b. The main emphasis is always on the construction of verbal descriptions of
behaviour or experience but quantitative data may be collected.
c. High levels of detail are provided.
2. Narrowly focused.
a. Typically a case study offers a description of only a single individual, and
sometimes about groups.
b. Often the case study focuses on a limited aspect of a person, such as their
psychopathological symptoms.
3. Combines objective and subjective data
a. i.e. the researcher may combine objective and subjective data: All are
regarded as valid data for analysis, and as a basis for inferences within the
case study.
i. The objective description of behaviour and its context
ii. Details of the subjective aspect, such as feelings, beliefs, impressions
or interpretations. In fact, a case study is uniquely able to offer a
means of achieving an in-depth understanding of the behaviour and
experience of a single individual.
4. Process-oriented.
a. The case study method enables the researcher to explore and describe the
nature of processes, which occur over time.
b. In contrast to the experimental method, which basically provides a stilled
‘snapshot’ of processes, which may be continuing over time like for example
the development of language in children over time.
Effects of isolation in young children1
Mason (1947) The case study of Isabelle who had been kept in isolation in a dark room
with her mother who was deaf and without speech gives insight into the development of
children by an extraordinary case. Isabelle had not been given an adequate diet and had
severe rickets. During her isolation she communicated with her mother using gestures.
The mother escaped from the isolation when Isabelle was about six years old. On her
admission to hospital Isabelle behaved like a wild animal and only made croaking
Cardwell,M. et al. (1996) Psychology for A level. London:Collins Educational p.380)
sounds. After one week in the hospital she started to make speech sounds and seemed to
pass rapidly through the normal stages of speech. After 18 months she had a vocabulary
of over 2000 words, could read and write, and could compose imaginative stories.
Koluchova (1976) This case study involves Czechoslovakian, male, identical twins
whose mother died after giving birth. The twins went to a children’s home for eleven
months, then spent six months with their aunt, and then went to live with their father and
stepmother. The father was of low intelligence and the stepmother was exceptionally
cruel. The boys were never allowed out of the house and were kept either in a small
unheated closet or in a cellar. They were discovered at the age of seven, and they could
hardly walk, had acute rickets, were very fearful and their spontaneous speech was very
poor. After placement in a hospital and later in a foster home excellent gains were made.
The children are now adults and appear well adjusted and cognitively able.
Curtiss (1977) Genie was found when she was 13 years old. Her history was one of
isolation, severe neglect and physical restraint; she was kept strapped to a child’s potty in
an attic. Her father punished her if she made any sound. On discovery her appearance
was of a six-or seven-year-old child. She was described by Curtiss as an "unsocialised,
primitive, and hardly human;" she made virtually no sounds and was hardly able to walk.
Genie has not achieved food social adjustment or language despite intervention and being
placed in a foster home.
Corkin (1984) H.M. was 27 when brain surgeons removed most of his hippocampus and
part of the amygdala in a last attempt to relieve the patient’s severe and life-threatening
epilepsy. The operation did achieve its goal, because the seizures were milder and could
be managed with medication. His memory, however, had been affected dramatically.
Although H.M. could recall most of the events that had occurred before the operation, he
could no longer remember new experiences for much longer than 15 minutes. The
declarative memories (i.e. memories of facts and events) vanished like water down the
drain. With sufficient practice, H.M. could acquire new skills, such as solving a puzzle or
playing tennis (this kind of memory is called procedural memories), but he could not
remember learning these skills. Nor could he learn new words, songs, stories, or faces.
H.M.’s doctors had to reintroduce themselves every time they saw him. It seems that
H.M.’s terrible memory deficits involve a problem in transferring explicit memories from
short-term storage into long-term storage in the first place. He would read the same issue
of a magazine over and over again without realising it. He could not recall the day of the
week, the year, or even his last meal. Today, many years later, H.M. will occasionally
recall unusually emotional evens, such as the assassination of someone named Kennedy.
He sometimes remembers that both his parents are dead, and he knows he has memory
problems. But according to Suzanne Corkin, who has studied H.M. extensively, these
“islands of remembering” are the exceptions in a vast sea of forgetfulness. He still does
not know the scientists who have studied him for decades. Although he is now in his
seventies, he thinks he is much younger. This good-natured man can no longer recognise
a photograph of his own face; he is stuck in a time warp from the past.
Advantages of the case study method (Searle 1999)2
1. Stimulating new research. A case study can sometimes highlight extraordinary
behaviour, which can stimulate new research. For example, Luria’s study of the
memory man “S” enabled researchers to begin to investigate cases of unusual
memory abilities, and the cognitive mechanisms, which made such phenomena
possible. Without the case study, it is unlikely that this area of research would have
been opened up in the same way.
2. Contradicting established theory. Case studies may sometimes contradict
established psychological theories. Searle cites the case study of severely deprived
Czechoslovak twins, and the remarkable recovery they showed when placed in a
caring social environment, as an example of a case study which challenged the
established theory of the early years of life being a critical period for human social
3. Giving new insight into phenomena or experience. Because case studies are so rich
in information, they can give insight into phenomena, which we could not gain in any
other way. For example, the case of S.B., a blind man given sight in adulthood, gave
researchers a particularly detailed insight into the processes and experiences of
perception, highlighting aspects of the experience, which had not yet previously been
4. Permitting investigation of otherwise inaccessible situations. Searle claimed that
the case study gives psychological researchers the possibility to investigate cases,
which could not possibly be engineered in research laboratories. One example of this
is the case of Genie, the severely deprived child whose case enabled researchers to
study the effect of extreme social deprivation continued from infancy to puberty. To
create such a situation for research purposes would be totally unethical and not
possible but when Genie was discovered by social workers, the use of case-study
methodology permitted much deeper insights into the mechanisms, processes and
consequences of her experience and recovery.
Disadvantages of the case study method
Searle (1999) identified a number of disadvantages to case study research.
1. Replication not possible. Uniqueness of data means that they are valid for only one
person. While this is strength in some forms of research, it is a weakness for others,
because it means that findings cannot be replicated and so some types of reliability
measures are very low.
2. The researcher’s own subjective feelings may influence the case study
(researcher bias). Both the collection of data and the interpretation of them. This is
particularly true of many of the famous case studies in psychology’s history,
especially the case history reported by Freud. In unstructured or clinical case studies
the researcher’s own interpretations can influence the way that the data are collected,
i.e. there is a potential for researcher bias.
3. Memory distortions. The heavy reliance on memory when reconstructing the case
history means that the information about past experiences and events may be
notoriously subject to distortion. Very few people have full documentation of all
Hayes, N. (2000) Doing Psychological Research. Gathering and analysing data. Buckingham: Open
University Press. p. 133.
various aspects of their lives, and there is always a tendency that people focus on
factors which they find important themselves while they may be unaware of other
possible influences.
4. Not possible to replicate findings. Serious problems in generalising the results of a
unique individual to other people because the findings may not be representative of
any particular population.
Ethical aspects of the case-study method
In a case study, the researcher often obtains deeply personal information, which is not
usually shared with other people. The nature of the study means that some of this
information will eventually be published, or at least written up as a research report. It is
therefore essential that anyone conducting a case study is very protective of their research
participant’s identity and that they must try to obscure details that can lead to deduction
of identity. Also it is important that the researcher has the professional competence to
deal with the problems of the case study, e.g. in the case of child abuse or anorexia
nervosa. Therefore the ethical guidelines such as informed consent, no deception, right to
withdraw, debriefing and confidentiality must always be observed.
Content analysis
Content analysis is a research method in which answers are categorised into different
types, and the number of each type is counted up. Content analysis can be used in many
different areas. In interviews general themes can be identified and the number of times
they appear can be counted. If you analyse for example diaries or other documents, you
could begin by counting how many times the specific topics you are interested in are
referred to.
Some times content analysis is referred to as a form of qualitative analysis. This is,
however, a controversial classification, since content analysis is really a type of
numerical coding. In the days before the richness of qualitative analysis was widely
recognised in psychology, though, content analysis was the main technique researchers
used for dealing with complex meanings.
The content analysis method consists of establishing a number of different content
categories, and counting up the number of times items relevant to each of them occurs in
a particular set of data.
Content analysis is really a way of using summary tables to describe qualitative data, i.e.
data which don’t appear in the form of numbers but as words or other meaningful
information- but in a quantitative form. However, content analysis isn’t really qualitative
analysis, event though it is used with qualitative data. Instead, it is a way of converting
that qualitative data into quantitative information- of describing it using numbers.
The essence of content analysis is categorisation. A content analysis describes a set of
data in terms of a set of categories, and how many examples have been counted in each
category. That information is usually presented as a summary table, with the categories
forming the columns, and the set of data forming the rows. The numbers, which appear in
the cells of the table, are the frequencies- the result of counting up how often that
category occurs in the data set.
Content analysis turns qualitative information into quantitative data
What content analysis does, then, is to turn qualitative information into quantitative data,
by converting it into numbers. In doing so, it describes the information but it also opens
the way for a researcher to perform additional statistical tests on the material, if that
seems appropriate. The most commonly used one is chi-square, because a content
analysis gives us nominal data.
An example of content analysis: Hacker and Swan (1992)3 focused on different aspect
of campaign strategy in forms of television advertisements paid for by political parties as
a means of ‘selling’ their candidate. Some of these ads aim to promote a candidate’s
strengths while at the same time highlighting the opponent’s weaknesses. Hacker and
Swan suggested that such advertisements have a stronger influence than other TV spots
because they are watched by a wider cross-section of the population than political debates
and are presented in simpler terms. The two researchers videotaped 17 campaign
advertisements in autumn 1988 and randomly selected five from each campaign for
analysis. The researchers devised a coding system by watching other advertising spots,
where the focus was on ‘mutually exclusive and mutually exhaustive’. Coding units were
single messages (for example, a specific isolated scene, a statement about a candidate or a
scene). Each was classified in terms of the media dimension: oral, visual, written,
candidate nonverbal (NV) and special effects. And each was classified in terms of 14
different message appeal categories such as positive or negative trait, nationalism, family,
humanitarian interest, mission statement, or fear. All coding was assessed using intercoder reliability (0.89), and for messages for which where there was not agreement were
discussed and if no agreement could be reached, they were treated as uncodeable. The
results showed a difference in the sense that the Bush campaign used significantly more
positive messages than the Dukakis campaign. One other difference was that the
Dukakis’ campaign emphasised the visionary appeal of the candidate. These may have
been perceived as irrelevant because of the insufficient number of positive images, or it
may be that many members of the electorate simply find such appeals irrelevant.
The researchers presented some of the results in a summary table where the F ratio (or
variance ratio) showed the differences between mean scores of the two groups and the
variation of scores within each data set in order to see if the kind of differences should be
expected just from random variation or individual differences. The F ratio (or variance
ratio) is a statistic that expresses the ratio between the two different types of variation in
the data, and it does this by dividing the between-groups variation by the within-groups
Table 1. Comparison of appeals vs. campaigns
Positive record
Negative record
Positive trait
Negative trait
Ideal vision
Positive issue
Negative issue
Standard deviation
F ratio
Summary tables are used in almost all kinds of quantitative research. They are ways of
summarising more than one set of data, so that the similarities and differences produced
by variables or factors can be seen as easily as possible. Summary tables often use
measures of central tendency and measures of dispersion. The convention used for
summary tables in research papers is to list the variables of the study along the left-hand
side, so that they form the rows of the table, and the statistical measures along the top, so
that they form the columns. If we had to draw a table that reported the study of two
different teaching methods (A and B) to teach children to read, the two methods would be
listed as the rows of the table, and the mean scores and standard deviations obtained from
our test results at the top. Reading along a row would then say how a single method
scored; reading down the ‘mean’ column would tell us whether the means of the two
groups were very different; and reading down the ‘standard deviation’ column would
enable us to compare the standard deviations of the two sets of scores.
Table 2. Methods of teaching reading: results from a reading accuracy test.
Method A
Method B
Related flashcards
Cultural studies

21 Cards


20 Cards

Create flashcards