The DSM –
how valid and reliable is it as a
tool for diagnosis?
A2 unit 4 Clinical Psychology
Reliability of the diagnosis of schizophrenia
Validity of the diagnosis of schizophrenia
Pre-reading check
• By now you should
have an
understanding of what
the DSM is.
• We now need to
consider how reliable
and valid it is.
Learning outcomes:
a) To be able to describe and evaluate the
reliability of the diagnosis of Sz
b) To be able to describe and evaluate the
validity of the diagnosis of Sz
c) To be able to describe and evaluate
cultural issues in the diagnosis of Sz . (in
all three cases using the findings of
studies).
• The Diagnostic and DSMStatistical Manual of
Mental Disorder (Edition
5), was last published in
2013.
• The DSM is produced by
the American Psychiatric
Association.
• It is the most widely used
diagnostic tool in
psychiatric institutions
around the world
IV
ICD - 10
• There is also the
International Statistical
Classification of
Diseases (known as
ICD).
• It is produced by the
World Health
Organisation (WHO) and
is currently in it’s 10th
edition.
DSM – a multi-axial system
• Axis I
• Axis II
• Axis III
• Axis IV
• Axis V
Disorders, clinical and mental eg
schizophrenia
Personality (underlying) including
mental retardation
Medical and Physical conditions
Environmental factors
Global functioning
•
•
•
•
•
Dad
Purchased
Most
Extraordinary
Glasses
• The upcoming fifth edition of the
Diagnostic and Statistical Manual of
Mental Disorders (DSM-5) makes several
key changes to the category of
schizophrenia and highlights for future
study an area that could be critical for
early detection of this often debilitating
condition.
Changes to the Diagnosis
• Schizophrenia is characterized by delusions, hallucinations,
disorganized speech and behaviour, and other symptoms that
cause social or occupational dysfunction. For a diagnosis,
symptoms must have been present for six months and include at
least one month of active symptoms.
• DSM-5 raises the symptom threshold, requiring that an individual
exhibit at least two of the specified symptoms. (In the manual’s
previous editions, that threshold was one.) Additionally, the
diagnostic criteria no longer identify subtypes. Subtypes had
been defined by the predominant symptom at the time of
evaluation. But these were not helpful to clinicians because
patients’ symptoms often changed from one subtype to another
and presented overlapping subtype symptoms, which blurred
distinctions among the five subtypes and decreased their validity.
Some of the subtypes are now specifiers to help provide further
detail in diagnosis. For example, catatonia (marked by motor
immobility and stupor) will be used as a specifier for schizophrenia
Evaluation of the DSM
Strengths
1. Allows for common diagnosis (although many
revisions) because it has stood the test of time
When 2 or more doctors use the DSM, they
should come close to the same diagnosis.
2. Evidence suggests that it is reliable ~Goldstein
looked at the reliability between DSM-II and
DSM-III is (1988)
She found there was evidence of reliability within
the DSM-III (but less so between DSM-II and
DSM-III)
Evaluation of the DSM
Weaknesses
1. The DSM is seen as a confirmation of the medical
state of mental disorder, as suffers are ‘patients’
and ‘treatment’ is suggested.
Mental health issues are ‘disorders’ and ‘illnesses’
so ‘cures’ are looked for.
However, it might be said that some mental disorders
are simply ways of living …. who is to say whether
it is ‘illness’ or not. (e.g. schizophrenics may be
trying to get back to their normal self)
Evaluation of the DSM
Strengths:
• It’s the best attempt at diagnosis that there is and
it allows a common diagnosis
• There are studies which support its reliability and
validity
Weaknesses
• It can be considered a way of labelling people
whose behaviour we see as “different”
• In the US some people argue by inventing mental
illnesses psychiatrists can make more money
Reliability and the diagnosis of
mental disorders
The DSM’s reliability rests on the question
of
whether one person’s set of symptoms
would lead to a common diagnosis by
different physicians
If different doctors give different diagnosis
for the same set of symptoms (e.g. for the
same person), then the diagnosis are not
reliable and the treatment may not work
Studies looking at the reliability of
the DSM
Goldstein (1988)
• tested DMS-III for reliability and found that there was
reliability
• She looked at the effect of gender on the experience of
schizophrenia
• she re-diagnosed 199 patients, originally diagnosed
using DSM–II; some differences
• she asked two other experts to re-diagnoses (single
blind) a random sample of 8 of the patients using the
case histories with all indication of previous diagnoses
removed
• high level of agreement/consistency of diagnosis
Goldstein (1988)
• Goldstein realised that she was biased as she
knew her hypothesis so she asked two experts
to re-diagnose a random sample of 8 patients.
• The experts received the original case notes
with the diagnosis removed
• She found a high level of agreement and interrater reliability.
• This suggests that DSM-III is a reliable tool.
Other research exploring
Reliability
• Using text books, research other studies
that have explored the reliability of the
DSM
• What have these studies shown?
Rosenhan (1973)
use for reliability and validity
• Because the diagnosis was the same
across all 12 of the hospitals presumably
using the current DSM at the time, we
could say this shows the DSM to be
reliable
• Because the diagnosis of healthy people
was schizophrenia, if they were using the
DSM this means it lacks any validity
Evaluation of validity issues
STRENGTHS
• The DSM has been shown to be valid
across a variety of studies covering a
range of different conditions
• Because it is reliable it is likely to be valid
too
• Much work has been done to increase its
validity as it has been rewritten
WEAKNESSES
• It is hard to diagnose people who are
suffering from more than one condition (comorbidity) when using the DSM
• It can be considered to be reductionist to
break down a condition into a series of
symptoms, so we shouldn’t over concentrate
on Axis 1
• Questionnaires and interviews such as in the
Kim-Cohen study may find what they are
looking for
CULTURAL ISSUES and the
DSM
Culture does not affect
diagnosis
Culture does affect
diagnosis
• It’s scientific, and if we
clearly define our
symptoms then it can
work all over the world eg
Lee(2006) in Korea
• Schizophrenia is more
similar across cultures
than different
• Some times symptoms
mean different things in
different cultures eg hearing
voices can make you
“special” in a positive way
(spiritual)
• There are cultural
differences in symptoms
• Eg more auditory
hallucinations in Mexico,
more grandiosity in white
Americans,
What should we do about the
cultural
problems in using
the DSM?
•We should be aware of the cultural
problems in diagnosis
•Concentrate less on first rank
(positive) symptoms which tend to be
more cultural
•Concentrate more on negative
symptoms which are less culturebound and easier to measure
objectively
Kirk and Kutchins (K&K) (1992)
• In their review paper Kirk and Kutchins argued that there
are methodological problems with the studies used to
test the reliability of the DSM up until 1992… these then
limit the generalisability of the findings.
• The studies outlined used interviews and questionnaires
to gather data; K&K argued that training and supervision
of interviews was insufficient and that they lacked the
commitment and skills to be accurate.
They also pointed out that the studies they looked at
tended to take place in specialised research settings…
meaning that their findings might not relate to clinicians
in normal clinical settings.
Generally speaking, an unreliable diagnostic tool would
lack validity…. K&K suggested the DSM could also lack
validity.
Evaluation of K&K (1992) points
Points about interviewing - such as that
different interviewers may affect the
situation and lead to different data – might
be important when considering
generalising findings from studies
Goldstein (1988) did not, however, use
interviewing to test reliability – she used
re-diagnosis using secondary data, and
also found reliability
Evaluation of K&K (1992) points
The patients in the studies looked at were
not all from research settings
The ‘gamblers’ in Stinchfield’s study were on
a gambling treatment program, not in a
research situation.
The patients in Brown et al.’s study were out
patients in a hospital
= possible ecological validity?
Evaluation of K&K (1992) points
• K&K’s study took place before Brown et al.
and Stinchfield’s studies which showed
that DSM-IV-TR could be regarded as
reliable.
Possible to therefore conclud that further
work has been done since DSM-II and
reliability has improved.
• Goldstein (1988), Brown et al. (2003) and
Stinchfield (2003) all provide evidence that
diagnosis is reliable.
Questions: you also have questions in your
pack ~ page 5 that need to be completed
1) What do these studies tell us about the
reliability of the DSM in diagnosing mental
disorders?
2) How are Kirk and Kutchin’s findings
different from the majority of the other
studies? What does this tell us about the
reliability of the DSM?
3) How can the reliability of the DSM be
improved?
Validity and the diagnosis of mental
disorders
• If the DSM were not reliable it would not
be valid either.
• If a diagnosis was done again and the
DSM provided a different one, then it
would not be a valid diagnosis (it would
not be measuring what it claimed to
measure) so therefore reliability and
validity go together
Validity and the diagnosis of mental
disorders
Outline what is meant by each of the
following terms:
– Construct validity
– Etiolical validity
– Concurrent validity
– Predictive validity
– Convergent validity
Construct
validity
• If the DSM is to define mental disorders, then
mental disorders need to be operationalised.
Lists of symptoms and behaviour are the result
of making a mental disorder measurable.
• It has been argued that in operationalising a
concept such as depression, something is lost
from the understanding of the nature of the
whole experience of depression, which means
that the DSM is not a valid tool. There is a lack
of construct validity, in that the constructs
drawn up, for example to represent depression,
might not be representative enough.
Etiological validity
If it has etiological validity, a group of
people who have been diagnosed with the
same disorder will have the same
symptoms or factors causing it.
e.g. schizophrenia is sometimes caused by
too much of the neurotransmitter
dopamine….
…..so in order to have etiological validity,
people diagnosed as schizophrenic should
all have an excess of dopamine in their
brain
Concurrent validity
• For a diagnosis to have concurrent
validity, symptoms that form part of the
disorder but are not part of the actual
diagnosis, should be found in those
diagnosed.
e.g. schizophrenics often have problems with
personal relationships, but this is not a
characteristic that is looked at when
diagnosing them according to the
classification system.
Predictive
validity
• Predictive validity is present if diagnosis can
lead to a prediction of future behaviours caused
by the disorder.
• If a diagnosis has predictive validity we should
be able to say whether the person is likely to
recover or whether the symptoms will continue
• It should also be possible to predict how someone
with a specific disorder will respond to specific
treatments.
e.g. the drug lithium carbonate is effective for bipolar
disorder, but not effective for other mental
disorders. If a classification system has good
predictive validity and diagnoses someone with
bipolar disorder, they should respond to lithium
carbonate.
Convergent validity
• Convergent validity is when a test results
converges on (gets close to) another test result
that measures the same thing.
• A correlation test would be carried out ~ If two
scales measure the same construct, for
example, then a person’s score on one should
converge with (correlate with) their score on the
other.
Difference between convergent,
predictive and concurrent
• Convergent validity ~ the 2 measures
should be measuring exactly the same
thing
• Predictive and concurrent can have
different ways of measuring each case
Task
• Complete the gap fill sheet regarding the
many types of validities relevant to the
DSM.
• Check your answers with the person
beside you or ask for the answer sheet.
Studies looking at the validity of the
DSM
• Next, in pairs read one of - Studies looking
at the validity of the DSM from Kim-Cohen
(2005), Hoffmann (20020 or Lee (2006)
• For the chosen study read it and then
report back to the class on what they
found.
• Evaluate the strengths and weaknesses of
the validity of the DSM.
Strengths – validity of diagnosis
• The studies show the DSM is valid in its
diagnosis. Different mental health issues
were cited in the different studies, which
reinforces the conclusion. It is likely that
symptoms for disorders are well
established, given that the DSM has had
many revisions. Different research
methods have yielded data that also
match the DSM criteria.
Strengths – validity of diagnosis
• The claim that the DSM is valid is
supported by the claim that it is reliable, as
reliability and validity go together. If the
DSM is not reliable it will not be valid.
• Great efforts have been made to make the
DSM-IV-TR more valid, such as adding
culture-bound syndromes.
Weaknesses – validity of diagnosis
• It has been said that co-morbidity – the state of
having more than one mental disorder – is hard
to diagnose using the DSM, a system which
relies on the user choosing the one closest
match from lists of symptoms and features.
• It could be claimed that splitting a mental
disorder into symptoms and features is
reductionist and that a holistic approach might
be more valid.
Weaknesses – validity of diagnosis
• It is possible that the questionnaires and
interviews produce the findings they are
searching for. For instance, if it is well
known that ‘children with ADHA are
impulsive and hyperactive’, and teachers
know which children have that label, they
will then say, ‘those children are impulsive
and hyperactive’. The diagnosis is selffulfilling.
Cultural issues
• Culture does not affect diagnosis – mental
disorders are ‘scientific’
• The DSM was developed in the USA and
is used widely in many other cultures. It is
valid if mental disorders are clearly defined
with specific features and symptoms.
• Mental disorders are scientifically defined
illnesses that are explained in a scientific
way.
Cultural issues
• Culture does affect diagnosis – a spiritual model
• There are studies that have shown that culture can
affect diagnosis.
• e.g. symptoms that are seen in western countries as
characterising schizophrenia (such as hearing
voices) are interpreted in other countries as showing
possession by spirits, which renders someone
special in a positive way, not a negative ‘disorder’.
• Depending on cultural interpretations of what is
being measured, the DSM is not always valid.
• A clinician from one culture must be aware that a
patient from another culture is guided by their own
frame of reference.
Questions
1) What are the two main theories
surrounding culture and diagnosis of
mental disorders? Be able to outline
each.
2) Using your Student book, outline what
two studies regarding culture and
diagnosis of mental disorders.
Homework
• Research schizophrenia – what are three
symptoms of this mental disorder?