eAppendix - Information on ASD diagnosis reliability and validity across sites
Validation of registry-reported diagnosis of autism
This online eAppendix summarizes validation efforts on the registers used for defining autism cases.
Denmark
Diagnoses of autism spectrum disorder (ASD) are entered in the Danish Psychiatric Central Register1 which
includes information on all inpatient admissions to psychiatric hospitals and psychiatric wards in general
hospitals in Denmark since 1969 and all outpatient contact since 1995. All registered diagnoses are made
by psychiatrists. Children who are suspected of having ASD are referred by general practitioners or school
psychologists to a child psychiatric ward where they are evaluated by a multidisciplinary team and assigned
a final diagnosis by a child psychiatrist. All cases of ASD are registered in the psychiatric register once a
formal diagnosis is established and without regard to need for treatment or educational provisions. The
International Classification of Diseases, Eighth Revision (ICD-8), was the diagnostic instrument used by
physicians in Denmark from 1969 through 1993. In 1994, the ICD-8 was replaced by the Tenth Revision
(ICD-10), which is still being used.
The autistic disorder (AD) diagnosis in the psychiatric register has been validated2 ; after evaluating 499
medical records of children diagnosed with AD, 94% met the criteria for AD. The quality of ASD has not
been validated, but is generally believed to be high3 .
Israel
Israel has a universal health care system and medical insurance is compulsory. All children attend well-baby
and child clinics. Children with developmental problems are referred for further assessment in Child
Development Centers.
Eligibility for receiving treatment and government benefits for individuals with ASD requires registration
with the Israeli Ministry of Social Affair4. All children and adolescents receiving benefits are included in the
Ministry of Social Affairs registry. Although the majority of individuals suffering from ASD and diagnosed
during childhood or adolescence are registered with the Ministry of Social Affairs1, some do not. Therefore,
ASD cases in the Ministry of Social Affairs registry are likely skewed towards the more severe end of the
autism spectrum.
The Ministry of Social Affairs registry data have not been formally validated, but diagnosis of ASD follows
guidelines provided by the Israeli Ministry of Health5 . According to the guidelines diagnosis of ASD should
be made according to DSM-IV criteria by a child psychiatrists, paediatric neurologists, or paediatricians in
Child Development Centers. In addition, the guidelines require that intellectual ability (IQ) be determined,
and level of functioning assessed by a child or a developmental psychologist. Assessment using the Autism
Diagnostic Interview – Revised (ADI-R) and Autism Diagnostic Observational Schedule (ADOS) are
recommended. A diagnostic consensus between the physician and psychologist is required for a child to be
eligible for benefits. The diagnostic report is sent to the Ministry of Social Affairs registry were it is
reviewed and the diagnosis is confirmed by a psychiatrist. In questionable cases the child and parents are
referred for further evaluation by a certified multidisciplinary team of trained diagnosticians.
1 (6)
05-Nov-2014
eAppendix - Information on ASD diagnosis reliability and validity across sites
Norway
The receipt of benefits from the Norwegian National Insurance System (NNIS) is based on a clinical
diagnosis and a written statement from a specialist within the field of relevance. For ASD cases, the
diagnoses and statements are made by paediatricians or child psychiatrists. Norway has a governmentfunded healthcare system with complete population coverage. Health services are free of charge for
children below seven years of age. For children aged seven and older, there is a co-payment of maximum
$350 per year, but anything above that is covered by the government. All evaluations and follow-up of
children with ASD occur at government-owned institutions.
All children in Norway attend a set check-up schedule at public well-baby clinics from birth until five years
of age. There are no mandatory check-ups after that, but children with developmental problems are
referred to school educational services. These services are obliged to refer children for medical evaluations
if underlying diagnoses is suspected. When children are diagnosed with ASDs, they are eligible for benefits
from the NNIS if they have special needs. Consequently, the population of ASD cases captured by the NNIS
will be skewed towards the severe end of the autism spectrum, and there is likely to be an underascertainment of high-functioning children with ASDs.
All children receiving benefits are recorded in the NNIS registry. The NNIS data have not been formally
validated, but a validation study has been conducted of ASD diagnoses assigned by Norwegian specialist
health services, from which the NNIS diagnoses are derived. This validation study showed that an ASD
diagnoses assigned by specialist health services have a PPV of 97%, i.e., 97% of registered cases met the
DSM-IV criteria for ASD at an in-person research-standard assessment6 .
2 (6)
05-Nov-2014
eAppendix - Information on ASD diagnosis reliability and validity across sites
Sweden
Since long the Swedish health system is publically financed and utilised. For the studied time period, private
care for psychiatric and mental disorders was almost non existent. All infants and preschool children in
Sweden are regularly seen at well-child care clinics and undergo routine medical and developmental
screening. At age 4 a mandatory developmental assessment (motor, language, cognitive and social
development) is conducted. Children with suspected developmental disorder are referred for further
assessment by a specialized team in a child psychiatry unit or habilitation service. Diagnostic information is
reported to the Patient Register. In the registers the international classification of diseases, 9th and 10th
revisions are.
The identification of cases treated for autism or autism spectrum disorder on a national basis has been
possible through the Swedish Patient Register, which has a complete coverage since 1987 of all psychiatric
hospital inpatient discharges and out-patient care since 2001. Each record contains admission and
discharge dates, dates for out-patient care, the main diagnosis and up to eight secondary diagnoses.
Diagnoses are established by the treating physician, usually a specialist within child psychiatry. In an
external review and validation study of the Swedish Patient Registerr7 concluded that the reporting of the
correct diagnoses (the positive predictive value) was generally in the range of 85 to 95 percent. For
psychiatric diagnoses, such a high level of agreement has been shown for the adult psychiatric disorders
schizophrenia8,9 and bipolar disorder10 . The diagnosis of Swedish ASD cases has previously been validated
within the Stockholm County11 with 96% of the ASD cases being verified.
In addition we have performed a separate validation of AD on a small sub-sample as described below.
Validation study
Aims and methods
The aim of the validation study was to evaluate the clinical specificity of autistic disorder diagnosis in a
subsample of the Swedish patient register. Medical records in Sweden are computerized and contain
notations from psychiatrists, psychologists, neurologists, social workers, and nurses for inpatient and
outpatient treatment. From the Swedish patient registry we retrieved a random sample of 68 individuals
treated with infantile autism according to the International Classification of Diseases, ICD-9 and ICD-10. The
individuals were assessed by and treated as inpatients during the years 1987 to 1998 in Uppsala and
Gävleborg county with approximately 300.000 inhabitants each. Medical journals were collected for the
patients and diagnostic information abstracted in accordance with DSM-IV criteria. The record review
adopted the approach developed and described by the Centers for Disease Control and Prevention based
on the published description12 and adapted for use in a similar register in Denmark2. A child psychologist
(Maria Henning), specialized in autism, reviewed the selected medical records to validate the reported
diagnoses and consensus was attained with a specialist in clinical psychology (Christina Hultman) and a
team member of the Uppsala Habilitation Centre (Sara Pettersson).
3 (6)
05-Nov-2014
eAppendix - Information on ASD diagnosis reliability and validity across sites
Results
The sample consisted of 68 individuals born between 1961 to 1995. Of those 39 individuals were born
between 1986 to 1994. The sample represented 9 female and 59 male patients.
Medical record review substantiated the presence of DSM-IV autistic disorder in 63 (93%) individuals,
whereas 5 (3 male, 2 female) cases were not. The 5 patients not validated as having autistic disorder were
diagnosed with developmental psychomotor disorder, OCD, language disorder, cerebral paresis and
developmental delay. The assessment of the medical records revealed that the diagnostic procedures for
patients included parental interviews, observations and a variety of developmental and communicative
tests to determine the ASD diagnosis.
Diagnosed genetic disorders known to be associated with autistic features (2 individuals with Fragile X, 3
with Down syndrome and one with Kleinfelter syndrome) were identified in 5 out 68 validated cased (7%).
Conclusions
The presented material showed that the diagnostic procedures were in accordance with clinical practice
guidelines. The documented procedures resulting in an autistic disorder diagnosis could be confirmed
having a high degree of specificity.
4 (6)
05-Nov-2014
eAppendix - Information on ASD diagnosis reliability and validity across sites
Western Australia
Population-based ascertainment of ASD in Western Australia involves the identification of cases from
overlapping sources. The primary sources used are:
1. The Disability Services Commission of WA database (the government agency that is the primary
service provider and assessment agency for children with autism and intellectual disability)
2. The WA Register of Autism Spectrum Disorders, a prospective surveillance system of newly
diagnosed cases.
Standardized procedures for multidisciplinary evaluation (paediatrician or psychiatrist, psychologist and
speech pathologist) for ASD were established in the early 1990s. Children may be diagnosed either at a
government-based assessment agency (multidisciplinary team) or a non-government private practice
(paediatrician). However to be eligible for ASD-specific early intervention funding children are required to
have their diagnosis confirmed through a multidisciplinary assessment. The primary diagnostic tool used up
to the mid-1990s was the DSM-IIIR and in 1994 this was superseded by the DSM-IV criteria and in 2000 by
the DSM-IV-TR. There have not been any specific validation studies of the accuracy of the reported ASD
diagnoses based on the standardized procedures established in the early 1990s13.
References
1 Munk-Jørgensen P, Mortensen PB. The Danish Psychiatric Central Register. Dan Med
Bull 1997; 44: 82–84.
2 Lauritsen MB, Jørgensen M, Madsen KM, Lemcke S, Toft S, Grove J et al. Validity of
childhood autism in the Danish Psychiatric Central Register: findings from a cohort
sample born 1990-1999. J Autism Dev Disord 2010; 40: 139–148.
3 Petersen DJ, Bilenberg N, Hoerder K, Gillberg C. The population prevalence of child
psychiatric disorders in Danish 8- to 9-year-old children. Eur Child Adolesc Psychiatry
2006; 15: 71–78.
4 Gal G, Abiri L, Reichenberg A, Gabis L, Gross R. Time trends in reported autism
spectrum disorders in Israel, 1986-2005. J Autism Dev Disord 2012; 42: 428–431.
5 Israeli Ministry of Health. Diagnostic Practice Guidelines: Autism Spectrum Disorders.
2013.
6 Surén P, Bakken IJ, Aase H, Chin R, Gunnes N, Lie KK et al. Autism spectrum disorder,
ADHD, epilepsy, and cerebral palsy in Norwegian children. Pediatrics 2012; 130: e152–
158.
7 Ludvigsson JF, Andersson E, Ekbom A, Feychting M, Kim J-L, Reuterwall C et al.
External review and validation of the Swedish national inpatient register. BMC Public
Health 2011; 11: 450.
5 (6)
05-Nov-2014
eAppendix - Information on ASD diagnosis reliability and validity across sites
8 Dalman C, Broms J, Cullberg J, Allebeck P. Young cases of schizophrenia identified in
a national inpatient register--are the diagnoses valid? Soc Psychiatry Psychiatr
Epidemiol 2002; 37: 527–531.
9 Ekholm B, Ekholm A, Adolfsson R, Vares M, Osby U, Sedvall GC et al. Evaluation of
diagnostic procedures in Swedish patients with schizophrenia and related psychoses.
Nord J Psychiatry 2005; 59: 457–464.
10
Sellgren C, Landén M, Lichtenstein P, Hultman CM, Långström N. Validity of bipolar
disorder hospital discharge diagnoses: file review and multiple register linkage in
Sweden. Acta Psychiatr Scand 2011; 124: 447–453.
11
Idring S, Rai D, Dal H, Dalman C, Sturm H, Zander E et al. Autism spectrum
disorders in the Stockholm Youth Cohort: design, prevalence and validity. PloS One
2012; 7: e41280.
12
Bertrand J, Mars A, Boyle C, Bove F, Yeargin-Allsopp M, Decoufle P. Prevalence of
autism in a United States population: the Brick Township, New Jersey, investigation.
Pediatrics 2001; 108: 1155–1161.
13
Nassar N, Dixon G, Bourke J, Bower C, Glasson E, de Klerk N et al. Autism
spectrum disorders in young children: effect of changes in diagnostic practices. Int J
Epidemiol 2009; 38: 1245–1254.
6 (6)
05-Nov-2014