Validation of self-administered
single item screening questions
(SISQs) for unhealthy alcohol and
other drug use in primary care
patients at two sites
Jennifer McNeely, Charles M. Cleland, Shiela M. Strauss,
Joseph J. Palamar, John Rotrosen, Marc N. Gourevitch,
Richard Saitz
No relevant financial relationships to disclose
Objectives
1. Describe the need for a self-administered
approach to substance use screening
2. Single Item Screening Questions (SISQs) for
alcohol and drug use
3. Present results of a validation study in
primary care
4. Discuss feasibility and application
Screening for substance
use in primary care
• Medical providers fail to identify clinically
D’Amico, Medical Care 2005
relevant substance use
• Barriers to screening:
o
o
o
o
o
Time
Workflow
Knowledge/Training
Discomfort
Attitudes
Sterling, Addict Med Clin Pract 2012
Friedmann, J Gen Int Med 2000
Friedmann, Arch Int Med 2001
Anderson, Alcohol Alcoholism 2004
McCormick, J Gen Int Med 2006
Self-administered screening is
a more feasible approach
Screening:
Screening
SISQ-alcohol and SISQ-drug
+
Assessment
Low Risk
Moderate Risk
Education
Monitoring
Office-based
counseling
High Risk or
Dependence
Treatment
Single Item Screening Questions
• SISQ-alcohol
How many times in the past year have you had X or
more drinks in a day?
(X=5 for men, and X=4 for women)
• SISQ-drug
How many times in the past year have you used an
illegal drug or used a prescription medication for nonmedical reasons (for example, because of the
experience or feeling it caused?
Prior validation of SISQs
• Adult primary care patients (N=286)
• Single site, urban safety net medical center
• Good sensitivity and specificity for detection
of unhealthy use
• SISQ-alcohol: Sensitivity 82%, Specificity 79%
• SISQ-drug: Sensitivity 85%, Specificity 96%
Smith et al., JGIM 2009
Smith et al., Arch Int Med 2010
Current Study
Screening (computer)
•
•
SUBS
SISQ-alcohol, SISQ-drug
Validation Measures (interviewer)
•
•
•
Timeline Follow Back
SIP-A and SIP-D
MINI-Plus
•
•
REALM
Demographics
Second Consent
Saliva drug screen
Referrals
Incentive
Reference standard measures
Timeline
followback (30d)
Alcohol
Unhealthy
use
+
SIP-A
SIP-D
+
MINI-Plus MINI-Plus Intercept
oral fluid
screening abuse or
dep
test*
+
Disorder
Drugs
Unhealthy
use
Disorder
+
+
+
+
+
+
* Collected at Site A only
Statistical Analysis
1. Comparison of SISQs to composite
reference standards
2. Examined site differences
3. Calculate sensitivity, specificity, AUC:
oUnhealthy use
oSubstance use disorder
4. Subgroup analyses
Study Sites and
Recruitment
•
•
•
•
Adult primary care clinics
2 urban safety net hospitals
Patients presenting for medical visits
Consecutive recruitment
Eligibility Criteria:
• Age 21-65
• Current clinic patient
• Fluent in English
• No disability preventing computer use
Participant Recruitment
Screened: N = 2131
Eligible: N = 915
1216 were excluded
Language: 679
Age: 306
Not a patient: 168
Other: 115
453 declined
No time: 363
Other: 90
1 lost data
Completed interview: N = 459
Site A: 265*
Site B: 194
*230 (87%) Site A participants agreed to saliva test
Characteristics of the 459 participants
Age (years)
Mean = 46, SD = 12
Range = 21-65
Sex (%)
Male
Female
Transgender
48.4
51.2
0.4
Race/Ethnicity (%)
Black/African American
Hispanic
White/Caucasian
Other
51.8
20.2
19.1
8.6
Country of Birth (%)
United States
Outside of United States
64.6
35.3
Education and Health Literacy
Highest Level of Education
25%
14%
Less than high
school
High school
diploma/GED
61%
Health Literacy Level
College
degree
41%
< High
school
59%
High school
or higher
Prevalence of substance use
Substance
Alcohol
Drugs
Specific drug categories
Illicit drugs
Marijuana
Cocaine
Heroin
Hallucinogens
Prescription drugs (non-medical use)
Opioids
Benzodiazepines
Stimulants
Past year use
(MINI)
N (%)
103 (22.3)a
114 (24.7)c
108 (23.4)
----21(4.6)
----
Past month use
(TLFB)
N (%)
89 (19.3)b
73 (15.8)c
58 (12.6)
12 (2.6)
10 (2.2)
1
5
3
2
Unhealthy use
Substance
+ on SISQ
N (%)
+ on
Reference
Alcohol
155 (34)
N (%)
146 (32)
Drugs
107 (23)
122 (27)
Oral fluid test results:
Sensitivity
Specificity
AUC
%
%
(95% CI)
(95% CI)
(95% CI)
73.3
(65.3, 80.3)
84.7
(80.2, 88.5)
0.79
(0.75, 0.83)
71.3
(62.4, 79.1)
94.3
(91.3, 96.6)
0.83
(0.79, 0.87)
8 tested positive, all reported use on SISQ
 No change to results
Substance use disorder
Substance
+ on SISQ
N (%)
+ on
Reference
Alcohol
155 (34)
N (%)
60 (13)
Drugs
107 (23)
74 (16)
Sensitivity
Specificity
AUC
%
%
(95% CI)
(95% CI)
(95% CI)
86.7
(75.4, 94.1)
74.2
(69.6, 78.4)
0.80
(0.76, 0.85)
85.1
(75.0, 92.3)
88.6
(85.0, 91.6)
0.87
(0.83, 0.91)
Subgroup Analysis
Subgroups anticipated to have greater difficulty with
self-administered screening:
• Male
• Age greater than 50
• Hispanic/Latino
• Primary language other than English
• Born outside US
• Education or health literacy lower than high school
level
Subgroup Analysis
• No differences for SISQ-alcohol
• Lower sensitivity of SISQ-drug among:
Primary language other than English (p<0.01)
Sensitivity
Specificity
English
74.3 (65.1, 82.2)
94.4 (90.7, 96.9)
Non-English
46.2 (19.2, 74.9)
94.3 (87.1, 98.1)
Less than high school education (p<0.01)
Sensitivity
Specificity
High school
79.0 (66.8, 88.3)
95.2 (91.0, 97.8)
< High school
63.3 (49.9, 75.4)
93.3 (88.0, 96.7)
50%
Required assistance
40%
Other
Computer-related
Comprehension/ reading
30%
20%
10%
0%
Site A
Site B
Limitations
• Safety net primary care populations
• English speaking only
• Tested in research context, with
assurance of confidentiality
Conclusions
• SISQs accurately identified unhealthy
substance us in primary care patients
• Lower sensitivity and specificity than
interviewer-administered versions
• Efficiency, fidelity, and patient comfort may
be advantages to self-administered
approach
Acknowledgements
Funding:
• K23 Career Development Award
NIDA K23 DA030395
• NYU-HHC CTSI Pilot Grant
NIH/NCATS UL1 TR000038
• The MITRE Corporation (contract
from ONC and SAMHSA)
Staff and others:
• Seville Meli
• Jacqueline German
• Ritika Batajoo
• Catherine Federowicz
• Marshall Gillette
• Charlie Jose
• Emily Maple
• Keshia Toussaint
• Julianne Cameron
• Arianne Ramautar
• Derek Nelsen
• Linnea Russell
• Study participants