The Garrett Lee Smith (GLS) Suicide Prevention National Outcomes Evaluation is supported through contract no. HHSS283201200007I/HHSS28342002T (reference no. 283-12-0702) awarded to ICF International by the Center for Mental Health Services (CMHS), Substance Abuse and
Mental Health Services Administration (SAMHSA), US Department of Health and Human Services (HHS).
NATIONAL OUTCOMES EVALUATION :
FINDINGS FROM THE SMSS AND MIS,
AND PREPARING FOR THE STUDENT
BEHAVIORAL HEALTH FORM (SBHF)
May 2, 2016
Megan Brooks, MA
ON TODAY’S AGENDA
• What we have learned from the National Outcomes
Evaluation and prior campus evaluation efforts
• SMSS background and results
• Management Information System (MIS) results
• Moving forward with the Student Behavioral Health
Form (SBHF)
• Youth Exploratory Services Interview (YESI)
2
SHORT MESSAGE SERVICE
SURVEY
ANALYSES CONDUCTED BY HOPE SOMMERFELDT, MA
WHAT IS THE SHORT MESSAGE SERVICE SURVEY
(SMSS)?
Text message (SMS) survey to students at GLS funded
campuses
• Gather data about exposure to on-campus suicide
prevention interventions, suicidal ideation, and past
suicide attempts
•
4
BENEFITS OF SMS FOR DATA COLLECTION
• 98% of Americans aged 18 – 29 have a cell phone1
• 2014 Harris Poll shows that 83% of college students
regularly use a Smart Phone.2
• 97% of persons ages 18 – 29 send or receive text
messages3
• Students can respond any time and place
• Can start at one time and finish later
• Private (if they secure their phone)
1. Pew Research Center’s Internet Survey Project., January 9 – 12, 2014. From: http://www.pewinternet.org/files/2014/02/PIP_25th-anniversary-of-theWeb_0227141.pdf
2. Harris Poll Pearson Student Mobile Device Survey, February 13 – March 12, 2014. From http://www.pearsoned.com/wp-content/uploads/Pearson-HE-StudentMobile-Device-Survey-PUBLIC-Report-051614.pdf
3. Pew Research Center’s Internet & American Life Project Spring Tracking Survey, April 17 – May 19, 2013. From
http://www.pewinternet.org/2013/09/19/additional-demographic-analysis/
5
SMSS RECRUITMENT METHODS
Intercept
School Listserv
Student Email
addresses
Student Cell
Phone Number
Sample of currently enrolled students were
stopped, by university or evaluation staff inperson to text in to the survey (University Staff
or Evaluation Team)
Currently enrolled students were invited to
provide a phone number through a school
sponsored listserv posting (University Staff)
An email was sent to students from currently
enrolled on campus asking them to provide a
phone number (Evaluation Team)
Random sample of phone numbers for currently
enrolled students received a text message
invitation to participate (Evaluation Team)
6
SMSS DATA COLLECTION
During the past 12 months, did
you attend class at [CAMPUS]?
• During the past 12 months did
you see, read, or hear of any
suicide prevention effort at
[CAMPUS]?
• During the past 12 months, did
you ever seriously consider
attempting suicide?
• During the past 12 months, how
many times did you actually
attempt suicide?
•
7
SMSS DATA COLLECTION – INCREASING RESPONSE
RATE
Previous data collection through a ‘cold’ email had very
low response
• $5 gift code incentive to complete the survey
• Incentive delivered right to their phones via text at the
end of the survey
• A reminder after 60 minutes of
idle time once the participant
begins responding to questions
•
8
CRISIS PROTOCOL
ICF monitors
Responses that
indicate risk
Student requests
HELP via text
•
•
•
•
ICF alerts local crisis center
Follow up
ICF communicates
with center about
Trained
the student of
professionals follow
concern
up with the student
Warning text prior to suicidal ideation questions
Option to skip suicidal ideation questions is clear
The National Suicide Prevention Lifeline number given
24 hour monitoring for crisis situations
9
EXPOSURE, IDEATION, AND ATTEMPTS
(WAVE 1)
10
SMSS DATA COLLECTION (WAVE 2)
• Data collection began in February 2016 and is
ongoing (goal of 14 Cohort 7 Campuses that
completed wave 1)
• To date, responses from 454 students at 6 GLS
funded campuses
• Between 18 and 112 responses per campus
(mean = 81, median =87.5)
11
SNEAK PEAK AT FUTURE ANALYSIS WAVE 1
AND WAVE 2 *
90%
81.7%
80%
70%
65.7%
60%
50%
40%
30%
20%
12.8%
10.2%
10%
3.3%
2.4%
0%
Exposure
Ideation
Wave 1
Attempt
Wave 2
* Includes only grantees that have completed both waves. Wave 1 N = 413; Wave 2 N = 454
12
POSSIBLE FUTURE SMSS ANALYSIS
Tests of significance between Wave 1 and Wave 2
• Inclusions of prevention strategies to see if different
strategies had an impact on exposure, ideation, and
attempt
•
13
USE OF STUDENT MENTAL
HEALTH SERVICES ON GLS
CAMPUSES
ANALYSES CONDUCTED BY YE XU, MA, MS
RESEARCH QUESTIONS
Is there an increase in mental health service use after receipt
of GLS award?
• Is the rate of mental health service use associated with the
number of students, faculty, and/or staff trained on suicide
prevention within the campus community?
• Is the rate of mental health service use associated with the
strategies implemented by campus grantees?
•
15
DATA SOURCES AND MEASURES
•
Management Information System (MIS)
– Number of students using mental health services, per year
•
Training Activity Summary Page (TASP)
– Number of individuals trained by role (student, staff, faculty), per year
•
Prevention Strategies Inventory (PSI)
– Implementation of prevention strategies, in particular Screenings and Public Awareness
Campaigns
•
Secondary Data Sources
– Integrated Postsecondary Education Data System (IPEDS) for student enrollment, number
of faculty and staff
– Carnegie Classification of Institutions of Higher Education (campus size and setting
categories)
16
DESCRIPTION OF MODEL
Students, Faculty, Staff
Trained
Implementation of
Screening or Public
Awareness Campaign (PAC)
Mental
health
service
use
17
SAMPLE
•
Panel data on 84 GLS campuses from cohorts 5 - 8
–
–
–
–
•
Cohort 5 campuses initially funded in year 2011
Cohort 6 campuses initially funded in year 2012
Cohort 7 campuses initially funded in year 2013
Cohort 8 campuses initially funded in year 2014
On average we have data for 3 years preceding the
initial funding year and 2 years following that initial
funding year
18
METHOD
•
Outcome of interest
– Student mental health service use, reported annually by GLS grantees using MIS
abstraction
•
Predictors
– Year of GLS award
– Pre and post GLS award
– Types of gatekeepers trained on suicide prevention, and supplemented by data from the
Integrated Postsecondary Education Data System (IPEDS)
– Implementation of Screening and Public Awareness Campaign on campus
19
DESCRIPTIVE
N
Mean
SD
Total number of students registered on campus
(MIS)
466
17,264
12,425
Average number of faculty on campus (IPEDS)
466
2,115
2,458
Average number of staff on campus (IPEDS)
466
2,385
3,041
Average number of students trained (TASP/TES)
459
106
323
Average number of faculty trained (TASP/TES)
452
7
22
Average number of staff trained (TASP/TES)
457
22
54
Average number of students receiving mental health
service on campus (MIS)
466
1,081
1,107
20
DESCRIPTIVE
Pre-GLS award
Average Training and
Mental health rate on
campus
Post-GLS award
Four
years
prior to
funding
year
Three
years
prior to
funding
year
Two years
prior to
funding
year
One year
prior to
funding
year
Funding
year
Funding
year1
Funding
year2
Average percent of
students trained on
campus
0
0
0
0
0.7%
4.3%
3.4%
Average percent of
faculty trained on
campus
0
0
0
0
2.2%
6.3%
1.8%
Average percent of staff
trained on campus
0
0
0
0
4.4%
7.6%
5.2%
5.9%
6.0%
6.2%
6.4%
6.9%
7.5%
7.9%
Average percent of
students receiving
mental health services
on campus
21
DESCRIPTIVE
Pre-GLS award
Post-GLS award
Average percent of students
trained on campus
--
2.5% (N=202)
Average percent of faculty
trained on campus
--
3.4% (N=202)
Average percent of staff trained
on campus
--
5.6% (N=202)
Average percent of students
receiving mental health services
on campus
6.1%
(N=264)
7.3%
(n=202)
22
DESCRIPTIVE
Number of campuses
% of Campuses
with PAC
% of Campuses
with Screening
Cohort 5 (n=19)
73.7%
78.9%
Cohort 6 (n=32)
65.6%
62.5%
Cohort 7 (n=19)
68.4%
57.9%
Cohort 8 (n=14)
35.7%
71.4%
Total (n=84)
63.1%
66.7%
23
DESCRIPTIVE
Average percent of
students receiving
mental health
services on campus
Campuses
without PAC
Campuses with
PAC
5.5%
7.3%
24
DESCRIPTIVE
Average percent of
students receiving
mental health
services on campus
Campuses
without Screening
Campuses with
Screening
6.5%
6.7%
25
DESCRIPTIVE
Number of students
Average percent of
students receiving
mental health
services on campus
Small campuses
Medium campuses
Large campuses
PreGLS
award
Post-GLS
award
Pre-GLS
award
PostGLS
award
Pre-GLS
award
PostGLS
award
(n=24)
(n=25)
(n=85)
(n=73)
(n=155)
(n=104)
8.9%
12.2%
5.8%
6.6%
5.9%
6.7%
26
DESCRIPTIVE
Number of trainees
Small
campuses
(n=25)
Medium
campuses
(n=73)
Large
campuses
(n=104)
Average percent of
students trained on
campus
5.9%
3.2%
1.3%
Average percent of faculty
trained on campus
16.9%
2.2%
1.4%
Average percent of staff
trained on campus
14.2%
5.5%
3.8%
27
ANALYSIS
Mixed model
◦ Fixed effect
 Year of GLS award
 Pre and post GLS award
 Implementing PAC and Screening activities on campus (timeinvariant)
 Percent of student, faculty and staff trained on campus (timevarying)
◦ Random effect
 Intercept
 Year of GLS award
28
RESULTS
•
•
•
Significant increase in student mental health
service use rates over time (p=0.027)
Campuses that have more staff trained in
GLS show greater increases in student
mental health service use rate (p=0.013)
Campuses that have more students trained in
GLS show greater increases in student
mental health service use rate (p=0.066)
29
RESULTS
Estimate
Standard
Error
DF
t Value
Pr > |t|
Intercept
0.07481
0.00621
81
12.05
<.0001
Years of GLS award
0.00173
0.00077
83
2.26
0.0267
Pre and Post GLS award indicator
0.00465
0.00279
277
-1.66
0.0973
Implementing PAC
0.02173
0.01008
277
-2.15
0.0321
Implementing Screening
0.00054
0.01031
277
-0.05
0.9584
GLS * Percent of student
trained on campus
0.03389
0.01838
277
1.84
0.0662
GLS * Percent of faculty trained on
campus
-0.02363
0.01236
277
-1.91
0.057
GLS * Percent of staff trained
on campus
0.04553
0.01812
277
2.51
0.0126
GLS * Implementing PAC
0.004652
0.0038
277
1.22
0.2219
GLS * Implementing Screening
0.003756
0.00382
277
0.98
0.3268
Effect
30
RESULTS
Percent of Students Using Mental Health Services
on Campus
20%
18%
16%
14%
12%
10%
8%
6%
4%
2%
7.48%
7.94%
Pre-GLS award
Post-GLS award
0%
Out of the total enrollment on GLS-funded campuses included in this
analysis (N=1,382,556), an additional 0.46% of students received mental
health services following GLS award (p=0.097)
31
RESULTS
10% staff trained
7.94%
10% students
trained
7.82%
Pre-GLS
7.48%
7.2%
7.3%
7.4%
7.5%
7.6%
7.7%
7.8%
7.9%
8.0%
Percent of Student Using Mental Health Services on Campus
32
AD-HOC: MODEL RESULT BY SIZE OF
CAMPUS
Size of Campus
Small Campuses Medium Campuses Large Campuses
Effect
Estimate
p
Estimate
p
Estimate
p
GLS * Percent of students
trained on campus
0.070
0.001
-0.014
0.823
-0.099
0.051
GLS * Percent of faculty
trained on campus
-0.056
0.002
-0.010
0.863
0.020
0.498
GLS * Percent of staff trained
on campus
0.102
0.001
0.042
0.333
0.026
0.265
33
DISCUSSION
•
•
•
There is evidence supporting the increase in mental
health service use over years, however, no strong
increase after the receipt of GLS award.
The increase in mental health service use appears
to be related to the number of staff trained on
suicide prevention on campus.
Similarly, the number of students trained was
marginally associated with the increase in mental
health service use.
34
DISCUSSION
•
•
•
Staff are potentially more knowledgeable about the referral
process and how to navigate university policies and
procedures regarding mental health services for students.
Staff may be more likely come into contact with a broader
spectrum of students and therefore are more likely to
encounter youth who are in need of mental health services
than student gatekeepers.
Staff may be more likely to serve in traditional gatekeeper
roles (e.g., student counseling or advisor), thus leading to the
significant effects for staff rather than faculty
35
DISCUSSION
•
•
It seems that the relationship between training and
student mental health use is likely driven by small
campuses
Small campuses have higher training rates of
students, faculty and staff, and therefore are
potentially more likely to reach “saturation” than
medium and large campus
36
DISCUSSION
•
Future exploration will investigate “saturation”
threshold training rates across campuses
37
MOVING FORWARD WITH THE
SBHF
PURPOSE OF SBHF
•
Includes collection of information on the implementation
of Suicide Safer Environment care practices from
campus health care providers as well as the annual
number of suicide attempts and deaths as a source of
long-term outcome data
39
SBHF IMPLEMENTATION AND LOGISTICS
•
All campus grantees will participate in the SBHF in early summer
•
The SBHF is a web-based form on the SPDC
•
The SBHF administrator will receive a password to access the form
•
Data in the SBHF will be reported for the current academic year (approx.
September 2015- May 2016) and the FOUR academic years prior to the grant
•
At the end of the grant, you will have data for a total of 7 academic years
40
SBHF CONTENT
•
The instrument includes a mix of quantitative questions (e.g., how
many suicide attempts occurred on campus?) and policy related
questions (e.g., after a suicide attempt, what happens?).
41
SBHF DATA ELEMENTS
•
•
•
•
•
•
Student use of behavioral health services (including counts
of specific services)
Campus use of standardized screenings (and screening tool)
Linkages to community providers
Follow-up support
Student suicide attempts
Student deaths by suicide
42
SBHF TIMELINE
Identify a SBHF
Administrator to
compile and enter data
Determine sources of
information for data
elements– this may
require coordination
with multiple on and
off-campus sources
Now
An email will be sent to
the project staff to
identify the SBHF
administrator; please
respond!
SBHF Administrator will
receive an email with a
password for the webbased survey
The Administrator will
have several weeks to
complete the data entry
for the current year and
the four years prior to the
grant
Next few
weeks
Approximately
June
TAL will
confirm SBHF
Administrator
Administrator
will complete
data for
academic
year
Spring 2017Spring 2018
43
QUESTIONS ON SBHF?
44
YOUTH EXPLORATORY SERVICES
INTERVIEW (YESI)—CAMPUS VERSION
PURPOSE OF YESI
•
The YESI—Campus Version includes information on the
Continuity of Care experiences of students identified as
at risk for suicide by GLS trained campus gatekeepers.
The protocol is used to understand information about the
(1) identification (gatekeeper) and referral process, (2)
perceptions of the identification, (3) follow-up care
receipt or barriers to receipt (4) current emotional and
overall health, and (5) participant demographics.
46
YESI IMPLEMENTATION AND LOGISTICS
•
Campus grantees will be solicited by the Campus TAL and the YESI Study Lead to
determine interest in participation
•
ICF will work with selected grantees (4-5 campuses) to secure local level
approvals and train local campus counselor center staff on YESI protocols (May
2016 – September 2016)
•
Nine YESI interviews will be conducted from October 2016 – May 2017 by ICF
(via trained crisis counselors)
•
YESI data will be analyzed, synthesized, and reported to SAMHSA in aggregate
47
YESI CONTENT
•
The instrument includes a mix of quantitative (e.g., how many times
have you been seen by campus counseling center services?) and
qualitative related questions (e.g., thinking back on that
identification by the campus gatekeeper, how, if at all, would you
have changed that experience, process or discussion?).
48
YESI NEXT STEPS
•
We’ll be focusing first on selecting grantees to participate
Need to have a campus counseling center and conduct gatekeeper
trainings
• Grantee and broader school needs to be interested/willing to
participate
•
•
To begin this process, your TAL, Jessica Wolff will send a broad
communication about the YESI to all campuses
• The YESI lead may also reach out to specific campuses based on our
reviews of grantee programs
• If you are interested, or would like to hear more information, talk to me
or Christine Walrath here at the meeting, or reach out to Jessica Wolff
directly
49