Promoting Quality
Prevention Counseling
Project:
What have we learned?
Spring 2005 Texas Tour
Dallas, Fort Worth, Houston, Midland,
Austin
Agenda
 Welcome
and Introduction
 Background
 Overview of project
 Implementation experiences by sites
 General evaluation findings
 Next Steps
Q & A
Background
 Revised
HIV
Counseling,
Testing and
Referral (CTR)
Guidelines,
November
2001
Background Cont’d
 CDC’s
Project RESPECT:
Evidence-based intervention
showing significant reduction of
STDs with protocol-based HIV
prevention counseling
 RESPECT-2: Refined HIV
prevention counseling protocol &
further developed “counseling
quality assurance” methods
RESPECT Methodology
 5758
heterosexual, HIV-negative
patients older than 14 years who
came in for STD examination
 Five public STD clinics (Baltimore,
Denver, Long Beach, Newark and
San Francisco)
Project RESPECT Results*:
New STD diagnoses
HIV Prevention Counseling Effective
250
225
200
175
150
125
100
75
50
25
0
211
173
149
107
12 months*
6 months*
RESPECT
Control
(*p<0.05)
Kamb, M.L., et al (1998) Efficacy of risk-reduction counseling to prevent human
immunodeficiency virus and sexually transmitted diseases, JAMA, 280 (13):1161-1167
How could we translate this
intervention into a real-world
setting?
 Risk
Reduction Specialist
support
 Supervisor support
 Practical tools
Goals of the Project
 Develop
and evaluate tools to
support protocol-driven prevention
counseling based on the RESPECT
model
 Develop and evaluate QA
procedures
 Better understand the barriers and
facilitators of good prevention
counseling
Definitions

Evidence-based interventions
 Interventions
that have demonstrated desired
outcomes through rigorous research

Core elements
 Components
of the intervention that are believed
to be essential to achieve the desired behavior
change

Protocol
A
structured approach to achieve core elements
Definitions

Client-centered prevention counseling
 One-on-one
interactions with risk-reduction as its
primary goal

Risk Reduction Specialist
A
trained specialist responsible for maintaining the
focus on a client’s specific risk reduction needs

Tools
 Job
aides to ensure fidelity to core elements of
protocol
Site Locations
Tarrant County
Health Dept.
Ft. Worth, TX
Resource Center
Of Dallas
Dallas, TX
City of Laredo
Health
ccDept.
Laredo, TX
Valley AIDS Council,
McAllen,
cc TX
Brownsville, TX
What was introduced during the
project?
Counseling protocol
 Training on the protocol
 Counseling tools

 Spiral
book with goals and sample
questions
 Laminate “wheel”
 Documentation form with space for RR
plan and referrals
 Personal review form
Session Documentation Form
What QA activities were part of
the project?

QA protocol
 Emphasizing
standardized preceptorship,
observation, routine meetings, documentation
review, and feedback on observations and
documentation

QA tools
 Supervisor
observation tool
 RRS self assessment
 Chart abstraction and summary tools
Quality Assurance Tools
Comparison of Core Elements
GOAL
PCPE
RESPECT
Protocol
1:1 Counseling
Yes
Yes
Yes
2-Session Model
Yes
Yes
Yes
Follow Protocol
No
Yes
Yes
Focus: Client Risk Behavior
Yes
Yes
Yes
Increase self-perception Risk
Yes
Yes
Yes
Negotiate Realistic RR Step
Yes
Yes
Yes
Provide Referrals
Yes
Yes
Yes
Support Test Decision
Yes
Yes
Yes
Interpret test results
Yes
Yes
Yes
Partner Elicitation/ Referral
Yes
No
Yes
Standardized QA Procedure
No
Yes
Yes
QA Tools Specified
No
No
Yes
Session Tools
No
Yes
Yes
Implementation
Training developed for protocol, tools and
QA
 Supervisors and Risk Reduction Specialists
trained in October 2003
 On-site and off-site TA provided for startup
 Staggered and tailored implementation of
protocol, tools, and QA
 Additional ongoing TA after start-up

What Do We Want to Learn?



Can you implement protocol-based
prevention counseling with existing
resources?
Did the protocols and tools help them
implement the intervention with fidelity?
What were some of the facilitators and
barriers of the implementation of the
protocols?
Evaluation Design
Two data collection phases: Pre- and Postintervention
 Evaluation data triangulation: 9
complementary quantitative and qualitative
instruments

Quantitative Instruments
Supervisor time logs (pre and post)
 Client Questionnaires (pre and post)
 Counseling chart reviews (pre and post)
 Observations of counseling by evaluators
(pre and post)

Qualitative Instruments





Risk Reduction Specialist (pre and post)
Supervisors (pre and post)
Site Program Managers (post only)
WAP (post only)
TDH (post only)
Results
Lessons Learned
Did the protocols and tools help
implement the intervention with
fidelity?
Observations: Initial Session
Goals
1
0.9
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0
t
or
RR
r*
de
in
em
tr
p'
*
Ap
on
i
*
ls
cis
rra
de
fe
st
re
Te
nd
ta
or
pp
Su
an
pl
RR
s*
rn
tte
*
Pa
pp
Su
*
sk
Ri
*
cd
nt
ce
Re
hn
En
*
tro
In
*P<.05
Post
Pre
Observations: Follow-up Session
Goals
1
0.9
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0
m
Su
pp
Su
&
&
e*
os
Cl
s*
al
rr
fe
Re
*
ep
st
*
lts
su
re
*P<.05
e
iz
ar
t
or
te
tia
nd
*
ep
st
ta
Post
m
go
Ne
n
rie
ew
vi
Re
O
Pre
Client survey: Initial session
Goals
100
90
80
70
60
50
**
isk
R
*P<.05
R
R
st
Pa
k
ris
n
tio
ua
*
of
b
jo
n
tio
ep
sit
t
en
fe
ec
R
Li
rc
Pe
e
os
rp
Pu
in
a
pl
Ex
Post
Pre
Client survey: Initial session
Goals (cont’d)
100
90
80
70
60
50
t
en
tm
n
n
oi
a
pl
p
Ap
RR
*
t*
or
l
ra
er
f
Re
pp
Su
*
Pre
Post
*P<.05
Client survey: Follow-up session
Goals
100
90
80
70
60
50
R
R
*P<.05
an
pl
l
ra
er
ef
R
*
rt
o
pp
Su
k
or
ly
*
an
pl
w
d
ie
hy
W
Tr
r
ea
ob
J
p.
Post
ts
ul
es
R
Ex
Pre
Client Surveys: Client Participation
Me
Both
RRS
100%
80%
60%
40%
20%
0%
*P<.05
Talked more
(Pre)
Talked more
(Post)*
Made plan
(Pre)
Made plan
(Post)
Chart reviews
Goals
100
90
80
70
60
50
40
30
20
10
0
Pre
Post
Percp.
Risk*
Recent
risk*
Pattern*
Past RR*
*P<.05
Significant changes seen in initial sessions
Goals
Observation
Client Report
Documentation
Introduction/Orientati
on
X
Enhanced risk percept.
X
Recent risk discussed
X
Reviewed past RR
X
X
Sum up pattern of risk
X
X
X
X
Risk reduction plan
X
X
Support and referral
X
Test decision
counseling
X
Appt and reminder
X
X (2)
X
X
33
Significant changes seen in follow up sessions
Goals
Observation
Orient and give results
X
Review RR efforts
X
Risk reduction plan
X
Support and referral
X
Summarize and close
X
Client Report Documentation
X
X
X
X
34
“That you had to follow every single task
even though they didn't all apply to
everybody [is a problem]. Protocol doesn't
allow for individual counseling styles or use
of skills RRS's have received at prior
trainings…it seems cumbersome and
redundant to use this protocol with clients
with very few risks - although it's easy
enough to move through the protocol by
saying this doesn't really apply to you [for
certain tasks].”
-Risk Reduction Counselor
“[The protocol improved the quality of
my counseling] because I had a
structure to make sure I wasn’t leaving
anything out.”
-Risk Reduction Counselor
Observations: Initial Visit
Use of Counseling Skills
3
2.5
2
1.5
1
0.5
0
e
os
cl
&
e
iz
ar
m
m
*
Su
tC
or
s
*
ill
pp
sk
ns
Su
io
ild
ct
bu
re
di
pp
ot
O
n
l
ns
ta
io
en
pt
O
em
dg
ju
nNo
fo
in
e
pl
ng
m
ni
Si
te
lis
e
tiv
d*
Ac
de
en
n
*
pe
RR
O
on
te
s
ta
cu
ls
Fo
na
io
ot
em
C
Pre
Post
*P<.05
Observations: Follow-up Session
Use of Counseling Skills
3
2.5
2
1.5
1
0.5
0
e*
os
cl
&
e
iz
ar
m
m
Su
tC
or
s*
*
ill
pp
sk
ns
Su
io
ild
ct
bu
ire
pp
td
O
no
l
ns
ta
io
en
pt
O
em
dg
ju
nNo
fo
in
e
pl
ng
m
ni
Si
te
lis
e
tiv
d
Ac
de
en
n
*
pe
RR
O
on
te
s
ta
cu
ls
Fo
na
io
ot
em
C
Pre
Post
*P<.05
Client Surveys: Client experience
100
90
80
70
60
50
i
Go
od
*
n
hi
lan
*P<.05
op
et
m
so
*
od
to
rs
de
t
ng
un
Post
d
ne
ar
Le
lt
Fe
Pre
g
What were some of the
facilitators and barriers of the
implementation of the protocols?
Overall themes and feedback
 Delivery
of protocol-driven prevention
 Provided structure
 Improves with practice
 Aided in identifying risk behaviors and
patterns
 Protocol questions felt rigid
 More training and TA is essential
Overall themes and feedback
(cont’d)
Spanish version tools are needed
 Supervisor buy-in is essential
 Supervisor’s other responsibilities need to be
considered due to time constraints of quality
assurance
 Difficult with certain clients (such as low
risk, outreach, drug treatment and jail)

Using the Counseling Tools
Most of the RRS found the cards to be the
most helpful of the tools (72%)
 Cards help ensure you cover everything in
order (44%)
 Wheel was not as helpful (83%)
 Spanish version of the tools is needed

Counseling QA by RRS
 Most
helpful
 Observation
by supervisor
 General feedback
 Role play
 Observation by peer/peer
observation/document review
 Case conference
Counseling QA by RRS (cont’d)
 Least
helpful
 Documentation review
 Observation by peer
 Case conference/self assessment
Counseling QA by Supervisor
 Most helpful
 Observation
by supervisor
 General feedback
 Document review
 Case conferences
 Least helpful
 Observation by peer
 Self-assessment
R
ec
ei
ve
Su
d
p
su
Fe
O
p
bs
ed
fe
ba
ed
ck
ba
ck
fo
rm
us
ed
R
ol
e
Pl
D
ay
oc
s
R
ev
D
oc
ie
w
fe
ed
ba
D
oc
ck
w
/
fo
rm
Ca
se
Co
nf
100%
90%
80%
70%
60%
50%
40%
30%
20%
10%
0%
QA Activities by RRS
Pre
Post
“[The supervisor observation form] is better
because it is less subjective and more structured.
…the priorities of the tasks are made clear by the
forms and that feedback using these forms makes
the whole process self-reinforcing [the process of
understanding the expectations of the protocol,
using the protocol, and getting feedback—all
have the same language, structure, and
expectations]…the new feedback is less stressful
for everybody, including the observer for the
stated reasons.”
---Risk Reduction Specialist
I think this new protocol is great—fabulous!
Before when they first told us about the program
and we went to training, we were all ‘iffy’ and
said ‘it’s not gonna work’ ‘no way in heck’ it
would be accepted by the people. Now that we
are implementing it, we are doing a great job.
When you have to write steps, the clients leave
with RR plan in hand, a referral, an appointment
card with the date on it in hand. As for review
forms used by the supervisor on documentation,
etc. You have the form yourself to be able to
discuss ‘met’ or ‘not met.’
---Risk Reduction Specialist
What has been done?

Changes to training
Trainers have bought in
 Preceptorship is done first, then attend training
 More time for role play
 Develop their own questions for each step
 Not a gripe session


Sites learning from each other
Role plays/Peer observations for practice
 Sign in waiting room for length of sessions
 Regularly scheduled QA sessions

What Now?

State-wide roll-out begins May 2, 2005

All DSHS HIV/STD contractors

Roll-out completed by July 2006

Protocol Based Counseling Training (PBCT) replaces
PCPE by August 2006 as the state mandated
prevention counseling course for risk reduction
specialists
HOW??

11-week training and technical assistance cycles
One month of supervisory training and
development
 Two week employee preceptorship
 Three weeks of employee training and TA
 Two weeks of independent implementation and TA


Ongoing technical assistance and support
7 training staff dedicated to rollout initially
 Field operations and regional staff support

When?

Cycle 1 – May 2

SE Texas area agencies

Cycle 2 – July 18
Cycle 3 – September 19
Cycles 4-6 in 2006

Agency selection for cycle based on:





Epi data
Field Ops
Agency Readiness
Programmatic




PBC is the prevention counseling model that
must be used if contracted to perform PCPE or
an ILI as a component of your GLI.
Once committed - no going back to old PCPE.
Changes in the RFP and contract language.
o State 2005/6
o Federal 2006 Federal 2006
Big competitive RFP released Spring 2006 to
start funding state 9/1/06 and federal 1/1/07
Points to Ponder…




Structural - your program overall? Buy-in from
your administration, capacity?
Staffing – Supervision requirements, hiring,
vacancies, current staff
Time for supervision, time for staffing, time for
QA activities, time to perform the sessions.
Who needs to be trained? PCM, EBI, TCADA,
Case Managers?
Budgets





Budgets – how to pay for upcoming trainings,
how to compensate your staff ?
 Outreach workers’ conference, OraSure,
salary savings
One week of Austin training for supervisor and
any team leaders.
All PCPE staff one week in possibly local area.
Possible budget amendments
Start thinking about new budgets for 2006
Workplans






Where are you doing your PC? What populations are you
serving?
What does your PC look like now?
How does this change the structure of the work that you
do?
What type of changes will you need to make in your
workplan?
Look at settings and time and how this will work?
Partner with your fellow providers to perform activities in
various settings.
THINGS YOU CAN DO NOW!




Look at the quality assurance guidelines and
your contract. Are you doing as required now?
Review the PBC tools QA. Please do not use
them until you are trained
What does your orientation plan look like?
Some do a type of preceptorship already.
Are you using the TDH documentation guide
and sample? Are you using a PCPE review tool?
Get them off our web.
Things to do now..



Review your objectives, what do you need to
meet your return rates, link to EI, and PE?
What does your counseling look like now?
Clean RECN data
TA and Monitoring



FO staff will go through Mega-training with
their program
TA provided with Training staff
Monitoring schedule will start six months after
all trained date
Q&A