2013
Kentucky
Victim
Assistance
Academy
Western Kentucky
University
Department of Social
Work
KENTUCKY VICTIM ASSISTANCE ACADEMY:
2013 EVALUATION REPORT
Submitted by:
Patricia Desrosiers, Ph.D., LCSW
Principal Investigator/ Project Manager
Christopher James, MSW
Co-Evaluator/Assistant Project Manger
Western Kentucky University
Department of Social Work
1
KENTUCKY VICTIM ASSISTANCE ACADEMY: 2013 EVALUATION REPORT
Introduction
Last month, the inaugural Kentucky Victim Assistance Academy, KVAA, took place at Barren
River State Park in Lucas, Kentucky. This academy has been developed over the past two years
by professionals who work with victims from different areas across the state. Victim Assistance
Academies started at the national level in 1994 and have since branched out into individual state
academies. There were approximately 50 people who attended this year’s academy in Barren
County.
This week-long training provided victim advocates with tools and skills they could apply in their
work with victims of crime. Before arriving at the academy, each student was required to
complete approximately eight hours of pre-work that covered introductory material and the
history of victim advocacy in the United States. The week began by covering the basics of
victim advocacy; including topics such as the Victim Bill of Rights, confidentiality, mandatory
reporting, and trauma informed care were covered. The academy then focused on the complex
ins and outs of the criminal justice system and how a working knowledge of this system can be
beneficial for victim advocates. On day 3 attendees learned about victim compensation, the
parole board, restorative justice, and the many impacts of crime. The next main section of the
academy gave attendees the opportunity to learn about how to effectively work with victims who
have disabilities, immigration or language access challenges, or cultural barriers that reduce
service utilization. The academy concluded with information on crisis management, vicarious
trauma, and provider self-care.
Throughout the week there were several activities which deserve highlighting. These include a
question and answer session with a panel of veteran victim advocates, a mock crime scene which
led to a mock trial, and even a visit from a nationally recognized victim advocate who helped
start the first National Victim Assistance Academy, Anne Seymour. We received a great deal of
positive feedback from participants and have begun work on preparing for next year’s academy.
2
KENTUCKY VICTIM ASSISTANCE ACADEMY: 2013 EVALUATION REPORT
Presentation of Results
The evaluative results from the 2013 academy are displayed in four sections. These sections are
divided into the pre-test and post-test comparison, evaluation data submitted by participants, and
recommendations for next year’s Academy.
I. Pre-test and Post-test Comparison
At the beginning of the academy, each participant submitted a completed pre-test. This test was
composed of 27 multiple choice and true or false questions. Participants were instructed to
complete this test before they began work on their pre-work assignments. The graph below
shows the average score for each question on the pre-test in blue. On the last day of the
academy, participants were required to complete a post-test. Again, this test was 27 multiple
choice and true or false questions, but was different that the pre-test. Although the specific
questions varied from the pre-test, they did come from the same content areas which allowed for
a direct comparison between the test questions. The average scores for each section on the posttest are displayed in the graph below in red. Investigators chose to use two different
examinations to safeguard against threats to validity, specifically the testing effect.
Investigators utilized a t-test to determine if the differences between the mean scores on the pretest and post-test were significant. The pre-test mean with 48 tests (N=48) was 67.28 %; the posttest mean with 45 (N= 45) was 73.09%. The pre-test scores ranged from 41 to 85%, and the posttest scores ranged from 48 to 100%. An independent samples t-test showed that the two groups
significantly differed in their average change of test score, t (1, 91) = -2.607, p = .011.
Percent Correct
Pre/Post Test Question Comparison
1
0.9
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0
PreAvg
PostAvg
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27
Test Question Number
3
It should be noted that 6 participants left on Thursday afternoon skipping the Friday morning
training. Those participants did complete the post-test, but some of them did so in less than 5
minutes.
There was one comment on the pre-test:
“Pre test was insightful and helpful.”
II. Participant Evaluation Forms
In this section, participant evaluations of each module will be described. A graph of the ratings
for each module will be offered. The average ratings for module content and module presenters
will be separated. Additionally, these ratings will be discussed, and select participant comments
will be offered to clarify the ratings further. Lastly, a discussion of the overall academy ratings
will be provided.
A. Pre-Work Module
Rating
Pre-Work Module
6
5.5
5
4.5
4
3.5
3
1
2
3
4
5
6
Module Number
The table above depicts the pre-work modules 1-6. The average module scores ranged
from 4.43 to 4.66 with 5 being the most commonly given score for these modules. The
pre-work modules were self-study, so there were no presenters. The median score for the
pre-work modules was also 5.
Comments about the pre-work predominantly related to the amount of work being an
unrealistic expectation. Many stated that the pre-work lacked organization. One
commenter summed these views up well. They wrote:
4
“Unrealistic to expect to have 8 hours to do this. Plus, very difficult to follow and
decipher pre-work sections. Numbers did not correlate. One page was missing entirely
from my handout.”
Four participants reported a dislike of the Intimate Partner Abuse online training. In
addition, several participants found the pre-work to be “informative” and “insightful.”
B. Module #1
Module #1 - Content
6
Ratings
5.5
5
4.5
4
3.5
3
1
2
3
4
5
6
Section
The table represents the average scores for the content that was presented in module 1.
As you can see, the averages range from 4.5 to 4.7. The most common score for content
sections 1.1 to 1.4 was 5, section 1.5 was 4, and section 1.6 was 6. The median for all
content sections in module 1 was also 5. Content section 1.1 had a range of 2, sections
1.2 through 1.5 produced a range of 3, and section 1.6 had a range of 4.
Module #1 - Presenters
6
Ratings
5.5
5
4.5
4
3.5
3
1
2
3
4
5
6
Presenter
5
This chart represents the average scores for each of the presenters in module 1. The
average ratings ranged from a minimum of 4.52 to a maximum of 4.81. Presenter 1
received the highest rating, while presenter 2 received the lowest rating of this group of
presenters. The median and mode scores for all presenters in module 1 were both 5.
Ratings for presenter 1 had a range of 2; presenters 2 through four had a range of 3, while
presenters 5 and 6 had a range of 4.
Written comments focused on needing more direct explanation of the week’s
organization, content being dry and lengthy, and suggestions for change. Participants
suggested “more interactive activities”, “moving the panel to the end of the week because
knowledge stems questions,” and “have someone give an overview of the topics to be
covered over the five days to begin.” The instructors for this day were characterized as
“very warm and engaging” and “good speakers!!”, and the content was called “a great
deal of good information.”
C. Module #2
Module #2 - Content
6
Rating
5.5
5
4.5
4
3.5
3
1
2
3
4
5
Section
6
7
8
9
As evident by the chart above, content areas in module 2 had more variation in ratings as
compared to the previous two modules. The average content ratings for module 2 ranged
from 3.87 to 5.4. The median score for content sections 2.1 to 2.7 was 5, section 2.8 was
4, and section 2.9 was 5.5. The mode for sections 2.1 through 2.6 was also 5; while the
mode for section 2.8 was 4 and sections 2.7 and 2.9 was 6. Content sections 2.1, 2.3, 2.4,
2.6, and 2.7 all produced a range of 3, while sections 2.2 and 2.4 both produced a range
of 4. Section 2.8 had the highest range at 5.
6
Module #2 - Presenters
6
Ratings
5.5
5
4.5
4
3.5
3
1
2
3
4
5
6
7
8
Presenter
9
10
11
12
13
This chart represents the average ratings each of the presenters in module 2 received from
participants. The average scores ranged from a low of 3.87 to a high of 5.41. The
median rating for the majority of presenters in module 2 was 5. Exceptions include NM
who received a median rating of 3.5 and KS and TLW who both received median ratings
of 6. The mode of the ratings for the most of the presenters in this module was also 5.
However, the mode of ratings for NM was a 4, while the mode of ratings for VW, KS,
MLW, JH, and MB was 6. The range of ratings for presenters in module 2 was primarily
(75%) 3 and 4, with there being slightly more presenters with a range of 3.
Comments centered around the varying presentation styles of this module. The material
when coupled with interactive activities, such as the mock grand jury, the mock trial, and
the jeopardy game, seemed to better clarify material for participants and keep their
attention. Participants stated:
“Grand jury info was a little confusing. The mock grand jury did help. The mock trial
was very good, kept everyone’s attention.”
“The mock trial was not only entertaining, but also very informative. Speakers on this
day were very interesting and energetic.”
Participants appreciated having hard copies of the PowerPoint presentations that were
being used. Suggestions from participants included having the Kentucky collegiate mock
trial team perform the mock trial, have a Kentucky parole board representative speak on
their section, and they wanted more information from the law enforcement and
community advocacy sides of working with victims.
7
D. Module #3
Rating
Module #3 - Content
6
5.5
5
4.5
4
3.5
3
1
2
Section
Module 3 contained only two different sections of content. The average ratings for the
sections in this module were the highest among training modules and were both
calculated at approximately 5.7. In addition, the median and mode scores for content in
module three were also highest and were both 6. The ranges of ratings in this section
were both 3.
Module #3 - Presenter
6
Rating
5.5
5
4.5
4
3.5
3
1
2
Presenter
The two different sections of content in module 3 were both presented by the same
presenter. Similar to the content ratings, the presenter in module 3 received the highest
ratings among all the presenters at the conference. The average ratings for each section
the presenter covered were 5.76 and 5.79. The median and mode of the ratings was
calculated for both sections and were found to be 6. In addition, the range of scores for
this presenter was 2 for both sections that were presented.
8
Participants seemed to thoroughly enjoy hearing from the national victim advocate
expert, Anne Seymour, and not one negative comment received. They particularly
enjoyed the “interactive” and “energetic” presentation style and several commented that
they hope she is able to return in the future. To quote one participant, they found this
module to be “awesome, informative, and holistic.”
E. Module #4
Rating
Module #4 - Content
6
5.5
5
4.5
4
3.5
3
1
2
3
4
Section
The content areas in module 4 received average ratings ranging from 4.5 to 5.19. The
median score of the content ratings for each section is this module was 5. The mode of
the ratings was 5 for sections 4.1, 4.2, and 4.4, while section 4.3 had a mode of 4. The
range of ratings for section 4.1 was 2, section 4.2 was 5, and ratings for sections 4.3 and
4.5 had a range of 4.
Module #4 - Presenters
6
Rating
5.5
5
4.5
4
3.5
3
1
2
3
Presenter
4
5
There were a total of five different presenters in module 4. The average ratings for
presenters in this section ranged from approximately 4.7 to 5.3. The median and mode of
the ratings was 5 for presenters 1, 2, 3, and 5, while presenter 4 received a median and
9
mode rating of 6. The ratings for presenters 1 and 2 had a range of 2, presenter 3 had a
range of 3, and scores for presenters 4 and 5 had a range of 3.
Again, the participants reported an overall appreciation for the interactive activities,
having PowerPoints available to follow along, and felt the information was applicable.
There were a few participants who reported the immigration and disabilities sections of
this module were too lengthy. A good representative comment given by a participant
was:
“I enjoyed the activities that the disabilities presenters had us do. I enjoyed the video clip
and information on when/how to use an interpreter.”
F. Module #5
Rating
Module #5 - Content
6
5.5
5
4.5
4
3.5
3
1
2
Section
3
Module 5 had a total of 3 content sections. The average ratings for these sections ranged
from 4.9 to 5.1. Similar to some of the previous sections, the median and mode scores of
the ratings for each of these sections were all the same; the median of the scores was 5
and the mode was 6. In addition, the range of the ratings for each of these sections was 3.
Rating
Module #5 - Presenters
6
5.5
5
4.5
4
3.5
3
1
2
3
4
Presenter
10
There were 2 different presenters for module 5. Presenter 1, 2, and 4 in the chart is the
same person. The average ratings for these two presenters ranged from 5.22 to 5.34. The
median score of the ratings for the first presenter was 5, while the ratings for the second
presenter had a median score of 5.5. The mode of the ratings for both presenters was 6.
The rating scores for both presenters primarily had a range of 3; however, presenter one’s
scores for the first section had a range of only 2.
Comments about module five were a good mixture of both positives and suggestions for
improvement. Several participants reported they enjoyed the small group activities and
found the videos helpful to apply the information. There were also several comments
about providing more emphasis on vicarious trauma and self care when working as a
victim advocate. One participant had particularly strong feelings about this module,
stating:
“It’s not okay to push vicarious trauma and self care out for more information on other
subjects. That’s too often what we do anyway; push that important stuff out for other
work stuff. Maybe keep Friday all about these topics. Don’t set a bad example!”
G. Overall
Rating
Overall Evaluation
6
5.5
5
4.5
4
3.5
3
1
2
Section
3
At the end of the evaluation form, there were three questions that addressed the academy
as a whole. The average score of all three of these questions was 4.66. The first question
asked participants to rate the organization of the academy. The median score of these
ratings was 5, while the mode was 6. The second question in this section asked
participants to rate whether or not the academy was a good use of their time. The median
and mode of these ratings were both 5. The third question in this section asked the
participants to rate whether or not the content presented at the academy was appropriate
11
to their need. The median and mode scores of these ratings were both 4. The range of
scores for all three of these questions was 4.
Ratings
Overall Averages
6
5.5
5
4.5
4
3.5
3
Content
Presenters
Averages
The overall average of the content from all modules, including the pre-work, was 4.95.
The overall average of the ratings for the presenters was slightly higher at 5.05.
At the end of the academy, participants provided a great deal of suggestions and
comments on how to improve for future years. One comment that seemed to reoccur was
that the advocates would have liked more information on EPOs and DVOs. Another
suggestion that was given numerous times was to provide an opportunity for break-out
sessions so the advocates can choose to learn more on topics they are interested in.
Several participants also reported that the quality of the audio in the conference room was
very poor. A few participants felt it would have been helpful to have actual victim’s
stories that applied to each particular section throughout the week. Many of the
participants also noted in their comments that they felt like the conference was a great
networking opportunity. Comments that summarized the group’s feedback about the
academy included:
“Improve the audio. I absolutely enjoyed this academy. Being such a new advocate, this
material was very needed.”
“The sessions were good, however, it would have been nice to have modules to choose
from, as much of the material was previously known while other parts were completely
new. Small breakouts would have been nice. Overall, good first effort.”
“…EPOs, DVOs, more education on domestic violence. Overall, great and very
informative. Great Job!”
12
“Valuable networking. Good, basic information on basic information. Excited about
ability to get provisional national certification (NOVA). Great handouts. Appreciate
speakers who have included emails and business cards for future questions.”
IV. Recommendations for Next Year
A. From Anne Seymour
Anne had the following recommendations for next year’s academy:
1. The general rule for adult learning is to offer at least 10 minutes of interactive
learning activities for every 50 minutes/hour of training (this equates to one-or-two
learning activities per session).
2. It helps faculty if students have name plates, so they can call on participants by their
actual names. You can prepare these in advance with the student’s first name in LARGE
print, and have them “decorate their name plates” when they first arrive in the training
room.
3. At the NVAA and SVAAs I’ve participated in, we move students EVERY day, and
explain that the purpose is to ensure that they meet the entire student body and improve
their network of colleagues (AND shhhh….get out of their “comfort zone” with friends!).
4. RE: seating of students: You can determine up front from student applications, those
who have a few more years in the field. Throughout the week, make sure one of the more
“seasoned” students is at each table (“Teacher’s pets” a bit, but helpful in mentoring the
newbies !)
5. It’s helpful to schedule a five-minute “review/preview” each morning to review what
was learned on the previous day; and highlight what will be learned today.
6. It’s helpful to have a “parking lot” for students to put up sticky notes with
questions/issues they have (which can be reviewed each morning during review/preview).
A parking lot is also helpful for faculty members, who can say “we don’t have time to
address that now, but let’s put it on the parking lot and we’ll make sure to cover it in
review/preview.”
7. Music is essential! In advance, students can be asked to bring their Ipods and play
their music. It is the best way to send folks on breaks, and they know to return when the
music stops.
13
8. It’s helpful for faculty/staff to sit at each student table, instead of in the back of the
room, to the degree possible. They are there to help facilitate any individual/small group
activities, and constantly “check the pulse” of students (to make sure they still have one
)!
9. For a large room, one standard microphone and two lavaliers are needed to make sure
all lectures/comments/questions are heard.
10. I have a large cadre of decorative posters and inspirational quotations that can be
hung high on all walls in the training room – they add a pop of inspiration and color!
11. It’s a good idea to invest in three-to-four boxes of Mr. Sketch markers – have four
markers on each table, and a bunch up front for presenters.
12. AND if faculty/staff are willing, it’s really nice to have a morning or evening walk,
led by a faculty or staff member. 30 minutes and slow, good bonding activity!
I personally agree with all of Anne Seymour’s recommendations, and they would cost
less than $500 to implement. I recommend we implement all 12.
B. From Participants
At the end of the academy, participants provided a great deal of suggestions and
comments on how to improve for future years. These comments were summarized in a
previous section but will be repeated here.
1. More use of the Participant Manual/PreWork concepts and materials during the
training
2. Less, no, or improved PreWork
3. Break-out sessions so the advocates can learn about choice topics
4. Improved audio quality
5. Actual victim’s stories or a victim panel that applied to each particular section
throughout the week
6. More networking opportunities, have participants tell their name each time they
speak
7. Resource handout for advocates
8. More information on EPOs and DVOs, trauma informed care, vicarious trauma &
self-care, law enforcement role, crisis management, KCVC, and the resort, but less
content in general.
9. Less information on immigration, death notification, psychiatric specifics —too
detailed
10. Less antagonism towards offenders—felt the mother of 6 activity helped
14
11.
12.
13.
14.
15.
16.
17.
More interactive activities, games, movement (yoga/meditation)
PowerPoint handouts
More time with advocate panel and at the end of the week
Shorter days, more frequent and longer breaks, free coffee
KY Parole Board there in person
Anne Seymour to come back
Graduation at the end of the Academy with certificate presentation
These recommendations are reasonable, but somewhat conflicting. It is a challenge to
provide more information on 7 topics (less on 2) in less time with more breaks. It is
difficult to integrate PreWork concepts when they don’t have time to do PreWork. It is
hard to network more when you are divided into breakout sessions.
C. From Dr. Desrosiers
Taking into account all of the feedback given from Anne Seymour, the participants, and
the steering committee members my recommendations are as follows:
1. Curriculum
A. Shorten the Immigration Visa section from 2 hours to 1 hour making it more basic
B. Add that 1 hour onto the Vicarious Trauma and Self Care Module
C. Change the Victim Scenario into a DV situation with an EPO required to pull in
that content.
D. Ensure that each 50-60 minutes of class includes at least 10 minutes of
movement/art/interaction. It is best if that occurs in the middle of the hour.
E. Hold a Train the Trainer full-day event in February. We have wonderfully
passionate and knowledgeable experts, and it would be best for them to have the
skills to develop their own activities. Train the Trainer would benefit all. Anne
Seymour has agreed to do this if we schedule her immediately. Alternatively, that
is a service that OVC TTAC can provide for free. It is referred to as the “Ultimate
Trainer”, and we can request Anne with no guarantee of getting her. We may get
a different trainer, and that service also requires 5-6 months advance notice.
F. Revise the instructor manual following the Train the Trainer event in February. At
the event the Trainers will add their activities and complete their PowerPoints
providing them to KVAA staff at that time.
G. Add 2 Breakout sessions (a 1 hour on Wednesday and a 2 hour on Friday) to give
participants choices about what information they need.
2. Academy Training Schedule
A. Format of the Academy could be 5 full days of 9-noon and 1-4 with 4-5pm
reserved at the end of the day for completing quiet work. Participants may choose
15
to take this work home for completion each day. 8-9am daily will be reserved for
networking activities, sign in, and review of learning each morning.
B. Add a first training day and mid training day 2.5 hour required evening activity.
The first training night will be a great Keynote Speaker following a victim panel
with either a variety of victims or a specific type of victim (other Academies have
had great luck with that). Mid training will be a fun activity like the history
jeopardy. Mandatory lunch on the second training day will include a special
speaker on a select topic. It would be wonderful if we could get a sponsor for that
second training day lunch and/or the evening activities. This equals 40 hours face
to face time if you count the 4 hours of quiet work and the Graduation Ceremony
(and I do).
C. Move the advocate panel to the last morning.
D. Rework the sequencing of the curriculum placing lighter topics next to heavier
ones to improve the flow of the training and reduce participant fatigue. See the
suggested Agenda for specifics.
E. Add a graduation ceremony at the end of the Academy where certificates are
given out and the week is celebrated.
3. Venue and Instructors
A. Keep the venue the same to reduce the time and effort involved in building a
relationship and learning the processes of a new vendor.
B. Provide participants with specific instructions related to comfort at the venue
(such as bring a sweater or jacket and snacks) as well as information about local
restaurants and points of interest.
C. Implement morning walks Tuesday to Friday to increase participant energy levels
and networking.
D. Have all Instructors go through one central person for instructions on
booking/payment information/etc. to reduce confusion.
E. Have a different type of victim advocate as the resident expert each day to make
sure that both community and legal advocacy get equal time.
F. Purchase or rent a second lavalier, standing microphone, and some good speakers
to use at this yearly training. We need better sound for sure. We should also offer
music between sessions to up the energy level and signal session is starting.
G. Get topic specific posters for the room, and bring Tanya’s Training Kit.
4. Participant Manual, handouts, and PreWork
A. Reduce PreWork due prior to the Academy from 8 to 3 hours requiring only the
Kentucky Crime Victim Bill of Rights, Federal Court System, AND PreTest
sections prior to the Academy. Place the other 4 PreWork Modules on the end of
each day as work they will do in class during a quiet study time.
B. Improve the quality of the Participant Manual by clarifying the instructions,
explicitly stating the purpose of the information/PreWork, and adding a resource
16
guide with names, addresses, and phone numbers of advocate resources across the
state.
C. Provide PowerPoint slide handouts OR Note Taking Guides for participants to
facilitate note taking and reduce anxiety of participants. These can be provided
online as we did this year with the PreWork.
These recommendations are lengthy; however, I think it is reasonable to implement them all over
the course of the coming year. The large number of suggestions will certainly facilitate a fruitful
steering committee discussion as we determine the most appropriate improvements to implement
for the 2014 Kentucky Victim Assistance Academy curriculum.
With the success of the first year academy, it is tempting to leave the well enough alone. It is
likely that these revisions and refinements will make this Academy one of the best in the
country. Thanks to everyone involved for your hard work in making the 2013 KVAA a huge
success.
This document was produced by the Justice and Public Safety Cabinet and
Western Kentucky University under 2011VFGXK004, awarded by the Office for
Victims of Crime, Office of Justice Programs, U.S. Department of Justice. The
opinions, findings, and conclusions or recommendations expressed in this
document are those of the contributors and do not necessarily represent the
official position or policies of the U.S. Department of Justice.
17