Palm Beach County Cultural Council
Building Learning
Communities Through
Arts and Culture
Final Report Prepared by
Escoffery Consulting Collaborative, Inc.
May 30, 2011
Escoffery Consulting Collaborative, Inc.
Final Report
Palm Beach County Cultural Council
Building Learning Communities Through Arts and Culture
Final Report
Table of Contents
Introduction
1
Evaluation Activities Conducted
1
Student Data
Grantee Agency Data
2
7
Documentation of Activities
Review and Recommendations
9
10
List of Tables
Table 1 – Students Completing Survey
Table 2 – Students’ Interest and Involvement in the Arts
2
2
Palm Beach County Cultural Council
Building Learning Communities Through Arts and Culture
Final Report
Introduction
The Building Learning Communities Through Arts and Culture (BLCTAC) is a
program developed and implemented by the Palm Beach County Cultural Council. The
program is in its second funding cycle and the consulting firm of Escoffery Consulting
Collaborative, Inc. was retained and asked to develop an evaluation plan, provide
outcome data, and present findings and recommendations for program improvement.
Escoffery Consulting presented an interim report in January 2011 and offered a
detailed description of the program’s aims and activities as well as evaluation
methodologies and deliverables. Therefore, this information is not presented in this
report.
The final report presents a review and analyses of the final data and results
pertaining to changes in students, teachers, and funded agencies. The reports
addresses also the four evaluation questions developed at the beginning of the 201011 funding cycle. Finally, the report presents recommendations for program
improvement based on the data and lessons learned.
Evaluation Activities
The main activities conducted during the third quarter (period covered by the
final report) were the following:
1. Gathering and analyzing data about the students, teachers, and grantee agencies.
 Student Data. The participating students from the three schools completed the
Student Interest in the Arts Survey that asked for their behaviors and interest in
the different art forms. The instrument was administered by the teaching artists
at the beginning (pre-test) and at the end (post-test) of the activities. The data
was analyzed and compared to determine differences in each of the 27 items.
 Teacher Data. The participating teachers were contacted to schedule a ½ hour
telephone interview during the month of May (end of the year interviews).
 Grantee Data. The executive directors, key staff and teaching artists of the three
grantee agencies were contacted to schedule a ½ hour telephone interview during
the month of May (end of the year interviews).
2. Gathering and reviewing documentation of activities.
 Student journals, art samples, and culminating events. The evaluator asked the
teachers and teaching artists for the students’ journals, samples of the artwork,
and documentation of the culminating events.
 Teachers’ lesson plans. At the January workshop and via emails the teachers were
asked to submit a lesson plan showing integration of the arts.
1
Student, Teachers, and Grantee Agencies Data
Student Data
Staff from the grantee agencies asked teachers and/or teaching artists to
distribute the student survey during the months of November and December (pretest) and March (post-test). The pre-tests were completed by 126 students and 91
students completed the post- tests.
Table 1 – Students Completing Student Interest in the Arts Survey
Students Completing Pre-test
Carver
18
Conniston
42
Roosevelt
66
Students Completing Post-test
18
38
35
The completion of the post-tests presented challenges that hinder the
interpretation of the results:
 One of the grantees did not distribute the surveys until after the students had
received the intervention (that is, the teaching artist had started her lessons).
 The number of students completing the pre and post-test was different for two
schools (Conniston and Roosevelt).
 All the students were given the post-test in March whereas one group of students
had completed the funded activities the month before and the other students had
completed the activities in December/January.
The survey responses were analyzed to answer the question: What was the status
of the students’ interest in the arts and their involvement in different art forms prior and
at the end of the BLCTAC Initiative? However, the challenges described make the
interpretation of results difficult.
The table below provides the percent of students selecting “Yes” and “Somewhat
Agree/“Strongly Agree” as their responses to each survey item.
Table 2 – Students’ Interest and Involvement in the Arts
Carver Middle School
Percent of Students Selecting Yes
Survey Item
Pre
Post
With the exception of a music performance at your school, did
33
67
you go to a music performance during the LAST 6 MONTHS?
With the exception of a stage performance at your school, did you
33
50
go to a stage performance during the LAST 6 MONTHS?
With the exception of a dance performance at your school, did
39
50
you go to a dance performance during the LAST 6 MONTHS?
During the LAST 6 MONTHS, did you visit an art museum or
28
22
gallery?
During the LAST 6 MONTHS, did you visit an art fair or festival, or
28
33
a craft fair or festival?
2
Carver Middle School
Percent of Students Selecting Yes
During the LAST 6 MONTHS, did you watch or listen to a
88
72
recorded (TV, Internet, radio, iTunes, MP3) music performance?
During the LAST 6 MONTHS, did you watch or listen to a
65
56
recorded (TV, Internet, radio, iTunes, MP3) stage performance?
During the LAST 6 MONTHS, did you watch or listen to a
89
61
recorded (TV, Internet, radio, iTunes, MP3) dance performance?
Number of students selecting “Strongly Agree” or “Agree”
Pre
Post
I like to watch plays
61
72
I like to act in plays
50
56
I like talking about plays
33
6
I like to help write plays
22
28
Acting in or writing plays makes me happy
44
44
I like to watch dancing
83
89
I like to dance
94
89
I like talking about dances and/or dancing
78
72
I like to create dances
61
72
Dancing makes me happy
72
78
I like to listen to music
100
94
I like to play music or sing
83
89
I like talking about music
78
83
I like writing music or songs
61
61
Listening to music, playing music or singing makes me happy
94
83
I like to look at drawings, paintings or photographs
67
78
I like to draw, paint or take photographs
89
78
I like talking about paintings, drawings or photographs
39
40
Drawing, painting or taking photographs makes me happy
56
56
Note: percentages highlighted in yellow indicate a positive change
Conniston Middle School
Percent of Students Selecting Yes
Survey Item
Pre
Post
With the exception of a music performance at your school, did
57
55
you go to a music performance during the LAST 6 MONTHS?
With the exception of a stage performance at your school, did you
45
53
go to a stage performance during the LAST 6 MONTHS?
With the exception of a dance performance at your school, did
33
46
you go to a dance performance during the LAST 6 MONTHS?
During the LAST 6 MONTHS, did you visit an art museum or
31
68
gallery?
During the LAST 6 MONTHS, did you visit an art fair or festival, or
33
47
a craft fair or festival?
During the LAST 6 MONTHS, did you watch or listen to a
83
76
recorded (TV, Internet, radio, iTunes, MP3) music performance?
3
Conniston Middle School
Percent of Students Selecting Yes
During the LAST 6 MONTHS, did you watch or listen to a
64
63
recorded (TV, Internet, radio, iTunes, MP3) stage performance?
During the LAST 6 MONTHS, did you watch or listen to a
64
50
recorded (TV, Internet, radio, iTunes, MP3) dance performance?
Number of students selecting “Strongly Agree” or “Agree”
Pre
Post
I like to watch plays
76
74
I like to act in plays
64
50
I like talking about plays
38
42
I like to help write plays
38
40
Acting in or writing plays makes me happy
48
50
I like to watch dancing
81
71
I like to dance
64
55
I like talking about dances and/or dancing
33
26
I like to create dances
50
42
Dancing makes me happy
74
71
I like to listen to music
98
97
I like to play music or sing
88
90
I like talking about music
71
79
I like writing music or songs
50
58
Listening to music, playing music or singing makes me happy
93
90
I like to look at drawings, paintings or photographs
81
82
I like to draw, paint or take photographs
76
84
I like talking about paintings, drawings or photographs
55
58
Drawing, painting or taking photographs makes me happy
69
79
Note: percentages highlighted in yellow indicate a positive change
Roosevelt Middle School
Percent of Student Selecting Yes
Survey Item
Pre
Post
With the exception of a music performance at your school, did
52
40
you go to a music performance during the LAST 6 MONTHS?
With the exception of a stage performance at your school, did you
47
37
go to a stage performance during the LAST 6 MONTHS?
With the exception of a dance performance at your school, did
57
43
you go to a dance performance during the LAST 6 MONTHS?
During the LAST 6 MONTHS, did you visit an art museum or
44
40
gallery?
During the LAST 6 MONTHS, did you visit an art fair or festival, or
47
43
a craft fair or festival?
During the LAST 6 MONTHS, did you watch or listen to a
86
69
recorded (TV, Internet, radio, iTunes, MP3) music performance?
During the LAST 6 MONTHS, did you watch or listen to a
77
66
4
Roosevelt Middle School
Percent of Student Selecting Yes
recorded (TV, Internet, radio, iTunes, MP3) stage performance?
During the LAST 6 MONTHS, did you watch or listen to a
82
66
recorded (TV, Internet, radio, iTunes, MP3) dance performance?
Number of students selecting “Strongly Agree” or “Agree”
Pre
Post
I like to watch plays
60
49
I like to act in plays
37
43
I like talking about plays
40
31
I like to help write plays
35
17
Acting in or writing plays makes me happy
39
40
I like to watch dancing
85
80
I like to dance
68
60
I like talking about dances and/or dancing
42
27
I like to create dances
62
57
Dancing makes me happy
65
63
I like to listen to music
99
94
I like to play music or sing
85
83
I like talking about music
83
86
I like writing music or songs
62
43
Listening to music, playing music or singing makes me happy
88
91
I like to look at drawings, paintings or photographs
80
77
I like to draw, paint or take photographs
83
71
I like talking about paintings, drawings or photographs
59
49
Drawing, painting or taking photographs makes me happy
72
63
Note: percentages highlighted in yellow indicate a positive change
Teacher Data
All the participating teachers were contacted via email and telephone to schedule
the end-of the year interview. The evaluator was able to conduct ½ hour interviews
with three teachers representing Carver and Roosevelt Middle Schools (Tonisha Cook
and Perry Stokes from Carver and Todd LaVogue from Roosevelt). Two of the
teachers did not participate in the interview process: Ms. Donna Mellius (Conniston)
did not return telephone calls and emails and Ms. Angela Price (Roosevelt) was
scheduled twice for the interview but did not answer the telephone at the scheduled
time.
The interview protocol included questions about the program’s impact and
suggestions for improvement. The completed interviews were analyzed and the
evaluator identified the common themes and differences.
Common Themes
 Level of comfort incorporating arts in the social studies curriculum. All the
teachers stated feeling more comfortable now than they did at the beginning of
5
the school year. All the teachers now incorporate at least one art form into their
lessons plans and some examples include:

Todd La Vogue: Africa – students created their own Kente patterns using
paper and each African “family” created its own pattern and colors. Greece –
students made an imaginary dinner party (Dinner at the Parthenon) and the
students created the invitation and researched the guests to be included in
the guest list. Rome – a majority of the students chose to create their own
board games (questions and answers on the subject they were studying).
Tonisha Cook: Greece – students “created” a Greek tragedy incorporating the
lesson and performed it.
 Benefits of integrating the arts in the social studies curriculum. Integrating the
arts requires more time and resources but all the teachers expressed it is worth
the investment as students learn differently:

“Hands on learning helps students get the learning. Art helps them
explain what they have learned. The children understand the lesson
because they are able to connect them to real life.”
 Students’ understanding of social studies. Teachers stated that their students
understanding of social studies has not only increased, now the students enjoy
learning and show enthusiasm for their lessons. In the IB class the students
presented their products in the IB fair and the products were at the level or
higher of what they should have been.
 Students’ interest in the arts. Students have increased their interest and
understanding of the arts. In Ms. Cook’s class they will create and present another
performance (Living Museum Project with One Love One Community) on June 19
in Downtown Delray. Mr. Stokes students’ went to a YSPB performance and
learned that the arts are available to everyone or that, “you do not have to be rich
to see a show”. Some of the students have asked Mr. Stokes about how to join the
YSPB.
 Parental involvement. The parents did not become involved in the program
activities and offered limited feedback. However, some parents indicated that
they loved the program and were very thankful. One parent sent a note to Ms.
Cook writing about how much she liked that the students were being exposed to
different cultures. Mr. LaVogue heard from some parents about how much they
had liked the board games.
Differences
 Challenges in integrating the arts. The teachers expressed different challenges
when integrating the arts; mainly, the time required, getting the buy-in from
administrators and teachers, and the money needed to buy the materials.
 Indicators pointing to changes in learning. Two of the teachers (Ms. Cook and Mr.
Stokes) stated that IB assessments or quizzes indicated that the students had
improved.
 Resources provided by the grantee agency. While all teachers agreed that the
6
grantee agencies had provided invaluable support, each teacher indicated a
different value: the teaching artists, the staff, and students being able to attend a
performance.
 Feedback from teachers/administrators. Each teacher had a different experience:
some teachers were very supportive, others were somewhat jealous, and some
were not supportive or enthusiastic.
 Suggestions for improvement. The teachers offered useful suggestions to improve
the program: offer it to all the students in the school, increase the number of
sessions with the teaching artist, and implement the program during a full
academic year.
Grantee Agency Data
Teaching Artists
Common Themes
The three teaching artists interviewed represented the three schools involved in
the program. They were all very satisfied with the program and agreed that they had
achieved their goals. They agreed on the following as well:
 Grantee agencies’ contribution. The grantee agencies most important
contributions had been offering students a different way of learning and
teachers a different way of teaching.
 Parental involvement. Parents were not very involved with the program and
the teaching artists expressed that it was important to get their support.

Outcomes. The main outcome described by the teaching artists was that
students really learned:
“Students experienced learning not just from text books but as a physical
experience (experiential learning). The program helps students get immersed
in learning and becoming part of the learning process. Learning as part of
their being not as an external thing that happens to them. They learn
concepts in class but with this program they feel their place in history and
society learning”
Differences
 Support from teachers and administrators. The teaching artists experienced
different levels of support from other teachers and the administrators. For
example, Ms. Gosselin indicated that:
“The principal wanted to make this happen and the head of the IB
program was very supportive. This support was probably due to the good
experience the school had with year 1 of the program.” However, the
experience at Conniston was different as “the school did not embrace the
program, administrators were not committed and they dropped the ball.”
 Suggestions for improvement. Given their high level of involvement, the teaching
7
artists offered well-thought suggestions for improvement:







Provide clear information at the beginning of the activities about budget,
roles, and expectations.
Focus on parental involvement and get parents’ support at the beginning of
the school year.
Schedule and budget for more time to plan, give lessons, prepare culminating
event, provide incentives for teacher involvement, and for documenting
lessons and performances.
Identify teachers before the program starts and make sure that they are really
interested in participating. Teachers should not “be told” of their participation
but really get their buy-in.
Prepare the teachers: the teaching artist provides support for learning and
teachers should not take the teaching artist’s visit as a time to relax.
Conduct more public relations and education in the community: maybe the
Palm Beach Post and other media are more widely read in West Palm Beach
than the Palm Beach Daily News.
Select schools for the project that have already made the commitment to
change the culture of learning in their schools. The results will be better and
the program will last beyond the funding years.
Grantee Agencies’ Staff
All the staff interviewed expressed being very satisfied with the program and felt
they had been very successful.
Common Themes
 Factors contributing to the program’s success: the teaching artist “fit” with the
teacher, the teaching artist’s experience, relationship with grantee agency,
personal or professional relationships between staff (agency/school) and
teaching artists.
 Influence on the community. The staff indicated that the community is better off
as a result of the program: the school community became aware of the role of the
arts and the teaching artist, students expanded their horizons and learned about
the arts, and the external community saw the students’ work. The YSPB invited
the students to a performance and some of the students had just arrived from
Haiti – the exposure to music helped them integrate into the class and provided a
“safe and beautiful” community experience.
Differences
 Program outcomes. Staff expressed achieving different outcomes: learning and
retention of information, exposure to the arts, student growth, and collaboration
between teacher and teaching artist.
 Factors contributing to the program’s lack of success included a lack of buy-in
from teachers and/or principals, trying to cover too much material or reach too
8
many students with insufficient staff, lack of involvement from the teachers
during the teaching artists’ visits, and difficulty in establishing communication
with teachers.
 New activities/programs. Two of the agencies have created or become involved
with new programs and/or activities: Resource Depot has given presentations at
the Florida Arts Alliance Educators Conference and Palm Beach County School
District Multicultural Language and Arts Conference; and after school programs
have asked them to collaborate by providing resources, giving workshops to
teachers, and working with students. The YSPB was awarded a grant for an after
school program to bring choral music to 4-6 grades in 6 sites (each site has a
teaching artist for 2 hours a day/2 days per week).
 Suggestions for improvement




Training. Several suggestions were made regarding training: (1) Provide
more training for the teaching artists (allocate more time with Gail and work
on what arts integration really is with the novice teaching artists). (2) Agency
staff should not be required to go to the workshops – they are geared to
teachers and teaching artists. (3) School administrators need to have more
information, maybe allocate a couple of hours during a workshop to talk
about roles, requirements, etc. Let administrators leave after this session and
continue with the program for the rest of the participants. (4) Teachers need
to be trained on their roles when the teaching artist is in the classroom.
Clarification of roles. All the participants need to understand their roles better
and sooner. It is important to get buy-in from all participants before starting
the program and involving the principal from the beginning as he/she will get
the teachers involved and enthusiastic.
Review the grant proposal submission process and implementation. The
deadline for submission of proposals might not be good time for the
prospective applicants. Also, this should be a three year program – the first
year is really for learning. It is important to figure out a way to schedule the
lessons reflecting the teaching flow and the teachers’ lesson plans. It is
difficult to get everything done in the short time frame (for example, teaching
artists working on Mesopotamia and the teachers have moved to Egypt).
Community involvement. Bring in the community (school and external)
through the arts installation. Make the culminating event more visible by
making them movable and presentable outside of the school.
Documentation of Activities
 Student journals, art samples, and culminating events. The evaluator reviewed
students’ journals and artwork when conducting site visits and received a video
documenting a culminating event. The documentation provided could not be used
to assess changes in the students’ level of appreciation, knowledge, understanding
of social studies through arts integration because: (a) no common rubric to
produce or assess the artwork has been developed, and (b) not all the teachers
9
and teaching artists provided documentation. The evaluator received the
following documentation:



Student journals were obtained from one school (provided by Gaynelle
Gosselin, teaching artist at Conniston).
Student artwork was observed during the site visits to two schools.
Documentation of the culminating event was provided by one teaching artist
(Erica Barnes).
 Teachers’ lesson plans. At the January workshop and via emails the teachers were
asked to submit a lesson plan showing integration of the arts. The evaluator
received one lesson plan (Tonisha Cook from Carver) and did not receive lesson
plans from the other teachers. Therefore, the evaluator could not use lesson plans
to assess teachers’ level of comfort and knowledge with incorporating the arts
into curriculum.
Review and Recommendations
Evaluation Questions
Question 1: Have the programs increased the students’ level of appreciation,
knowledge, understanding of social studies through arts integration?
Question 2: Has the teachers’ level of comfort and knowledge with incorporating
the arts into the curriculum increased?
Question 3: Have parents increased their involvement in their child’s education?
Question 4: Have the target organizations increased their capacity to deliver
outcome-based arts and cultural education?
To address the first evaluation question, the evaluator reviewed the data from
student surveys and site visits checklists. As explained in prior sections, the student
survey data might not be reliable or valid. The information from the site visits does
indicate that the students have increased their understanding and knowledge of
social studies. The teachers interviewed indicated also that students’ knowledge and
understanding of social studies had increased as demonstrated by the quizzes or IB
assessments.
The second evaluation question pertains to teachers and the evaluator analyzed
data from site visits as well as teacher workshop surveys and interviews. The data
clearly indicates that the teachers’ level of comfort and knowledge had increased with
incorporating the arts into the social studies curriculum.
The third evaluation question pertains to the parents and the information from
the interviews points to a lack of parental involvement. However, the three schools
involved in the program have traditionally experienced low parental involvement.
The teachers involved in the BLCTAC program have indicated that parents have
approached them to show their appreciation for the activities and products. Many
parents have attended the culminating events or contributed with refreshments. In a
10
culture of no parental involvement, these actions and behaviors can be interpreted as
signs that the parents have become more involved.
Finally, a review of the interviews indicates that the grantee organizations have
increased their capacity to deliver outcome-based arts and cultural education.
Furthermore, two of the organizations have expanded their activities into after school
programs. The Palm Beach County Cultural Council also has changed due to the
program as staff has become more involved with the schools and the school system.
Mr. Todd LaVogue (Roosevelt) provided the following example of how the Council is
creating changes for the schools:
“Alyx showed the School Board a music video of what the kids had
done (Mesopotamia) and a presentation that he and Jennifer had done – all
this caused an impression. A teacher called me to let me know and the
principal congratulated me. Alyx has been a great support (a “champion”)
for the program, the teachers and their work. Principals love good publicity
and Alyx gave him this with the presentation at the School Board. This will
help get him onboard for other programs”
Recommendations
The interim report offered recommendations about documentation and
assessment, expectations, strategies to support program activities throughout the
school year, implementation logistics, grant process, and parental and school
administrators involvement. The data and results from the final quarter of activities
validate these recommendations and highlight the need to implement programmatic
changes. Mainly, it is recommended that the BLCTAC program (a) modify its structure
to include characteristics of successful programs and (b) refine its approach to
program evaluation.
Program duration, intensity and frequency. The BLCTAC program should offer
sufficient program exposure in terms of duration, intensity and frequency: students
need to be exposed to enough of the program for it to have an effect. The amount of
exposure to produce results varies based on the participants’ risk level, typically
more intensity is needed for higher risk participants to obtain positive and long
lasting effects. The students participating in the BLCTAC program are considered high
risk due to:
(a) High risk of not completing high school (leaving school early or graduating
without an adequate education is a problem closely identified with our nation's
children of color and/or those living in poverty).
(b) Propensity to engage in high risk health behaviors such as tobacco use,
unhealthy dietary behaviors; inadequate physical activity; alcohol and drug use;
sexual behaviors that contribute to unintended pregnancy and sexually transmitted
diseases, including HIV infection; and behaviors that contribute to unintentional
injuries and violence.
11
The characteristics of successful programs were identified by many of the
program stakeholders who stated the need for changes in the program’s duration,
intensity and frequency, including the need to offer the program year-round and
increasing the number of sessions with the teaching artists.
Program evaluation. The evaluation process for the 2010-11 funding cycle should
be used as a pilot evaluation that provided an understanding of how the BLCTAC
program works and what can be done to improve both the program and the
evaluation. Specifically, the 2011-12 funding cycle would benefit from a refined
program evaluation process that includes:





Building a theory of change model.
Refining the evaluation questions by seeking input from different stakeholders.
Identifying outcomes that accurately represent the program’s impact.
Developing assessment tools that measure the outcomes.
Developing tools for the grantees to track progress, monitor activities, and
document success.
Finally, the BLCTAC program provides activities to a student population that does
not have access to quality arts experiences and does not have many opportunities to
learn in an environment that fosters creativity, curiosity, and different approaches to
learning. The Palm Beach County Cultural Council has leveraged community
resources by effectively engaging community partners to increase opportunities for
all our students. The BLCTAC program is a new and innovative program with the
potential to become a successful program that is embraced by schools, the school
system, and the community at large.
12