Living Arts
Social Innovation Fund Grant Interim Report Year 2 (2013)
Submitted by Mary Lou Greene, M.F.A.
And Shlomo Sawilowsky, Ph.D.
Grantee: Living Arts is dedicated to strengthening the neighborhoods of Southwest Detroit by
cultivating an environment that sparks the imagination and fosters ingenuity through the arts and
community development initiatives.
 Core activity is in-school arts infused programming.
 Collaborative projects serving students in pre-school through grade 12
 Community artists and classroom teachers use hands-on methods to pilot innovative
curriculum and improve student performance.
Educational Theory
At-risk students (including English language learners and those at-risk for academic failure) do
not achieve academic success or school readiness through traditional modes of
instruction.Studies have shown that using arts-based strategies in the classroom creates a
stimulating environment can enhance a child’s cognitive, academic and social development. See
www.wolftrap.org/~/media/Files/PDF/Fairfax_Pages.ashx. .
Research Questions
Impact Evaluation:
Confirmatory:
The proposed research project will examine the following hypotheses to demonstrate the
effectiveness of Arts-Infused education:
Does the Living Arts intervention have an impact on advancing school readiness in prekindergarten children ages 3 & 4’s measured by the HighScope COR instrument?
Exploratory:
1. Are teachers’ use of the arts-infused strategies, after the artist residency, indicating systemic
change in instructional delivery?
2. Are parents using the arts-infused strategies at home with the children in the programs?
Original Overview of study:
The study was to include 97 intervention classrooms (n=1597) and 59 comparison classrooms
(n=999). The COR instrument was to be used for each child and include 3 observation records
per year in years 2-4. Later years were to include parent/teacher surveys created using the
blueprint approach for test construction, tested and administered to approximately 100 parents
and 20 teachers in years 4/5. In addition, a feedback instrument was to be developed and
administered to determine the impact of the parent/caregiver DVD intervention.
Study Components—Year 1
Description of instrument: HighScope Educational Research Foundation’s Child Observation
Record (COR) is a 32-item observation-based instrument providing systematic assessment of
young children's knowledge and abilities in all areas of development.
Living Arts
Social Innovation Fund Grant Interim Report Year 2 (2013)
Page |1
COR data is reported in six section and multiple sub-sections. Those include:
 Initiative
◦ Makes choices
◦ Solves problems
◦ Initiates play
◦ Personal needs
 Social Relations
◦ Relates to adults
◦ Relates to children
◦ Resolves conflicts
◦ Expresses feelings
 Creative Representation
◦ Makes & builds
◦ Draws and paints
◦ Pretending
 Movement/Music
◦ Moves in ways
◦ Moves with objects
◦ Steady beat
◦ Moves to music
◦ Singing
 Language/Literacy
◦ Listening
◦ Vocabulary
◦ Speech
◦ Sounds
◦ Books
◦ Letters
◦ Reading
◦ Writing
 Math/Science
◦ Sorting
◦ Patterns
◦ Comparing
◦ Counting
◦ Position
◦ Sequence
◦ Materials
◦ Living things
Proposed analysis approach: Following the logic model, we will be using a randomized
repeated measures Multivariate Analysis of Covariance approach to determining the impact of
the combined Wolf Trap and Living Arts intervention on specific domains and subscales of the
COR that pertain to creativity, arts, and language.
Results in Year 1 and Problems to Resolve:
From the evaluation team:
1) Upon initial analysis, it appeared that the WolfTrap program did have a significant
impact over the comparison classrooms: The analyses showing Wolf Trap (full day) is
statistically significantly better than the comparison for 27 of 30 subscales. (On the
remaining 3, there were no statistically significant differences.) The average effect size
was partial eta square = .029. A value of .02 is considered small, so the overall treatment
effects are small but detectable. Subsequently, it was learned that the Wolf Trap full-day
intervention was being compared with a combination of full-day and part-day
Comparison groups. Research into effectiveness of full-time versus part-time
kindergarten has been undertaken by numerous researchers, including Jill Walston and
Jerry West’s Full-Day and Half-Day Kindergarten in the United States: Findings from
the Early Childhood Longitudinal Study, Kindergarten Class of 1998-99, NCES 2004078 and Tiffany Field, Wendy Masi, Sheri Goldstein, Susan Perry’s Infant day care
facilitates preschool social behavior, Early Childhood Research Quarterly, Volume 3,
Living Arts
Social Innovation Fund Grant Interim Report Year 2 (2013)
Page |2
2)
3)
4)
5)
6)
Issue 4, December 1988, Pages 341-359, ISSN 0885-2006,
http://dx.doi.org/10.1016/0885-2006(88)90034-8.
(http://www.sciencedirect.com/science/article/pii/0885200688900348)
Due to the proven difference between full-day and part-day kindergarten, we separated
the classrooms to compare intervention full-day to its equal comparison and the same for
half-day.
Another series of univariate ANCOVAs were conducted on the posttest scores, with the
pretest scores used as the covariate. This part-time data favored the Comparison group, as
did the full-time.
Nevertheless, those findings are inconsequential. Although the comparison groups did
not receive the intervention this year, all of the comparison groups were from schools
where teachers have received El Arte training and the intervention over numerous years
in the past.
This would suggest that teachers have incorporated the methodology as a matter of
course (which could be an answer to the first implementation question), but needs to be
tracked more rigorously through a more appropriate comparison group.
In subsequent years, the comparison group must be comprised of teachers who have had
no training in this methodology nor will the classrooms or schools have had any
intervention.
From the director of the organization: Significant changes in program implementation and data
collection in year 1 included:
1) Living Arts engaged 39 VNHS classrooms and 2 DPS classrooms in year 1, instead of 40
VNHS classrooms and 20 DPS classrooms as planned. VNHS provided COR data for
their intervention classrooms but DPS did not provide any data. In fact, the DPS Office of
Research, Evaluation and Assessment still has not approved (or denied) Living Arts'
evaluation proposal (submitted August of 2012). Living Arts must now choose a new
partner for years 2-5.
2) Living Arts oriented potential DPS teachers in the fall, but the spring workshop was
cancelled (by DPS).
3) Although Hartford Head Start (a DPS school) agreed last year to share their COR data as
comparison data for Year 1 of the project, they reversed their decision at the end of the
school year. Presumably, this occurred due to change in management as opposed to any
material reason pertaining to the intervention. Therefore, we are using comparison data
from VNHS classrooms. However, these comparison classrooms are not strong because
most of the teachers in these classrooms received Living Arts intervention last year or
two and may be using the Living Arts methodologies already. Also, a small percentage of
the students in these comparison classrooms are returning students who received Living
Arts intervention last year.
4) In years 2-5, Living Arts will replicate its Wolf Trap Early Learning Through the Arts
program at additional Detroit Head Start Centers (currently managed by CDI) and at two
Wayne Metropolitian Head Start Centers(to replace DPS sites), possibly adding
additional partners in the course of the project.
Year 2 Changes :
Living Arts
Social Innovation Fund Grant Interim Report Year 2 (2013)
Page |3
From the director of the organization:
1) Number of VNHS residencies that will provide evaluation data in SIF Year 2: 28
2) Number of expansion residencies (never had before)that will provide evaluation data in
SIF year 2: 21 (possibly up to a total of 40)
3) Additional match residencies that will NOT provide evaluation data, but will build
capacity in SIF Year 2: up to 15 more (11 are definite)
4) Basically, we can only count on 49 classrooms for evaluation data in SIF Year 2, but it's
possible we'll add up to 22 more evaluation classrooms for a total of up to 71 evaluation
classrooms. In SIF Years 3 to 5, we plan to sustain at least 49 evaluation classrooms.
From the evaluation team:
As noted in the SEP, the COR (a High-Scope Educational Research Likert Scale) instrument was
used to compare Wolf-Trap with full and part time Comparison classrooms. Initially, Wolf Trap
classrooms were shown to have statistically significantly higher scores on 27 of 30 subscales
pertaining to Movement/Music and Language/Literacy. However, those improvements dissipated
when results were disaggregated in comparison with full time only Comparison classrooms.
Hence, in Year 2, two adjustments will be undertaken:
(1) the COR scores will be used to examine broader outcomes, such as the Initiative, Social
Relations, Creative Representation, and Math/Science subscales,
(2) the Comparison groups in Year 1 were from schools where El Arte training has taken
place in the past, and therefore, in Year 2, Comparison groups will be selected where such
cross-contamination is precluded, and
In addition, Living Arts is investigating the feasibility to developing a criterion-references
instrument to more carefully target the assessment of desired programmatic outcomes. We will
look at developing and possibly piloting the instrument this year to use next year if the COR
reports do not give us the targeted information needed.
 The criterion referenced test will be created with input from WolfTrap National and El
Arte trained personnel, in conjunction with artists and teachers. Prof. Shlomo
Sawilowsky, as the measurement expert, will facilitate the development following the
blueprint approach to test construction. The test blueprint will delineate the competencies
and their weights, a table of specifications (i.e., hierarchical cognitive taxonomy) will be
developed, a task analysis will be conducted to delineate prerequisite and enabling skills.
Item writers and validators will consist of content experts (i.e., teachers and artists),
Living Arts staff, and Wolf Trap representatives.
 Literature links will be provided to substantiate content validity, and the resulting
criterion-referenced test will be subjected to classical measurement theory psychometrics,
such as Cronbach alpha with Hoyt’s correction, based on a pilot classroom. Subject to
item deletion based on psychometric analyses, the resulting criterion-referenced test will
then be invoked as a Post-test only instrument at the end of Year 2.
 Following exploratory factor analysis to substantiate the internal factor structure, the final
version will then be used as a pretest-posttest instrument for Years 3 and 4.
Summary: Initially, there were to be 97 intervention classrooms (n=1597) and
Living Arts
Social Innovation Fund Grant Interim Report Year 2 (2013)
Page |4
59 comparison classrooms (n=999). Now…..there will be at least 49 intervention classrooms
(n=833) and approximately 50 comparison classrooms (n=850). We will continue to use the
COR report each year, but will also begin an investigation into the feasibility of development of
a more targeted instrument beginning January 2014, to be piloted/tested throughout the 2014
school year, and ready for use in years 3-5.
These changes should give us measurable data to substantiate the impact of this methodology.
Living Arts
Social Innovation Fund Grant Interim Report Year 2 (2013)
Page |5