Iowa Support System for Schools in Need of Assistance
Audit Phase:
Audit Narrative
Introduction
The audit process is the first phase in the Iowa Support System for Schools in Need of
Assistance (SINA). It may also be considered a potentially dynamic and important first step in
other important improvement initiatives for schools and districts such as Comprehensive School
Improvement Planning (CSIP), planning for the Iowa Professional Development Model (IPDM),
High School Reform, and Instructional Decision Making (IDM). The goal for working through
the audit process and the introductory steps in other improvement initiatives is to precisely
identify the key areas of potential influence on student achievement that need to be considered in
a school or district. We frame this process by looking at two key questions: “What data do we
have on student outcomes and what does it tell us?” (Remember that student outcomes include
both academic achievement data and behavioral data.) and “What is our system’s capacity to
improve student outcomes?”
In the Iowa Support System for Schools in Need of Assistance (SINA), the audit phase is
comprised of a series of events with supporting actions that will be completed in approximately
fifteen (15) school days. Each event has specific activities and purposes. Each event also has a
set of important considerations for successful accomplishment. These considerations are
described below.
Event 1 – Meet with the district and/or building team
The purpose of this meeting is to introduce the audit team from the AEA that will be working
with the identified district/building, using the Iowa Support System for Schools in Need of
Assistance (SINA). Based on our experiences, this meeting typically takes one hour to cover the
needed material and information. It is important that all necessary and key administrators from
the district/building attend this meeting so they are clearly apprised of the Iowa Support System
for Schools in Need of Assistance (SINA) and the scope of the audit team’s work. If doing a
building-level audit, key administrators might include the principal, assistant principals and/or
deans, curriculum coordinator or director, special education coordinator or director, etc. If doing
a district-level audit, key administrators might include the aforementioned list representing each
building in the district and the superintendent, associate superintendent, etc.
This meeting should begin with introductions of all persons and their respective roles in assisting
with the Iowa Support System for Schools in Need of Assistance (SINA). This allows all
participants to gain insight into the strengths and capacities that are available for further actions.
It will be very important at this first meeting to share the overall framework of the Iowa Support
System and to help district/building personnel gain an understanding of the commitment and
involvement necessary over the entire process. It is vital that the audit team share explicitly that
the audit process is not a process for finding fault nor is the team present to tell the district/
building what to do. Rather, the audit is designed to allow a district/building an opportunity to
closely scrutinize what they are currently doing (or not doing) to improve student learning and
provide suggestions for further areas of study.
Audit Phase: Narrative
©2005
Audit - 72
Iowa Support System for Schools in Need of Assistance
Of considerable importance at the first meeting is an in-depth discussion of the events of the
audit process and the development of the district/building profile as a result of the audit. Begin
by sharing the audit phase’s Series of Events page and carefully reviewing each event. As each
event is shared, distribute copies of the appropriate documents related to that event. For
example, Event 2 refers to reviewing district/building documents and creating the first draft of
the building profile. During the discussion of this event, the blank copy of the Audit Profile
should be distributed and shared, as well as the list of the audit’s Questions for Developing the
District/Building Profile, the audit phase’s Documents for Review checklist, and Leverage Points
for a Strong System for Learning. The Practices Inventory should be explained as well,
including that it is an optional tool for collecting data from all of the teachers in the
district/building, the advantages of completing it, how the data will be used, who will have
access to this data, and how teachers will access and complete it (see the Practices Inventory
information sheet on the Iowa Support System’s web page).
It will be important to establish timelines for completion of the Practices Inventory and the
collection of documents during this initial meeting. In our experience, a building may take
anywhere from two weeks to two months (or more!) to collect documents. However, if a
building has waited until official notification from the Iowa Department of Education in August,
be very clear that they do not have two months to collect their documents because the 90-day
clock is ticking. As part of the discussion on timelines and collection of documents, it will also
be important to clarify who will be responsible for collecting which documents. Before this
initial meeting ends the team might set future meetings to continue the next events in the audit
process.
Event 2 – Review district/building documents
The purpose of this event in the audit process is to begin completing the district/building profile,
answering the audit questions based on the information provided in the collected documents,
which may include data from the Practices Inventory. A separate piece of the profile will be to
summarize student achievement data in relation to the area of identified concern. This event
should be completed by the audit team, rather than an individual, in approximately five school
days from the time the documents are collected from the district/building.
All documents gathered need to be carefully labeled and checked for dates. Checking the dates
on district/building documents will assure use of the most current information and help resolve
questions if there are several documents with the same title (e.g., several sets of standards and
benchmarks, professional development plans, data analysis summaries). In our experience, it is
very useful to organize the materials by domain and element. The materials may be organized
further by placing them into a binder or even in an electronic file. The background skills and
knowledge of the audit team should be carefully considered when reviewing the various resource
documents so the team members with the most experience in a given domain or element will be
reviewing those documents.
Once the documents are collected, members of the audit team will carefully match the provided
documents to the audit questions and begin to answer the questions. Answers to the audit
questions will need to be recorded on the building profile in the Essential Findings column and
substantiated by listing the document(s) in which that answer was found in the Source column.
As the team proceeds through the questions, they may find that there is no document to which to
refer, or that the information in the document does not answer the question. When this happens,
Audit Phase: Narrative
©2005
Audit - 73
Iowa Support System for Schools in Need of Assistance
there is no further need to ask about the document or search for it unless the team desires to do
so. Rather, note in the Essential Findings column, “No evidence at this time,” craft a question
regarding the lack of evidence for the upcoming interviews with the administration and/or
teachers, and move on to another area. Potential interview questions can be listed in the
Comments column.
If a district/building has completed the Practices Inventory, this data should be summarized in
the district/building profile. One approach to incorporating data from the Practices Inventory is
to do the following:
(1) Consider each element in the profile and the corresponding page(s) of the data report for
the Practices Inventory.
(2) Cite any data that directly support a question under that element.
(3) Cite any data that are significant but may not be precisely related to a question under that
element.
(4) If there are any data from the Practices Inventory that are unclear, conflict with data from
the documents, or warrant further study, craft a question about the data in the Comments
column of the profile for use during the interviews with administrators and/or teachers.
Event 3 – Interview the district/building administration
The purpose of this meeting is to engage the district/building administrators in clarifying any
findings and answering any questions raised as a result of the review of district/building
documents in Event 2. In our experience, this meeting will take approximately one to one and a
half hours. The emphasis of the meeting is on clarification, not in-depth diagnosis. Members of
the audit team should remain neutral and minimize any evaluative or judgmental comments.
Prior to the meeting with the administrator(s), questions should be clearly identified that need
further clarification so that time is used efficiently and productively.
It is recommended that at least two members of the audit team attend this activity so that one
person can ask the questions, and another can capture the responses using a computer and
projector so everyone can see and clarify what is being written, and others can observe as
administrators respond to the questions. To assist the recorder, it is helpful that the recorder have
an electronic copy of the interview questions prior to the interview, and that during the interview
the interviewer should paraphrase responses from the administrator(s).
Event 4 – Interview the teachers
The purpose of this meeting is to engage teachers in clarifying any findings and answer any
questions raised as a result of the review of district/building documents in Event 2. In our
experience, the ideal meeting will take approximately two hours; however, the typical interview
with teachers will last 45 minutes to an hour. This lack of sufficient time for interviewing
teachers is a concern because audits often provide between 20 – 30 questions that need
clarification. Since there is a limited amount of time for the interviews, especially with teachers,
prioritizing the interview questions is essential. One strategy for addressing this lack of time for
interviewing teachers is to conduct several sets of interviews.
In order to maximize efficiency, the interview with teachers should occur in a group setting. We
have found the optimum group size to be between 6-8 teachers. However, the groups may vary in
size and the number of groups to interview may vary depending on time of year and building
Audit Phase: Narrative
©2005
Audit - 74
Iowa Support System for Schools in Need of Assistance
schedule. It is important that the dynamics of the group allow everyone to freely engage in the
process and get their responses heard. If asked for guidance in selecting teachers to interview,
we suggest that a stratified sample be used. In other words, teachers should represent different
grade levels, content areas, general, remedial, and special education, and should be familiar with
the identified area (i.e., reading, math). However, it is not uncommon to find teachers selected
completely at random, be strictly volunteers, or simply be members of the district/building
leadership team. We strongly recommend that neither district/building administrators nor AEA
staff serving the district/building be a part of the team of teachers being interviewed to assure
that teachers are comfortable providing unbiased responses.
As mentioned at the end of Event 3, it is recommended that this activity be accomplished by at
least two members of the audit team so that one person can ask the questions and that another
can capture the responses using a computer and projector so everyone can see and clarify what is
being written, and others can observe as teachers respond to the questions. To assist the
recorder, it is helpful that the recorder have an electronic copy of the interview questions prior to
the interview, and that during the interview the interviewer should paraphrase responses from the
teacher(s).
Event 5 - Create a final building profile
The audit team members complete the final district/building profile by compiling existing
information from the documents, interviews, and Practices Inventory. If the first four steps are
completed in a thorough and thoughtful manner, this part of the process will essentially be done.
Areas for Further Study should first be clearly identified with an “X” in the last column and in
the row by the appropriate question. Next, the concern should be framed using a question that
gets at the heart of the concern and would lead to further diagnostic work. This question should
be placed in the Comments column. Finally, strengths should also be explicitly noted in the
Comments column. (Refer to the sample profile for examples of these kinds of questions and
strengths.) AEA staff that are not members of the support team working with the district may be
invited to review the final district/building profile to ensure that it is written in an understandable
and non-evaluative manner.
As the team completes the profile, they need to begin prioritizing the Areas for Further Study.
There are several ways to complete this prioritization. A team may divide the Areas for Further
Study into those that are most proximal (or closest) to impacting student achievement (e.g.,
standards, curriculum, instruction, and assessment) and those that are more distal (or furthest),
especially since a district/building’s success under No Child Left Behind is based on proficiency
as measured by the Iowa tests (ITBS & ITED). Another way might be to follow Maslow’s
hierarchy by focusing on student behavioral issues first, then work on instructionally relevant
elements. Caution the team not to over focus on behaviors issues to the exclusion of changes in
instructional practices. The main point is that the team needs to be clear on how they will
prioritize the Areas for Further Study and that they share their rationale supporting the prioritized
list with the district/building administration and leadership team(s).
The audit team may want to produce an executive summary of the Areas for Further Study based
on the actual profile. This summary should have the following sections and is generally limited
to four pages: a guide for administrators and teachers to independently review the final version
of the profile (two pages – one with directions and one for recording notes), a list of identified
strengths, general recommendations, and content-specific recommendations (note that the
Audit Phase: Narrative
©2005
Audit - 75
Iowa Support System for Schools in Need of Assistance
content-specific recommendations are limited to the identified area(s) of concern). The purpose
of the executive summary is to keep Event 6 focused on the key findings instead of spending
time flipping through the profile and becoming distracted by individual findings or questions.
The team may wish to refer to the sample executive summary for ideas on how to craft their
own.
The approximate time for this event is eight hours spread over three or four days because the
team will want to take time to reflect on the data and craft precise questions.
Event 6 - Meet with district/building administrators and/or leadership team
The purpose of the final event in the audit process is to share the results with district/building
administrators and possibly a teacher leadership team. In our experience, this final sharing has
consistently included both the district/building administrators and the appropriate
teacher/building leadership team. This meeting will take approximately one (1) hour.
A typical meeting begins by reviewing the Iowa Support System for Schools in Need of
Assistance (SINA) and in particular the purpose for the audit (emphasizing that this is not an
evaluation, nor an attempt to tell a district/building what they should do). Next the Practices
Inventory summary report is shared, with the focus on showing the meeting participants how to
interpret the summary report and analyze the data. Then the final version of the profile is shared,
with the audit team showing participants how to read the report and emphasizing that the
administrators and teachers should independently review the profile using the guide found in the
first two pages of the executive summary. Finally, the last pages of the executive summary are
shared. These pages contain strengths, general recommendations, and content-specific
recommendations.
These latter items, the strengths and recommendations, should be the focus for the bulk of the
meeting time. The goal is to go through each strength and recommendation by sharing the data
from which it was derived, citing relevant research that helped inform the recommendation, and
possibly providing some actions that the district/building might take. It is very common for the
meeting participants to gloss over their strengths and want to stop and thoroughly discuss each
recommendation. Therefore, the audit team should take care to emphasize the strengths and
coach the meeting participants to hold their discussion of the recommendations until the end of
the meeting or at another meeting scheduled after they each have had time to independently
review the profile. In our experience, it is effective to refer to district/building strengths as part
of the possible actions a district/building might take to address the suggested recommendations
in order to capitalize on the good things that the district/building is already doing.
It is also common that the district/building administrator and/or members of the leadership team
will be overwhelmed with the number of recommendations contained in the executive summary.
Therefore, it may be prudent to take some time during this meeting to coach the district/building
leadership team into identifying some preliminary next steps or even a sketchy multi-year action
plan as a way to reduce this feeling of having too many recommendations to address and no time
to work on them. This coaching is not designed to produce a working action plan but rather to
help members of the leadership team to see that they do not need to be overwhelmed by the
number of recommendations and that they will be able to address many of the recommendations
through the good things they are already doing and during the Diagnosis and Design Phases of
the process.
Audit Phase: Narrative
©2005
Audit - 76
Iowa Support System for Schools in Need of Assistance
Finally, it will be important to include diagnosis team members in this meeting, if they are not
already part of the audit team, so that they can become engaged in the process as this meeting
also serves as the initial step of the diagnosis phase.
Audit Phase: Narrative
©2005
Audit - 77