2014 Self-Study Workshop
Characteristics of
Good Self-Study Reports
[NOTE: Updated January 2015
to reflect changes in Handbook]
Tom Tanner
ATS Commission Staff
Before we begin, a caution…
If you’re going to work that hard, do the right
car…
ATS Commission on Accrediting: 2014 Self-Study Workshop
Characteristics of Good Self-Study
Reports
Five things to remember to do the “right”
report:
1.
2.
3.
4.
5.
The 4 purposes of the report
The 3 qualities of the report
2 3 4
The 5 features of good/weak reports 1
The 4 parts of the report
5
The 3 last things to remember
ATS Commission on Accrediting: 2014 Self-Study Workshop
Characteristics of Good Self-Study
Reports
Five things to remember to do the “right”
report:
1.
2.
3.
4.
5.
The 4 purposes of the report
The 3 qualities of the report
The 5 features of good/weak reports 1
The 4 parts of the report
The 3 last things to remember
ATS Commission on Accrediting: 2014 Self-Study Workshop
Characteristics of Good Self-Study
Reports
1. The 4 purposes of the report (www.ats.edu)
[formerly Handbook of
Accreditation]
ATS Commission on Accrediting: 2014 Self-Study Workshop
Characteristics of Good Self-Study
Reports
1. The 4 purposes of the report
1) Demonstrate ability to analyze
institution’s own effectiveness
2) Develop plans for improvement
3) Provide evidence of meeting
standards
4) Form basis for external review
ATS Commission on Accrediting: 2014 Self-Study Workshop
Characteristics of Good Self-Study
Reports
1. The 4 purposes of the report
1) Demonstrate ability to analyze
institution’s own effectiveness
Good long look in the mirror
The mirror has two faces:
• School’s own mission
• ATS Commission standards
ATS Commission on Accrediting: 2014 Self-Study Workshop
Characteristics of Good Self-Study
Reports
1. The 4 purposes of the report
2) Develop plans for improvement
Report must not only “look around” but “look ahead”
Assessment (self-study) should
show
ATS Commission on Accrediting: 2014 Self-Study Workshop
Characteristics of Good Self-Study
Reports
1. The 4 purposes of the report
3) Provide evidence of meeting
standards
Weak report: “Our seminary clearly meets this standard.”
Good report:
“Evidence of how our seminary meets
this standard may be summarized in these
three ways. First,…”
ATS Commission on Accrediting: 2014 Self-Study Workshop
Characteristics of Good Self-Study
Reports
1. The 4 purposes of the report
4) Form basis for external review
The self-study report is your story…
but told to strangers (so don’t assume)
who are sympathetic, but objective
so you must present and
persuade
ATS Commission on Accrediting: 2014 Self-Study Workshop
Characteristics of Good Self-Study
Reports
Five things to remember to do the “right”
report:
1.
2.
3.
4.
5.
The 4 purposes of the report
The 3 qualities of the report
The 5 features of good/weak reports
The 4 parts of the report
The 3 last things to remember
ATS Commission on Accrediting: 2014 Self-Study Workshop
2
Characteristics of Good Self-Study
Reports
2. The 3 qualities of the report*
1) Provide sufficient description
2) Document appropriate evaluation
3) Identify strategic recommendations
* Page 11 of Chapter Two of the Handbook
NOTE: Under heading “General Characteristics”
ATS Commission on Accrediting: 2014 Self-Study Workshop
Characteristics of Good Self-Study
Reports
2. The 3 qualities of the report
1) Provide sufficient description
Weak report: “We have sufficient faculty, and
they are all sufficiently qualified.”
Good report: “We have seven full-time faculty;
six have PhD degrees and one has
a DMin (see this link for detailed
list).”
ATS Commission on Accrediting: 2014 Self-Study Workshop
Characteristics of Good Self-Study
Reports
2. The 3 qualities of the report
1) Provide sufficient description
Weak report: “Our SLAP efforts have produced
significant improvements.”
Good report: “Our Student Learning Assessment
Program (affectionately called SLAP by
students) has produced these three
improvements in student learning this
past year. First,...”
ATS Commission on Accrediting: 2014 Self-Study Workshop
Characteristics of Good Self-Study
Reports
2. The 3 qualities of the report
1) Provide sufficient description
2) Document appropriate evaluation
3) Identify strategic recommendations
ATS Commission on Accrediting: 2014 Self-Study Workshop
Characteristics of Good Self-Study
Reports
2. The 3 qualities of the report
2) Document appropriate evaluation
Weak report: “Our MDiv program addresses the ‘cultural context’
requirement (Degree Program Standard A, section
A.2.3) fairly well, but could use improvement.”
Good report: “We evaluate the ‘cultural context’ component of
our MDiv program in two ways: (1) a cultural
awareness survey (an indirect measure) and (2) a
cultural engagement project (a direct measure).
The results for these two assessments are
provided in Table A.2 below, followed by a
discussion of how those results have been used to
improve our MDiv students’ learning in this critical
area.”
ATS Commission on Accrediting: 2014 Self-Study Workshop
Characteristics of Good Self-Study
Reports
2. The 3 qualities of the report
2) Document appropriate evaluation
Weak report:
“We try hard not to spend more than we take
in.”
Good report: “The seminary has balanced its budget four of the
last five years, as Table 8.6 below demonstrates.
The deficit of $74,892 in FY12 represented only a
2.5% overage due to an unexpected need to
replace a furnace. We have since established a
‘contingency fund’ for such emergencies, with a
line-item budget of $50,000 per year.”
ATS Commission on Accrediting: 2014 Self-Study Workshop
Characteristics of Good Self-Study
Reports
2. The 3 qualities of the report
1) Provide sufficient description
2) Document appropriate evaluation
3) Identify strategic recommendations
ATS Commission on Accrediting: 2014 Self-Study Workshop
Characteristics of Good Self-Study
Reports
2. The 3 qualities of the report
3) Identify strategic recommendations
Weak report: “We know more should be done to address our lack
of board diversity, and we are working on that.”
Good report: “Given our global mission and given that one fourth
of our students and staff are ethnic minorities, while
no one on the board is, our trustees have set a goal
of recruiting one new ethnic trustee each year for the
next five years until our board more fairly represents
our constituency and our mission. Our first African
American trustee will begin her term this fall.”
ATS Commission on Accrediting: 2014 Self-Study Workshop
Characteristics of Good Self-Study
Reports
2. The 3 qualities of the report
3) Identify strategic recommendations
Weak report: “As you will see scattered throughout this report,
we have identified dozens and dozens of different
areas to improve.”
Good report: “We have organized each chapter of the report
(one per standard) into four parts: (1) a brief
description of that area, (2) a critical evaluation of
that area, (3) a list of strengths in that area, and (4)
a set of two or three strategic recommendations
per area to address key concerns identified. All
recommendations are summarized and prioritized
in the last chapter.”
ATS Commission on Accrediting: 2014 Self-Study Workshop
Characteristics of Good Self-Study
Reports
Five things to remember to do the “right”
report:
1.
2.
3.
4.
5.
The 4 purposes of the report
The 3 qualities of the report
The 5 features of good/weak reports
The 4 parts of the report
The 3 last things to remember
ATS Commission on Accrediting: 2014 Self-Study Workshop
3
Characteristics of Good Self-Study
Reports
3. The 5 features of good [vs. weak]
reports
1) Are organized coherently
2) Describe the self-study process
3) Present the data crucial to understanding
4) Comprehensive as necessary; brief as
possible
5) Demonstrate objectivity and honesty
ATS Commission on Accrediting: 2014 Self-Study Workshop
Characteristics of Good Self-Study
Reports
3. The 5 features of good/weak
reports
1) Are organized coherently
Must be organized by the standards

If joint visit with one report, must have detailed “cross walk”
[Consult with Commission staff about joint visits]
Must also have Introduction and Conclusion
 Will discuss later under “The 4 parts of the report”
ATS Commission on Accrediting: 2014 Self-Study Workshop
Characteristics of Good Self-Study
Reports
3. The 5 features of good/weak
reports
2) Describe the self-study process
Introduction describes process
Process engages the community
Reflected in report and interviews
Caution:
Summarize process in 1-2 pages
If use people’s names, use titles too
ATS Commission on Accrediting: 2014 Self-Study Workshop
Characteristics of Good Self-Study
Reports
3. The 5 features of good/weak reports
3) Present the data crucial to understanding
Be clear and concise with data
Use qualitative and quantitative data
Use tables and charts to support narrative
Caution: Clearly label each item
Put extensive data in Documents Room
and/or hyperlink in narrative
But include summary data in text
When using links, make easy to use
ATS Commission on Accrediting: 2014 Self-Study Workshop
Characteristics of Good Self-Study
Reports
3. The 5 features of good/weak reports
3) Present the data crucial to understanding
4.9
Chart 8.2
Revenue vs.
Expenses
4.8
4.7
4.6
Revenue
Expenses
4.5
4.4
4.3
[in millions]
4.2
4.1
FY09
FY10
FY11
FY12
FY13
ATS Commission on Accrediting: 2014 Self-Study Workshop
Characteristics of Good Self-Study
Reports
3. The 5 features of good/weak reports
3) Present the data crucial to
understanding
Weak report:
“Our students are very diverse.”
Good report:
“Our 150 students, whose ages range
from 24 to 64 (with most in their 30s), are 45%
female, and 30% ethnic minorities, with a
dozen different denominations represented
(see Table 6.3 below for demographic details).
This is what one recent graduate said about
her experience here regarding diversity:
ATS Commission on‘[quote].’”
Accrediting: 2014 Self-Study Workshop
Characteristics of Good Self-Study
Reports
3. The 5 features of good/weak
reports
3) Present the data crucial to
understanding
Weak report:
“Most of our MDiv students are
placed.”
Good report:
“Over the last five years our MDiv
graduates have averaged a 90% placement
rate, with a steady increase each year, as the
ATS Commission on Accrediting: 2014 Self-Study Workshop
Characteristics of Good Self-Study
Reports
3. The 5 features of good/weak
reports
3) Present the data crucial to
understanding
Weak report:
“We evaluated our student services
through a survey last year; see data in
Appendix 6.”
Good report:
“Faculty and staff reviewed the results
of a recent student survey (found in
Documents Room; File 6.1 Student Survey for
ATS Commission on2014;
Accrediting:
2014 Self-Study
see also
this link)Workshop
and drew these four
Characteristics of Good Self-Study
Reports
3. The 5 features of good/weak
reports
4) Comprehensive as necessary; brief as
possible
Weak reports:
• Missing standards (e.g., Degree Program Standards)
• Too few pages: 50 pages won’t cover the standards
• Too many pages: 175+ pages overwhelm reviewers
• Printed incorrectly: Single-sided/double-spaced
reports double the length and the weight (remember
your traveling readers)
ATS Commission on Accrediting: 2014 Self-Study Workshop
Characteristics of Good Self-Study
Reports
3. The 5 features of good/weak
reports
4) Comprehensive as necessary; brief as
possible
Good reports:
• 100-135 pages is “ideal range”




5-10 page Introduction
75ish pages for General Institutional Standards (1-8)
15-40 pages for Educational and Degree Program Standards
5-10 page Conclusion
• Don’t overwhelm text with supplemental
ATS Commission
on Accrediting: 2014 Self-Study Workshop
information
Characteristics of Good Self-Study
Reports
3. The 5 features of good/weak reports
5) Demonstrate objectivity and honesty
Weak report: “Morale on campus is much better.”
Good report: “Three years ago faculty morale hit a low point
when a doctrinal controversy led to the dismissal
of a popular professor. The seminary followed its
procedures (documented in our personnel files),
but it was a very difficult time. Since then, the
board and administration have worked hard to
rebuild unity, with significant success—as noted in
the recent faculty survey (see Documents Room,
File 5.4: Faculty Survey, item #8; see also this
link).”
ATS Commission on Accrediting: 2014 Self-Study Workshop
Characteristics of Good Self-Study
Reports
3. The 5 features of good/weak
reports
5) Demonstrate objectivity and honesty
Weak report: “The seminary has experienced some
administrative turnover since the last visit.”
Good report: “We have had three presidents and four
deans in the last decade. These turnovers
have occurred for various reasons and with
varying results. We will summarize those
reasons and results here, but the evaluation
committee is also encouraged to ask about
this during their on-site interviews to have a
more nuanced understanding of this issue.”
ATS Commission on Accrediting: 2014 Self-Study Workshop
Characteristics of Good Self-Study
Reports
3. The 5 features of good/weak reports
5) Demonstrate objectivity and honesty
Weak report:
[no mention of recent public controversy]
Good report:
“Just prior to submitting this self-study report to
our trustees for their review, five students filed a
complaint against a faculty member alleging
sexual harassment. Unfortunately, portions of
that complaint have surfaced recently in social
media. We have initiated our formal complaint
process, and we will inform the evaluation
committee of any significant developments that
we are permitted legally and ethically to share.”
NOTE: Boards are to “receive” self-study reports (Chapter Two, p. 10)
ATS Commission on Accrediting: 2014 Self-Study Workshop
Characteristics of Good Self-Study
Reports
Five things to remember to do the “right” report:
1.
2.
3.
4.
5.
The 4 purposes of the report
The 3 qualities of the report
The 5 features of good/weak reports
The 4 parts of the report
The 3 last things to remember
ATS Commission on Accrediting: 2014 Self-Study Workshop
4
Characteristics of Good Self-Study
Reports
4. The 4 parts of the report*
1) Introduction
2) Main Narrative
3) Conclusion
4) Appendices (not printed; on flash drive only)
* Pages 13-14 of Chapter Two of the Handbook
[NOTE: These 4 parts were discussed, in part, earlier]
Reminder: For how to copy and distribute the report and the
appendices, see Chapter Three, p. 12 and p. 16)
ATS Commission on Accrediting: 2014 Self-Study Workshop
Characteristics of Good Self-Study
Reports
4. The 4 parts of the report
1) Introduction
2) Main Narrative
3) Conclusion
4) Appendices
ATS Commission on Accrediting: 2014 Self-Study Workshop
Characteristics of Good Self-Study
Reports
4. The 4 parts of the report
1) Introduction (5-10 pages)
 Introduce the school (mission/history)
 Review accreditation history (since last
visit)
 Give overview of the self-study
process
 Identify any special features of the
report
 Summarize any major changes (since last
ATS Commission on Accrediting: 2014 Self-Study Workshop
Characteristics of Good Self-Study
Reports
4. The 4 parts of the report
2) Main Narrative (90-115 pages)
Part 1: General Institutional Standards (75 pp.)
Standards 1-8—must cover each one
Part 2: Educational and Degree Program Standards (15-40
pp.)
Educational Standard—cover only applicable sections
Degree Programs Standards—cover only applicable
standards
NOTE:
Can cover
assessment
ES.6
or under section 1 of
ATS
Commission
on Accrediting:
2014under
Self-Study
Workshop
Characteristics of Good Self-Study
Reports
4. The 4 parts of the report
2) Main Narrative (90-115 pages)
One suggested structure per chapter (per earlier
slide):
 brief description of school’s resources for that standard
 critical evaluation of school’s alignment with standard
 list of strengths of the school in terms of that standard
 strategic recommendations to improve in that standard
NOTE: If not meeting standard, must describe plans to do so.
* Do not need to cover every subsection in every standard: group things.
ATS Commission on Accrediting: 2014 Self-Study Workshop
Characteristics of Good Self-Study
Reports
Standard 4 Library (as
example)
Weak Report:
Discuss each of 10
subsections
Good Report:
Discuss each of 3 sections
ATS Commission on Accrediting: 2014 Self-Study Workshop
Characteristics of Good Self-Study
Reports
4. The 4 parts of the report
3) Conclusion (5-10 pages)
 Summarize overall findings of selfstudy
 Organize and prioritize strengths
 Organize and prioritize concerns
 Organize and prioritize
recommendations
 Conclude with desired action
ATS Commission(optional)
on Accrediting: 2014 Self-Study Workshop
Characteristics of Good Self-Study
Reports
4. The 4 parts of the report
4) Appendices (lengths vary, but try to keep under 250
pages)
 Put only these items in the appendices (PDF on
USB);
put everything else in Documents Room and/or
hyperlink
1) Current strategic plan
2) Current budget
3) Most recent audit
**
4) Handbooks*
5) Academic Catalog*
6) Targeted Issues Checklist
* Trustee, faculty, staff, and student handbooks
** Also provide Word version of TIC for committee to complete
ATS Commission on
Accrediting:
2014
Self-Studyinto
Workshop
Appendices
must
be combined
one PDF and bookmarked
Characteristics of Good Self-Study
Reports
4. The 4 parts of the report
Documents Room (pp. 15-16 of Chapter Three of Handbook)
“Historically, schools … have been required to provide paper
copies of various documents in a Documents Room…
Increasingly, schools are asking if some or all of those
documents could be provided instead electronically…
A Virtual Documents Room is acceptable, provided these
five conditions are met…” [listed on pp. 15-16 of Chapter Three]
KEY: Easily accessible and clearly
marked
ATS Commission on Accrediting: 2014 Self-Study Workshop
Characteristics of Good Self-Study
Reports
Five things to remember to do the “right”
report:
1.
2.
3.
4.
5.
The 4 purposes of the report
The 3 qualities of the report
The 5 features of good/weak reports
The 4 parts of the report
The 3 last things to remember
ATS Commission on Accrediting: 2014 Self-Study Workshop
5
Characteristics of Good Self-Study
Reports
5. The 3 last things to remember
Tell your story
in light of the standards
to improve your students and
school
Where will this
journey take
you?
ATS Commission on Accrediting: 2014 Self-Study Workshop