Enhancing Excellence in
Assessment: Being
Successful with MSCHE
Standards 7 & 14
SUNY Council on Assessment
November 2012 Workshops
Deborah Moeckel
Nancy Willie-Schiff
SUNY Assessment Policy - 2010
Campuses expected to meet or exceed MSCHE’s
standards by assessing:
• general education (based on common SUNY
SLOs)
• academic programs at all levels (on a 5-7 year
cycle, or with programmatic accreditation)
• institutional effectiveness.
http://www.suny.edu/provost/academic_affairs/assessment.cfm
2
Middle States Commission on Higher Education
Characteristics of Excellence in Higher
Education: Requirements of Affiliation
and Standards for Accreditation
(Updated 2011)
http://www.msche.org/publications/CHX-2011-WEB.pdf
3
Crosswalk of Standards
to Federal Regulations
• One each for Standards 7 and 14.
• Explains the source of each standard’s fundamental
elements.
• Produced by Middle States staff.
4
MSCHE Standard 7
Institutional Assessment
The institution has developed and implemented an assessment process that
evaluates its overall effectiveness in achieving its mission and goals and its
Compliance with accreditation standards.
FUNDAMENTAL ELEMENTS
• documented, organized, and sustained assessment process to evaluate and
improve the total range of programs and services; achievement of institutional
mission, goals and plans; and compliance with accreditation standards that
meets the following criteria:
o a foundation in the institution’s mission and clearly articulated institutional,
unit-level, and program-level goals that encompass all programs, services
and initiatives and are appropriately integrated with one another;
osystematic, sustained, and thorough use of multiple qualitative and/or
quantitative measures that:
maximize the use of existing data and information;
clearly and purposefully relate to the goals they are assessing;
5
MSCHE Standard 7
Institutional Assessment
FUNDAMENTAL ELEMENTS (continued)
are of sufficient quality that results can be used with confidence to
inform decisions;
o support and collaboration of faculty and administration;
o clear realistic guidelines and a timetable, supported by appropriate
investment of institutional resources;
o sufficient simplicity, practicality, detail, and ownership to be sustainable;
o periodic evaluation of the effectiveness and comprehensiveness of the
institution’s student learning assessment processes;
• evidence that assessment results are shared and discussed with appropriate
constituents and used in institutional planning, resource allocation, and
renewal…to improve and gain efficiencies in programs, services and processes…
• written institutional (strategic) plan(s) that reflect(s) consideration of
assessment results.
6
MSCHE Standard 14
Assessment of Student Learning
Assessment of student learning demonstrates that, at graduation, or other
appropriate points, the institution’s students have knowledge, skills and
competencies consistent with institutional and appropriate higher education goals.
FUNDAMENTAL ELEMENTS
• clearly articulated statements of expected learning outcomes, at all levels
(institution, degree/program, course), and for all programs that aim to foster
student learning and development…
• documented, organized, and sustained assessment process to evaluate and
improve student learning that meets the following criteria:
o systematic, sustained, and thorough use of multiple qualitative and/or
quantitative measures that:
 maximize the use of existing data and information;
 clearly and purposefully relate to the goals they are assessing;
 are of sufficient quality that results can be used with confidence to
inform decisions;
7
MSCHE Standard 14
Assessment of Student Learning
FUNDAMENTAL ELEMENTS (continued)
 include direct evidence of student learning;
o support and collaboration of faculty;
o clear, realistic guidelines and timetable, supported by appropriate
investment of institutional resources;
o support and collaboration of faculty;
o sufficient simplicity, practicality, detail, and ownership to be sustainable;
o periodic evaluation of the effectiveness and comprehensiveness of the
institution’s student learning assessment processes;
• assessment results that provide sufficient, convincing evidence that students are
achieving key institutional and programmatic learning outcomes;
• evidence that student learning assessment information is shared and discussed
with appropriate constituents and is used to improve teaching and learning; and
• documented use of student learning assessment information as part of
institutional assessment.
8
Helpful Middle States Publications
•Team Visits: Conducting and Hosting an Evaluation Visit
(Appendix 1)
• Assessing Student Learning and Institutional Effectiveness
•Evaluating Student Learning Assessment processes (rubric)
•Examples of Evidence of Student Learning
•Follow up Reports and Visits
•Summary of Actions a Team May Take
9
Resources
for
Campuses:
Advantages
of
“Systemness”
10
http://www.suny.edu/provost/academic_affairs/assessment.cfm
AND MUCH MORE…
11
http://www.suny.edu/provost/academic_affairs/RegAccred.cfm
AND MUCH MORE…
12
SUNY Listservs
Campus Assessment Contacts
SUNYASSESSSO-L@ls.sysadm.suny.edu
(STATE-OPERATED, INSTANT)
SUNYASSESSCC-L@ls.sysadm.suny.edu (
COMMUNITY COLLEGES, INSTANT)
SUNYASSESS-L@ls.sysadm.suny.edu
(ALL CAMPUSES, WEEKLY DIGEST)
Campus Accreditation Liaison Officers
SUNYALOSO-L@ls.sysadm.suny.edu
(STATE-OPERATED, INSTANT)
SUNYALOCC-L@ls.sysadm.suny.edu
(COMMUNITY COLLEGES, INSTANT)
SUNYALO-L@ls.sysadm.suny.edu
(ALL CAMPUSES, INSTANT)
13
http://www.sunyassess.org/index.html
SUNY Council on Assessment (SCoA)
•
Enhances the assessment of institutional effectiveness
and student learning by building, leading, and
supporting a SUNY-wide assessment community that
fosters collaboration, reduces duplication of effort and
provides guidance and resources to campuses
•
Plans and leads professional development
opportunities for faculty, staff and administrators in
the area of assessment
14
http://www.oneonta.edu/anny/
Assessment Network of New York (ANNY)
•
Founded in 2010, as a private, non-profit professional association
•
Over 200 members by fall 2011, more than 60% from SUNY
•
Stimulates dialogue and the exchange of ideas related to assessment
across MSCHE-accredited institutions and provides support and resources to
those committed to meaningful assessment of student learning and institutional
effectiveness
•
First Executive Board elected in June 2012 (11 members, 6 from SUNY, including 3
State Ops and 3 Community Colleges)
•
SERVICES: Listserv, Newsletter, Conference, Regional Workshops
•
Free, charter memberships still available
15
SUNY’s MSCHE Study
Purpose
• Assess SUNY progress toward goals
• Inform campuses for improvement
Data Sources
• Documents from 41 MSCHE reviews in 2010 – 2012
• Decennial visits, follow ups & PRRs
• Commission actions or team reports
Method
• Identify and classify each finding
Caveats
• No documents for a few additional reviews
• Team reports not definitive
16
Distribution of 41 MSCHE Reviews
in 2010 - 2012 by Type
PRR, 20%
Follow Up, 17%
Decennial, 63%
17
Distribution of MSCHE's 621 Findings
in 2010 - 2012 by Type
Requirements, 4%
Recommendations,
41%
Commendations,
55%
18
Middle States Commission on
Higher Education - Definitions
Commendation: Significant accomplishments, significant progress, or
exemplary/innovative practices.
Suggestion: Non-binding Findings for Improvement.
Recommendation: Institutional action needed for the institution to
continue to meet the standards in Characteristics
of Excellence in Higher Education. (Must be reported on in PRR or
Monitoring Reports.)
Requirement: Institutional actions needed to achieve compliance with the
standard; “requirements” necessitate Commission action of postponement,
warning, probation, or show cause. (Followed by additional monitoring
reports and small team follow-up visit.)
19
Distribution of MSCHE's
341 Commendations in 2010 - 2012
by Standard
14%
12%
10%
8%
6%
4%
2%
0%
20
Number of Campuses with Commendations
in 2010 - 2012 by Standard
25
20
15
10
5
0
21
Distribution of MSCHE's
254 Recommendations in 2010 - 2012
by Standard
25%
20%
15%
10%
5%
0%
22
Number of Campuses with Recommendations
in 2010 - 2012 by Standard
30
25
20
15
10
5
0
23
Distribution of MSCHE's
26 Requirements in 2010 - 2012
by Standard
45%
40%
35%
30%
25%
20%
15%
10%
5%
0%
24
Number of Campuses with Requirements
in 2010 - 2012 by Standard
9
8
7
6
5
4
3
2
1
0
25
Areas for Improvement for Standards 7 and 14
SETTING GOALS
● Use of SLOs for all programs and courses including graduate programs
● Goals for all functional units
● Incorporation of student learning outcomes in institutional effectiveness assessment
PLANNING
● Formal assessment plan document
● Use of quantitative measures, direct measures, and data
● Alignment of all assessments (curriculum mapping)
● Staff and institutional support for assessment
IMPLEMENTATION
● Culture of assessment/faculty engagement
● Communication of assessment results
● Closing the Loop
● Linking assessment to planning and budgeting
● Sustainability of assessment efforts and degree and consistency of implementation
26
Omnibus Requirements (10Q) and
Recommendations (2R)
•Appear most frequently in situations where:
1. assessment efforts appear random and
inconsistent
2. evidence of assessment is lacking
3. the institution has been cited previously for
assessment, and progress seems insufficient
•Are most helpful in delineating all of the elements that
an assessment plan and process need to indicate.
27
Sustainability of efforts, degree or
consistency of implementation (18R)
These recommendations are used primarily when
assessment is underway, but:
•
the process is so complex or labor intensive that it
cannot be sustained long term
•
the process is applied incompletely or inconsistently
28
Linking assessment to
planning and budgeting (13R) and
closing the loop (12R, 1Q)
These recommendations tend to appear when:
• assessment does not seem to be used to create institutional
or departmental budgets
• assessment does not appear to be used in strategic or unit
planning
The process is also especially important for academic planning:
•
implementation of new programs
•
elimination of programs
29
Alignment of all assessments,
including curriculum mapping
(12R, 5Q)
Curricular and assessment alignment: clear linkages
among student learning goals at the program, general
education and course level.
Some teams have looked for:
• evidence that assessment of student learning outcomes
specifically informs faculty grading policy
•
evidence that course syllabi contain this information
30
Development of SLOs for all programs
and courses including graduate
programs (6R)
•
ALL programs and courses must have clearly
identified and published student learning
outcomes (SLOs).
• Program-level information must be available
in catalogs and on websites.
31
Staff and institutional support for
assessment (6R)
These recommendations are primarily evident where:
•
there is no formal ownership of the assessment process in a
particular organizational structure
•
various campus leaders are unresponsive to the need for their
participation in assessment processes
•
appropriate assessment processes lack sufficient institutional
research support
•
processes for assessment are particularly labor intensive
32
Use of quantitative measures, direct
measures, and data (6R)
•
Use of multiple measures is recommended.
•
Quantitative measures in sufficient quantity must be
used.
•
Direct measures of student learning must be
included (as opposed to use of grades or surveys
alone).
33
Communication of assessment results (6R)
•
These recommendations appear in situations where
assessment results appear to be owned by a few
people.
•
The need for access to these results for Boards of
Trustees and College Councils has also been noted.
•
Communication of results is seen as an important
part of linking assessment with planning and
budgeting.
34
Goals for all functional units –
administrative assessment (5R)
•
Implement assessment of all college services.
•
All offices including Business Offices, Financial Aid,
Public Safety, Human Resources, among others,
should have documented unit goals and assessment
plans.
35
Culture of assessment/faculty
engagement (5R, 1Q)
•
Some recommendations have
indicated a need for all faculty to
be actively engaged in assessment
activities.
36
Incorporation of student learning outcomes
in institutional effectiveness assessment (5R)
• The primary purpose of Standard 7 is to
assess whether or not an institution is meeting
its mission and goals. Since the primary mission
of any institution of higher education is student
learning, these outcomes are considered to be
appropriate to this standard.
37
Formal assessment plan document (3R, 1Q)
The assumption that ‘everybody knows’ what needs to be done for
assessment and what the processes are isn’t enough.
Need a document which outlines:
• all of the processes and how they fit together,
• who is responsible,
• what measures are being used,
• what the timeline is, and
• how the results have been used for improvement.
See the omnibus recommendations and requirements for more
detail on what these plans should include. Also helpful is the SUNY
Memorandum to Presidents 2010-02 on the SUNY Website.
38
Questions?
Comments and
suggestions
welcomed!
Deborah.Moeckel@suny.edu
Nancy.Willie-Schiff@suny.edu
39