Webinars for Chairs - Accreditation Council for Business Schools

advertisement
Site Visit Team Chairs
Role and Responsibilities
January 2013
ACCREDITATION COUNCIL FOR BUSINESS
SCHOOLS AND PROGRAMS
AGENDA ITEMS:
 Welcome/Introductions
 Role
of the chair
 Resources available to site visit team
 Scheduling of Consensus Calls
 Travel arrangements
 Communication with institutional contact
 Site Visit Schedule
WEBINAR AGENDA, CONT.
 Working
with first-time evaluators
 Preparing feedback report – format,
editing, timetable, etc.
 Submission of feedback report
 Submission of expense reports and other
forms
 Any other topics
ROLE OF THE CHAIR
Receive self-study of the institution to be visited.
Review self-study for completeness and content.
Identify those areas in need of intensive scrutiny by
the team while on site.
 Coordinate with team members, commissioner,
and mentor to schedule consensus call.
 Determine how the self-study should be divided
among the team members for on site review.
 Prepare tentative assignments for team members
and communicate those assignments to each
member.

ROLE OF THE CHAIR




Contact team members to obtain travel itineraries for
arrival to and return from the site to be visited.
Discuss tentative self-study assignments and areas of
concern identified in the self-study.
Contact institutional coordinator to confirm visit dates.
Determine transportation and lodging
accommodations.
Provide the coordinator with each team member’s
itinerary.
ROLE OF THE CHAIR, CONT.




Plan and finalize schedule of visit to include meetings with
president, head of business unit, faculty, and other key
personnel with the institutional coordinator.
Provide institutional coordinator with a list of sites and
units to be visited by the team.
Provide institutional coordinator with the requirements for
the team work room, including all materials to be provided,
computer and clerical support, and telephones.
Develop a schedule of activities, visits, and inspections for
each day of the visit to be distributed to institutional
representatives and to the visiting team.
ROLE OF THE CHAIR, CONT.
Schedule a meeting of the evaluators for the
afternoon/evening prior to the initiation of the site visit to
- finalize the schedule of activities,
- discuss the self-study,
- finalize the distribution/assignment of individual team
member responsibilities,
- establish the procedure to follow in summarizing
observations,
- determine the format of the exit conference,
- discuss preparation of the team report, and finalize the
time frame for completion of all activities.
ROLE OF THE CHAIR, CONT.
Include time frames in each day’s schedule when the
team can convene to determine progress, identify
problems or needs, and schedule additional interviews or
data gathering as needed.
Communicate to institutional coordinator the
undesirability of scheduling social events unless it is
agreed that such events will facilitate interaction with key
personnel, i.e., board members, alumni, employers,
community residents, etc., who would otherwise be
unavailable.
ROLE OF THE CHAIR, CONT.

Utilize ACBSP staff or the Commissioners as resource
persons to clarify any problems of interpretation, etc., which
may confront the team during the course of the visit.

Assure that each team member performs his/her tasks in
such a manner as to validate the content of the self-study
and not to function as consultants. Do not express personal
comments as Chair nor allow team members to express such
comments regarding their perceptions that the institution
does/does not meet standards, or how the business unit
should be structured, etc., in contrast to what is observed or
documented.

References to how business activities are conducted at team
member’s institutions should not be used as the criterion for
judging the quality of the business unit being examined.
ROLE OF THE CHAIR, CONT.



As team Chair, you will be expected to prepare a
final report of the team visit that will reflect the
consensus of the team as a whole.
The Chair will be expected to prepare an Executive
Summary of the visit to provide an overview of the
entire visit and to express the extent to which the
institution accurately described the business unit
and the extent to which the team was able to
validate the information contained in the self-study
document.
This Summary should be concise (about 3 pages)
and cover all the major standards, e.g., Leadership,
Strategic Planning, etc.
RESOURCES AVAILABLE
The CD enclosed with the site visit materials
contains several folders with these files:
Evaluation and Site visit handouts
ACBSP Observation Guidelines
Scoring Guidelines
Practical Guide for Evaluators (which includes)
Site Visit Issue Worksheet

EVALUATION AND SITE VISIT HANDOUTS, CONT.
Site Visit Questions
 Site Visit Issue Strategy

Evaluator Workbook
 Executive Summary
 Feedback Report


Forms to sign and return – evaluator agreement
form, expense form, and evaluation form
(electronically).
EVALUATOR RESOURCES ON CD, CONT.

Accreditation Process Manuals
Standards and Criteria
 (most current version)


Electronic copy of the self-study when provided
CHANGES IN WORKBOOK FORMAT :
ASSOCIATE DEGREE WORKBOOK EXAMPLE

STANDARD 1: LEADERSHIP


Business Unit administrators and faculty should lead and be involved in creating and sustaining values,
business school or program directions, performance expectations, student focus, and a leadership system
that promotes performance excellence. Values and expectations should be integrated into the business
unit’s leadership system to enable the business unit to continuously learn, improve, and address its societal
responsibilities and community involvement.



Criterion 1.1 - Leadership Strategies
Explain the business unit leadership strategies or systems to foster a high-performance work environment
that results in meeting the mission of the business unit.


Explain how the business unit maintains effective communications and develops participation throughout the
business unit.


Observations of Evaluator:


Is there evidence that all areas in this Criterion have been addressed?


Strengths:





OFIs:
 Yes
 No
CHANGES IN WORKBOOK FORMAT :
BAC/GRAD DEGREE WORKBOOK EXAMPLE

Criterion 1.1.c The business school or program must have processes in place for evaluating the performance
of both administrators and faculty.

Explain how the performances of administrators and faculty are evaluated.



Observations of Evaluator:


Is there evidence that all areas in this Criterion have been addressed?
 Yes
 No


Strengths:





OFIs:

CHANGES IN FORMAT – NO MORE TABLE FORMAT TO ENTER INFORMATION
– NO USE OF + AND – FOR COMMENTS.

Advantage: After each team member has finalized their comments, it is a
simple copy and paste into the feedback report. You do not have to remove
the table formatting.
FEEDBACK REPORT TEMPLATE
NEW FOR ASSOCIATE DEGREE


No changes for Baccalaureate/Graduate Degree Reports. The Executive Summary is
part of the feedback report.


EXECUTIVE SUMMARY



Key summary findings from the feedback report:


The most important strengths or outstanding practices are:






The most significant opportunities for improvement are:
FEEDBACK REPORT TEMPLATE, CONT.



COPY AND PASTE FINAL COMMENTS FROM EVALUATOR WORKBOOK:
STANDARD #1. Leadership
Business Unit administrators and faculty should lead and be involved in creating and
sustaining values, business school or program directions, performance expectations,
student focus, and a leadership system that promotes performance excellence.
Values and expectations should be integrated into the business unit’s leadership
system to enable the business unit to continuously learn, improve, and address its
societal responsibilities and community involvement.




Strengths
Example:
The business unit administrators communicate program directions and
performance expectations to their faculty as part of the faculty handbook and during
the mandatory training provided at the beginning of the academic year.




Opportunities for Improvement
Example:
While performance expectations are clearly communicated, there is no evidence that
the business unit has a process in place to use results from administrator and
faculty evaluations to make improvements.
NEW EVALUATOR CRITIQUE FORM
Excel Worksheet rather than Word Document
Process can now be evaluated by
Pre-site visit performance
Site visit performance
Post Site visit performance
Individual team member performance
The new format will allow ACBSP to provide each team
member with feedback from the team as well as
commissioners.
NEW FORMAT FOR EVALUATOR CRITIQUE:
SAMPLE CONSENSUS CALL AGENDA:

Introduction of Participants / Self-introductions (5 min)
 Name, School, Location
 History with ACBSP / Experience as an evaluator
 Cell phone contact info


Review of Planned Schedule (5 min)
 Plans for arrival in destination city
 Review of Divide and Conquer plans (attached)
 Travel Plans to Satellite Campuses (if applicable)
 Institution Accommodation forms


Discussion of Scoring Summary Worksheet (30 min)
 Discussion led by Lead Reviewer for each Standard
 Goal = Preliminary score of BiC, VG/E, VG, G, IN, MIN for
each examination item (worksheet attached)
SCHEDULING CONSENSUS CALLS
ACBSP requires that all site visit teams
conduct a consensus call prior to the site
visit. Once you have finalized the date
and time, contact
Maria Hallerud at mhallerud@acbsp.org
to schedule calls.
SAMPLE CONSENSUS CALL AGENDA:

Identification of Additional Info Needs from Institution (10 min)

Roundtable – Open Discussion / Other Topics (5 min)
 Complete/submit Program Evaluator Agreements prior to Site Visit
 Goal: Complete Draft Feedback Report with Strengths and OFI’s before
departure on Wednesday
Wrap-Up / Adjourn (2 min)










Divide & Conquer Plan
Standard 1 - Chair
Standard 2 - Team Member 2
Standard 3 - Team Member 3
Standard 4 - Chair
Standard 5 - Team Member 2
Standard 6 - Team Member 3
Backup = Team Member 2
Backup = Team Member 3
Backup = Chair
Backup = Team Member 2
Backup = Team Member 3
Backup = Chair
SCORING BAND USED DURING CONSENSUS
CALL:







Individual Assessment Dimensions (See Qualitative
Scoring Band Dimensions and Assessment
Guidelines for definitions)
Best in Class
Very Good to Excellent
Very Good
Good
Improvements Needed
Major Improvements Needed
SCORING BAND
Qualitative
Dimensions
Approach/ Deployment
Quantitative
Dimensions
Results
Poor
* no systematic approach evident; anecdotal information
Poor
* no results or poor results in areas reported
Major
Improvements
Needed
* beginning of a systematic approach to the primary
purposes of the item
* early stages of a transition from reacting to problems to
a general improvement orientation
* major gaps exist in deployment that would inhibit progress in achieving the primary purposes of the item
* a sound, systematic approach, responsive to the primary purposes of the item
* a fact-based improvement process in place in key
areas; more emphasis is placed on improvement than
on reaction to problems
* no major gaps in deployment, though some areas or
work units may be in very early stages of deployment
Major
Improvements
Needed
* early stages of developing trends; some improvements
and/ or early good performance levels in a few areas
* results not reported for many to most areas of importance to the applicant’s key business requirements
Good
* a sound, systematic approach, responsive to the overall purposes of the item
* a fact-based improvement process is a key management tool; clear evidence of refinement and improved
integration as a result of improvement cycles and analysis
Good to
Excellent
* improvement trends and/ or good performance levels
reported for many to most areas of importance to the
applicant’s key business requirements
* no pattern of adverse trends and/ or poor performance
levels in areas of importance to the applicant’s key
business requirements
* some trends and/ or current performance levels -evaluated against relevant comparisons and/ or good to
very good relative performance levels
* current performance is good to excellent in most areas of
importance to the applicant’s key business requirements
* most improvement trends and/ or performance levels are
sustained
* many to most trends and/ or current performance levels -evaluated against relevant comparisons and/ or bench-
Good
Good to
Excellent
Best in Class
* approach is well-deployed, with no major gaps; deployment may vary in some areas or work units
* a sound, systematic approach, fully responsive to all
the requirements of the item
* a very strong, fact-based improvement process is a
key management tool; strong refinement and integration -- backed by excellent analysis
* approach is fully deployed without any significant
weaknesses or gaps in any areas or work units
Best in Class
marks -- show areas of leadership and very good relative
performance levels
* current performance is excellent in most areas of
importance to the applicant’s key business requirements
* excellent improvement trends and/ or sustained excellent
performance levels in most areas
* strong evidence of industry and benchmark leadership
demonstrated in many areas
ASSOCIATE DEGREE SCORING SUMMARY WORKSHEET
Evaluator Name:
School Name:
SUMMARY OF EXAMINATION ITEMS
1 LEADERSHIP
2 STRATEGIC PLANNING
3 STUDENT, STAKEHOLDER, AND MARKET FOCUS
4 MEASUREMENT, ANALYSIS, AND KNOWLEDGE MANAGEMENT
4.1 Student Learning Outcomes Assessment
4.2 Program Evaluation
4.3 Student Assessment
5 FACULTY AND STAFF FOCUS
5.1 Human Resource Planning
5.2 Faculty Qualifications
5.3 Faculty Composition
5.4 Faculty Deployment
5.5 Faculty Load
5.6 Faculty Evaluation
5.7 Faculty and Staff Professional Development & Scholarly Activities
5.8 Faculty Instructional Development
5.9 Faculty Operational Policies, Procedures, & Practices
6 PROCESS MANAGEMENT
6.1 Curriculum
6.2 Professional Component
6.3 General Education Component
6.4 Business Major Component
6.5 Off-Campus Operations and Unique Items
6.6 Minimum Grade Requirement
6.7 Learning and Academic Resources
6.8 Support Services
6.9 Educational Innovation
6.10 Articulation and Transfer Relationships
Individual Assessment Dimension
Baccalaureate/Graduate Degree Scoring Summary Worksheet
Examiner Name:
School's Name:
SUMMARY OF EXAMINATION ITEMS
1 LEADERSHIP
2 STRATEGIC PLANNING
3 STUDENT AND STAKEHOLDER FOCUS
4 MEASUREMENT AND ANALYSIS OF STUDENT LEARNING AND PERFORMANCE
4.1 Selection and Use of Information and Data
4.2 Selection and Use of Information Results
4.3 Selection and Use of Comparative Information and Data
4.4 Continuous Process Improvement
5 FACULTY AND STAFF FOCUS
5.1 Human Resource Planning
5.2 Employment
5.3 Faculty Deployment
5.4 Faculty Size and Load
5.5 Faculty Evaluation
5.6 Faculty and Staff Development
5.7 Faculty Operational Procedures, Policies, and Practices
5.8 Scholarly and Professional Activities
Individual Assessment
Dimension
TRAVEL ARRANGEMENTS:
Airfare – ACBSP is trying to reduce expenses for
our member institutions. Contact Steve or Diana
prior to booking a flight if airfare is over $500.
Location will determine airfare.
 Hotel arrangements – some schools will arrange
for direct billing. Others will ask that each team
member be responsible for payment and get
reimbursed from ACBSP. Please ask your
institutional contact how they want to handle this.

TRAVEL ARRANGEMENTS:
Meals – ACBSP appreciates your efforts to
keep expenses down during your stay. Try to
avoid the most expensive items on the menu
and purchasing alcohol as some schools do not
cover alcoholic beverages for meal
reimbursements.
 Share transportation when possible.

INSTITUTIONAL CONTACT
The chair serves as the primary contact for the
team. Any questions for the school should
come through you so that the contact person is
not overwhelmed with multiple messages and
to avoid conflicting information.
 If you have difficulty getting responses from the
institutional contact or other team members,
please let ACBSP know.

INSTITUTIONAL CONTACT
As you know, some institutions are more
prepared for a site visit than others.
You can help by:
 providing a list of documents you will need in
the resource room,
 asking where the resource room is located,
 where you should park, get lunch, etc.
SITE VISIT SCHEDULE
The Practical Guide for Evaluators - good
outline of how the schedule should be
arranged. Your schedule may vary depending
on the availability of administrators, advisory
groups, etc.
 Build in time for the team to regroup and
discuss any additional items that have yet to be
covered or need clarification.

WORKING WITH FIRST-TIME EVALUATORS
Make sure they know you are available for
questions and assistance.
 Provide examples of how you conduct your
review.
 Explain that institutions provide examples of
how they meet criteria in various areas of the
self-study, not necessarily where they are asked
to address the specific criteria.

WORKING WITH FIRST-TIME EVALUATORS
Do they understand what has been provided or
need guidance?
 Reinforce the importance of following the
schedule and meeting deadlines.
 Suggest wording of questions if they are
struggling with getting information.
 Review the comments early in the process to
see if the comments sufficiently address the
standard.

PREPARING THE FEEDBACK REPORT
Executive Summary – no more than three
pages – cover the major strengths and OFI’s.
 The feedback report should
 Be written in complete sentences with correct
grammar and punctuation.
 State that “there is no evidence” rather than “it
is not clear”. Any questions regarding a
standard or criteria should be clarified during
the site visit.

FEEDBACK REPORT, CONT.
The feedback report should
 Use the same style and font throughout the
document. After comments are copied and
pasted, make sure format is the same.
 Be reviewed by the entire team with a final edit by
the chair prior to submitting to ACBSP. Please
complete the final review after returning home and
submitting two to three days after the site visit, but
no later than 10 days following the visit.

FEEDBACK REPORT, CONT.
The feedback report should not indicate an OFI if
evidence of meeting a standard or criteria was
provided on site even though not covered in the
self-study. It is an OFI if the business unit does not
provide evidence that the standard or criteria was
met.
 Example: While the business unit did not provide
information about Standard Four, Criteria 4.4 in
the self-study, some evidence of improvements
were provided on site for accounting, but no
improvements were observed in the management
discipline.

FEEDBACK REPORT CONTINUED:

THINGS TO REMEMBER:
Comments are based on the requirements of the standards and criteria,
not personal opinions or how processes are followed at other institutions.
Comments should be clear and concise and should address the situation,
how the business unit does (for strengths) or does not meet (for OFI’s) the
standard, and explain the significance of the situation.
Comments should not be prescriptive unless you are using language
directly from the standards and criteria.
Comments should be written in complete sentences and should use proper
grammar and punctuation.
Avoid the use of slang or local expressions others may not be familiar with,
especially when on international visits. Example: “pig in a poke”.
FEEDBACK REPORTS - STRENGTHS:
If the institution has a strong strategic plan that includes both short and
long-term goals and
has a good approach,
has been deployed or implemented,
demonstrates learning,
and identifies improvements, your comment could state:

“The John Doe School of Business has a strategic plan that has
identified both short and long-term goals. The plan is systematic with
an approach that has been implemented for all disciplines, provides
examples of learning and evidence of improvements the school has
made. The strategic plan helps the school to plan for increasing
enrollment and the replacement of faculty members as they retire.”
WRITING A STRENGTH:
If the institution has an assessment instrument and provides evidence of a
systematic process and examples of how the results are used to make
improvements, you could provide a comment such as:
 “The John Doe School of Business has administered the MFT test for all
disciplines for five years. Each department documents their review of the
pre and post results each semester during department meetings and makes
improvements based on the results. For example, the accounting
department added additional projects for preparing financial statements in
the ACCT 3300 course.”
FEEDBACK REPORTS - OPPORTUNITY FOR
IMPROVEMENTS (OFI’S):
Suppose the institution has identified:
 - increasing enrollment and a large number of faculty
scheduled for retirement as strategic challenges.
 - strategic objectives in the strategic plan are for one to three
years, and faculty are not consulted in the development of the
objectives.
The opportunity for improvement could read,
“While the John Doe School of business has strategic objectives
for up to three years, there was no evidence of planning beyond
three years or involvement by faculty in the development of the
objectives. Developing long term goals and involving more
stakeholders such as faculty could provide resource planning to
meet the increasing enrollment and replacement of faculty
members scheduled to retire in the next five years.
WRITING AN OPPORTUNITY FOR IMPROVEMENT:
Suppose the institution administers the Major Field Test (MFT) to
all students as a pre and post test.
 There is no evidence, either in the self-study or on-site, that the
results are reviewed to indicate that there is learning or
subsequent improvements are made. In addition, the school
does not use the data for comparative purposes
Your opportunity for improvement comment could read:
“The John Doe School of Business provided three years of data for
the Major Field Test (MFT). However, there is no evidence that the
school is using this data to measure how their students are
performing in each discipline and how the results compare to
other institutions with a similar mission. Analyzing the results to
identify program strengths and weaknesses could allow the
school of business to make improvements where needed and
provide an external source of comparative data.”
FEEDBACK REPORT, CONT.
Make sure your comments are written so that
the business unit understands why an OFI was
provided.
 There is a big difference (conditional
accreditation v deferral) between providing an
OFI because the business unit does not have
results from an assessment and having results,
but not using the results to make
improvements.

FEEDBACK REPORT, CONT.
Once all team members have reviewed the
feedback report in the entirety and it has been
edited with a spell-check and checked for
grammar,
 Email the report as a Word document to:
 reports@acbsp.org. Please request a read
receipt so you know it was received. We do not
need a hard copy.

SUBMISSION OF EXPENSES, ETC.
Submit expense form and all receipts as soon
as possible following the site visit. This can be
sent to Mary Riley via email at
maryriley@acbsp.org or fax at 913-339-6226.
 Payments are processed on the 15th & 30th
each month.
 Submit evaluator agreement forms and
evaluation forms via email at
reports@acbsp.org or fax to 913-339-6226.

BEST PRACTICES – WHAT DO YOU DO?

Please share practices you have found to be
helpful on site visits.
WRAP-UP

Final Questions

Thank you for giving your time and helping
ACBSP continuously improve our processes.
Download