REPORT OF THE WASC VISITING TEAM
EDUCATIONAL EFFECTIVENESS REVIEW
UNIVERSITY OF HAWAI’I AT HILO
March 17 – 19, 2004
In Partial Fulfillment of the Requirements for
Reaffirmation of Accreditation
James E. Lyons, Sr., Chair
Katharyn W. Crabbe, Assistant Chair
Dean Elias
Dan Hocoy
Denise M. Lucy
Nancy Teskey
The evaluation team in conducting its review was able to evaluate
the institution according to Commission Standards and Core
Commitments and therefore submits this Report to the
Accrediting Commission for Senior Colleges and Universities of
the Western Association of Schools and Colleges for action and to
the institution for consideration.
Ta
bl
e
of
Contents
Overview and Context……………………………………………………………....1
Quality of the Educational Effectiveness Presentation and
Alignment with the Proposal………………………………………………..3
Preparatory Review Update…………………………………………………………5
Evaluation of Educational Effectiveness
Theme #1 Organizing to Provide and Support Student Learning…………..5
Theme # 2 Becoming a Learning Organization…………………………….12
Theme # 3 Striving for a Collective Vision of Educational Attainment……15
General Education…………………………………………………………………..18
Evaluation of the Institution’s Systems for Enhancing Teaching Effectiveness
And Learning Results……………………………………………………….20
Summary of Conclusions……………………………………………………………22
Recommendations…………………………………………………………………...23
Overview and Context
IA
Description of Institution and Visit
The University of Hawaii at Hilo (UH Hilo) is a public comprehensive university
located in the city of Hilo on the island of Hawaii. Organized in 1970 as part of the tencampus system of the University of Hawaii, UH Hilo is one of three public baccalaureate
degree granting institutions in the state. It is the only accredited four-year institution on
the island of Hawaii.
UH Hilo's 35 bachelor's degree programs and its two master's degree programs
are organized into four colleges: Arts and Sciences; Agriculture, Forestry and Natural
Resource Management; Hawaiian Language; and Business and Economics. The
University also has a College of Continuing Education and Community Service.
The University is experiencing a period of slow but steady growth, reporting
student headcount increases from 2,639 in Fall 1997 to 3,305 in Fall 2003, an overall
increase of 25%.
The Hilo campus of the University of Hawaii was granted candidacy for
accreditation in 1970 as Hilo College. It was accredited by the Western Association of
Schools and Colleges in 1973 and has maintained its accreditation since that time.
In 1992, the Substantive Change Committee of WASC reviewed the separation of
Hawaii Community College from UH Hilo, a separation which, it was noted, should have
been subject to prior review by the Substantive Change Committee. In 1998, the
Committee referred a proposal for a Master's Degree in Hawaiian Language and Culture
to the WASC Comprehensive Visit Team that was scheduled to visit the campus in spring
1
1998. The WASC Senior Commission expressed concern that the University had again
taken an action that required WASC prior approval before securing that approval.
In 1998, the Commission approved the MA in Hawaiian Language and Literature
and required prior approval for any additional master's level programs.
In 1999, the Substantive Change Committee acted to defer a decision on a
proposed MA in China-US Studies and to approve the MA in Education.
In 2000, the Substantive Change Committee approved the MA in Counseling
Psychology and the MA in China-US Studies, noting that the required site visits for these
master's programs would, if possible, occur during the 2003-04 campus visits for the
preparatory and educational effectiveness reviews required by WASC under the 2001
Handbook. The programs were reviewed in spring 2003 as a part of the preparatory
review, and the China-US Studies program is scheduled for implementation in Fall 2004,
the Counseling Psychology program in Fall 2005.
In March 2003, the Preparatory Review Team visited the Hilo campus and
prepared a report that included seven recommendations, five of which the university
could attend to immediately and the last two of which the team acknowledged would
likely take some time to accomplish:
1.
That student learning outcomes for all programs and for General
Education be stated in assessable terms;
2.
That the University compile a set of approved syllabi for all courses in the
current catalog;
3.
That a more inclusive and transparent budget planning process be adopted;
2
4.
That the University draw on the 2002-2010 Strategic Plan and institutional
data for decisions on budget cuts and reallocations;
5.
That the University undertake a focused effort to increase student retention
and graduation rates;
6.
That the University take steps to abolish the College Senates and replace
their functions with a University-wide Faculty Congress;
7.
That the University adopt a more efficient and effective organizational
structure, perhaps by dividing the College of Arts and Sciences into
smaller and more manageable units.
In May 2003, the Commission acted to receive the Preparatory Review Report
and to continue the accreditation of the University of Hawaii at Hilo. The action letter
identified four issues that were to be addressed as a part of the Educational Effectiveness
Report:
1. Mission, Planning and Institutional Resources;
2. Institutional Governance and Organizational Structure;
3. Institutional and Educational Effectiveness; and
4. Diversity.
IB
Quality of the Educational Effectiveness Presentation and Alignment with
the Proposal
The University of Hawaii at Hilo Educational Effectiveness Review is organized
around three themes:
1.
Organizing to provide and support student learning;
2.
Becoming a learning organization;
3
3.
Striving for a collective vision of educational attainment.
The WASC team agrees that the institutional review is consistent with the
campus’s proposal for the review.
The degree of constituency involvement in developing the Educational
Effectiveness report is worthy of commendation. The report was prepared by the
Accreditation Steering Committee, drawing on information provided by various campus
committees as well as the offices of academic and administrative units. The draft report
was circulated to administrators and select faculty in early versions in November 2003
and posted on the UH Hilo website on December 1, 2003, and comments and corrections
were invited. This is the same methodology the campus employed in the development of
the strategic plan and in the development of the Preparatory Review. It seems to work
effectively for the purpose of including faculty in the discussions.
The primary method of institutional inquiry employed in the report is culturally
appropriate; it is the identification and telling of stories relevant to the three themes.
Although the approach does not feature a pointed formulation of a typical “research
question,” it does give the campus sufficient clarity as to which issues are important to
allow for the identification of relevant evidence and revelatory stories.
The campus has demonstrated its ability to identify, collect and analyze data to
assess its effectiveness in the three areas of their review. The WASC team agrees that the
report, its appendices, and the campus website are rich sources of data and that the report
makes good use of the data that has been gathered.
4
IC
Preparatory Review Update
Based on information gleaned from the campus website; work products of budget
hearings and committee meetings; the Strategic Plan Performance Indicators; University
of Hawaii Hilo Faculty, Staff, Student Statistics, Fall 2003; the University of Hawaii
Strategic Plan for Diversity 2002-2003; the University of Hawaii Hilo Affirmative Action
Plan 2000-2001; and campus interviews, the Team concludes that the University of
Hawaii Hilo has accomplished a great deal in the period since the Preparatory Review
visit and has demonstrated responsiveness to most of the recommendations in the report
of the visiting WASC Team and the Commission action letter. The one exception to the
above is in the area of faculty Diversity. Based on the information we received during
the campus interviews, it appears that the work of the Office of Equal Employment
Opportunity/Affirmative Action is not well integrated with University planning and
funding; and, the Office has insufficient authority to enforce existing policy in promoting
faculty diversity.
II.
Evaluation of Educational Effectiveness
Theme #1
Organizing to Provide and Support Student Learning addresses the
institution’s need for documented procedures and guidelines in administrative and
academic operations. Underpinning the campus’s achievement in this area is the
Strategic Plan 2002-2010 with its clearly stated goal of having UH Hilo become Hawaii’s
comprehensive university and its commitment to have UH Hilo lead in addressing the
social, economic and workforce needs of the Big Island and the state of Hawaii.
Imbedded in the strategic plan is a set of more than thirty performance indicators. The
5
campus Academic Policy Committee and the Budget and Planning Committee have
prioritized nineteen strategic initiatives using principles established in the Strategic Plan.
Strategic Plan
Commendation: (CFR 4.1) The team commends UH Hilo for its significant
progress in developing a strategic plan with persuasive performance indicators for
achieving its educational objectives. The success metrics include specific data gathering
and means of reporting for each objective. Interviews with faculty and staff reveal that
the plan involved multiple constituencies in the reflection and planning process and had
significant faculty participation. The annual status report on each objective is valuable. It
is less clear how the campus is engaged in the process of determining the successful
completion of strategic planning objectives, and how that progress is communicated to
the campus community.
Many of the performance indicators are assessable and reflect the University’s
movement toward forming a culture of evidence. There are a number of performance
indicators stated in vague terms with measures of success that are not definitive. There is
also an open-ended timeframe for completing the objectives with few corresponding
target completion dates.
The Strategic Plan has the potential to be a very useful tool for inspiring change
and monitoring progress. The capacity to realize the vision is limited by the severe
budget challenges facing UH Hilo as part of the UH System. Nonetheless, the Strategic
Plan offers a tool to advance educational effectiveness through charting a direction in
alignment with the vision.
6
Information Sources: EE plan, EE plan appendix D, group and individual interviews with
administrators and faculty.
Academic Program Review
The second major strand of campus activity under theme #1 is the revision and
implementation of the academic program review process. The academic program review
procedures were first adopted in spring 2002 and were revised in spring 2003. As of
spring 2004, UH Hilo has adopted a program review process that culminates in a formal
Memorandum of Understanding between the program and the Vice Chancellor for
Academic Affairs for specific program improvements to remedy the issues identified in
the review.
The concluding step of each program review now incorporates the use of a
“Memorandum of Understanding,” signed by all faculty in the program and the Vice
Chancellor for Academic Affairs. This memorandum includes such information as the
promise to long-term commitment levels on the part of the institution to the program and
the commitment levels of the Vice Chancellor for Academic Affairs to physical and/or
personnel resources. Additionally the memorandum lists the agreed upon areas in the
review that demand further diligence and efforts on the part of the faculty. Suggestions
may be made by the Vice Chancellor for Academic Affairs for future actions to be taken
by the department or program. Given the tight budgetary restrictions on the institution,
this form helps track where resources have been promised and assists in prioritizing
allocation of funds for better educational effectiveness tied to findings in program
reviews.
7
The process as described is quite rigorous, utilizing an external reviewer who
visits the campus, does an assessment, and writes a formal report. The reports reviewed
in the team room (mathematics, computer science) were impressive.
Especially helpful in attaining good program review data is the practice of having
the Institutional Research Officer assist the departments as they begin to gather data for
their reviews. In this way they can see what the office of Institutional Research has
already banked. Placement of much of the data on easily accessible websites has greatly
decreased workloads associated with the gathering and dissemination of this kind of
information.
As of the date of the Educational Effectiveness report, only the B.S. program in
the College of Agriculture, Forestry and Natural Resource Management (CAFNRM) has
completed the entire process, although many departments are engaged in it, and cohorts
of departments are entering the process on time and in orderly fashion.
The results of the process in CAFNRM will include discontinuation of lowdemand curriculum options, combining of options, and the creation of new
interdisciplinary and multi-disciplinary options to respond to changes in student demand.
By the site visit, the Nursing Department, which had recently completed its NLN
accreditation review successfully, also had a Memorandum of Understanding with the
VCAA. Similar examples of program review incorporating data to drive continuous
improvement and responsiveness to student needs can be found in the natural sciences
departments of Biology, Chemistry, Computer Science, Geology, and Mathematics,
although those departments had not quite completed the process at the time the report was
8
submitted, at least in part because of the multiple levels of faculty governance review
required in the College of Arts and Sciences.
The University has made significant progress since the Preparatory Review to
proceed with using program review as a key component of its planning paradigm. The
University governance organizations have publicly embraced the decision to utilize
program review evidence of student learning in the decisions which align academic,
personnel, fiscal, physical and technological needs with the University’s strategic
objectives.
The program review process, using quantitative and qualitative information, is
reported to be linked to the budget allocation process. The current budget process
requires departments to report data on using Board of Regents’ performance indicators as
well as those indicators required for program review. The first program review under the
new system has been completed. Some funding has been made available through the
process of reprioritizing. However, faculty and administrators concur with the concern
that a dedicated budget is needed to support the several improvements proposed as a
response to the program review assessments.
Reviews of the work products of the academic program review process and the
resulting memoranda of understanding demonstrate that departments are making use of
institutional data such as enrollment patterns, student performance on external exams and
in capstone courses, and student portfolios to identify areas of the curriculum that require
attention and to set performance goals for the next cycle.
With regard to “organizing to provide and support student learning,” there is
evidence that UH Hilo has engaged the multiple constituencies of which it is comprised
9
in institutional reflection and planning. The upper administration of UH Hilo also appears
committed to a continuous process of review that informs strategic planning.
Discussions with administration and faculty leaders surfaced an ongoing issue
regarding a lack of clarity around lines of authority and responsibility within the
governance structures and its potential to interfere with educational effectiveness.
Individuals who were interviewed expressed concern that the lack of clarity around the
roles of the Faculty Congress and the College Senates could slow progress in program
review evaluations and/or development of general education design and implementation.
Should the slow down keep the energy and efforts of some of the best minds on the
faculty from being validated and implemented, the educational effectiveness of the
University may suffer. While tremendous strides have been accomplished in this arena,
the team recognizes the danger that past achievements may be minimized in light of the
need for governance structures that enable and support the teaching and learning
community.
Use of Data
Outside Academic Affairs proper, the campus points to the on-going practices of
data gathering and analysis to drive planning and budgeting in the Mookini Library, the
Office of Student Affairs, and the Office of Technology and Distance Learning. In sum,
it appears that UH Hilo has been successful in creating a community that has
institutionalized policies and procedures for on-going use of data in the context of
program review to drive planning, budgeting, and program improvement.
To support the diversity initiative and to respond to changes in the collective
bargaining agreement, the campus has updated information related to faculty hiring,
10
reappointment, tenure and promotion, including developing assessable outcomes for
teaching effectiveness and effectiveness in research and scholarship. A handbook for
faculty hiring has been developed and is in use, and provision for continuous feedback
and refinement has been made.
The campus has demonstrated its ability to disaggregate data to inform discussion
and planning to improve retention and graduation rates. Examples are the work being
done by the Office of Student Affairs to address the disproportionate attrition among
mainland Caucasian students and the work of the Department of Mathematics, the interim
Dean of the College of Arts and Sciences, the ad hoc Committee on Remediation, and the
ad hoc Math Needs Group in addressing the mathematics learning needs of students who
enter the University unready to enroll in pre-calculus.
Also in the area of student records, the University catalog has been revised to be
more student-centered and user-friendly. The new student records system (Banner) has
been implemented and the campus is working its way through the myriad challenges that
accompany a records system conversion. The full potential of the system to provide data
on enrollment patterns, student demand, etc., is yet to be realized because of conversion
issues.
To support the decision of the system to allow UH Hilo to expand its graduate
mission, a Graduate Council has been appointed and is in the process of developing
graduate program policies and procedures.
The achievements addressed in the Preparatory Review Update above also
demonstrate campus progress in organizing to provide and support student learning
11
through having greater clarity and efficiency in governance and budgeting, although work
still needs to be done to clarify the academic governance structure.
Enrollment Growth
The University is experiencing a period of slow but steady growth. An overall
enrollment increase of 25% has taken place between 1997 and 2003. Everyone with
whom we spoke was proud of this growth but expressed a concern that the financial
support from the System has not increased proportionately. During several of the
interviews and group sessions, individuals raised concerns that the campus needs to
consider the cost of growth, not only in financial terms, but also in terms of educational
effectiveness. The cost of asking the learning community to work “at the edge” of its
capacity for such an extended period of time could be dangerous. Either growth rates
need to be modified or planning for increased numbers of students must be limited to
areas where the impact on programs will not become an obstacle to educational
effectiveness. (std/ 2 CFR 2/3)
Theme #2 Becoming a Learning Organization Where Theme #1 addresses
the campus’s progress in creating systems that support data- and need-driven decision
making, Theme #2 takes advantage of the opportunity to demonstrate what UH Hilo
faculty, staff, and administrators have learned about their students and themselves and
how they have used what they have learned to achieve continuous improvement .
Data in Support of Educational Effectiveness
Commendation: UH Hilo faculty, staff and administrators are doing an
exemplary job of using data gathered over time through the National Survey of Student
12
Engagement (NSSE) and their Graduating Seniors Survey (GSS) to develop and revise
curriculum in support of student learning and increased retention and graduation rates.
For example, as the University expresses its commitment to use the Island as a
learning laboratory in many ways, 60% of UH Hilo students report participating either in
an internship or in a service learning experience before graduation, a strikingly high
number. The Service-Learning Coordinator described a spectrum of opportunities for
students – from one-day events for working students who cannot make longer
commitments (e.g Make a Difference Day), to weeklong service commitments over
Spring Break (Breakthrough Adventures) to continuing and potentially credit-bearing
experiences (e.g. tutoring ).
Information Sources: EE Report, on-site interviews.
The same spirit and intent is evident within academic affairs as 72% of graduating
seniors reported in Spring 2003 that they had participated in internships, many paid
through Federal grants (an increase of 7% in one year). Virtually all natural science and
CAFNRM faculty involve students in research and writing. For example, through an
NSF-Science Technology Engineering Math program, in Summer 2003 seven student
interns conducted research with faculty, nine more worked during Fall 2003, and five
more joined them in Spring 2003.
Information Sources: EE Report, Grant documents on file in the team room, several
interviews.
The Incoming Student Survey (ISS) is being used to identify predictors for
students at high risk for dropping out or transferring from the institution. These data are
a key part of the enrollment management team’s work in developing a recruitment plan
13
that will bring to UH Hilo students for whom UH Hilo is a good fit, and who intend to
graduate from Hilo. The University has correctly concluded that it will be difficult to
improve graduation rates with a freshman class of which only 43% intend to graduate
from Hilo.
University personnel have also analyzed enrollment data to identify an issue
around declining percentages of students of Hawaiian ancestry and of Filipino ancestry.
It appears that as the campus enrollment has grown, the numbers of students in these two
groups has not kept pace, resulting in a percentage decline.
Using its array of data gathering and its analysis of those data, the campus has
identified several other issues that they are in the process of addressing or are developing
tools that will allow them to address. These issues include the following:
Low scores on NSSE items regarding student/faculty interaction for freshmen;
Student dissatisfaction with academic advising; and
Revising course evaluation surveys to put more emphasis on effectiveness in
supporting student learning.
In many cases feedback from alumni and employers has been included in the
assessment of student learning, primarily via anecdotal means. Some of the program
reviews have tracked their alumni progress into their careers. Academic departments
report that they would welcome the opportunity to utilize alumni in the continuous
improvement process, but the University’s infrastructure to support this goal has not yet
been developed. In summary, UH Hilo is clearly demonstrating its ability to gather
and use institutional data, both aggregated and disaggregated, to drive continuous
program improvement including planning and resource allocation. They are in the early
14
years of this activity, but they have systems and people in place to make their efforts
fruitful and they understand that they will have to continue to make investments in
Institutional Research. It is the opinion of the visiting team that much success awaits
them.
Here it must be noted that, as the campus report states, increased support for the
institutional research function will be required. As the campus begins to look to that
office more frequently, a one-person professional staff will not be able to meet the
campus’s needs for data. In time, some routine retrieval of data (e.g., number of majors,
degrees granted, etc.) may be pushed out to end users once the campus masters the use of
the new integrated records system.
Theme #3, Striving for a Collective Vision of Educational Attainment UH
Hilo is actively working to achieve a collective vision of educational attainment. There
are some very good examples of the campus’s efforts. A significant change is supporting
student learning through the efforts of expanding the number of companies willing to
work with the UH Hilo to link students with a company seeking new employees or
internships. Students once found a paucity of industries willing to interview them on the
island. Now over one hundred and fifty companies are working with the career center on
campus to find “best fit” candidates for open positions in their organizations. The
amount of effort and expertise required to achieve the observed increase in opportunities
for students to find jobs and internships locally is commendable.
Perhaps the closest the campus as a whole has come to attaining a collective
vision of educational attainment is in realizing the vision of the university as an island
learning laboratory. There are examples too numerous to mention of academic programs
15
that are distinctive because of the uniqueness of place and culture—in the College of
Agriculture, Forestry, and Natural Resource Management, in the Education Department
and the Performing Arts Department, in Biology and Environmental Studies, to name a
few.
In fact, the team discovered what seems like a common vision and intended
outcomes from the University’s work during meetings that included administrators,
faculty, and community representatives. The conversations embraced a broad definition
of who may be appropriately called student learners, and who may be the intended
beneficiaries of the University’s educational mission. In effect, what was described is
UH Hilo as a communiversity. The intended beneficiaries of both the curricular and cocurricular activities of UH Hilo include the Island (considered as an ecosystem) as a
whole, as well as specific endangered communities: For example, a UH Hilo student
research project devoted to understanding the biology and social significance of the fish
“Moi” in turn was translated into a curriculum for a local Hawaiian immersion school.
A second example: Hawaiian elders come into classes on campus to engage
students about the history, traditions, and native Hawaiian language. Further, the
University is committed to develop a professional workforce for the State, e.g. with a
focus on Nursing, Education, and the Hawaiian language. In short, we observed what
seems like positive synergy between UH Hilo and the Island community, and an
unexpressed assumption that the mission of UH Hilo is integrally connected with the
health and future of the Island, and the vitality of UH Hilo is dependent on an enlivening
relationship with the Island.
16
Information Sources: EE Report, reports of student work on file in the team room,
multiple interviews on-site.
One area in which the institution continues to struggle toward a collective vision
as revealed in personal interviews is in defining its responsibility to students entering UH
Hilo with deficits in algebra skills. There are no remedial courses in mathematics; thus,
students needing the non-college level assistance must enroll in the community college
located on the Hilo campus for the coursework
Some students do not enroll at the community college even though their assessed
mathematics skills would indicate such placement as wise. These students opt to use the
Math Lab and tutoring to make up their lack of algebra preparation while they are
enrolled in pre-calculus. Most recently a special section of pre-calculus has been
designated specifically for these under-prepared students. In 2004-2005, the department
will submit a new course proposal for a college algebra course that is expected to meet
Senate approval, with the new course being offered as early as Spring 2005. It is most
encouraging to see that the Mathematics Department, using the program review process
as a vehicle, has been able to develop a plan to address the needs of students not
adequately prepared for pre-calculus at entrance. It is also encouraging to note that the
faculty and administration worked collaboratively to develop a multi-pronged plan with
students and student needs at the center.
The commitment to improving programs in support of students and faculty is
demonstrated by administrators and faculty. Evaluation and inquiry are part of the ethos
of the institution. However, there is wide variance among the faculty as to whether
professional judgment should remain as the primary or sole means of evaluation. This
17
mindset is juxtaposed to that of other departments that have embraced and are using
multiple modalities of assessment seeking to illustrate outcomes of student learning, as
well as the need to collect examples of student achievement. There is debate by some
faculty that their departments should even be required to have a mission statement or set
of learning goals for their majors.
Not surprisingly, then, relatively few faculty are including clear statements of
student learning objectives phrased in assessable form in their syllabi. It is difficult to
determine the extent to which this situation is attributable to lack of knowledge or
whether it is a reaction against “externally mandated” requirements. Whatever its origin,
the lack of clear articulation of student learning goals, in assessable form, represents a
significant challenge to the campus.
Self-reports from interviewees indicate that there is significant faculty
involvement in the process of inquiring about whether their teaching is successfully
achieving the intended learning for their students. The new movement toward assessment
practices has ignited discussions in several departments regarding the content of courses
and how these courses fit within a cohesive whole of particular programs. Interviewees
reported that the assessment discussions have helped departments identify duplications
and overlap among courses within a major. These discussions have been energizing and
have identified new ways to revitalize overall programs.
General Education
The General Education (GE) Committee has worked hard to establish the
“hallmarks” (“makia”) for the UH Hilo graduate and has attempted to align these
descriptors with course features and learning outcomes. While no faculty body will ever
18
have complete agreement with the decisions on curricular mandates, the time to move
forward and make choices is upon the University. The vision of what should distinguish
a graduate of UH Hilo can be made clear by campus discussions, and a program that is
both unique in design and contour as well as consistent over time in its effectiveness will
give substance to the idea of a collective vision of UH Hilo education
The process for developing agreement on General Education makia, using them
for course input criteria, and then developing courses and syllabi with the support of the
Congress Assessment Support Committee seems well-conceived and, if successful, may
provide a model for other campuses in the region who are also striving to develop a
collective vision of educational attainment. It is encouraging to see that the plan includes
systematic assessment, feedback, and program improvement and the structures to achieve
them (e.g., syllabus review). A key indicator will be the campus’s ability to come to
agreement on the hallmarks and the syllabus review process within a reasonable period of
time.
The campus has formulated a thoughtful plan for moving forward on the issues of
student learning objectives and syllabi. Structures such as the Congress Assessment
Support Committee and the workshops presented by nationally known experts should
result in progress if the program review process, with its memoranda of understanding
and its link to planning and budgeting, supports their faculty development efforts.
In addition to the work of the Congress Assessment Support Committee, the work
of the General Education Committee will make an important contribution to progress in
the articulation of student learning objectives as well as bringing to the campus one
answer to the question, “What is distinctive about a UH Hilo graduate?”
19
Although the campus has not yet achieved its ambitious goals for identifying and
measuring student learning outcomes as laid out in the Educational Effectiveness
proposal, some departments are demonstrating a high level of engagement and success in
putting student learning at the center of their collective work. The student work products
displayed in the team room, discussions with faculty of the processes employed to
analyze that work for purposes of increasing student learning, and the records of program
changes and the resulting improvements speak eloquently of engaged cadres of faculty in
every college.
IIB
Evaluation of the Institution’s Systems for Enhancing Teaching Effectiveness
and Learning Results.
UH Hilo has made great strides in the period since their last reaffirmation visit in
establishing and refining their systems for enhancing teaching effectiveness and learning
results. The initiative to have all faculty include clear, assessable statements of student
learning goals on their syllabi is well begun. The syllabus website is user-friendly and
accessible. The Faculty Congress Assessment Support Committee and a General
Education Committee are committed to helping faculty and departments master the skill
of expressing student learning objectives in assessable form and to demonstrate it in the
course of the periodic program reviews.
As a part of the Strategic Planning process, the campus has identified more than
thirty performance indicators that pertain to teaching effectiveness and learning results
and is monitoring and reporting on those indicators regularly.
The Office of Institutional Research has created a rolling calendar of datagathering exercises and posts the results of those exercises on its website. Key
20
contributions, such as the National Survey of Student Engagement, the Graduating Senior
Survey, and the Incoming Student Survey are regularly reviewed and discussed by the
campus community.
Since summer 2002, the campus has enjoyed the benefit of an annual New
Faculty Orientation. The two-day event, developed and offered by the Office of the Dean
of Arts and Sciences, focuses on teaching and students, and includes discussion of the
National Survey of Student Engagement and effective teaching practices.
A committee on faculty development and teaching was established in summer
2003. Comprised of distinguished faculty who have been recognized for their teaching
effectiveness, the Advising Coordinator, and an instructional technology professional, the
committee has developed an ambitious agenda of workshops, speakers, and other
resources to lead the campus in developing a shared understanding of teaching
effectiveness.
The quality assurance systems that the campus is developing are completely
congruent with the mission and vision of the campus as expressed in its strategic plan.
As is often the case, the way the campus guides the expectations for periodic program
review will determine the extent to which actual student work and learning results are a
part of the quality review process. A number of departments regularly have their majors
participate in an external examination such as ETS field tests. Even more have designed
a curriculum with a capstone course or experience. The extent to which the products of
those capstone experiences are reviewed in the light of program improvement rather than
in the light of individual student evaluation is not clear, but the tools are available for
such review.
21
The challenge for the campus continues to lie in the resistance of a significant
proportion of faculty to what they characterize as an “externally mandated” requirement
that they articulate, in assessable form, the learning goals they have for their students
and, having done so, assess to understand the extent to which those goals are being met.
This issue is one of culture change, and the WASC team believes that the campus has
been successful in creating and institutionalizing systems to help it make that change.
While there appears to be an authentic intention to utilize information and data to
improve decision making, the next phase of development will be for the University to
take better advantage of the significant amounts and types of data collected and
strategically incorporate those data into proactive decision making which will lead to
more effective performance indicators.
III.
Summary of Team Conclusions and Major Recommendation from the
Preparatory Review and Educational Effectiveness Review
Summary:
In addition to the commendations mentioned in the body of this Report, the
University is to be commended for the substantial progress it has made since WASC’s
visit last year and in its vigorous response to Commission and Team recommendations.
This progress has been made on many fronts. The University’s goal of continuous
improvement is demonstrated through its commitment to enhancing and advancing its
planning for achieving educational effectiveness. The August 2003 Vice Chancellor for
Academic Affair’s letter to the UH Hilo Congress indicates the issues of priority related
to assessment of student learning outcomes, as well as faculty governance structures and
campus academic advising.
22
The University is to be commended for the significant and quality progress it has
made in implementing its plan for Program Review. The new Program Review process is
well underway and is an example of one of the University’s intentional systems of quality
assurance with the goal of improving teaching and learning. Program review is taking
place not only in the academic units, but also in Student Development, the Library, the
Office of Student Affairs and the Office of Technology and Distance Learning.
The University is also to be commended for its progress toward realizing its mission
as a comprehensive University serving indigenous and other under-represented
populations from Hawaii and surrounding islands. For example, UH Hilo has won grants
used to retain and to provide academic support to students of Hawaiian ancestry to the
University (e.g., a $1.7M Title III grant), and the UH Hilo Agriculture Development
Program provides academic support for under-represented minorities aspiring to careers
in agriculture. The University has responded to requests for career-related degree
programs from islands lacking four-year institutions through its Distance Learning
project (e.g. Computer Science, English, and Marine Biology baccalaureate degree
programs provided through Interactive Video and online programs).
Recommendations:
1. While the Team stops short of recommending that the Board of Regents establish
a standing committee on the Four Year University (similar to the standing
committee on community colleges), it does strongly recommend that the
University request the creation of a special task force or a blue ribbon commission
to discuss the mission of UH Hilo as it seeks to grow and expand as a
comprehensive university serving the Big Island and the state of Hawaii.
23
2. The Team recommends that the University, as it moves forward with the
development of its planning and budget processes, find ways to address the results
of its program review system.
3. The Team encourages the campus to continue to seek external funding (both
philanthropic and in grants and contracts) and otherwise identify sources of funds
within the existing budget to help fund the anticipated enrollment growth.
4. The Team recommends that the institution continue to examine current
governance practices critically with an eye to increasing effectiveness. We
suggest that UH Hilo explore governance models in place at other institutions that
are acknowledged as exemplary, and using those models as a foil, examine the
UH Hilo governance structure with an eye toward clarifying lines of
communication and authority and simplifying your practices.
5. The Team endorses the University’s efforts to expand its assessment initiatives
across the campus. The University is encouraged to continue the expansion of its
faculty development. Faculty need to know that the ultimate purpose of
assessment is to improve teaching and learning and not simply to comply with a
perceived external mandate.
6. The Team recommends that the University continue to monitor the impact that
enrollment growth is having on the campus infrastructure, student services, and
staff morale, and put planning for growth under the lens of educational
effectiveness.
24
7. The Team recommends that the University take steps to ensure that the mandate
of the Office of Equal Employment Opportunity/Affirmative Action is broadly
understood and shared. This would include staffing, funding and broad
assignment for the recruitment and retention of a more diverse faculty to the
entire University community.
25