(3rd Year) Report on Assessment

advertisement
Public Administration Program
Assessment and Program Improvement Report
I.
PROGRAM
The Public Administration Program currently offers the Master's of Public
Administration (M.P.A.) as well as several undergraduate courses.
II.
PROGRAM MISSION
The mission of the Public Administration Program is to train students in
the governance, policy, and administration of the public and nonprofit sectors
through intellectual and practical interdisciplinary education. Through excellence
and innovation in teaching, the ultimate goal of the Program is to prepare
students to work in and manage the organization of tomorrow.
The Public Administration Program follows and supports the mission of the
University in:
 Attracting, retaining, and graduating a highly diverse student body of public
administration students;
 Providing a disciplinary and interdisciplinary professional education in public
administration that is academically rigorous and intellectually challenging;
 Providing a public administration curriculum that reflects all dimensions of
human diversity, and that encourages critical thinking and social and cultural
awareness;
 Fostering a collegial and cooperative intellectual environment for students
and faculty that includes recognition and appreciation of differing viewpoints
and promotes academic freedom within the Program community; and
 Serving the communities with which public administration students and faculty
are engaged.
III.
PROGRAM GOALS
Student Learning Goals
Goal 1: Provide students with substantive knowledge about the depth and
breadth of public administration today
Objective 1: Train students to develop an understanding of the
theoretical and intellectual knowledge underpinning the discipline of public
administration - the history and theories of the field, the importance of social,
economic, and political institutions, and the public policy process
Goal 2: Provide students with the skills necessary to manage today’s and
tomorrow’s organizations
Objective 1: Train students in the essential professional skills necessary to
succeed in public administration—research skills, budgeting and policy
analysis skills, skills in organizational development and change, and skills in
personnel management
Objective 2: Train students to write effectively and concisely for an audience
of professional public administrators
Objective 3: Train students to make effective oral presentations to small and
larger groups, provide both experience and the opportunity to develop
confidence while speaking in public
Objective 4: Train students to work in groups and teams, providing positive
experiences and the opportunity to assess the importance of using teams to
accomplish organizational tasks
Objective 5: Train students to utilize the predominant computer and Internet
applications as tools appropriate for the public and nonprofit workplace
Goal 3: Provide students with the ability to be sensitive and
knowledgeable about ethical and diversity issues in the public and
nonprofit sectors
Objective 1: Train students in the issues of public sector ethics and diversity,
focusing attention upon critical thinking and independent evaluations about
these important topics
Goal 4: Provide students with the substantive knowledge about a
specialized area of public administration
Objective 1: Provide students with specialized knowledge in one of the
important subfields of public administration
Programmatic Goals
Goal 5: Work collaboratively with other departments and programs within
the College, the University, and the community
Objective 1: Develop SPIA (the units of political science, international
relations, urban studies, and public administration) as a viable, collaborative
unit
Objective 2: Work with other departments and institutes (like the Public
Research Institute) around the University to create, out of existing courses,
other course concentrations and academic offerings and to work on other
projects
Objective 3: Work with community groups like the San Francisco Bay Area
Chapter of the American Society of Public Administration, other local
professional associations, and local governments interested in and related to
public administration
Objective 4: Utilize the University’s resources in a cost-effective fashion
while still meeting the needs of our students and providing for the professional
development of our faculty
Goal 6: Seek to be recognized as an effective public administration
program, remaining on the cutting edge of public administration, and
becoming and remaining a learning organization that changes when
change is required
Objective 1: Improve and enhance the management of the Public
Administration Program itself, thereby enhancing our effectiveness and
stretching existing resources.
Objective 2: Attain status as an accredited program by the National
Association of Schools of Public Administration and Affairs-FULLY
ACHIEVED 8/2000
Objective 3: Recruit and retain high quality students in both the public and
the nonprofit sectors
Objective 4: Develop Pi Alpha Alpha Chapter (the public administration
honorary society)-- FULLY ACHIEVED 1/2000
Objective 5: Establish a supportive environment for faculty to conduct useful
and important research (both applied and theoretical)
Objective 6: Establish a supportive environment for faculty to become and
stay meaningfully involved in community and professional service.
Objective 7: Continue to evaluate the Program and its successes and
failures in order to improve its operation and education
Objective 8: Seek out new, innovative ways of accomplishing our mission,
goals, and objectives
Objective 9: Seek to distinguish ourselves from other public administration
programs in the Bay Area and California in order to attract high quality
students
Objective 10: Take advantage of the Program’s location in San Francisco
and on the Pacific Rim.
Objective 11: Engage in a continuing strategic planning process,
reevaluating our accomplishments each year and revising our mission and
desired goals, objectives, and outcomes as we move towards accomplishing
our current goals and objectives
IV.
ASSESSING STUDENT ATTAINMENT OF LEARNING
OBJECTIVES
Beginning in Spring 1999, the Program required that students to compile a
Student Learning Portfolio to demonstrate their mastery of the knowledge and
skills required of effective public administrators. The process developed by
Public Administration Program faculty and students has now been recognized by
our national accrediting body, the National Association of Schools of Public
Affairs and Administration (NASPAA) as a "NASPAA Exemplary Practice".
The following areas of student learning outcomes were identified:




Knowledge of Public Administration theories and processes
Knowledge of a Public Administration sub-field
Skill in professional areas such as developing budget proposals, workforce
policies, and understanding statistical analysis
Effective written and oral communication skills, ability to work in teams, and
basic and advanced computer skills
For each of these areas, learning outcomes were identified. Then, work
products were selected that would allow for an assessment of the extent to which
students achieved each outcome. Finally, we ensured that students had the
opportunity to learn the material or skills, to produce each work product, and
have it assessed, in at least one course (often more). Following that activity,
the faculty member (or classmates) evaluates the activity according to a series of
criteria developed for each activity. Another characteristic that distinguishes
this portfolio process is that the portfolio is electronically managed. Student and
Program achievement are tracked by databases and all student materials are
stored in digital format. When students graduate, they will receive a CD-Rom
complete with all their portfolio products and assessments.
In this way, both the Program and the student directly benefit from the
Student Learning Portfolio.
The section below lists each learning goal and each accompanying
learning objective. Following each of these is a table that includes the learning
competency outcomes for each learning objective, where in the curriculum each
of those outcomes is addressed, what criteria is used to assess the outcome,
and the method of assessment.
a. COMPETENCIES-- LEARNING OBJECTIVES AND GOALS
Learning competencies relate directly back to the Student Learning Goals of
Section III, above.
a(i).
Knowledge of Public Administration Competencies
COMPETENCY OUTCOMES
Students posses a core body of knowledge about
public administration history, theories and current
issues in the field today
Students understand the importance and role of
economic, political and social institutions
WHERE ADDRESSED
Successful Completion of Case
Study Analysis in PA 8901.
Students develop an understanding of
organizations and how they work
Successful Completion of Case
Study Analysis in PA 890
Students understand the policy process, policy
issues, and public administrators' role in the
process
Students develop an understanding of human
resources management issues
Successful completion of policy
paper required in PA 730.
METHOD OF ASSESSMENT
Case Study Required in PA 890,
Part of Culminating Experience
Requirement
Case Study Required in PA 890,
Part of Culminating Experience
Requirement
Case Study Required in PA 890,
Part of Culminating Experience
Requirement
Written evaluation by faculty
member teaching PA 730.
Successful completion of paper in
PA 745.
Written evaluation by faculty
member teaching PA 730.
Successful Completion of Case
Study Analysis in 890
Criteria. Each of these competencies is judged upon a five point scale by
the following criteria:





On a scale of 1 (lowest) to 5 (highest), how would you judge the extent to
which student shows a development of knowledge in the area?
How would you judge the extent to which the student synthesizes this
knowledge?
How would you judge the extent to which the student has shown they can
apply their knowledge?
How would you judge the insight that the student has shown in the use
and application of their knowledge?
How would you judge the originality that the student has shown in the
use and application of their knowledge?
a(ii)
Professional Skill Development Competencies
COMPETENCY OUTCOMES
WHERE ADDRESSED
Students develop research skills and become
intelligent consumers of statistical and
qualitative data and research
Students learn to use electronic and other
search methods to develop bibliographic
materials
Students develop ability to conduct cost
benefit analysis
Successful completion of PA 710 homework
exercises and research paper.
Students are able to complete effective budget
proposals
Successful completion of budget development
project in PA 750.
Students develop ability to read and
understand public and nonprofit financial
statements
Students develop an understanding of human
Successful completion of financial statement
exercise in PA 750
Successful completion of paper in PA 710
Successful completion of cost benefit analysis
exercise in PA 720
Successful completion of workforce policies
METHOD OF
ASSESSMENT
Evaluated by faculty
member teaching PA 710.
Written evaluation by
faculty member teaching
PA 710
Written evaluation by
faculty member teaching
PA 720
Written evaluation by
faculty member teaching
PA 750.
Written evaluation by
faculty member teaching
PA 750
Written evaluation by
resources issues and workforce policies
exercise in PA 745
faculty member teaching
PA 745
Criteria. The criteria used to judge effective development of professional
skills is based upon a scale of 1 (lowest) to 5 (highest)-- how would you judge the
extent to which student has demonstrated they can use these skills?
a(iii)
Written Communication
OUTCOMES
Students have the ability to write concisely
and effectively
WHERE ADDRESSED
Successful completion of two (2) written
papers, the Case Study Analysis in PA 890,
and the Culminating Experience paper
(Subfield paper) in PA 895. (Other evidence
from other courses is possible.)
METHOD OF
ASSESSMENT
Written evaluation by
faculty in courses, PA 890,
and PA 895
Criteria. The criteria used to judge effective development of written
communication skills is based upon a scale of 1 (lowest) to 5 (highest)- has the
student:






demonstrated a working knowledge of theories, concepts and methods
appropriate to the discipline in their writing?
demonstrated critical thinking in their writing?
presented and developed a clear line of analysis (effective opening and
closing with well-organized points throughout the paper) in their writing?
was able to write concisely?
demonstrated good basic writing skills: grammar, spelling, and sentence
structure?
demonstrated originality in his or her writing and analysis?
a(iv).
Oral Communication
COMPETENCY OUTCOMES
WHERE ADDRESSED
Students have the ability to make effective
oral presentations
Successful completion of at least three (3) oral
presentations in selected courses throughout
the Program (PA 700, 710, 720, 740, 745 750).
At least one presentation should be
accompanied by a PowerPoint presentation.
METHOD OF
ASSESSMENT
Written evaluation by
faculty member teaching
in each course.
Criteria. The criteria used to judge effective development of oral
communication skills is based upon a scale of 1 (lowest) to 5 (highest)- does the
faculty member think the student:








was clear in the oral presentation?
was interesting and engaging in their oral presentation?
was well organized?
was attentive to the audience?
delivered the oral presentation in a professional manner?
demonstrated originality and creativity in the presentation
made use of useful and appropriate handouts?
used the time well (including staying within the allotted time)?
a(v).
Development of To Work Effectively in Teams
COMPETENCY OUTCOMES
Students have the ability to work
effectively in teams
WHERE ADDRESSED
Successful completion of two (2) team projects
in PA 740, PA 750, other core courses, or
electives.
METHOD OF
ASSESSMENT
Written evaluation by
faculty member teaching
in each course.
Criteria. The criteria used to judge effective development of team work
skills is based upon a scale of 1 (lowest) to 5 (highest)- did the student:
 contributed a fair share of the group’s work,
 made a valuable contribution to achieving group goals?
 shown reliability and timeliness in completing group tasks?
 demonstrated a willingness to compromise when making group decisions?
 exhibited a willingness to effectively listen to others in the group?
 demonstrated an ability to contribute good ideas that lead to group success?
 shown leadership skills in accomplishing tasks?
a(vi)
Computer Skills
COMPETENCY OUTCOMES
Students are able to make effective use of
these tools:

Email

Listserv

World Wide Web for research

Computer Conferencing
Students are able to use these computer
applications

Word Processors

Statistical Packages

Spreadsheets

Presentation Software

Databases
WHERE ADDRESSED
Evidence of successful use of :

Email messages with faculty

Participation in MPA Listserv

Use of Web for research in at least one
paper

Participation in discussion topic in
conference
Evidence of successful use in project in
(respectively)

PA 700

PA 710

PA 750

PA 750

PA 745
METHOD OF
ASSESSMENT
Form completed by
Program Director for each
skill, except for WWW
skill—completed by
faculty member in course
for which research took
place
Form completed by
faculty member in each
course reflecting level of
experience in each skill.
Criteria. The criteria used to judge the student's development of
computer skills is the extent to which the student has shown he or she can
effectively put this skill to use.
a(vii)
Understanding of Ethics and Diversity Issues
COMPETENCY OUTCOMES
WHERE ADDRESSED
Students develop an understanding of
public sector ethical issues and how they
apply to daily organizational life
Successful completion of paper defining or
discussing public or nonprofit ethics (PA 700,
740, or 745) and addressed in Culminating
Experience Case Analysis (PA 890)
Students develop an understanding of the
importance and complexity of diversity in
public sector and nonprofit organizations
and diversity issues apply to daily
organizational life
Successful completion of paper defining or
discussing diversity issues (PA 700, 740, 745,
790) and addressed in Culminating Experience
Case Analysis (PA 890)
METHOD OF
ASSESSMENT
Written evaluation by
faculty member teaching
in each course. Case
Study Required as
Culminating Experience
Written evaluation by
faculty member teaching
in each course. Case
Study Required as
Culminating Experience.
Criteria. The criteria used to judge the student's development of an
understanding of ethics and diversity issues is the extent to which written
work effectively and successfully illustrates an understanding of the
complexity of these issues.
a(viii)
Understanding of Specialized Area of Public Administration
COMPETENCY OUTCOMES
Students develop a more specialized body
of knowledge in a specific subfield
WHERE ADDRESSED
Successful completion of paper, initiated in core
concentration course and completed in PA 890,
to incorporate knowledge and skills developed in
other concentration courses and electives.
METHOD OF
ASSESSMENT
Paper completed in PA
890.
Criteria. Each of these competencies is judged upon a five point scale by
the following criteria:





On a scale of 1 (lowest) to 5 (highest), how would you judge the extent to
which student shows a development of knowledge in the area?
How would you judge the extent to which the student synthesizes this
knowledge?
How would you judge the extent to which the student has shown they can
apply their knowledge?
How would you judge the insight that the student has shown in the use
and application of their knowledge?
How would you judge the originality that the student has shown in the
use and application of their knowledge?
Results of Assessment
After significant database development and other implementation efforts,
this digitally managed portfolio contains over 90 students and 216 completed
portfolio elements. The development of this assessment process began over
three years ago and is constantly in a state of change as the faculty and students
learn more and more from the results and the implementation process itself.
To date, the Program has obtained the following results. Since the
beginning of the portfolio process, this data has been constantly updated and
enlarged. The numbers indicate the average score that all students in the
Program achieved on each portfolio item and then the percentage "grade" based
upon the criteria for that element and standardized across all students and
portfolio element.
Not all portfolio elements are represented here since the Program has
been phasing in the portfolio with newer students. For instance, students who
were at the culminating experience stage are having that element assessed this
semester and so, Knowledge of Public Administration will be assessed beginning
this semester.
From these results, it is clear that the Program and its faculty need to
focus more upon improving writing skills and the development of effective team
work skills. The Program is providing effective instruction in professional skill
development, computer skills, and oral presentations. Anecdotal evidence
suggests that knowledge of public administration, a public administration
subfield, and understanding of ethics and diversity issues are also being handled
effectively.
Professional Skills
 Social Science Bibliography (4.8 of 5- 96%)
 Financial Statements (4.69 of 5- 93.8%)
 Social Science research (4.6 of 5- 92%)
 Budget Project (4.57 of 5- 91.4%)
Written and Oral Communication
 Writing Assignment (24.76 of 30- 82.53%)
 Oral Presentation with PowerPoint (40.94 of 45- 90.9%)
Team Work Development
 Team Work (29.1 of 35-- 83.1%)
Computer Skills
 Database Software (5 of 5- 100%)
 Spreadsheets (4.94 of 5- 98.8%)


Presentation Software (4.87 of 5- 97.4%)
Statistical Software (4.67 of 5- 93.4%)
V.
OTHER ASSESSMENT ACTIVITIES AND THEIR RESULTS
The Public Administration Program has also utilized focus groups, alumni
surveys, reviews of alumni career paths, and surveys of both our alumni and
community advisory boards. These techniques are used to continually
update our strategic plan, seek input about our Program in general, and
improve our curriculum.
VI.
USE OF ASSESSMENT RESULTS FOR PROGRAM IMPROVEMENT
Even before any results of the assessment process were available, the
Public Administration Program used the process of developing the Student
Learning Portfolio to improve our Program. The process of identifying our
learning goals and objectives immediately led to changes in our curriculum
and in course content. For instance, feedback from alumni as the Strategic
Plan was being developed led to enhanced coverage of accounting and the
inclusion of understanding financial statements as an important learning goal
under the category of Professional Skills Development.
Since the early development of the Portfolio, we have continued to
constantly improve our curriculum and course content. The first actual
assessment results have only solidified that process. After seeing these
results, Program faculty are actively seeking ways and new curriculum to try
to help improve students' writing abilities and their team work skills.
Report Completed By:
Dr. Genie Stowers, Program Director
Date: 9/1/2000
Download