Public Administration Program Assessment and Program Improvement Report I. PROGRAM The Public Administration Program currently offers the Master's of Public Administration (M.P.A.) as well as several undergraduate courses. II. PROGRAM MISSION The mission of the Public Administration Program is to train students in the governance, policy, and administration of the public and nonprofit sectors through intellectual and practical interdisciplinary education. Through excellence and innovation in teaching, the ultimate goal of the Program is to prepare students to work in and manage the organization of tomorrow. The Public Administration Program follows and supports the mission of the University in: Attracting, retaining, and graduating a highly diverse student body of public administration students; Providing a disciplinary and interdisciplinary professional education in public administration that is academically rigorous and intellectually challenging; Providing a public administration curriculum that reflects all dimensions of human diversity, and that encourages critical thinking and social and cultural awareness; Fostering a collegial and cooperative intellectual environment for students and faculty that includes recognition and appreciation of differing viewpoints and promotes academic freedom within the Program community; and Serving the communities with which public administration students and faculty are engaged. III. PROGRAM GOALS Student Learning Goals Goal 1: Provide students with substantive knowledge about the depth and breadth of public administration today Objective 1: Train students to develop an understanding of the theoretical and intellectual knowledge underpinning the discipline of public administration - the history and theories of the field, the importance of social, economic, and political institutions, and the public policy process Goal 2: Provide students with the skills necessary to manage today’s and tomorrow’s organizations Objective 1: Train students in the essential professional skills necessary to succeed in public administration—research skills, budgeting and policy analysis skills, skills in organizational development and change, and skills in personnel management Objective 2: Train students to write effectively and concisely for an audience of professional public administrators Objective 3: Train students to make effective oral presentations to small and larger groups, provide both experience and the opportunity to develop confidence while speaking in public Objective 4: Train students to work in groups and teams, providing positive experiences and the opportunity to assess the importance of using teams to accomplish organizational tasks Objective 5: Train students to utilize the predominant computer and Internet applications as tools appropriate for the public and nonprofit workplace Goal 3: Provide students with the ability to be sensitive and knowledgeable about ethical and diversity issues in the public and nonprofit sectors Objective 1: Train students in the issues of public sector ethics and diversity, focusing attention upon critical thinking and independent evaluations about these important topics Goal 4: Provide students with the substantive knowledge about a specialized area of public administration Objective 1: Provide students with specialized knowledge in one of the important subfields of public administration Programmatic Goals Goal 5: Work collaboratively with other departments and programs within the College, the University, and the community Objective 1: Develop SPIA (the units of political science, international relations, urban studies, and public administration) as a viable, collaborative unit Objective 2: Work with other departments and institutes (like the Public Research Institute) around the University to create, out of existing courses, other course concentrations and academic offerings and to work on other projects Objective 3: Work with community groups like the San Francisco Bay Area Chapter of the American Society of Public Administration, other local professional associations, and local governments interested in and related to public administration Objective 4: Utilize the University’s resources in a cost-effective fashion while still meeting the needs of our students and providing for the professional development of our faculty Goal 6: Seek to be recognized as an effective public administration program, remaining on the cutting edge of public administration, and becoming and remaining a learning organization that changes when change is required Objective 1: Improve and enhance the management of the Public Administration Program itself, thereby enhancing our effectiveness and stretching existing resources. Objective 2: Attain status as an accredited program by the National Association of Schools of Public Administration and Affairs-FULLY ACHIEVED 8/2000 Objective 3: Recruit and retain high quality students in both the public and the nonprofit sectors Objective 4: Develop Pi Alpha Alpha Chapter (the public administration honorary society)-- FULLY ACHIEVED 1/2000 Objective 5: Establish a supportive environment for faculty to conduct useful and important research (both applied and theoretical) Objective 6: Establish a supportive environment for faculty to become and stay meaningfully involved in community and professional service. Objective 7: Continue to evaluate the Program and its successes and failures in order to improve its operation and education Objective 8: Seek out new, innovative ways of accomplishing our mission, goals, and objectives Objective 9: Seek to distinguish ourselves from other public administration programs in the Bay Area and California in order to attract high quality students Objective 10: Take advantage of the Program’s location in San Francisco and on the Pacific Rim. Objective 11: Engage in a continuing strategic planning process, reevaluating our accomplishments each year and revising our mission and desired goals, objectives, and outcomes as we move towards accomplishing our current goals and objectives IV. ASSESSING STUDENT ATTAINMENT OF LEARNING OBJECTIVES Beginning in Spring 1999, the Program required that students to compile a Student Learning Portfolio to demonstrate their mastery of the knowledge and skills required of effective public administrators. The process developed by Public Administration Program faculty and students has now been recognized by our national accrediting body, the National Association of Schools of Public Affairs and Administration (NASPAA) as a "NASPAA Exemplary Practice". The following areas of student learning outcomes were identified: Knowledge of Public Administration theories and processes Knowledge of a Public Administration sub-field Skill in professional areas such as developing budget proposals, workforce policies, and understanding statistical analysis Effective written and oral communication skills, ability to work in teams, and basic and advanced computer skills For each of these areas, learning outcomes were identified. Then, work products were selected that would allow for an assessment of the extent to which students achieved each outcome. Finally, we ensured that students had the opportunity to learn the material or skills, to produce each work product, and have it assessed, in at least one course (often more). Following that activity, the faculty member (or classmates) evaluates the activity according to a series of criteria developed for each activity. Another characteristic that distinguishes this portfolio process is that the portfolio is electronically managed. Student and Program achievement are tracked by databases and all student materials are stored in digital format. When students graduate, they will receive a CD-Rom complete with all their portfolio products and assessments. In this way, both the Program and the student directly benefit from the Student Learning Portfolio. The section below lists each learning goal and each accompanying learning objective. Following each of these is a table that includes the learning competency outcomes for each learning objective, where in the curriculum each of those outcomes is addressed, what criteria is used to assess the outcome, and the method of assessment. a. COMPETENCIES-- LEARNING OBJECTIVES AND GOALS Learning competencies relate directly back to the Student Learning Goals of Section III, above. a(i). Knowledge of Public Administration Competencies COMPETENCY OUTCOMES Students posses a core body of knowledge about public administration history, theories and current issues in the field today Students understand the importance and role of economic, political and social institutions WHERE ADDRESSED Successful Completion of Case Study Analysis in PA 8901. Students develop an understanding of organizations and how they work Successful Completion of Case Study Analysis in PA 890 Students understand the policy process, policy issues, and public administrators' role in the process Students develop an understanding of human resources management issues Successful completion of policy paper required in PA 730. METHOD OF ASSESSMENT Case Study Required in PA 890, Part of Culminating Experience Requirement Case Study Required in PA 890, Part of Culminating Experience Requirement Case Study Required in PA 890, Part of Culminating Experience Requirement Written evaluation by faculty member teaching PA 730. Successful completion of paper in PA 745. Written evaluation by faculty member teaching PA 730. Successful Completion of Case Study Analysis in 890 Criteria. Each of these competencies is judged upon a five point scale by the following criteria: On a scale of 1 (lowest) to 5 (highest), how would you judge the extent to which student shows a development of knowledge in the area? How would you judge the extent to which the student synthesizes this knowledge? How would you judge the extent to which the student has shown they can apply their knowledge? How would you judge the insight that the student has shown in the use and application of their knowledge? How would you judge the originality that the student has shown in the use and application of their knowledge? a(ii) Professional Skill Development Competencies COMPETENCY OUTCOMES WHERE ADDRESSED Students develop research skills and become intelligent consumers of statistical and qualitative data and research Students learn to use electronic and other search methods to develop bibliographic materials Students develop ability to conduct cost benefit analysis Successful completion of PA 710 homework exercises and research paper. Students are able to complete effective budget proposals Successful completion of budget development project in PA 750. Students develop ability to read and understand public and nonprofit financial statements Students develop an understanding of human Successful completion of financial statement exercise in PA 750 Successful completion of paper in PA 710 Successful completion of cost benefit analysis exercise in PA 720 Successful completion of workforce policies METHOD OF ASSESSMENT Evaluated by faculty member teaching PA 710. Written evaluation by faculty member teaching PA 710 Written evaluation by faculty member teaching PA 720 Written evaluation by faculty member teaching PA 750. Written evaluation by faculty member teaching PA 750 Written evaluation by resources issues and workforce policies exercise in PA 745 faculty member teaching PA 745 Criteria. The criteria used to judge effective development of professional skills is based upon a scale of 1 (lowest) to 5 (highest)-- how would you judge the extent to which student has demonstrated they can use these skills? a(iii) Written Communication OUTCOMES Students have the ability to write concisely and effectively WHERE ADDRESSED Successful completion of two (2) written papers, the Case Study Analysis in PA 890, and the Culminating Experience paper (Subfield paper) in PA 895. (Other evidence from other courses is possible.) METHOD OF ASSESSMENT Written evaluation by faculty in courses, PA 890, and PA 895 Criteria. The criteria used to judge effective development of written communication skills is based upon a scale of 1 (lowest) to 5 (highest)- has the student: demonstrated a working knowledge of theories, concepts and methods appropriate to the discipline in their writing? demonstrated critical thinking in their writing? presented and developed a clear line of analysis (effective opening and closing with well-organized points throughout the paper) in their writing? was able to write concisely? demonstrated good basic writing skills: grammar, spelling, and sentence structure? demonstrated originality in his or her writing and analysis? a(iv). Oral Communication COMPETENCY OUTCOMES WHERE ADDRESSED Students have the ability to make effective oral presentations Successful completion of at least three (3) oral presentations in selected courses throughout the Program (PA 700, 710, 720, 740, 745 750). At least one presentation should be accompanied by a PowerPoint presentation. METHOD OF ASSESSMENT Written evaluation by faculty member teaching in each course. Criteria. The criteria used to judge effective development of oral communication skills is based upon a scale of 1 (lowest) to 5 (highest)- does the faculty member think the student: was clear in the oral presentation? was interesting and engaging in their oral presentation? was well organized? was attentive to the audience? delivered the oral presentation in a professional manner? demonstrated originality and creativity in the presentation made use of useful and appropriate handouts? used the time well (including staying within the allotted time)? a(v). Development of To Work Effectively in Teams COMPETENCY OUTCOMES Students have the ability to work effectively in teams WHERE ADDRESSED Successful completion of two (2) team projects in PA 740, PA 750, other core courses, or electives. METHOD OF ASSESSMENT Written evaluation by faculty member teaching in each course. Criteria. The criteria used to judge effective development of team work skills is based upon a scale of 1 (lowest) to 5 (highest)- did the student: contributed a fair share of the group’s work, made a valuable contribution to achieving group goals? shown reliability and timeliness in completing group tasks? demonstrated a willingness to compromise when making group decisions? exhibited a willingness to effectively listen to others in the group? demonstrated an ability to contribute good ideas that lead to group success? shown leadership skills in accomplishing tasks? a(vi) Computer Skills COMPETENCY OUTCOMES Students are able to make effective use of these tools: Email Listserv World Wide Web for research Computer Conferencing Students are able to use these computer applications Word Processors Statistical Packages Spreadsheets Presentation Software Databases WHERE ADDRESSED Evidence of successful use of : Email messages with faculty Participation in MPA Listserv Use of Web for research in at least one paper Participation in discussion topic in conference Evidence of successful use in project in (respectively) PA 700 PA 710 PA 750 PA 750 PA 745 METHOD OF ASSESSMENT Form completed by Program Director for each skill, except for WWW skill—completed by faculty member in course for which research took place Form completed by faculty member in each course reflecting level of experience in each skill. Criteria. The criteria used to judge the student's development of computer skills is the extent to which the student has shown he or she can effectively put this skill to use. a(vii) Understanding of Ethics and Diversity Issues COMPETENCY OUTCOMES WHERE ADDRESSED Students develop an understanding of public sector ethical issues and how they apply to daily organizational life Successful completion of paper defining or discussing public or nonprofit ethics (PA 700, 740, or 745) and addressed in Culminating Experience Case Analysis (PA 890) Students develop an understanding of the importance and complexity of diversity in public sector and nonprofit organizations and diversity issues apply to daily organizational life Successful completion of paper defining or discussing diversity issues (PA 700, 740, 745, 790) and addressed in Culminating Experience Case Analysis (PA 890) METHOD OF ASSESSMENT Written evaluation by faculty member teaching in each course. Case Study Required as Culminating Experience Written evaluation by faculty member teaching in each course. Case Study Required as Culminating Experience. Criteria. The criteria used to judge the student's development of an understanding of ethics and diversity issues is the extent to which written work effectively and successfully illustrates an understanding of the complexity of these issues. a(viii) Understanding of Specialized Area of Public Administration COMPETENCY OUTCOMES Students develop a more specialized body of knowledge in a specific subfield WHERE ADDRESSED Successful completion of paper, initiated in core concentration course and completed in PA 890, to incorporate knowledge and skills developed in other concentration courses and electives. METHOD OF ASSESSMENT Paper completed in PA 890. Criteria. Each of these competencies is judged upon a five point scale by the following criteria: On a scale of 1 (lowest) to 5 (highest), how would you judge the extent to which student shows a development of knowledge in the area? How would you judge the extent to which the student synthesizes this knowledge? How would you judge the extent to which the student has shown they can apply their knowledge? How would you judge the insight that the student has shown in the use and application of their knowledge? How would you judge the originality that the student has shown in the use and application of their knowledge? Results of Assessment After significant database development and other implementation efforts, this digitally managed portfolio contains over 90 students and 216 completed portfolio elements. The development of this assessment process began over three years ago and is constantly in a state of change as the faculty and students learn more and more from the results and the implementation process itself. To date, the Program has obtained the following results. Since the beginning of the portfolio process, this data has been constantly updated and enlarged. The numbers indicate the average score that all students in the Program achieved on each portfolio item and then the percentage "grade" based upon the criteria for that element and standardized across all students and portfolio element. Not all portfolio elements are represented here since the Program has been phasing in the portfolio with newer students. For instance, students who were at the culminating experience stage are having that element assessed this semester and so, Knowledge of Public Administration will be assessed beginning this semester. From these results, it is clear that the Program and its faculty need to focus more upon improving writing skills and the development of effective team work skills. The Program is providing effective instruction in professional skill development, computer skills, and oral presentations. Anecdotal evidence suggests that knowledge of public administration, a public administration subfield, and understanding of ethics and diversity issues are also being handled effectively. Professional Skills Social Science Bibliography (4.8 of 5- 96%) Financial Statements (4.69 of 5- 93.8%) Social Science research (4.6 of 5- 92%) Budget Project (4.57 of 5- 91.4%) Written and Oral Communication Writing Assignment (24.76 of 30- 82.53%) Oral Presentation with PowerPoint (40.94 of 45- 90.9%) Team Work Development Team Work (29.1 of 35-- 83.1%) Computer Skills Database Software (5 of 5- 100%) Spreadsheets (4.94 of 5- 98.8%) Presentation Software (4.87 of 5- 97.4%) Statistical Software (4.67 of 5- 93.4%) V. OTHER ASSESSMENT ACTIVITIES AND THEIR RESULTS The Public Administration Program has also utilized focus groups, alumni surveys, reviews of alumni career paths, and surveys of both our alumni and community advisory boards. These techniques are used to continually update our strategic plan, seek input about our Program in general, and improve our curriculum. VI. USE OF ASSESSMENT RESULTS FOR PROGRAM IMPROVEMENT Even before any results of the assessment process were available, the Public Administration Program used the process of developing the Student Learning Portfolio to improve our Program. The process of identifying our learning goals and objectives immediately led to changes in our curriculum and in course content. For instance, feedback from alumni as the Strategic Plan was being developed led to enhanced coverage of accounting and the inclusion of understanding financial statements as an important learning goal under the category of Professional Skills Development. Since the early development of the Portfolio, we have continued to constantly improve our curriculum and course content. The first actual assessment results have only solidified that process. After seeing these results, Program faculty are actively seeking ways and new curriculum to try to help improve students' writing abilities and their team work skills. Report Completed By: Dr. Genie Stowers, Program Director Date: 9/1/2000