SJSU Annual Assessment Report Academic Year 2012-2013 Electronic Copy of Report Due July 1, 2013 Send to Undergraduate Studies (Kim.Huynh@sjsu.edu), with cc: to your College Associate Dean and College Assessment Facilitator Department/Program: Urban & Regional Planning/MUP Date of Report: June ‘13 Contact Person: Hilary Nixon email: hilary.nixon@sjsu.edu Phone: x45852 Program Accreditation (if any): Planning Accreditation Board Please refer to expanded instructions for each item. The narrative portion of the report should not exceed four pages. Appendices should be included as part of the report. 1. Overview and Context: As a result of new accreditation standards adopted by the Planning Accreditation Board, the Urban & Regional Planning department revised its Program Learning Outcomes in Summer 2012 during our annual assessment review. A list of all new PLOs is shown in Appendix A. In addition, a matrix showing how our revised PLOs align with our core MUP curriculum is shown in Appendix B and Appendix C includes our revised assessment schedule for 2012-2017. In order to obtain a baseline report for all PLOs, each PLO (except 1f and 3c) were assessed at least once during the 2012-2013 AY. 2. Use of Prior Assessment/Closing the Loop: Since we developed all new PLOs at the beginning of this AY, our focus has been on collecting baseline data and ensuring that our assessment procedures are working well. 3. Assessment Data: A detailed assessment report for all PLOs is shown in Appendix D. Methodologies/Measures: Our general approach has been to work with each individual instructor assigned to teach a class where assessment data is collected so that the assessment process is clear to all involved and to ensure that measurement tools and the resulting data can be effectively used. Measurement tools vary but tend to include the following: exams geared to the specific PLO and assignments or assignment components focused on the specific PLO. The use of rubrics (particularly for URBP 298B) have been used effectively to tease out student learning directly focused on a particular PLO, however, this approach is not consistently used across all courses. In addition, evidence from this year’s assessment for several PLOs (e.g. PLO 2e, 2f, 3b, 3d, and 3e) indicate that the measurement tools did not produce useful assessment data, so new measurement tools will be implemented in the future. Data: Data collected in AY 2012-2013 indicate that the vast majority of students achieve the stated PLO. In addition, in several instances, we have multiple assessments of the same PLO over several courses also showing consistent data. Of particular interest was the assessment of PLO 1e, where students were assessed on this PLO multiple times during the same course and evidence suggests that over the course of the semester, student learning on this PLO increased. Ideally, this type of assessment approach could be used for all PLOs, but it should be noted that this approach is time-consuming for instructors. Evaluation/Reflection: As noted above, and in the detailed assessment report, additional work on developing effective assessment tools is needed. A summer assessment review by department faculty is planned where faculty will work collaboratively to improve assessment processes. 4. Alignment of Course and Program Learning Outcomes: Please see Appendix B. 5. Recommendations for Student Learning: Our main effort this year will be to work more closely with individual instructors to share past assessment data and procedures so that not only are assessment procedures improved, but that we have a more consistent approach to using past data to improve future learning. To facilitate this, our department assessment coordinator recently worked with our chair to improve our “Instructor Course Guidelines” which provides course-specific information to each instructor regarding how each courses fits within our overall assessment plan, as well as information about conducting assessment. In addition, our department assessment coordinator will work with each individual faculty member prior to the beginning of the semester to clarify assessment expectations. 6. Plans 2013-14 Academic Year: As shown in Appendix C, we plan to collect data on all of our PLOs again this coming year, including PLO 1f and 3c which we were unable to collect data for this AY. Appendix A: MUP Program Learning Outcomes 1. General planning knowledge: The comprehension, representation, and use of ideas and information in the planning field, including appropriate perspectives from history, social science, and the design professions. Students will be able to: a) Describe and explain why planning is undertaken by communities, cities, regions, and nations, and the impact planning is expected to have. b) Describe and explain the behaviors and structures available to bring about sound planning outcomes, as well as the strengths and weaknesses of different ones. c) Describe and explain the legal and institutional contexts within which planning occurs. d) Describe and explain the growth and development of places over time and across space. e) Describe and explain the relationships between past, present, and future in planning domains, as well as the potential for methods of design, analysis, and intervention to influence the future. f) Describe and explain interactions, flows of people and materials, cultures, and differing approaches to planning across world regions. 2. Planning skills: The use and application of knowledge to perform specific tasks required in the practice of planning. Students will be able to: a) Assemble and analyze ideas and information from prior practice and scholarship, and from primary and secondary sources. b) Prepare clear, accurate and compelling text, graphics and maps for use in documents and presentations. c) Collect data, conduct analysis, and use modeling tools for forecasting, policy analysis, and design of projects and plans. d) Describe and assess when it is appropriate to use integrative tools for sound plan formulation, adoption, and implementation and enforcement. e) Describe and assess when it is appropriate to use different tools for stakeholder involvement, community engagement, and working with diverse communities. f) Describe and assess when it is appropriate to use tools for attention, formation, strategic decision-making, team building, and organizational/community motivation. 3. Values and ethics: Values inform ethical and normative principles used to guide planning in a democratic society. The program shall appropriately incorporate issues of diversity and social justice into all required courses of the curriculum. Students will be able to: a) Describe and explain key issues of planning ethics and related questions of the ethics of public decision-making, research, and client representation (including principles of the AICP Code of Ethics). b) Describe and explain the roles of officials, stakeholders, and community members in planned change. c) Describe and explain natural resource and pollution control factors in planning, as well as how to create sustainable futures. d) Describe and explain the roles of economic, social, and cultural factors in urban and regional growth and change. e) Describe and explain equity concerns in planning. Appendix B: Mapping Program Learning Outcomes to Curriculum Courses URBP 298A URBP 298B C URBP 297P URBP 236 A C C A A A C C C C A A C A C C A C A URBP 275G A URBP 241 A URBP 225 URBP 204 URBP 201 Key: C – material covered A – assessed Program Learning Outcomes 1. General planning knowledge: The comprehension, representation, and use of ideas and information in the planning field, including appropriate perspectives from history, social science, and the design professions. Students will be able to: a) Describe and explain why planning is undertaken by communities, cities, regions, and nations, and the impact planning is expected to have. b) Describe and explain the behaviors and structures available to bring about sound planning outcomes, as well as the strengths and weaknesses of different ones. c) Describe and explain the legal and institutional contexts within which planning occurs. d) Describe and explain the growth and development of places over time and across space. e) Describe and explain the relationships between past, present, and future in planning domains, as well as the potential for methods of design, analysis, and intervention to influence the future. f) Describe and explain interactions, flows of people and materials, cultures, and differing approaches to planning across world regions. 2. Planning skills: The use and application of knowledge to perform specific tasks required in the practice of planning. Students will be able to: a) Assemble and analyze ideas and information from prior practice and scholarship, and from primary and secondary sources. b) Prepare clear, accurate and compelling text, graphics and maps for use in documents and presentations. c) Collect data, conduct analysis, and use modeling tools for forecasting, policy analysis, and design of projects and plans. d) Describe and assess when it is appropriate to use integrative tools for sound plan formulation, adoption, and implementation and enforcement. e) Describe and assess when it is appropriate to use different tools for stakeholder involvement, community engagement, and working with diverse communities. f) Describe and assess when it is appropriate to use tools for attention, formation, strategic decision-making, team building, and organizational/community motivation. 3. Values and ethics: Values inform ethical and normative principles used to guide planning in a democratic society. The program shall appropriately incorporate issues of diversity and social justice into all required courses of the curriculum. Students will be able to: a) Describe and explain key issues of planning ethics and related questions of the ethics of public decision-making, research, and client representation (including principles of the AICP Code of Ethics). b) Describe and explain the roles of officials, stakeholders, and community members in planned change. c) Describe and explain natural resource and pollution control factors in planning, as well as how to create sustainable futures. d) Describe and explain the roles of economic, social, and cultural factors in urban and regional growth and change. e) Describe and explain equity concerns in planning. URBP 200 Core Courses A C C C A A A A C A A A A C A C C C A A A A A Appendix C: Assessment Schedule (2012-2017) PLOs 1a 1b 1c 1d 1e 1f 2a 2b 2c 2d 2e 2f 3a 3b 3c 3d 3e Courses for Assessment 200 200 225 200 200, 236 241 298B 200, 201, 204, 275G, 204, 298B 225, 298B 225 201 201 200 201 241 201, 236 201, 236 F2012 S2013 F2013 S2014 C C C C C D D D D C,D C C C C C C C C C,D C,D C,D D D D D D C C C,D C,D C,I C,I C,I C,I C,I C C,I C,I C,I C,I C,I C,I C,I C,I C C,I C,I D D D D C,D D C,D C,D C,D D D D D D D C,D C,D Codes: C=Collect assessment data; D= discuss by faculty; I= implement changes F2014 C,I C,I C,I C,I C,I C,I C,I C,I C,I C,I C,I C,I C,I C,I C,I C,I C,I S2015 F2015 S2016 F2016 S2017 D D D D C,D D C,D C,D C,D D D D D D D C,D C,D C,I C,I C,I C,I C,I C,I C,I C,I C,I C,I C,I C,I C,I C,I C,I C,I C,I D D D D C, D D C,D C,D C,D D D D D D D C,D C,D C,I C,I C,I C,I C,I C,I C,I C,I C,I C,I C,I C,I C,I C,I C,I C,I C,I D D D D C,D D C,D C,D C,D D D D D D D C,D C,D Appendix D: Detailed Assessment Report (2012-2013) PLO 1a: Describe and explain why planning is undertaken by communities, cities, regions, and nations, and the impact planning is expected to have. Assessment Course: URBP 200. During AY 2012-2013, two sections of URBP 200 were taught during Fall 2012 (Joe Kott and Laurel Prevetti). Data was collected from one section (Prevetti). Assessment Methodology/Measures: Students complete a literature review assignment and must complete part 3 of this assignment with a grade of “B” or better. Data: Ninety-two percent of students met the threshold for demonstrating knowledge of the PLO. Evaluation and Reflection: Students received detailed feedback on the assignment. No specific changes were noted by the instructor. Results from this assessment, including assignment guidelines, will be shared with future instructors. PLO 1b: Describe and explain the behaviors and structures available to bring about sound planning outcomes, as well as the strengths and weaknesses of different ones. Assessment Course: URBP 200. During AY 2012-2013, two sections of URBP 200 were taught during Fall 2012 (Joe Kott and Laurel Prevetti). Data was collected from one section (Prevetti). Assessment Methodology/Measures: Students complete a planning theory assignment and must complete this assignment with a grade of “B” or better. Data: Ninety percent of students met the threshold for demonstrating knowledge of the PLO. Evaluation and Reflection: Students received detailed feedback on the assignment. No specific changes were noted by the instructor. Results from this assessment, including assignment guidelines, will be shared with future instructors. PLO 1c: Describe and explain the legal and institutional contexts within which planning occurs. Assessment Course: URBP 225. During AY 2012-2013, one section of URBP 225 was taught during Fall 2012 (John Davidson) Assessment Methodology/Measures: A take-home midterm exam was used to assess student learning. Data: Sixty percent of students scored in the acceptable range (85 out of 100 possible points). Evaluation and Reflection: The course instructor indicated that in the future, a written in-class practice test will be administered so that students can get feedback on how to structure their arguments before the exam. PLO 1d: Describe and explain the growth and development of places over time and across space. Assessment Course: URBP 200. During AY 2012-2013, two sections of URBP 200 were taught during Fall 2012 (Joe Kott and Laurel Prevetti). Data was collected from one section (Prevetti). Assessment Methodology/Measures: Students much complete the historical person assignment with a grade of “B” or better. Data: One hundred percent of students met the threshold for demonstrating knowledge of the PLO. Evaluation and Reflection: Students received detailed feedback on the assignment. No specific changes were noted by the instructor. Results from this assessment, including assignment guidelines, will be shared with future instructors. PLO 1e: Describe and explain the relationships between past, present, and future in planning domains, as well as the potential for methods of design, analysis, and intervention to influence the future. Assessment Courses: URBP 200, 236. During AY 2012-2013, two sections of URBP 200 were taught during Fall 2012 (Joe Kott and Laurel Prevetti). Data was collected from one section (Prevetti). Two sections of URBP 236 were taught in Spring 2013 and used for assessment (Hing Wong/Steve Ross and Kathrine Richardson). Assessment Methodology/Measures: In URBP 200, students participate in an engagement assignment designed to assess this PLO. In URBP 236, a major focus of the class is on how planners can influence the future. Three assignments in both sections evaluated student learning and were designed to evaluate learning over the course of the semester. Data: In URBP 200, 100% of students met the threshold for demonstrating knowledge of the PLO (grade of “B” or better on the engagement assignment). In both sections of URBP 236, instructors were able to see measurable improvement in student learning related to this PLO over the course of the semester. During the initial evaluation (using Assignment 1), 80% to 90% of students met the threshold (“B” or better on the assignment). Assignment 3 continued to evaluated student learning on this PLO and 90% to 95% of students met the threshold. By the end of the semester, 100% of students met the threshold. Evaluation and Reflection: In URBP 200, students received detailed feedback on the assignment. No specific changes were noted by the instructor. Results from this assessment, including assignment guidelines, will be shared with future instructors. In URBP 236, instructors noted that using multiple assignments over the course of the semester enabled students to demonstrate a mastery of the PLO PLO 1f: Describe and explain interactions, flows of people and materials, cultures, and differing approaches to planning across world regions. Assessment Course: URBP 241. URBP 241 is a brand new course. It will be offered for the first time in Fall 2013. Assessment Methodology/Measures: Not assessed in 2012-2013. Data: Evaluation and Reflection: PLO 2a: Assemble and analyze ideas and information from prior practice and scholarship, and from primary and secondary sources. Assessment Course: URBP 298B. During AY 2012-2013, a total of 38 students completed URBP 298B and submitted final planning reports. Assessment Methodology/Measures: Students must complete a professional planning report in URBP 298B. All faculty teaching URBP 298B use a common rubric to evaluate student learning. Outcome 1 on the rubric addresses PLO 2a. Data: Thirty-eight students successfully completed URBP 298B. On their final report grading rubric for Outcome 1, students received an average score of 9.9 out of 12, with a range of scores from 8 to 12. Evaluation and Reflection: Faculty have worked consistently over many years to improve 298 instruction and advising, which is demonstrated through the fairly high level of student learning and achievement on this PLO. PLO 2b: Prepare clear, accurate and compelling text, graphics and maps for use in documents and presentations. Assessment Course: URBP 200 (writing), 201 (graphics/maps), 204 (graphics), 275G (maps), and 298B (overall). During AY 2012-2013, two sections of URBP 200 were taught during Fall 2012 (Joe Kott and Laurel Prevetti). Data was collected from one section (Prevetti). Two sections of URBP 201 were taught during Fall 2012 (Richard Kos and Justin Meek). Two sections of URBP 275G were taught (Ralph McLaughlin in Fall, and Richard Kos in Spring). Two sections of URBP 204 were taught in Fall 2012 (Shishir Mathur). Thirty-eight students successful completed URBP 298B. Assessment Methodology/Measures: Several courses in the department are used to assess student learning related to this PLO. URBP 200, 204, and 275G assess students during the early phase of their program, while URBP 201 and 298B evaluate students at the end of their program. In URBP 200, students are assessment through the writing component on part 3 of the literature review assignment and must complete this component with a grade of “B” or better. In URBP 204, students are assessed specifically on the quality of their charts and tables in the term project presentation, as well as their ability to properly link their data/statistical tests to the correct chart/table format. In URBP 275G, students critique existing published maps and develop their own maps using best practices in map design. In URBP 201, student learning was assessed using Assignment 4 that asked students to prepare posters reflecting the results of community assessment in 4 key areas (land use, transportation, community life and safety, and housing). Posters needed to present existing conditions in a visual format that included data and visual images. In URBP 298B, student complete a professional planning report and all faculty use a common rubric to evaluate student learning and achievement on this PLO. Outcome 3 on the rubric assesses PLO 2b. Data: In URBP 200, 12 out of 13 students met the department standard (a “B” grade or better on the writing component of part 3 of the literature review assignment). In URBP 204, 15 out of 21 students met the department standard (a “B” grade or better on this part of the assignment). In 275G, 20 out of 21 students achieved the threshold for demonstrating that they met the PLO (B+ or better on the assignment). In URBP 201, all 34 students met the learning outcome and were able to demonstrate their ability to prepare clear, accurate, and compelling graphics and maps. In URBP 298B, 38 students successfully completed URBP 298B. On their final report grading rubric for Outcome 3, students receive an average score of 10.1 out of 12, with a range of scores from 8 to 11.6. Evaluation and Reflection: In URBP 200, students received detailed feedback on the assignment. No specific changes were noted by the instructor. Results from this assessment, including assignment guidelines, will be shared with future instructors. In URBP 204, the instructor noted that most students meet this specific learning outcome for the course, but in the future a separate handout will be created to improve student learning, specifically as it relates to linking specific statistical tests/data with chart and table formats for presentation. The instructor for URBP 275G noted that assignment guidelines could be clarified to avoid confusion between critiquing published maps and creating original maps. Faculty have worked consistently over many years to improve 298 instruction and advising, which is demonstrated through the fairly high level of student learning and achievement on this PLO. PLO 2c: Collect data, conduct analysis, and use modeling tools for forecasting, policy analysis, and design of projects and plans. Assessment Course: URBP 204, 236, 298B. During AY 2012-2013, two sections of URBP 204 were taught in Fall 2012 (Shishir Mathur). Two sections of URBP 236 were taught in Spring 2013 and used for assessment (Hing Wong/Steve Ross and Kathrine Richardson). Thirtyeight students successful completed URBP 298B. Assessment Methodology/Measures: In URBP 204, the Term Project Analysis Report is used to evaluate student learning for PLO 2c and focuses on data collection and analysis. In URBP 236, two assignments were used to assess student learning focused on policy analysis over the course of the semester. In URBP 298B, student complete a professional planning report and all faculty use a common rubric to evaluate student learning and achievement on this PLO. Outcome 2 on the rubric assesses PLO 2c. In URBP 298B, student complete a professional planning report and all faculty use a common rubric to evaluate student learning and achievement on this PLO. Outcome 2 on the rubric assesses PLO 2c. Data: In URBP 204, 20 out of 24 students achieving the learning outcome (measured by a score of 20 out of 25 on the Term Project Analysis Report). In URBP 236, for the first assignment, 90% to 100% of students met the learning threshold (B or better on the Assignment 3) while 100% of students in both sections met the learning threshold by the end of the semester (B or better on the final assignment). In URBP 298B, 38 students successfully completed URBP 298B. On their final report grading rubric for Outcome 3, students receive an average score of 12.5 out of 16, with a range of scores from 10.25 to 15.75. Evaluation and Reflection: The instructor for URBP 204 felt that some students struggled with linking the research question to the appropriate statistical test and have trouble developing a well-reasoned research hypothesis. In the future, additional instruction time will be devoted to these topics. Faculty have worked consistently over many years to improve 298 instruction and advising, which is demonstrated through the fairly high level of student learning and achievement on this PLO. PLO 2d: Describe and assess when it is appropriate to use integrative tools for sound plan formulation, adoption, and implementation and enforcement. Assessment Course: URBP 225. During AY 2012-2013, one section of URBP 225 was taught during Fall 2012 (John Davidson). Assessment Methodology/Measures: URBP 225 assesses student learning for this PLO through the Urban Village Plan project. Data: All 13 students met the learning outcome (scoring 85 or better on their Urban Village Plan). Evaluation and Reflection: The instructor noted that students had the greatest challenge with developing policy recommendations related to the plan and presented possible solutions for future classes where students complete a semester-long journal to evaluate similar plan documents, conduct field visits to communities build under those plans, and evaluate what policies have the greatest impact on the look, feel, and function of a neighborhood. PLO 2e: Describe and assess when it is appropriate to use different tools for stakeholder involvement, community engagement, and working with diverse communities. Assessment Course: URBP 201. During AY 2012-2013, two sections of URBP 201 were taught during Fall 2012 (Richard Kos and Justin Meek). Assessment Methodology/Measures: In URBP 201, all students were asked to provide written responses to questions designed to assess their level of knowledge regarding this PLO. Data: There was significant inconsistency regarding student responses to this assignment, making effective analysis of individual student learning difficult. Evaluation and Reflection: Instructors noted that the method used to evaluate student learning did not lend itself to precise measurement and that, in the future, instructors need to devote more effort to integrating the PLO into specific course assignments and designing effective assessment tools. PLO 2f: Describe and assess when it is appropriate to use tools for attention, formation, strategic decision-making, team building, and organizational/community motivation. Assessment Course: URBP 201. During AY 2012-2013, two sections of URBP 201 were taught during Fall 2012 (Richard Kos and Justin Meek). Assessment Methodology/Measures: In URBP 201, all students were asked to provide written responses to questions designed to assess their level of knowledge regarding this PLO. Data: There was significant inconsistency regarding student responses to this assignment, making effective analysis of individual student learning difficult. Evaluation and Reflection: Instructors noted that the method used to evaluate student learning did not lend itself to precise measurement and that, in the future, instructors need to devote more effort to integrating the PLO into specific course assignments and designing effective assessment tools. PLO 3a: Describe and explain key issues of planning ethics and related questions of the ethics of public decision-making, research, and client representation (including principles of the AICP Code of Ethics). Assessment Course: URBP 200. During AY 2012-2013, two sections of URBP 200 were taught during Fall 2012 (Joe Kott and Laurel Prevetti). Data was collected from one section (Prevetti). Assessment Methodology/Measures: Students complete an ethics assignment and must received a grade of “B” or better on the specific component of the assignment used to assess learning related to this PLO. Data: One hundred percent of students met the department standard for this PLO. Evaluation and Reflection: Students received detailed feedback on the assignment. No specific changes were noted by the instructor. Results from this assessment, including assignment guidelines, will be shared with future instructors. PLO 3b: Describe and explain the roles of officials, stakeholders, and community members in planned change. Assessment Course: URBP 201. During AY 2012-2013, two sections of URBP 201 were taught during Fall 2012 (Richard Kos and Justin Meek). Assessment Methodology/Measures: In URBP 201, all students were asked to provide written responses to questions designed to assess their level of knowledge regarding this PLO. Data: There was significant inconsistency regarding student responses to this assignment, making effective analysis of individual student learning difficult. Evaluation and Reflection: Instructors noted that the method used to evaluate student learning did not lend itself to precise measurement and that, in the future, instructors need to devote more effort to integrating the PLO into specific course assignments and designing effective assessment tools. PLO 3c: Describe and explain natural resource and pollution control factors in planning, as well as how to create sustainable futures. Assessment Course: URBP 241. URBP 241 is a brand new course. It will be offered for the first time in Fall 2013. Assessment Methodology/Measures: Not assessed in 2012-2013. Data: Evaluation and Reflection: PLO 3d: Describe and explain the roles of economic, social, and cultural factors in urban and regional growth and change. Assessment Course: URBP 201, 236 (not assessed in 236 for 2012-2013). Assessment Methodology/Measures: In URBP 201, all students were asked to provide written responses to questions designed to assess their level of knowledge regarding this PLO. Data: For URBP 201, there was significant inconsistency regarding student responses to this assignment, making effective analysis of individual student learning difficult. Evaluation and Reflection: Instructors for URBP 201 noted that the method used to evaluate student learning did not lend itself to precise measurement and that, in the future, instructors need to devote more effort to integrating the PLO into specific course assignments and designing effective assessment tools. In order to more fully assess student learning, this PLO will also be assessed in URBP 236 in the future. PLO 3e: Describe and explain equity concerns in planning. Assessment Course: URBP 201, 236 (not assessed in 236 for 2012-2013). Assessment Methodology/Measures: In URBP 201, all students were asked to provide written responses to questions designed to assess their level of knowledge regarding this PLO. Data: For URBP 201, there was significant inconsistency regarding student responses to this assignment, making effective analysis of individual student learning difficult. Evaluation and Reflection: Instructors for URBP 201 noted that the method used to evaluate student learning did not lend itself to precise measurement and that, in the future, instructors need to devote more effort to integrating the PLO into specific course assignments and designing effective assessment tools. In order to more fully assess student learning, this PLO will also be assessed in URBP 236 in the future.