SJSU Annual Program Assessment Form Academic Year 2014-2015 Department: Communication Studies Program: B.A. Program College: Social Sciences Website: www.sjsu.edu/comm _ Check here if your website addresses the University Learning Goals. Program Accreditation: Contact Person and Email: Date of Report: Not Applicable Shawn Spano, shawn.spano@sjsu.edu June 1, 2015 Part A 1. List of Program Learning Outcomes (PLOs) There are five B.A. Program PLOs total, organized into three broad categories: Foundations, Inquiry, and Practice (see below). Department faculty developed and refined these PLOs over a series of faculty meetings, beginning in 2008 and continuing through AY 2014-15, including the development of assessable outcomes for each of the PLOs for indicating criteria for increasing levels of mastery. During this current review cycle, we conducted a fairly major revision of the PLOs, which included reducing the number from six to five, and altering the focus and scope of three of the PLOs. The revised PLOs are listed here: PLO CATEGORY 1: FOUNDATIONS are theoretical and conceptual frameworks for understanding and evaluating communication. Foundations PLO 1: Understanding Communication Theories Demonstrate an understanding of at least one major theory in the field of communication. PLO1, Assessable Outcome: Students can explain, apply, and critique a major theory that has shaped a specific area of Communication Studies. Beginning: Students can describe key concepts and principles of a communication theory. Developing: Students can explain the purpose for a communication theory and how it can be applied to social situations. Accomplished: Students can utilize a communication theory to make insightful observations and judgments about communication. Exemplary: Students can evaluate the strengths and limitations of a communication theory. PLO Category 2: INQUIRY consists of research methods that generate and evaluate new knowledge about communication 1 Inquiry PLO 2: Research Methods Demonstrate an understanding of methods of communication research and analysis such as rhetorical, critical, interpretive, performative and social scientific approaches and their ethical implications. PLO 2, Assessable Outcome: Students can explain the rationale underlying practices and approaches in communication research and their ethical implications. Beginning: Students can identify the key processes of at least two approaches to communication inquiry. Developing: Students can describe and compare the decisions that need to be made when navigating key processes and applications of at least two approaches to communication inquiry. Accomplished: Students can explain the nature (kind, scope) of knowledge produced by and the ethical implications of at least two approaches to communication inquiry. Exemplary: Students can justify the use of a particular approach to a communication question and articulate the ethical implications of the kind of knowledge claims it produces Inquiry PLO 3: Research Application and Critique Develop and apply analytical skills and ethical practices for understanding, conducting, and evaluating communication research studies. PLO 3, Assessable Outcome: Students can evaluate scholarly communication research and apply it to their own communication inquiry. Beginning: Students can locate appropriate scholarly communication research and summarize the results. Developing: Students can identify research questions that emerge from their review of literature and appropriate approaches for addressing those questions. Accomplished: Students can describe the strengths and limitations as well as ethical implications of their own communication inquiry. Exemplary: Students can present their research and incorporate feedback in ways that contribute to the communication field. PLO Category 3: PRACTICE is the use of communication skills and the application of theoretical frameworks and research methods in specific contexts. Practice PLO 4: Communication Competence: Demonstrate the ability to communicate competently and in a theoretically informed manner in a variety of contexts. 2 PLO 4, Assessable Outcome: Students can effectively create and implement communication strategies and skills in a contextually appropriate manner. Beginning: Students can plan the application of and demonstrate implementation of coursespecific communication strategies and skills in a satisfactory manner. Developing: Students can plan the application of and demonstrate implementation of course-specific communication strategies and skills in a polished, professional, and competent manner. Accomplished: Students can demonstrate through practice an understanding of the nuances of the course-specific context, and adapt their performance to that context. Exemplary: Students can demonstrate an understanding of the nuances of the coursespecific context, reflecting on the strengths and weaknesses of their performances and improving them accordingly. Practice PLO5: Social Responsibility: Demonstrate social responsibility, ethical awareness, and community engagement. PLO 5, Assessable Outcome: Students can engage in communication practices designed to produce positive community outcomes and can articulate the ethics that underwrite those practices. Beginning: Students can identify how a communication practice can lead to a positive community outcome and can begin to articulate the ethical implications of that practice. Developing: Students can attempt to enact communication practices designed to produce positive community outcomes, and can express the ethical implications of those practices. Accomplished: Students can engage in multiple communication practices, demonstrate how they produce positive community outcomes, and can articulate the ethical strengths and weaknesses of those practices. Exemplary: Students can adapt and revise multiple communication practices to different community contexts to produce a range of positive community outcomes, and can articulate the different ethical strengths and weaknesses of those practices. 2. Map of PLOs to University Learning Goals (ULGs) B.A. Communication Studies PLO 1 - Understanding Communication Theories: Demonstrate an understanding of at least one major theory in the field of communication ULG 1 (Specialized Knowledge) ULG 2 (Broad Integrative Knowledge) ULG 3 (Intellectual Skills) ULG 4 (Applied Knowledge) ULG 5 (Social and Global Responsibilities) X 3 PLO 2 - Research Methods: Demonstrate an understanding of methods of communication research and analysis such as rhetorical, critical, interpretive, performative and social scientific approaches and their ethical implications PLO 3 - Research Application and Critique: Develop and apply analytical skills and ethical practices for understanding, conducting, and evaluating communication research studies PLO 4 - Communication Competence: Demonstrate the ability to communicate competently and in a theoretically informed manner in a variety of contexts PLO5 - Social Responsibility: Demonstrate social responsibility, ethical awareness, and community engagement X X X X The map above was created by Professor Gao Ge (former Department Curriculum and Assessment Chair) in collaboration with Professor Hillary Nixon (College Assessment Facilitator) and the College of Social Sciences Assessment Committee. It was modified by Shawn Spano to accommodate the revised PLOs. 4 3. Alignment – Matrix of PLOs to Courses Core Course that Introduces Students to all the PLOs COMM 101C Junior Seminar: Theorizing Communication [Required for all Majors] COMM 198: Applied Activity in Communication [Required for all Majors] Core Course for Summative Assessment of all PLOs COMM 199C Senior Seminar: Synthesis and Application [Required for all Majors] Foundation Courses that Support PLO 1 [Majors select two from list] Inquiry Courses that Support PLOs 2 and 3 [Majors select two from list] Practice Courses that Support PLOs 4 and 5 [Majors select two from list] COMM 110F Interpersonal Comm COMM 117F Understanding Play and Games COMM 122F Performance Studies COMM 130F Social Movements Comm COMM 133F Ethical Problems in Comm COMM 144F Organizational Comm COMM 146F Comm and the Environment COMM 149F Rhetoric and Public Life COMM 160F Language, Meaning and Culture COMM 161F Comm and Culture COMM 164F Comm and Global Organizations COMM 170F Persuasion COMM 171F Visual Comm COMM 172F Multicultural Comm COMM 173F Intercultural Comm and Global Understanding COMM 175F Nonverbal Comm COMM 181F New Media/New World COMM 119I Researching Games COMM 123I Performance of Ethnodrama COMM 145I Rhetorical and Cultural Criticism COMM 150I Inquiry in Organizational Comm COMM 151I New Media/New Methods COMM 152I Comm in World Cultures COMM 155I Quantitative Comm Inquiry COMM 156I Qualitative Comm Inquiry COMM 169I The Media: Response and Criticism COMM 105P Comm, Self and Society COMM 111P Interviewing COMM 113P Games in Everyday Life COMM 114P Business and Professional Speaking COMM 115P Comm and Conflict COMM 116P Mediation: Theory and Practice COMM 120P Persuasive and Presentation Skills COMM 121P Performance as Practice COMM 124P Training and Development COMM 125P Ensemble Performance COMM 131P New Media/You Media COMM 140P Argumentation and Debate COMM 141P Small Group Comm COMM 147P Argumentation and Persuasion in Courts of Law COMM 176P Gender and Comm COMM 182P Comm in the Classroom 5 4. Planning – Assessment Schedule PLO PLO 1: Understanding Communication Theories: Demonstrate an understanding of at least one major theory in the field of communication. PLO 2: Research Methods: Demonstrate an understanding of methods of communication research and analysis such as rhetorical, critical, interpretive, performative and social scientific approaches and their ethical implications. When will this PLO be assessed? We will assess all five PLOs In AY 2015-16 PLO 3: Research Application and Critique: Develop and apply analytical skills and ethical practices for understanding, conducting, and evaluating communication research studies. PLO 4: Communication Competence: Demonstrate the ability to communicate competently and in a theoretically informed manner in a variety of contexts. PLO5: Social Responsibility: Demonstrate social responsibility, ethical awareness, and community engagement. Foundations PLO 1: Understanding Communication Theories Demonstrate an understanding of the major theories that have shaped the field of communication, including historical developments, ethical issues and current trends. AY 2016-2017 PLO5: Social Responsibility: Demonstrate socialTheories: responsibility, Foundations PLO 2: Critiquing Communication Demonstrate the ability to discuss the strengths and limitations of ethical awareness, and community engagement theoretical perspectives on communication. 6 Inquiry PLO3: Research Methods Demonstrate an understanding of methods of communication research and analysis such as rhetorical, critical, interpretive, performative and social scientific approaches and their ethical implications. AY 2017-2018 Inquiry PLO 4: Research Application and Critique Develop and apply analytical skills and ethical practices for understanding, conducting, and evaluating communication research studies. Practice PLO 5: Communication Competence: Demonstrate the ability to communicate competently and in a theoretically informed manner in a variety of contexts. AY 2018-2019 Practice PLO 6: Social Responsibility: Demonstrate social responsibility, ethical awareness, and community engagement. 5. Student Experience PLOs are communicated to students in two ways. First and foremost, through their major courses and syllabi. For example, the two main core courses (Comm 101C and Comm 199C) and all of the F, I, P courses in the major include the PLOs on their syllabi. In the case of the core courses, all five PLOs are included on the syllabi. Additionally, these courses are instrumental in introducing students to all of the PLOs (101C) and assessing how well students accomplish the PLOs in the program (199C). Also, each of the F, I, P courses lists the PLOs for their particular area on the syllabi (i.e. F courses include the two Foundations PLOs, I courses include the two Inquiry PLOs, etc.). The second way we communicate the PLOs to students is through our Department website. We have started to incorporate student feedback into the refinement of the PLOs through an “exit interview” assignment in Comm 199C. Specifically, we learned that students are not as familiar with the PLOs as they should be before taking the 199C senior capstone course. We also found that the PLOs, as originally worded, were not particularly accessible to students, in large part because they written by faculty for faculty using academic jargon and terminology. Part of our goal in revising the PLOs was to make them more accessible and meaningful for students (without compromising the scholarly integrity of the outcomes). 7 Part B 6. Graduation Rates for Total, Non URM and URM students (per program and degree) Total URM Non-URM All others First-Time Freshmen Undergraduate Transfer New Credential First-Time Graduate Fall 2008 Cohort: 6-Year Fall 2011 Cohort: 3-Year Fall 2011 Cohort: 3-Year Fall 2011 Cohort: 3-Year Graduation Rate Graduation Rate Graduation Rate Graduation Rate Universit Universit Universit Universit College College College College y y y y Averag Averag Averag Averag Average Average Average Average e Grad e Grad e Grad e Grad Grad Grad Grad Grad Progra Rate Progra Rate Progra Rate Progra Rate Progra Rate - All Progra Rate - All Progra Rate - All Progra Rate - All m All m All m All m All m Grad Students m Grad Students m Grad Students m Grad Students Cohort Studen Cohort Studen Cohort Studen Cohort Studen Rate Who Rate Who Rate Who Rate Who Size ts Who Size ts Who Size ts Who Size ts Who Entered Entered Entered Entered Entere Entere Entere Entere the the the the d This d This d This d This Universit Universit Universit Universit College College College College y y y y 15 46.7% 48.4% 49.7% 83 74.7% 68.1% 55.3% 0 /0 /0 8.3% 13 46.2% 51.0% 60.8% 100.0 3 66.7% 47.2% 40.7% 24 62.5% 67.5% 55.2% 0 /0 /0 12.2% 3 54.8% 65.2% % 10 40.0% 50.3% 53.3% 49 79.6% 70.3% 54.9% 0 /0 /0 8.0% 6 33.3% 50.0% 54.2% 2 50.0% 44.1% 52.9% 10 80.0% 63.7% 56.9% 0 /0 /0 4.9% 4 25.0% 50.0% 69.4% The “total” 6-yr graduation rate for first-time freshman was 46.7% in fall 2008, which is slightly below the university target (51.6%) and College and University averages. The “URM” 6-yr graduation rate for first-time freshman was 66.7% in fall, 2008, which is above the university target (47.8%). The “Non-URM” 6-yr graduation rate for first-time freshman was 40.0% in fall, 2008, which is below the university target (53.2%). 7. Headcounts of program majors and new students (per program and degree) Total BA MA Fall 2014 New Students Continuing Students Total FT AdmitNew TransfContinuingRetn.TranfReturningTrnst-Ugrd 40 133 533 4 3 3 716 23 133 518 4 2 3 683 17 15 1 33 Headcount trends in the B.A. Program for first-time freshmen and new undergraduate transfer students have increased steadily over years. Overall, the total number of majors in the B.A. Program has increased from approximately 467 students in fall 2009 (data not included here) to 683 in fall 2014. 8 8. SFR and average section size (per program) Fall 2014 Subject College University SFR SFR SFR Lower 26.7 Division Upper 21.8 Division Graduate 13.1 Division 40.0 31.0 26.8 25.5 12.2 20.8 Fall 2014 Subject College University Headcount Headcount Headcount per per per Section Section Section Lower Division 27.3 41.8 35.6 Upper Division 22.4 27.4 28.0 Graduate 7.7 8.9 15.8 Division SFR in the B.A. Program increased from 24.9 in lower division courses in fall 2009 (data not included here) to 26.7 in fall 2014. SFR also increased in upper division courses from fall 2009 (18.3) (data not included here), to fall 2013 (21.8). Department SFR averages are below College and University averages at both lower and upper-division levels. Department headcount in lower and upper sections is also below College and University averages. 9. Percentage of tenured/tenure-track instructional faculty (per department) Fall 2014 DepartmentDepartment College University FTEF # FTEF % FTEF % FTEF % Tenured/Tenuretrack Not tenure-track Total 9.1 25% 44.9% 42.8% 27.1 36.2 75% 100% 55.1% 100.0% 57.2% 100.0% In fall 2014 the department had a 33.5% ratio between Tenured/Probationary Faculty and Temporary Lecturers. This ratio is substantially lower than college and university ratios, indicating that the department, on average, has fewer T/TT faculty compared to temporary lecturers than other departments on campus. Part C 10. Closing the Loop/Recommended Actions According to our PLO assessment schedule from last year, we were supposed to conduct a formal assessment of all PLO during AY 2014-15. When we looked carefully at out assessment data, however, we found that the data and PLOs were not fully aligned enough to do rigorous and systematic assessment. So instead we decided to conduct a fairly major revision of the PLOs. This revision resulted in reducing the total number of PLOs from six to five, and altering the focus and scope of three of the PLOs (see the list of revised PLOs in Part, A, number 1). 9 While we revised the PLOs, we also initiated a revision of the PLO assessment data to better match the new PLOs (see Assessment data, number 11 below). While we are pleased with this effort, it did require us to alter the assessment schedule because we were not able to conduct a systematic assessment of the PLOs. We will do this in AY 2015-16 when the revised PLOs and revised assessment data are in place. Another effort we initiated to close the loop and ensure that students have a deeper understanding of the PLOs before they take the senior capstone course was to develop a PLO assignment for Comm 101C: Junior Seminar: Theorizing Communication. Comm 101C is the required introduction course to the major. The assignment requires students to familiarize themselves with the PLOs, and to write paper describing how particular F, I, and P courses address the PLOs. 11. Assessment Data Data collected for PLO assessment is derived primarily from our senior capstone course: Comm 199C (Senior Seminar: Synthesis and Application). Specifically, students complete a portfolio where they select a paper or assignment from one of their F, I, and P courses and reflect and synthesize what they learned. Students write a reflection essay for each of their F, I and P papers or assignments. We call these reflection essays “frames.” As noted above, we revised the requirements for the portfolios and frames to align more closely to the new PLOs. Because of these recent revisions we have not yet conducted a systematic assessment of the revised PLOs with the revised portfolio (assessment) data. 12. Analysis Here are the results of our informal analysis of the PLO assessment data, based on what were able to do given that we just recently revised the PLOs. The analysis reported here is preliminary; it is based on informal assessment and anecdotal information: PLO PLO 1: Understanding Communication Theories Assessable Outcome 1: Students can explain, apply, and critique a major theory that has shaped a specific area of Communication Studies. Achievement Notes This assessable outcome was recently revised; no analysis of data has been conducted yet. PLO2: Research Methods Assessable Outcome 2: Students can explain the rationale underlying practices and approaches in communication research and their ethical implications. This assessable outcome was recently revised; no analysis of data has been conducted yet. 10 PLO 3: Research Application and Critique Assessable Outcome 3: Students can evaluate scholarly communication research and apply it to their own communication inquiry PLO 4: Communication Competence Assessable Outcome 4: Students can effectively create and implement communication strategies in a contextually appropriate manner PLO 5: Social Responsibility Assessable Outcome 5: Students can engage in communication practices designed to produce positive community outcomes and can articulate the ethics that underwrite those practices This assessable outcome was recently revised; no analysis of data has been conducted yet. Overall, we believe that students are able to achieve this assessable outcome at beginning and developing levels, based on our informal analysis and anecdotal information. Our recent efforts in analyzing the assessment data revealed that we needed to revise the Comm 199C portfolio assignment in order assess this outcome. Based on our informal analysis and anecdotal information, we believe that students are able to achieve this outcome at beginning and developing levels, with some drop off drop at accomplished and exemplary levels Proposed changes and goals (if any) We believe we have improved our PLO assessment program and structure by revising the PLOs and assessment data. Our efforts for next year and beyond will focus on implementing the changes, conducting a formal and systemic assessment of the PLOs, and making any further changes based on our analyses. Additionally, we will work to integrate the PLOs more fully into the F, I, and P courses. Currently, the PLOs function as a secondary learning goals in theses courses, behind the more visible course-specific learning outcomes (CLOs). We intend to make the linkages between these two sets of outcomes more explicit so that students can see how the F, I, and P CLOs roll up and support the PLOs. 11