SJSU Annual Program Assessment Form Academic Year 2013-2014 Department: Communication Studies Program: B.A. Program College: Social Sciences Website: www.sjsu.edu/comm _ Check here if your website addresses the University Learning Goals. Program Accreditation: Contact Person and Email: Date of Report: Not Applicable Shawn Spano, shawn.spano@sjsu.edu June 1, 2014 Part A 1. List of Program Learning Outcomes (PLOs) There are six B.A. Program PLOs total, organized into three broad categories: Foundations, Inquiry, and Practice (see below). Department faculty developed and refined these PLOs over a series of faculty meetings, beginning in 2008 and continuing through AY 2013-14. Most recently, faculty developed assessable outcomes for each of the PLOs, indicating criteria for increasing levels of mastery. PLO CATEGORY 1: FOUNDATIONS are theoretical and conceptual frameworks for understanding and evaluating communication. Foundations PLO 1: Understanding Communication Theories Demonstrate an understanding of the major theories that have shaped the field of communication, including historical developments, ethical issues and current trends. PLO1, Assessable Outcome 1: Students can explain the major theories that have shaped a specific area of communication studies. Beginning: Name a minimum of three major theories in a specific area of communication studies. Developing: State and explain key concepts and principles of the theories. Accomplished: Describe and explain the historical context of theories and their relationships to contemporary research. Exemplary: Articulate key accomplishments of those theories, their impact for that area of the field and suggest an agenda for future research. PLO 1, Assessable Outcome 2: Students can provide a comprehensive analysis of at least one communication theory. Beginning: Identify theory author(s) and foundational principles of a major theory of communication. Developing: Articulate theory method and application. 1 Accomplished: Explain rationale for the theory within the field and the broader social context. Exemplary: Evaluate theory impact and propose insightful expansion and alternative uses. Foundations PLO 2: Critiquing Communication Theories: Demonstrate the ability to discuss the strengths and limitations of theoretical perspectives on communication. PLO 2, Assessable Outcome 1: Students can demonstrate their understanding of the process and responsibility of scholarly critique. Beginning: Students can identify the elements of scholarly critique appropriate to that particular course. Developing: Student can explain the elements of scholarly critique Accomplished: Students can articulate the need for and purposes of scholarly critique Exemplary: Students can contextualize scholarly critiques with a larger disciplinary conversation PLO 2, Assessable Outcome 2: Students can present a comparative analysis and critique of at least two major communication theories. Beginning: Students can identify fundamental differences between theoretical perspectives Developing: Students can explain specific similarities and differences between theoretical perspectives Accomplished: Students can use two or more theoretical perspectives to engage and investigate a specific site of communication or communication event Exemplary: Students can articulate the implications of a comparative analysis and critique for the study of communication and future theoretical development. PLO CATEGORY 2: INQUIRY includes research methods that generate and evaluate new knowledge about communication. Inquiry PLO3: Research Methods Demonstrate an understanding of methods of communication research and analysis such as rhetorical, critical, interpretive, performative and social scientific approaches and their ethical implications. PLO 3, Assessable Outcome 1: Students can articulate key methodological concepts and their ethical implications in published communication research. Beginning: Name a minimum of two key methodological concepts from at least two research methods. Developing: Explain, compare, and contrast the key methodological concepts and resulting ethical implications from at least two different research methods. 2 Accomplished: Describe and explain the nature (kind, scope) of knowledge and the ethical stakes produced by those research methods in our discipline. Exemplary: Articulate a strategic usage of two paradigmatically different research methods in the pursuit of specific research goals. Identify the ethical rationale for engaging in such research. PLO 3, Assessable Outcome 2: Students can explain the rationale underlying practices and procedures of communication research. Beginning: Name the key processes and procedures of at least two research methods Developing: State and explain the decisions that need to be made when navigating those key processes and procedures of at least two research methods Accomplished: By drawing on scholarly literature, justify and defend the key processes and procedures of at least two research methods. Exemplary: Troubleshoot the methodological and ethical challenges embedded in the key processes and procedures of at least two paradigmatically different research methods. Inquiry PLO 4: Research Application and Critique Develop and apply analytical skills and ethical practices for understanding, conducting, and evaluating communication research studies. PLO 4, Assessable Outcome 1: Students can write a critique of published communication research, including an examination of the study’s ethical practices. Beginning: Be able to locate appropriate peer-reviewed sources in communication studies and summarize the results. Developing: Be able to identify strengths and limitations, both theoretical and methodological, in appropriate peer-reviewed sources. Accomplished: Be able to synthesize patterns in research studies and explain gaps in the literature to create research questions. Exemplary: Be able to do all this in a compelling, novel, and ethical way. PLO 4, Assessable Outcome 2: Students can write an original research proposal. Beginning: Students can explain the parts of a proposal and how they contribute to its effectiveness. Developing: Students can identify research questions that emerge from the literature review and the appropriate methods for addressing those questions. Accomplished: Students can identify the strengths and limitations and ethical implications of their chosen research method for their proposed study. Exemplary: Students articulate how their proposal responds to relevant disciplinary questions in a novel manner. 3 PLO CATEGORY 3: PRACTICE is the use of communication skills and the application of theoretical frameworks and research methods in specific contexts. Practice PLO 5: Communication Competence: Demonstrate the ability to communicate competently and in a theoretically informed manner in a variety of contexts. PLO 5, Assessable Outcome 1: Students can effectively create and implement communication strategies in a contextually appropriate manner: Beginning: Plan the application of and demonstrate implementation of course-specific communication strategies in a satisfactory manner. Developing: Plan the application of and demonstrate implementation of course-specific communication strategies in a polished, professional, and competent manner. Accomplished: Demonstrate through practice an understanding of the nuances of the course-specific context, and adapt their performance to that context. Exemplary: Demonstrate through practice an understanding of the nuances of the coursespecific context, reflecting on the strengths and weaknesses of their performances and improving their performances accordingly. Practice PLO 6: Social Responsibility: Demonstrate social responsibility, ethical awareness, and community engagement. PLO 6, Assessable Outcome 1: Students can engage in communication practices designed to produce positive community outcomes and can articulate the ethics that underwrite those practices. Beginning: Students can identify how a communication practice can lead to a positive community outcome and can begin to articulate the ethical implications of that practice. Developing: Students can attempt to enact communication practices designed to produce positive community outcomes, and can express the ethical implications of those practices. Accomplished: Students can engage in multiple communication practices, demonstrate how they produce positive community outcomes, and can articulate the ethical strengths and weaknesses of those practices. Exemplary: Students can adapt and revise multiple communication practices to different community contexts to produce a range of positive community outcomes, and can articulate the different ethical strengths and weaknesses of those practices. 4 2. Map of PLOs to University Learning Goals (ULGs) The map above was created by Professor Gao Ge (former Department Curriculum and Assessment Chair) in collaboration with Professor Hillary Nixon (College Assessment Facilitator) and the College of Social Sciences Assessment Committee. 3. Alignment – Matrix of PLOs to Courses Core Course that Introduces Students to all the PLOs COMM 101C Junior Seminar: Theorizing Communication [Required for all Majors] COMM 198: Applied Activity on Communication [Required for all Majors] Core Course for Summative Assessment of all PLOs COMM 199C Senior Seminar: Synthesis and Application [Required for all Majors] Foundation Courses that Support PLOs 1 and 2 [Majors select two from list] Inquiry Courses that Support PLOs 3 and 4 [Majors select two from list] Practice Courses that Support PLOs 5 and 6 [Majors select two from list] COMM 110F Interpersonal Comm COMM 122F Performance Studies COMM 130F Social Movements Comm COMM 133F Ethical Problems in Comm COMM 144F Organizational Comm COMM 123I Performance of Ethnodrama COMM 145I Rhetorical and Cultural Criticism COMM 150I Inquiry in Organizational Comm COMM 151I New Media/New Methods COMM 152I Comm in World Cultures COMM 105P Comm, Self and Society COMM 111P Interviewing COMM 114P Business and Professional Speaking COMM 115P Comm and Conflict COMM 116P Mediation: Theory and Practice COMM 120P Persuasive and 5 COMM 146F Comm and the Environment COMM 149F Rhetoric and Public Life COMM 160F Language, Meaning and Culture COMM 161F Comm and Culture COMM 164F Comm and Global Organizations COMM 170F Persuasion COMM 171F Visual Comm COMM 172F Multicultural Comm COMM 173F Intercultural Comm and Global Understanding COMM 175F Nonverbal Comm COMM 181F New Media/New World COMM 155I Quantitative Comm Inquiry COMM 156I Qualitative Comm Inquiry COMM 169I The Media: Response and Criticism Presentation Skills COMM 121P Performance as Practice COMM 124P Training and Development COMM 125P Ensemble Performance COMM 131P New Media/You Media COMM 140P Argumentation and Debate COMM 141P Small Group Comm COMM 147P Argumentation and Persuasion in Courts of Law COMM 176P Gender and Comm COMM 182P Comm in the Classroom 4. Planning – Assessment Schedule PLO Foundations PLO 1: Understanding Communication Theories Demonstrate an understanding of the major theories that have shaped the field of communication, including historical developments, ethical issues and current trends. Foundations PLO 2: Critiquing Communication Theories: Demonstrate the ability to discuss the strengths and limitations of theoretical perspectives on communication. When will this PLO be assessed? We will assess all six PLOs In AY 2014-2015 Inquiry PLO3: Research Methods Demonstrate an understanding of methods of communication research and analysis such as rhetorical, critical, interpretive, performative and social scientific approaches and their ethical implications. Inquiry PLO 4: Research Application and Critique Develop and apply analytical skills and ethical practices for understanding, conducting, and evaluating communication research studies. Practice PLO 5: Communication Competence: Demonstrate the ability to communicate competently and in a theoretically informed manner in a variety of contexts. Practice PLO 6: Social Responsibility: Demonstrate social responsibility, ethical awareness, and community engagement. 6 Practice PLO 5: Communication Competence: Demonstrate the ability to communicate competently and in a theoretically informed manner in a variety of contexts. Practice PLO 6: Social Responsibility: Demonstrate social responsibility, ethical awareness, and community engagement. Foundations PLO 1: Understanding Communication Theories Demonstrate an understanding of the major theories that have shaped the field of communication, including historical developments, ethical issues and current trends. AY 2015-2016 Foundations PLO 2: Critiquing Communication Theories: Demonstrate the ability to discuss the strengths and limitations of theoretical perspectives on communication. Inquiry PLO3: Research Methods Demonstrate an understanding of methods of communication research and analysis such as rhetorical, critical, interpretive, performative and social scientific approaches and their ethical implications. AY 2016-2017 Inquiry PLO 4: Research Application and Critique Develop and apply analytical skills and ethical practices for understanding, conducting, and evaluating communication research studies. Practice PLO 5: Communication Competence: Demonstrate the ability to communicate competently and in a theoretically informed manner in a variety of contexts. AY 2017-2018 Practice PLO 6: Social Responsibility: Demonstrate social responsibility, ethical awareness, and community engagement. 5. Student Experience PLOs are communicated to students primarily through courses and syllabi. First, the two core courses and all of the F, I, P courses in the major include the PLOs on their syllabi. In the case of the core courses (Comm 101C and Comm 199C), all six PLOs are included on the syllabi. Additionally, these courses are instrumental in introducing students to all of the PLOs (101C) and assessing how well students accomplish the PLOs in the program (199C). Second, each of the F, I, P courses lists the PLOs for their particular area on the syllabi (i.e. F courses include the two Foundations PLOs, I courses include the two Inquiry PLOs, etc.). 7 We have not incorporated student feedback into the development or refinement of the PLOs at this time. Part B 6. Graduation Rates for Total, Non URM and URM students (per program and degree) The “total” 6-yr graduation rate for first-time freshman was 70.0% in fall 2007, which is well above the university target (51.6%), and roughly consistent with prior fall semesters. The “URM” 6-yr graduation rate for first-time freshman was 25.0% in fall, 2007, which is well below the university target (47.8%). However, the graduation rates were higher in six of the previous nine fall semesters, ranging from 33.3% to 100.0% (data not included here). The “Non-URM” 6yr graduation rate for first-time freshman was 100.0% in fall, 2007, which is well above the university target (53.2%). This rate is higher than most of the previous nine fall semesters, although the rate was also 100.0% in fall of 1990, 2002 and 2004 (data not included here). 7. Headcounts of program majors and new students (per program and degree) Headcount trends in the B.A. Program for first-time freshmen have steadily increased over the past five years, going from 26 students in fall semester 2009 (data not included here) to 45 in fall 2013. The trends for undergraduate transfer students show a more dramatic increase, going from 70 students in fall 2009 (data not included here) to 179 in fall 2013. Overall, the total number of majors in the B.A. Program has increased from approximately 467 students in fall 2009 (data not included here) to 654 in fall 2013. 8 8. SFR and average section size (per program) SFR in the B.A. Program increased from 24.9 in lower division courses in fall 2009 (data not included here) to 28.9 in fall 2013. SFR remained essentially the same in in upper division courses from fall 2009 (18.3) (data not included here), to fall 2013 (18.2). 9. Percentage of tenured/tenure-track instructional faculty (per department) In fall 2013 the department had a 31.6% ratio between Tenured/Probationary Faculty and Temporary Lecturers. This ratio is quite a bit lower than the university ratio (53.1%), indicating that the department, on average, has fewer T/TT faculty compared to temporary lecturers than other departments on campus. Part C 10. Closing the Loop/Recommended Actions Our primary assessment activity this year was developing assessable outcomes for our PLOs (see Part A, Item #1 above). We are pleased with those efforts and the progress we made. There are two primary actions that we will be taking next year and in subsequent years. One action is to ensure that students have a deeper understanding of the PLOs before they take the senior capstone, assessment course (Comm 199C). One of the ways we will accomplish this is to provide an explicit and systematic introduction of them in the core, introduction course to the major, Comm 101C. Another way to front load the PLOs is to integrate them more fully into the F, I, and P courses. Currently, the PLOs function as a secondary learning goals in theses courses, behind the more visible course-specific learning outcomes (CLOs). The linkages between these two sets of outcomes will be made more explicit so that students can see how the F, I, and P CLOs roll up and support the PLOs. The second action we will be taking is to be more systematic in the collection and analyses of program-level assessment data. As noted below, we have developed specific assignments in Comm 199C for program level assessment, but we have not been very systematic or rigorous in the collection and analyses of this data. In order to accomplish this we will have 199C instructors collect a designated sample of data each semester, and have them meet to review the data to more formally assess student learning. 9 11. Assessment Data Data collected for PLO assessment is derived primarily from our senior capstone course: Comm 199C (Senior Seminar: Synthesis and Application). Specifically, two assignments are used. One is a “course mapping activity” where students identify the specific F, I, and P courses they took, and how they accomplished the PLOs in those courses. Second, students complete a portfolio where they select a paper or assignment from one of their F, I, and P courses and synthesizing what they learned. This includes describing how assignments link to the PLOs. We collected data from these two assignments in fall 2013 and conducted an informal analysis only (see below), in large part because we have not been very systematic in collecting data, which in turn has not enabled us to be very rigorous in analyzing the data. 12. Analysis Here are the results of our informal analysis of the PLO assessment data: PLO PLO 1: Understanding Communication Theories Assessable Outcome 1: Students can explain the major theories that have shaped a specific area of communication studies. Assessable Outcome 2: Students can provide a comprehensive analysis of at least one communication theory. Achievement Notes Students are able to achieve the two assessable outcomes of this PLO at beginning and developing levels. While the performance drops off at the accomplished and exemplary levels, students still seem to do relatively well performing at these levels. PLO 2: Critiquing Communication Theories Assessable Outcome 1: Students can demonstrate their understanding of the process and responsibility of scholarly critique. Assessable Outcome 2: Students can present a comparative analysis and critique of at least two major communication theories. PLO3: Research Methods Assessable Outcome 1: Students can articulate key methodological concepts and their ethical implications in published communication research. Assessable Outcome 2: Students can Students are able to achieve the two assessable outcomes at beginning and developing levels for this PLO, though their performance is not as strong as with PLO 1. Similarly, there appears to be an even greater drop off at the accomplished and exemplary levels for this outcome compared to PLO 1. Overall, we are finding that students do better understanding communication theories (PLO 1) than critiquing them (PLO 2). Students perform well on the two assessable outcomes for this PLO at beginning and developing levels, with fewer students achieving the outcomes accomplished and exemplary levels. 10 explain the rationale underlying practices and procedures of communication research. PLO 4: Research Application and Critique Assessable Outcome 1: Students can write a critique of published communication research, including an examination of the study’s ethical practices. Assessable Outcome 2: Students can write an original research proposal. PLO 5: Communication Competence Assessable Outcome 1: Students can effectively create and implement communication strategies in a contextually appropriate manner PLO 6: Social Responsibility Assessable Outcome 1: Students can engage in communication practices designed to produce positive community outcomes and can articulate the ethics that underwrite those practices Students seem to be able to achieve the two assessable outcomes for this PLO at beginning and developing levels, with a stronger decline in performance at accomplished and exemplary levels compared to PLO 3. Overall, we are finding that students do better understanding communication research methods compared to applying and evaluating them. Students do very well achieving the one assessable outcome for this PLO at beginning and developing levels. While performance declines at accomplished and exemplary levels, the drop off does not seem to be as severe compared to the first four PLOs. Students do very well achieving the one assessable outcome for this PLO at beginning and developing levels. Similar to PLO 5, the drop at accomplished and exemplary levels is not as severe compared to the first four PLOs. Overall, we think students are achieving the two Practice PLOs (#5 and #6) at higher levels compared to the Foundations and Inquiry PLOs (#1, #2, #3, #4). Proposed changes and goals (if any) We believe we have an effective PLO assessment program and structure in place, so we are not proposing any changes as this time. Our efforts for next year and beyond will focus on improving our existing program as outlined in Item #10 above. 11