Chabot College Program Review Report DIGITAL MEDIA _X_ SLO Portion of Upcoming ’16’17 Program Review (Submitted May 2015 in Preparation for Oct 2015) ___ Revision to ’15-’16 Program Review (Originally Submitted Oct 2014) ___ Revision to ’14-’15 Program Review (Originally Submitted Oct 2013) Submitted on 5/1/15 Contact: Mark Schaeffer Appendix B: “Closing the Loop” Course-Level Assessment Reflections. Course Semester assessment data gathered Number of sections offered in the semester Number of sections assessed Percentage of sections assessed Semester held “Closing the Loop” discussion Faculty members involved in “Closing the Loop” discussion DIGM 31A/B, 32A/B, 34, 35A/B, 36A/B Spanning Fall 13, Spring 14, Fall 14, and Spring 15 1 or 2 Each 100% Spring 2015 Mark Schaeffer Form Instructions: Complete a separate Appendix B2 form for each Course-Level assessment reported in this Program Review. These courses should be listed in Appendix B1: Student Learning Outcomes Assessment Reporting Schedule. Part I: CLO Data Reporting. For each CLO, obtain Class Achievement data in aggregate for all sections assessed in eLumen. Part II: CLO Reflections. Based on student success reported in Part I, reflect on the individual CLO. Part III: Course Reflection. In reviewing all the CLOs and your findings, reflect on the course as a whole. PART I: COURSE-LEVEL OUTCOMES – DATA RESULTS CONSIDER THE COURSE-LEVEL OUTCOMES INDIVIDUALLY (THE NUMBER OF CLOS WILL DIFFER BY COURSE) Defined Target Scores* (CLO Goal) (CLO) 1: Effectively embody artistic concepts and themes in digital work See Narrative Below Actual Scores** (data from eLumen or your own tracking) See Narrative Below (CLO) 2: Demonstrate strong craftsmanship (using industry standard software and technology) in creating digital work See Narrative Below See Narrative Below (CLO) 3: If more CLOs are listed for the course, add another row to the table. * Defined Target Scores: What scores in eLumen from your students would indicate success for this CLO? (Example: 75% of the class scored either 3 or 4) **Actual scores: What is the actual percent of students that meet defined target based on the eLumen (or your own) data collected in this assessment cycle? PART II: COURSE- LEVEL OUTCOME REFLECTIONS A. COURSE-LEVEL OUTCOME (CLO) 1: 1. How do your current scores match with your above target for student success in this course level outcome? See Narrative Below 2. Reflection: Based on the data gathered, and considering your teaching experiences and your discussions with other faculty, what reflections and insights do you have? See Narrative Below B. COURSE-LEVEL OUTCOME (CLO) 2: 1. How do your current scores match with your above target for student success in this course level outcome? See Narrative Below 2. Reflection: Based on the data gathered, and considering your teaching experiences and your discussions with other faculty, what reflections and insights do you have? See Narrative Below C. COURSE-LEVEL OUTCOME (CLO) 3: 1. How do your current scores match with your above target for student success in this course level outcome? See Narrative Below 2. Reflection: Based on the data gathered, and considering your teaching experiences and your discussions with other faculty, what reflections and insights do you have? See Narrative Below D. COURSE-LEVEL OUTCOME (CLO) 4: 1. How do your current scores match with your above target for student success in this course level outcome? See Narrative Below 2. Reflection: Based on the data gathered, and considering your teaching experiences and your discussions with other faculty, what reflections and insights do you have? See Narrative Below E. COURSE-LEVEL OUTCOME (CLO) 5: ADD IF NEEDED. PART III: COURSE REFLECTIONS AND FUTURE PLANS 1. What changes were made to your course based on the previous assessment cycle, the prior Closing the Loop reflections and other faculty discussions? See Narrative Below 2. Based on the current assessment and reflections, what course-level and programmatic strengths have the assessment reflections revealed? What actions has your discipline determined might be taken as a result of your reflections, discussions, and insights? See Narrative Below 3. What is the nature of the planned actions (please check all that apply)? Curricular Pedagogical Resource based Change to CLO or rubric Change to assessment methods Other:______________________________________________________ ___________ Appendix C: Program Learning Outcomes Considering your feedback, findings, and/or information that has arisen from the course level discussions, please reflect on each of your Program Level Outcomes. Program: Digital Design – Certificate of Proficiency Digital Media – Certificate Graphic Design – AA Graphic Design – Certificate of Proficiency PLO #1: Effectively embody artistic concepts and themes in digital work PLO #2: Demonstrate strong craftsmanship (using industry standard software and technology) in creating digital work What questions or investigations arose as a result of these reflections or discussions? See Narrative Below What program-level strengths have the assessment reflections revealed? See Narrative Below What actions has your discipline determined might be taken to enhance the learning of students completing your program? See Narrative Below NOTES ON DIGITAL MEDIA STUDENT LEARNING OUTCOMES (CLOs and PLOs) Replaces Appendices B1, B2, and C For the past few years, Digital Media has been wrestling with how to implement Student Learning Outcomes in a meaningful way. As the only full-time faculty member in this discipline, I have been faced with a number of difficult issues: The technology that underlies Digital Media courses advances rapidly. As the technology changes, the material taught in the courses must change too, as must the way the material is taught. It’s safe to say that I rarely teach the same course twice, even if the course keeps the same name. As a result, lessons that are learned from one year of teaching a course may not have much relevance to the next. We offer a two-course sequence in Photoshop, Illustrator, web design, and digital video editing. However while the "A" courses are offered nearly every semester, the concluding "B" courses are only once every two years. This makes it hard to assess the "B" courses consistently and meaningfully. Our students come from diverse backgrounds and their goals can be wildly different. We have students who want to be professional graphic designers and plan on transferring. Some of our students are learning the software as part of their current career. Some intend to use the software for one particular purpose (such as designing textile patterns) and feel no need to learn the other material that a course may cover. We even have students who are taking our courses for a hobby. The SLO process that I was expected to follow grouped all of these students together, as if they all had the same goals. This severely limited the process’s usefulness. Technical problems with eLumen made it difficult to retrieve and manage the data needed to track SLOs effectively. When the SLO process was rolled out 5 or 6 years ago, we were told to create Course Level Outcomes first. Then we were told to create Program Learning Outcomes based on the Course Level Outcomes. Then we were told that we needed to map the CLOs to the PLOs to the College-Wide Learning Goals. This bottom up hierarchy is counterintuitive and doesn't lead towards successful integration of the levels. In an effort to make the SLO process more useful, I’ve constructed new outcomes for both the Program Level and the Course Level. Starting with two College-Wide Learning Goals: Communication and Critical Thinking, I instituted the following two PLOs: 1. (Communication) Effectively embody artistic concepts and themes in digital work. 2. (Critical Thinking) Demonstrate strong craftsmanship (using industry standard software and technology) in creating digital work. These two PLOs currently apply to our Digital Media Certificate program. Over the next three years, as responsibility for graphic design courses is gradually moved from the Art discipline to Digital Media, they will come to apply to three other programs as well: Digital Design — Certificate of Proficiency, Graphic Design — AA, and Graphic Design — Certificate of Proficiency. Instead of working backward from CLOs to PLOs, we can now much more naturally extend the PLOs to outcomes for individual courses. The only difference between the courses would be the media and the software. Having a consistent set of outcomes creates a strong integrating link between all our courses, even though (like JavaScript programming and video editing) they might have very different types of content. In addition to changing the Learning Outcomes, we will also group our A and B classes together. They use the same software and the same technology, and developing the artistic skill and craftsmanship is essentially the same for both. Fortunately, given the nature of digital media, all of the students’ coursework has been stored electronically and is available for review and reassessment. Here, then, are the SLO results for the past three years of classes: DIGM 31A/B (Photoshop) DIGM 32A/B (Illustrator) Effectively embody artistic concepts and themes in digital work 1: 0% 2: 4% 3: 20% 4: 41% 5: 35% 1: 0% 2: 0% 3: 7% 4: 32% 5: 61% Demonstrate strong craftsmanship (using industry standard software and technology) in creating digital work 1: 0% 2: 2% 3: 11% 4: 21% 5: 66% 1: 1% 2: 4% 3: 8% 4: 17% 5: 70% DIGM 34 (JavaScript for Designers) DIGM 35A/B (Building a Website) DIGM 36A/B (Video Editing) DIGM Program (average of all) 1: 6% 2: 0% 3: 11% 4: 11% 5: 72% 1: 1% 2: 2% 3: 9% 4: 33% 5: 55% 1: 2% 2: 3% 3: 8% 4: 34% 5: 54% 1: 2% 2: 2% 3: 11% 4: 30% 5: 55% 1: 6% 2: 6% 3: 12% 4: 24% 5: 53% 1: 3% 2: 8% 3: 17% 4: 19% 5: 54% 1: 2% 2: 5% 3: 8% 4: 20% 5: 66% 1: 2% 2: 5% 3: 11% 4: 20% 5: 62% Based on these new criteria, Digital Media students are doing very well — over 80% have scored a 4 or 5 for each of the two PLOs. (The scores come from assessment of the last two projects that the students were assigned in each course.) Realistically speaking, however, these high scores can’t be credited completely to good teaching. Students who are not succeeding as well as they would like tend to drop these courses midway through, so these assessments are based only on those students who are still turning in projects during the last third of each course — and those tend to be the better students. A few patterns worth noting: In DIGM 31A/B (Photoshop), students don’t do as well with embodying artistic concepts in their work as they do with demonstrating technical craftsmanship. That’s because the course it mostly technically oriented, and only people who have some graphic design experience are likely to use the tools in an artistically effective way. We’re addressing that by reviving the dormant Graphic Design program, and have begun the process of moving that program from Art to Digital Media. This will also help the students in our other visual-art course, DIGM 32A/B (Adobe Illustrator), although many of the students in that course already have some design experience from taking architecture courses. Students in DIGM 35A/B (formerly Dreamweaver, now Building a Website) have more trouble with the technical aspects, mostly because the technology behind the Web is undergoing rapid change. With the need to make websites compatible with a variety of devices besides web browsers, building a website has become a good deal more complicated, making the process more difficult to teach and to learn. I’m hoping that the development of web technology settles down in the next few years so that software, educational resources, and my own level of experience can catch up to it. Until then, it will likely remain a challenge for many students. DIGM 36A/B (formerly Final Cut, now Video Editing) has the opposite problem: The technology has become so ubiquitous, familiar, and user-friendly that students are able to master it quite easily. However, the aesthetic aspects of video editing are not as easy to absorb, and students have less patience for learning the characteristics of good vs. bad editing now that the technology lets them do fairly complex things quickly. Each time I teach the course, I try to put more emphasis on the artistic side as less is needed on the technical side, and am continuing to experiment with ways to do that effectively. DIGM 34 (JavaScript for Designers) is a very new course that has only been taught once so far. It replaces an earlier course, Flash ActionScript, which covered technology that has gone out of vogue. This is largely a programming course that teaches students to write the computer code that makes web pages fun, interactive, informative, and easy to use. The subject matter is difficult for our students who do not have any programming background, but those who are interested in doing serious web development are willing to take up the challenge I am currently teaching the course for the second time and am experimenting with different sequences and exercises for presenting concepts. The assessment results above represent just one class section with a small number of students, and I don’t think they’re especially meaningful. To better understand the needs of Digital Media students, I have begun to ask students at the beginning of each course to submit in writing their goals for the course — what they hope to be able to do by the time the course is over. At the end, I’ll ask them to rank (on a scale of 1 to 5) how close they’ve come to reaching those goals. This will help the SLO process to apply to students as individuals with different needs rather than as an undifferentiated group. This new data will support a third PLO/CLO — based on Development of the Whole Person — that will be assessed in the next cycle.