Chabot College Program Review Report 2014‐2015 Year Two of Program Review Cycle Astronomy Submitted on 11/1/13 Scott Hildreth 1 Appendix B2: “Closing the Loop” Assessment Reflections Course Semester assessment data gathered Number of sections offered in the semester Number of sections assessed Percentage of sections assessed Semester held “Closing the Loop” discussion Faculty members involved in “Closing the Loop” discussion Astronomy 20 Spring 2012 2 2 100% Fall 2012 Timothy Dave, Shannon Lee, Scott Hildreth Form Instructions:  Part I: CLO Data Reporting. For each CLO, obtain Class Achievement data in aggregate for all sections assessed in eLumen.  Part II: CLO Reflections. Based on student success reported in Part I, reflect on the individual CLO.  Part III: Course Reflection. In reviewing all the CLOs and your findings, reflect on the course as a whole. PART I: COURSE‐LEVEL OUTCOMES – DATA RESULTS CONSIDER THE COURSE‐LEVEL OUTCOMES INDIVIDUALLY (THE NUMBER OF CLOS WILL DIFFER BY COURSE) Demonstrate an understanding of the reasons for seasons
Defined Target Scores* (CLO Goal) 50% score 3 or 4
Actual Scores** (eLumen data) 36% Demonstrates understanding of Galileos' Discoveries and why each was important in overthrowing the geocentric model. List of possible answers includes seeing Venus' 50% score 3 or 4
40% phase and size changes, seeing Jupiter's 4 moons revolve around Jupiter, seeing Craters and Mountains on the Moon, Sunspots, Saturn's oblong face, distant (and prior, invisible) stars in the Milky Way. Critical Thinking ‐ Assignment ASTR 20 ‐ Demonstrate 50% score 3 or 4
40% understanding of HR diagram Critical Thinking ‐ Assignment ASTR 20 ‐ Demonstrate gain in understanding of key concepts in Astronomy 20 through 50% score 3 or 4
35% completion of 28‐question multiple choice survey quiz given at least once during the term.  If more CLOs are listed for the course, add another row to the table. * Defined Target Scores: What scores in eLumen from your students would indicate success for this CLO? (Example: 75% of the class scored either 3 or 4) **Actual scores: What is the actual percent of students that meet defined target based on the eLumen data collected in this assessment cycle? 2 PART II: COURSE‐ LEVEL OUTCOME REFLECTIONS A. COURSE‐LEVEL OUTCOME (CLO) 1: 1. How do your current scores match with your above target for student success in this course level outcome? We are below the target goal for students demonstrating an understanding of seasons, based on this CLO and when it is assessed in the class. 2. Reflection: Based on the data gathered, and considering your teaching experiences and your discussions with other faculty, what reflections and insights do you have? We use a variety of metrics to assess this CLO, including Mastering Astronomy scores based on an initial homework assignment. Students may not yet understand the concept at the time of this assessment. We could consider giving this assessment later in the term, or using a different metric, including essays in a midterm. Not all faculty use this method, so comparing results might not be effective or reasonable. B. COURSE‐LEVEL OUTCOME (CLO) 2: 1. How do your current scores match with your above target for student success in this course level outcome? We are below the target goal for students demonstrating an understanding of Galileo’s work, based on this CLO and when it is assessed in the class. 2. Reflection: Based on the data gathered, and considering your teaching experiences and your discussions with other faculty, what reflections and insights do you have? We use a variety of metrics to assess this CLO, including Mastering Astronomy scores based on an initial homework assignment. Students may not yet understand the concept at the time of this assessment. We could consider giving this assessment later in the term, or using a different metric, including essays in a midterm. Not all faculty use this method, so comparing results might not be effective or reasonable. 3 C. COURSE‐LEVEL OUTCOME (CLO) 3: 1. How do your current scores match with your above target for student success in this course level outcome? We are below the target goal for students demonstrating an understanding of HR diagrams, based on this CLO and when it is assessed in the class. 2. Reflection: Based on the data gathered, and considering your teaching experiences and your discussions with other faculty, what reflections and insights do you have? We use a variety of metrics to assess this CLO, including Mastering Astronomy scores based on an initial homework assignment. Students may not yet understand the concept at the time of this assessment. We could consider giving this assessment later in the term, or using a different metric, including essays in a midterm. Not all faculty use this method, so comparing results might not be effective or reasonable. D. COURSE‐LEVEL OUTCOME (CLO) 4: 1. How do your current scores match with your above target for student success in this course level outcome? We are below the target goal for students demonstrating an overall understanding of the key concepts in our class, based on this CLO and whether, how, and when it is assessed in the class. 2. Reflection: Based on the data gathered, and considering your teaching experiences and your discussions with other faculty, what reflections and insights do you have? We need to enable students to do this assessment both on paper and online to add more data for the analysis. Currently it is deployed online only, at the end of the term, and we are not getting 100% participation from students. We’ll give the assessment on paper, and collect them for the final exam, in Spring, and see what happens. 4 PART III: COURSE REFLECTIONS AND FUTURE PLANS 1. What changes were made to your course based on the previous assessment cycle, the prior Closing the Loop reflections and other faculty discussions? 1) We continue to use new visual resources in the planetarium to help students both learn the concepts and apply them to current events. 2) We changed CLO’s after the first few years of trying more complex assessments and finding the eLumen system unworkable. 2. Based on the current assessment and reflections, what course‐level and programmatic strengths have the assessment reflections revealed? What actions has your discipline determined might be taken as a result of your reflections, discussions, and insights? We are continuing to discuss what online tool sets make the most sense, and the most positive difference, with our students. We will continue to investigate how to use those online systems to help us track student success. In particular, we are continuing to investigate using newer online homework/lecture/discussion portals that offer combined assessment and interaction beyond what Blackboard can do, and different from the current system in use, Mastering Astronomy, supported by the publisher of our primary textbooks. We still struggle significantly with the CLO process, the difficulty in data entry and inconsistency with the various reporting tools published online. We would like to make much more fluid changes to our CLOs, but those entries don’t always seem to populate the online system. 3. What is the nature of the planned actions (please check all that apply)? X Curricular X Pedagogical X Resource based X Change to CLO or rubric X Change to assessment methods
5 Appendix B2: “Closing the Loop” Assessment Reflections Course Semester assessment data gathered Number of sections offered in the semester Number of sections assessed Percentage of sections assessed Semester held “Closing the Loop” discussion Faculty members involved in “Closing the Loop” discussion Astronomy 30 LAB Spring 2012 2 2 100% Fall 2012 Timothy Dave, Shannon Lee, Scott Hildreth Form Instructions:  Part I: CLO Data Reporting. For each CLO, obtain Class Achievement data in aggregate for all sections assessed in eLumen.  Part II: CLO Reflections. Based on student success reported in Part I, reflect on the individual CLO.  Part III: Course Reflection. In reviewing all the CLOs and your findings, reflect on the course as a whole. PART I: COURSE‐LEVEL OUTCOMES – DATA RESULTS CONSIDER THE COURSE‐LEVEL OUTCOMES INDIVIDUALLY (THE NUMBER OF CLOS WILL DIFFER BY COURSE) Defined Target Scores* (CLO Goal) 50% score 3 or 4
Actual Scores** (eLumen data) Demonstrate an understanding of the difference between 66% precision and accuracy through a lab report or capstone survey Demonstrate an understanding of the nature of angular size and resolution with telescopes through a lab activity, write‐up, project, or hands‐on demonstration using a 50% score 3 or 4
66% telescope  If more CLOs are listed for the course, add another row to the table. * Defined Target Scores: What scores in eLumen from your students would indicate success for this CLO? (Example: 75% of the class scored either 3 or 4) **Actual scores: What is the actual percent of students that meet defined target based on the eLumen data collected in this assessment cycle? 6 PART II: COURSE‐ LEVEL OUTCOME REFLECTIONS C. COURSE‐LEVEL OUTCOME (CLO) 1: 3. How do your current scores match with your above target for student success in this course level outcome? We are above the target goal for students demonstrating an understanding of precision and accuracy, based on this CLO and when it is assessed in the class. 4. Reflection: Based on the data gathered, and considering your teaching experiences and your discussions with other faculty, what reflections and insights do you have? We will continue to stress the difference between the two concepts, and illustrate that difference in every lab. D. COURSE‐LEVEL OUTCOME (CLO) 2: 3. How do your current scores match with your above target for student success in this course level outcome? We are above the target goal for students demonstrating an understanding of angular size, resolution with telescopes, based on this CLO and when it is assessed in the class. 4. Reflection: Based on the data gathered, and considering your teaching experiences and your discussions with other faculty, what reflections and insights do you have? We will continue to find new ways to work on, and practice with, the concepts of angular size, angular resolution, and construction of telescopes to maximize that resolution. 7 PART III: COURSE REFLECTIONS AND FUTURE PLANS 4. What changes were made to your course based on the previous assessment cycle, the prior Closing the Loop reflections and other faculty discussions? 1) We are adding a new lab activity based on the Solar Dynamics Observatory Data. 2) We changed CLO’s after the first few years of trying more complex assessments and finding the eLumen system problematic. 5. Based on the current assessment and reflections, what course‐level and programmatic strengths have the assessment reflections revealed? What actions has your discipline determined might be taken as a result of your reflections, discussions, and insights? Students are getting the ideas presented, as we stress their application in multiple labs across the curriculum rather than in just one particular exercise on one night. Our new SDO lab has been field testing in both sections in Fall 2012, and with adjustments will be tried again in Spring 2013. We will assess that new lab in the next round, as well as look at changing or adding CLOs. 6. What is the nature of the planned actions (please check all that apply)? X Curricular X Pedagogical X Resource based X Change to CLO or rubric X Change to assessment methods 8 Appendix C: Program Learning Outcomes Considering your feedback, findings, and/or information that has arisen from the course level discussions, please reflect on each of your Program Level Outcomes. Program: Astronomy Lecture Courses for GE Physical Science Students (Note – this is not a program per se, because students typically take only one lecture course, not a sequence of courses. We have chosen to look at the “Program” as more general goals for student learning in either lecture course, over and above specific concept‐related CLOs) 
PLO #1: Students will be able to explain key principles of astronomy using applicable vocabulary; including employing the scientific method to organize, prioritize, and problem solve. Astronomy (10, 20) 
PLO #2: Students will explain how and where the human species fits into the immense, complex and ever‐
changing universe. Astronomy (10, 20) What questions or investigations arose as a result of these reflections or discussions? We are investigating how to use multi‐media resources and online course tools to address these PLOs. One faculty member is “flipping” the classroom with extensive video clips made available outside of lecture, delivered through a new online tool. One faculty member is utilizing an online homework/media tool with lined YouTube clips to encourage student application and extension. What program‐level strengths have the assessment reflections revealed? The planetarium as a visual theater with immersive 3D capabilities is a resource that still needs to be utilized even more. What actions has your discipline determined might be taken to enhance the learning of students completing your program? Actions planned: We want to solicit more funds to enable purchase of 3D movies for the Spitz projector that will benefit our students by offering immersive visual learning experiences. We will also continue to investigate new online content delivery systems. 9 Program: Astronomy 30 Lab Course for GE Physical Science Students  PLO #1: Students will be able to explain key principles of observational and theoretical astronomy using applicable vocabulary and relating them to hands‐on laboratory activities; including employing the scientific method to organize, prioritize, and problem solve. What questions or investigations arose as a result of these reflections or discussions? We want to give our students even greater opportunities to do real science with real data, rather than just do pre‐planned activities that represent the process of science. What program‐level strengths have the assessment reflections revealed? We have connections with Stanford and NASA that are enabling the creation of new labs utilizing real‐time data and opportunities for students to create their own investigations. What actions has your discipline determined might be taken to enhance the learning of students completing your program? 1) Continue to develop and publish the SDO lab activities. 2) Continue to investigate access and use of robotic telescopes. 3) Continue to push for a roof‐top observatory location on campus so that students might do actual data collection in the lab. 10