Central Washington University Assessment of Student Learning Department and Program Report Please enter the appropriate information concerning your student learning assessment activities for this year. Academic Year of Report: __2013-2104_________ College: _____COTS________ Department ______Geology_________ Program: _____Geology B.S., Environmental Geology, B.S., and Geology B.A._ 1. What student learning outcomes were assessed this year, and why? We are in the process of refining two assessment tools that address the following three outcomes: 1. Students will be able to calculate quantitative problems in the discipline. 2. Students will be able to interpret representations of data, including graphs, maps, cross-sections. 3a. Students will be able to demonstrate knowledge of core areas of geology (Geology B.A. and B.S.). 3b. Students will be able to demonstrate knowledge of core areas of environmental geology, e.g. processes and risks associated with natural and anthropogenic environmental hazards (Environmental Geology B.S.). 2. How were they assessed? We used two assessment tools in core courses at the beginning of our program (GEOL 210, Intro to Field Methods) and at the end of our program (GEOL 487, End-of-Major Review). The GEOL 200 course is taught fall term and had 16 students. The GEOL 487 course was taught in the winter and had 12 students. To assess outcome 1 above, we used the same math assessment tool that we have used in previous years (attached). This tool assesses a few basic skills and knowledge in a few areas (e.g. scientific notation, exponents, simple algebra, trigonometry and geometry). As described below, we are working on refining our assessment tool so that it is more specific to geology and so that it incorporates a larger range of math areas. The second assessment tool is a Geoscience Literacy Exam (GLE), that was developed by the assessment team for InTeGrate (Interdisciplinary Teaching about Earth for a Sustainable Future), an NSF-funded program that promotes geoscience teaching in the context of societal issues. The GLE is based on the four literacy documents developed by the scientific communities of Earth, atmospheric, climate, and ocean scientists. These documents can be viewed at http://nagt.org/nagt/teaching_resources/literacies.html. The InTeGrate assessment team developed a set of three questions to go with each big idea or essential principle: a level 1 question with a single multiple choice answer, a level 2 question with a multiple-answer multiple choice question, and an essay question with a 3-pt rubric spelled out. For our department assessment tool, we selected 24 of the multiple choice questions at levels 1 and 2 and three essay questions. These questions are meant to measure geoscience literacy as defined by the community of scientists. Although this is a fairly basic level for our majors, we chose it as a good starting point to see what we could learn from this tool. We are in the process of deciding how we should define proficiency with this new tool. 3. What was learned? In terms of outcome 1, what we have been seeing for years is that, despite taking math courses through Calculus II, many of our students do not have a solid knowledge of basic concepts. In particular, they consistently miss questions that require knowledge of exponent laws and thus have problems with metric system calculations. They also have trouble solving fairly basic algebra equations and do not all have the area of a circle committed to memory. For our new GLE assessment tool, we compared in detail the results for the GEOL 210 students versus the GEOL 487, thinking that we should at least see some improvement in geoscience literacy between the two levels. The results of this analysis are shown in Table 1. Table 1 -- Geoscience Literacy Exam Results Comparision of score between 210 and 487 Question # 210 487 Multiple Choice % % % difference 1 81 92 11 2 68 31 -37 3 44 100 56 4 6 62 56 5 56 69 13 6 44 15 -29 7 94 100 6 8 85 92 7 9 100 92 -8 10 60 15 -45 11 56 62 6 12 88 38 -50 13 81 100 19 14 77 15 -62 15 77 62 -15 16 38 31 -7 17 92 92 0 18 88 100 12 19 78 62 -16 20 63 46 -17 21 5 0 -5 22 81 69 -12 23 59 53 -6 24 58 38 -20 Short Answer 25-210 25-487 difference 3pts % 6 23 17 2pts % 19 38 19 1pt % 31 31 0 0pts % 44 8 -36 26-210 26-487 difference 38 69 31 31 23 -8 12 8 -4 19 0 -19 Overall Class Average 210 56% 487 54% difference -2% In this table, the % difference column shows the change between Geology 210, the sophomore-level course, and Geology 487, a senior-level course. Note that it is comparing two different sets of students in the same year, not the same students over time. One notable result is that there are clearly questions that the GEOL 210 students did considerably better in (questions 2, 6, 10, 12) and questions that the GEOL 487 students did better in (questions 3, 4). The Geology 487 did substantially better in both of the short answer questions. As described below, we are currently looking in detail at the questions on our assessment tool to try to understand why one set of students did better than another on a given question. 4. What will the department or program do as a result of that information? Based on both the departmental assessment as well as assessment data from individual classes, we know that our students are not as strong as we would like in quantitative skills and continue to work to improve our students’ quantitative skills by a number of means: 1) We added math prerequisites to a number of classes forcing students to take their math requirements earlier. We are now working to incorporate more quantitative activities that use the specific math level into these courses. 2) We are preparing a new math assessment that is more specific to geological problems and links to our Quantitative Methods course (GEOL 305). 3) We have examined all of our classes in terms of the type of math skills that are needed for each one (which courses require algebra, statistics, calculus, etc.). The attached table (below the math assessment) summarizes this analysis. Our revised math assessment will have assessment tools related to each area. We plan to use these tools at various points in the major to measure student progress. We will also collaborate on how we incorporate these quantitative skills so that we can help students make connections between different courses and set expectations for increasing mastery of these skills. For the Geoscience Literacy Exam, we are currently examining the exam results in detail to try to understand what they are telling us. In particular, we are looking at the questions that the sophomore-level class did better on. In some cases (question 2), this may be because the content for the question had been taught to them very recently. In other cases (possibly question 6), the older students are over-thinking the multiple-choice question. Once we have better understood these results, we will take two approaches: 1) refine our assessment tool so that it better measures the skills and knowledge we want to assess; 2) adjust our curriculum and course assignments in order to improve student performance, particularly in the senior-level class. 5. What did the department or program do in response to last year’s assessment information? Because of a general concern for our students’ writing skills that is also reflected in our assessment of writing, we have added a new elective course, Communicating Geoscience (GEOL 306).The course description is as follows: “Exploration of effective communication skills in the geosciences. Includes readings from the primary and popular literature, writing, making figures, and oral presentation. Emphasis on peer review and revision.” This course will be offered for the first time next winter (2016). In GEOL 306, we hope to work on both oral and written communication skills. We will assess students at the beginning and end of this course to determine the areas of improvement and track these students in our departmental writing assessment to see if they perform better than their peers who have not taken this elective. For the graduate students, we feel that our current emphasis on writing literature reviews in GEOL 502, including the rubrics that we use for those assignments, are fairly successful. In two of our required graduate courses, we work with the students a great deal on writing. It did not appear that any action is needed as a result of the 2012-2013 assessment for the M.S. program other than continuing with what is working. 6. Questions or suggestions concerning Assessment of Student Learning at Central Washington University: Name ________________________________ Geol 487 Math Assessment (NO CALCULATORS) This is to help us learn about our majors’ ‘working knowledge’ of elementary mathematics. Name, City, State of High School _______________________________________ Geology Degree: _____________________________ Date & name of last Math class_______________________________ 1. Arithmetic (a) 1/0.1 = (b) 2 3 = (c) 641/ 2 = (d) 2-2 = (e) 25 ´10 3 = 5 ´10-5 (f) SHOW WORK 231/7 = 2. Express in scientific notation (a) 0.00012 (b) 300,000 GO TO NEXT PAGE 3. Geometry and Trigonometry (a) The formula for the area of a circle is: (b) Using the diagram at the right, cos = Circle the correct answer _a_ _a_ _d_ b c a _ b_ c _c_ b 4. Algebra (a) PV = nRT Solve for T. (b) y = __x__ (1-x) Solve for x. (c) _a_ = _b_ x c Solve for x in terms of a, b, and c. DONE! THANKS!!! logical course Analysis Basics Functions Statistics current Multivariable and pre- Functions Probability calculus rec's 3 of the 101 yes b's 3 of the 103 yes b's 3 of the 107 yes b's 3 of the 108 yes b's all of the 200 yes b's 210 yes ----- f-4 3 of the 302 305 320 346 yes yes yes yes b's all of f-1, f-2, athe b's f-3 all of the f-1, f-2,f- b's 3 all of the f-1, f-2, b's f-3, f-4 mf-1, sp-1 b's f-1, f-4, mf-1, sp-1, sp-3 all of the f-1, f-3, sp-1, sp-2, b's f-4 sp-3 sp-1, sp-3 all of the 360 370 yes yes all of the 380 yes b's sp-1? all of the 377 yes 386 yes 388 yes 415 423 yes yes b's all of the f-1, f-2, b's f-3 sp-1,? all of the all of the sp-1, sp-2, b's f's mf-1, mf-4 sp-3, sp-3 all of the sp-1, sp-2, b's sp-3 100-C 425 yes all of the f-1, f-2, all of the b's f-3, f-5 sp's all of the 432 yes b's mf-1, mf-3, f-3, f-4 mf-5 sp-1, sp-4 all of the 434 yes b's f-3, f-4 all of the 441 445 452 453 yes yes yes yes b's f-1, f-4 all of the f-1, f-2, b's f-3, f-5 mf-1 sp's all of the all of the all of the all of the all of b's f's mf's sp's the c's all of the all of the all of the b's f's mf's logical course 454 Analysis yes 456 yes 463 yes 474 yes Basics Functions all of the 154 154 all of sp-1 the c's 173 Statistics current Multivariable and pre- Functions Probability all of the all of athe b's sp's all of the all of the b's f's all of the f-1, f-2, sp-1, sp-3, b's f-3, sp-3 calculus rec's all of mf-1, mf-4 the c's co-172 b-2, b-6, 475 yes b-7 all of the 478 480 yes b's yes all of the sp-1, sp-2, b's sp-3 all of the 483 484 la yes b's yes all of the f-2, f-3, b's f-4 logical analysis la-1 sp-4 problem solving sp-2, sp-4 153 c-1, c-2 172 b basic b-1 graphing, graph interpretation, rates b-2 unit conversion b-3 dimensional analysis b-4 back of the envelope, sanity check, order of magnitude qualitative understanding of algebraic equations, and how varying variables affects f mf b-5 unknown b-6 sig-figs b-7 measurements, accuracy and precision functions f-1 dependent vs independent variables f-2 separation of variables (solve for x) f-3 types of functions (linear, power, exponential, logarithmic) f-4 periodic (trig) f-5 solving systems of equations multivariable functions mf-1 vectors