CH-152 Spring 2013 Course assessment Prepared by Moni Chauhan, David Sarno and Jun Shin Date: Spring 2013 Department: Chemistry Course: CH-152 Curriculum or Curricula: LS1, PE1, HS1, EH1, SF1, BY1 PART I. STUDENT LEARNING OBJECTIVES For Part I, attach the summary report (Tables 1-4) from the QCC Course Objectives Form. TABLE 1. EDUCATIONAL CONTEXT This course is the second part of general chemistry and is intended to provide students with a fundamental knowledge of the modern theory in general and inorganic chemistry. With an emphasis on developing problem-solving skills as well as on concepts and theories, the course also covers topics in thermodynamics, kinetics and equilibrium that are essential background material to many disciplines in science and technology. These include: enthalpy, entropy, and free energy; molecular kinetic theory of gases; rates of chemical reactions; equilibrium in gaseous and aqueous systems, especially acids and bases; electrochemistry; and an introduction to nuclear chemistry. The students in Honors Classes will give 10 – 15 minute oral presentations using Power Point on topics and concepts chosen from the course material and write a term paper on the topic of the presentation. This course makes extensive use of computers and requires the development of scientific communication skills. TABLE 2. CURRICULAR OBJECTIVES Note: Include in this table curriculum-specific objectives that meet Educational Goals 1 and 2: Curricular objectives addressed by this course: Demonstrate proficiency in factual knowledge and conceptual understanding required for transfer to the junior year in a baccalaureate program in natural science, mathematics, engineering, or computer science or any other program in health sciences. (LS1, PE1) Demonstrate skills in mathematics to the minimum level of basic calculus concepts, including their applications to science and/ or engineering. (LS1) Demonstrate proficiency in communication skills, including technical writing and oral presentation. (LS1) Apply concepts through use of current technology. (LS1) Demonstrate an understanding of the professional, ethical, and social responsibilities related to the fields of natural science, forensic science, mathematics, engineering, and /or computer science. (LS1, PE1, SF1) Demonstrate proficiency in acquiring, processing and analyzing information in all its forms as related to the field of concentration. (LS1) 1 CH-152 Spring 2013 Course assessment Prepared by Moni Chauhan, David Sarno and Jun Shin Use analytical reasoning skills and apply logic to solve problems. (PE1) Use quantitative skills and mathematical reasoning to solve problems. (PE1) Demonstrate effective skills in technical writing and oral presentation (PE1); Students will communicate effectively through reading, writing, listening and speaking. (SF1) Demonstrate a strong foundation in the core engineering fundamentals of general chemistry. (PE1) Students will demonstrate competency in the concepts and methods of the foundation general chemistry courses required for transfer to the junior year in Forensic Science at John Jay College. (SF1) Students will apply concepts learned in the classroom and make conclusions based on scientific thinking. (SF1) Students will work collaboratively in the laboratory to provide reasonable analysis of data obtained and to solve problems. (SF1) Students will integrate the knowledge and skills gained in previous courses with subsequent courses to establish an all-around scientific background. (SF1) Demonstrate mastery of mathematics and science required for transfer to the junior year in a baccalaureate program in Environmental Health or a related program. (EH1) Demonstrate an understanding of the principles of chemistry and how they are fundamental to all living systems. (HS1) TABLE 3. GENERAL EDUCATION OBJECTIVES Gen Ed objective’s ID General educational objectives addressed by this course: Select number from list (1-10) from preceding list. #2 Use analytical reasoning to identify issues or problems and evaluate evidence in order to make informed decisions #3 Reason quantitatively and mathematically as required in their fields of interest and in everyday lifelong learning TABLE 4: COURSE OBJECTIVES AND STUDENT LEARNING OUTCOMES Course objectives Learning outcomes 2 CH-152 Spring 2013 Course assessment 1. Students will solve qualitative and quantitative problems in chemistry. Prepared by Moni Chauhan, David Sarno and Jun Shin a. Students will use the varied forms of mathematical communication: language, symbolic notation, graphs, charts, to formulate quantitative ideas and patterns. b. Students will interpret and solve single-step and multi-step word problems. c. Students will interpret diagrams and models as they relate to qualitative concepts and quantitative problem-solving. a. Students will understand how energy is associated with chemical 2. Students will study reactions. thermochemistry b. Students will be able to calculate the heat energy changes in any of chemical chemical process using calorimetry and thermochemical reactions. equations. 3. Students will learn about the kinetics of chemical reactions. a. Students will determine the order and rate law of chemical reactions by mathematical, graphical, and deductive methods. b. Students will calculate reaction rate and understand its temperature dependence. c. Students will understand basic principles of reaction mechanisms. d. Students will understand basic principles of catalysis. 4. Students will learn a. Students will learn to deduce and interpret the equilibrium constant, and determine the direction a reaction will run. and apply b. Students will calculate equilibrium concentrations. equilibrium c. Students will learn and apply LeChatelier’s Principle to equilibria. concepts to chemical reactions. a. Students will learn and apply basic principles of acid-base chemistry and equilibria. 5. Students will b. Students will calculate the pH values and percent ionization of demonstrate strong and weak acids and bases. knowledge of Acids c. Students will interpret molecular structures and pictorial models and Bases to understand and describe the strength of acids and bases. d. Students will apply acid-base concepts to salts. a. Students will understand how buffers function and be able to determine their pH. 6. Students will b. Students will evaluate pH when titrating acids and bases, demonstrate knowledge of acid c. Students will calculate molar solubility and solubility of sparingly and base equilibria soluble salts. and solubility d. Students will predict the outcome of precipitation reactions. equilibria. e. Students will understand the common ion effect as it applies to acid-base equilibria and solubility equilibria. 3 CH-152 Spring 2013 Course assessment Prepared by Moni Chauhan, David Sarno and Jun Shin a. Students will learn and apply basic principles of entropy and determine the entropy of chemical reactions. 7. Students will study b. Students will comprehend and apply the second and third Laws of Entropy, Free Thermodynamics. Energy and c. Students will learn and apply basic principles of Free Energy and Equilibrium of predict whether reactions are spontaneous or not. chemical reactions d. Students will demonstrate knowledge of the relationship between equilibrium and thermodynamics. 8. Students will learn the introductory concepts of Electrochemistry. a. Students will learn to balance redox reactions. b. Student will explain different batteries and cells. c. Students will comprehend standard reduction potential and evaluate standard electromotive force (Emf) of oxidationreduction reactions. d. Students will evaluate Spontaneity of redox reactions. e. Students will understand the effect of concentration on the Emf of electrochemical cells. f. Students will apply concepts of electrochemistry to the phenomenon of corrosion. a. Students will learn to balance Nuclear Reactions. 9. Students will learn b. Students will comprehend the kinetics of radioactivity decay and about the concepts perform related calculations. of Nuclear c. Students will understand basic principles of Nuclear Chemistry. Transmutation, Nuclear Fission and Nuclear Fusion. 4 CH-152 Spring 2013 Course assessment Prepared by Moni Chauhan, David Sarno and Jun Shin PART II. ASSIGNMENT DESIGN: ALIGNING OUTCOMES, ACTIVITIES, AND ASSESSMENT TOOLS For the assessment project, you will be designing one course assignment, which will address at least one general educational objective, one curricular objective (if applicable), and one or more of the course objectives. Please identify these in the following table: TABLE 5: OBJECTIVES ADDRESSED IN ASSESSMENT ASSIGNMENT Course Objective(s) selected for assessment: (select from Table 4) 1. Students will solve qualitative and quantitative problems in chemistry. 2. Students will learn about the kinetics of chemical reactions. 3. Students will demonstrate knowledge of acid and base equilibria and solubility equilibria. 4. Students will study Entropy, Free Energy and Equilibrium of chemical reactions. 5. Students will learn and apply equilibrium concepts to chemical reactions. 6. Students will demonstrate knowledge of Acids and Bases. Curricular Objective(s) selected for assessment: (select from Table 2) 1. Demonstrate proficiency in factual knowledge and conceptual understanding required for transfer to the junior year in a baccalaureate program in natural science, mathematics, engineering, or computer science or any other program in health sciences. (LS1, PE1) 2. Demonstrate skills in mathematics to the minimum level of basic calculus concepts, including their applications to science and/ or engineering. (LS1) 3. Demonstrate proficiency in acquiring, processing and analyzing information in all its forms as related to the field of concentration. (LS1) 4. Use analytical reasoning skills and apply logic to solve problems. (PE1) 5. Use quantitative skills and mathematical reasoning to solve problems. (PE1) 6. Students will demonstrate competency in the concepts and methods of the foundation general chemistry courses required for transfer to the junior year in Forensic Science at John Jay College. (SF1) 7. Demonstrate mastery of mathematics and science required for transfer to the junior year in a baccalaureate program in Environmental Health or a related program. (EH1) General Education Objective(s) addressed in this assessment: (select from Table 3) GE#2: Use analytical reasoning to identify issues or problems and evaluate evidence in order to make informed decisions GE#3: Reason quantitatively and mathematically as required in their fields of interest and in everyday lifelong learning In the first row of Table 6 that follows, describe the assignment that has been selected/designed for this project. In writing the description, keep in mind the course 5 CH-152 Spring 2013 Course assessment Prepared by Moni Chauhan, David Sarno and Jun Shin objective(s), curricular objective(s) and the general education objective(s) identified above, Also in Table 6, please a) identify the three to four most important student learning outcomes (1-4) you expect from this assignment b) describe the types of activities (a – d) students will be involved with for the assignment, and c) list the type(s) of assessment tool(s) (A-D) you plan to use to evaluate each of the student outcomes. (Classroom assessment tools may include paper and pencil tests, performance assessments, oral questions, portfolios, and other options.) Note: Copies of the actual assignments (written as they will be presented to the students) should be gathered in an Assessment Portfolio for this course. TABLE 6: ASSIGNMENT, OUTCOMES, ACTIVITIES, AND ASSESSMENT TOOLS Briefly describe the assignment that will be assessed: American Chemical Society Assessment Exam for General Chemistry II Student will take the national standardized exam at the end of the semester. Five of the exam questions have been selected to analyze the understanding of concepts in General Chemistry II. The selected problems emphasize on critical thinking, visual interpretation, and multiple step problems solving involving math applied to chemical concepts. The questions chosen are key to determining the students’ understanding of this course and also represent some of the topics that are critical to student success in STEM fields in general. The exam questions are multiple choice and each choice can be correlated to a certain level of understanding or mastery of the concepts. Desired student learning outcomes for the assignment (Students will…) List in parentheses the Curricular Objective(s) and/or General Education Objective(s) (1-10) associated with these desired learning outcomes for the assignment. Briefly describe the range of activities student will engage in for this assignment. What assessment tools will be used to measure how well students have met each learning outcome? (Note: a single assessment tool may be used to measure multiple learning outcomes; some learning outcomes may be measured using multiple assessment tools.) 1. Students will use the varied forms of mathematical communication: language, symbolic notation, graphs, charts, to formulate quantitative ideas and a. Students will attend class to learn necessary concepts, including chemical terminology, visualization of matter from a chemical A. Student responses to five selected exam problems on the ACS assessment exam will be analyzed. Students’ choices on the multiple choice exam will 6 CH-152 Spring 2013 Course assessment 2. 3. 4. 5. 6. 7. 8. patterns. Students will interpret and solve single-step and multistep word problems. Students will interpret diagrams and models as they relate to qualitative concepts and quantitative problemsolving. Students will determine the order and rate law of chemical reactions by mathematical, graphical, and deductive methods. Students will learn to calculate the molar solubility and solubility of sparingly soluble salts. Students will demonstrate knowledge of the relationship between equilibrium and thermodynamics. Students will learn and apply LeChatelier’s Principle to equilibria. Students will understand the relationship between ionization and acid strength. Learning outcome 1 is associated with Curricular Objectives #1, 2, 5, 6, 7 in Table 5 and Gen Ed Objective #3. Learning outcome 2 is associated with Curricular Objectives #1, 2, 4, 5, 6, 7 in Table 5 and Gen Ed Objective #2, 3. Learning outcome 3 is associated with Curricular Objectives #3, 4, 6 in Table 5 and Gen Ed Objective #2 Prepared by Moni Chauhan, David Sarno and Jun Shin perspective, and methods for solving logical and mathematical problems b. Students will engage in problem solving through graded and ungraded assignments with feedback from the instructor c. Students will perform laboratory experiments that require understanding and application of chemical principles d. Students will take the ACS Assessment Exam for General Chemistry II at the end of the semester. be correlated to their level of understanding of the particular concepts. 7 CH-152 Spring 2013 Course assessment Prepared by Moni Chauhan, David Sarno and Jun Shin Learning outcomes 4, 5 are associated with Curricular Objectives #1, 2, 4, 5, 6, 7 in Table 5 and Gen Ed Objective #3. Learning outcomes 6, 7, 8 are associated with Curricular Objectives #1, 3, 4, 6, in Table 5 and Gen Ed Objective #2, 3. PART III. ASSESSMENT STANDARDS (RUBRICS) TABLE 7: CH-152: ASSESSMENT STANDARDS (RUBRICS) Brief description of assignment: (Copy from Table 6 above) American Chemical Society Assessment Exam for General Chemistry II Student will take the national standardized exam at the end of the semester. Five of the exam questions have been selected to represent fundamental concepts in General Chemistry II. The selected problems emphasize logical reasoning, visual interpretation, and application of mathematical methods to chemical concepts. They also represent some of the topics that are critical to student success in subsequent chemistry courses. The exam questions are multiple choice and each choice can be correlated to a certain level of understanding or mastery of the concepts. Desired student learning outcomes (Copy from Column 1, Table 6 above; include Educational Goals and/or General Education Objectives addressed) Assessment measures for each learning outcome (Copy from Column 3,Table 6 above) Standards for student performance: Describe the standards or rubrics for measuring student achievement of each outcome in the assignment. Give the percentage of the class that is expected to meet these outcomes If needed, attach copy(s) of rubrics. 1. Students will use the varied forms of mathematical communication: language, symbolic notation, graphs, charts, to formulate quantitative ideas and patterns. A. Student responses to five selected exam problems on the ACS assessment exam will be analyzed. Students’ choices on the multiple choice exam will be correlated to their Each question requires more than one step to solve or requires the student to make use of assumed fundamental knowledge. Each response on the selected multiple choice questions is assigned 8 CH-152 Spring 2013 Course assessment 2. Students will interpret and solve single-step and multistep word problems. 3. Students will interpret diagrams and models as they relate to qualitative concepts and quantitative problem-solving. 4. Students will determine the order and rate law of chemical reactions by mathematical, graphical, and deductive methods. 5. Students will learn to calculate the molar solubility and solubility of sparingly soluble salts. 6. Students will demonstrate knowledge of the relationship between equilibrium and thermodynamics. 7. Students will learn and apply LeChatelier’s Principle to equilibria. 8. Students will understand the relationship between ionization and acid strength. Learning outcome 1 is associated with Curricular Objectives #1, 2, 5, 6, 7 in Table 5 and Gen Ed Objective #3. Learning outcome 2 is associated with Curricular Objectives #1, 2, 4, 5, 6, 7 in Table 5 and Gen Ed Objective #2, 3. Learning outcome 3 is associated with Curricular Objectives #3, 4, 6 in Table 5 and Gen Ed Objective #2 Learning outcomes 4, 5 are Prepared by Moni Chauhan, David Sarno and Jun Shin level of understanding of the particular concepts. a performance level (point value) of 0-3 based on how completely the question has been answered. Three (3) points indicates that the student can successfully solve the problem and is able to work with the information that is given, as well as with assumed contextual knowledge based on prior experience in the course. Two (2) points indicates that the student understands most of the necessary concepts but could not make the final connection that would completely solve the problem. One (1) point indicates that the student may have recognized a step to solving the problem but could not make any additional conceptual connections. Zero (0) points indicate that the student either did not recognize the type of problem presented or did not know how to begin solving it. See rubric below. 9 CH-152 Spring 2013 Course assessment associated with Curricular Objectives #1, 2, 4, 5, 6, 7 in Table 5 and Gen Ed Objective #3. Learning outcomes 6, 7, 8 are associated with Curricular Objectives #1, 3, 4, 6, in Table 5 and Gen Ed Objective #2, 3. Prepared by Moni Chauhan, David Sarno and Jun Shin Projected outcomes: Question #7 (Learning outcomes #1, 2 and 4) 40% expected to be 3 25% expected to be 1 35% expected to be 0 Question #21 (Learning outcomes #2 and 5) 30% expected to be 3 30% expected to be 2 30% expected to be 1 10% expected to be 0 Question #27 (Learning outcomes #1, 2 and 6) 35% expected to be 3 50% expected to be 2 15% expected to be 1 Since 1, 2 and 3 add up to 100%, 0% will be 0 Question #30 (Learning outcomes #2 and 7) 60% expected to be 3 25% expected to be 1 15% expected to be 0 Question #33 (Learning outcomes #1, 3 and 8) 40% expected to be 3 20% expected to be 2 40% expected to be 1 Since 1, 2 and 3 add up to 100%, 0% will be 0 10 CH-152 Spring 2013 Course assessment Prepared by Moni Chauhan, David Sarno and Jun Shin RUBRIC FOR SELECTED QUESTIONS ON ACS ASSESSMENT EXAM FOR CH-152 Question 7 Choice D B A C Performance Evaluation level Correct response; able to deduce the order of the reaction based on 3 the exponential relationship between initial rate and concentration. Student may have determined the order of the entire reaction rather 1 than just for the single reactant. (It is 1st order in A, 1st order in B and 2nd order overall). Student has set the coefficient of the species in the chemical reaction 0 equal to its order, or may have partially applied the correct method and rounded the result. Student unable to determine the relation between concentration and 0 initial rate and/or does not know the method used to solve the problem. Question 21 Choice C A D B Performance Evaluation level Student has correctly determined how the compound dissociates and 3 applied the proper method to calculate its molar solubility. Student may have correctly determined how the compound 2 dissociates, but did not completely solve the problem (calculated the cube root of given Ksp, but left out an algebraic step) Student solved the problem based on an incorrect dissociation of the 1 compound (calculated as square root of Ksp) Student has not shown any clear understanding of the dissociation or 0 how solubility is commonly determined. Question 27 Choice C A, D Performance Evaluation level Student understands that the largest equilibrium constant will come from the largest negative Free Energy change, which requires a 3 favorable (negative) Enthalpy change and a favorable (positive) Entropy change. Student may understand that these choices can give a negative free energy change, and thus a large equilibrium constant. However, since 2 only one of the factors (ΔH or ΔS) is favorable, the magnitude of the free energy and the corresponding equilibrium constant will necessarily be smaller than in choice C. It is also possible that the 11 CH-152 Spring 2013 Course assessment B 1 Prepared by Moni Chauhan, David Sarno and Jun Shin student does not know how equilibrium constant is related to free energy or how free energy is related to Enthalpy and Entropy, or both. Student may not know how to deduce the positive or negative signs with integers. This combination will always give a positive free energy change and therefore the smallest equilibrium constant. It is possible that the student does not know how equilibrium constant is related to free energy or how free energy is related to Enthalpy and Entropy, or both. Student may not know how to deduce the positive or negative signs with integers. Question 30 Choice D B, C A Performance Evaluation level Understands Le Chatelier’s Principle and also comprehends the 3 ionization of substances in water. Recognizes that the removal of a species from the equilibrium mixture 1 will shift the equilibrium, but has incorrectly applied Le Chatelier’s Principle. 0 Fails to understand that catalysts do not shift the equilibrium position. Question 33 Choice A D B C Performance Evaluation level Understands the ionization concept of acids in water as it applies to 3 acid strength and can visually depict it with a model. Reverse order; student recognizes a pattern from the models, but does 2 not correctly apply it to the concept of acid ionization or strength. Student is partially correct in applying the visual model based on 1 strength of acids. Student is partially correct in applying the visual model based on 1 strength of acids. 12 CH-152 Spring 2013 Course assessment Prepared by Moni Chauhan, David Sarno and Jun Shin PART IV. ASSESSMENT RESULTS TABLE 8A: CH-152: SUMMARY OF ASSESSMENT RESULTS, SPRING 2013 N=82 STUDENTS/6 SECTIONS Question 7 Performance Level A 0 B 1 Evaluation Student has set the coefficient of the species in the chemical reaction equal to its order, or may have partially applied the correct method and rounded the result. Student may have determined the order of the entire reaction rather than just for the single reactant. 18 22 14 28 22.0 (25%) 26.8 (25%) 17.1 (10%) 34.1 (40%) 22.8 % (25%) 38 % (25%) 18.5 % (10%) 20.7 % (40%) A 2 B 0 C (correct) 3 D 1 # of Students Outcome: sp2013 Actual (Expected) Outcome: fa2012 Actual (Expected) Question 21 Performance Level Evaluation # of Students Outcome: sp2013 Actual (Expected) Outcome: fa2012 Actual (Expected) Student may have correctly determined how the compound dissociates, but did not completely solve the problem (calculated the cube root of given Ksp, but left out an algebraic step) Student has not shown any clear understanding of the dissociation or how solubility is commonly determined C 0 Student unable to determine the relation between concentration and initial rate and/or does not know the method used to solve the problem. Student has correctly determined how the compound dissociates and applied the proper method to calculate its molar solubility. D (correct) 3 Correct response; able to deduce the order of the reaction based on the exponential relationship between initial rate and concentration. Student solved the problem based on an incorrect dissociation of the compound (calculated as square root of Ksp) 4 20 33 25 4.9% (30%) 24.4 % (10%) 40.2 % (30%) 30.5 % (30%) 21.7% (30%) 22.8 % (10%) 34.8 % (30%) 20.7 % (30%) Question 27 Performance Level A 2 B 1 C (correct) 3 D 2 Evaluation Student may understand that these choices can give a negative free energy change, and thus a large equilibrium constant. However, since only one of the factors (ΔH or ΔS) is favorable, the magnitude of the free energy and the corresponding equilibrium constant will necessarily be smaller than in choice C. It is also possible that the student does not know how equilibrium constant is This combination will always give a positive free energy change and therefore the smallest equilibrium constant. It is possible that the student does not know how equilibrium constant is related to free energy or how free energy is related to Enthalpy and Entropy, or both. Student may not know how to deduce the positive or negative signs with integers. Student understands that the largest equilibrium constant will come from the largest negative Free Energy change, which requires a favorable (negative) Enthalpy change and a favorable (positive) Entropy change. Student may understand that these choices can give a negative free energy change, and thus a large equilibrium constant. However, since only one of the factors (ΔH or ΔS) is favorable, the magnitude of the free energy and the corresponding equilibrium constant will necessarily be smaller than in choice C. It is also possible that the student does not know how equilibrium constant is 13 CH-152 Spring 2013 Course assessment Prepared by Moni Chauhan, David Sarno and Jun Shin related to free energy or how free energy is related to Enthalpy and Entropy, or both. Student may not know how to deduce the positive or negative signs with integers # of Students Outcome: sp2013 Actual (Expected) Outcome: fa2012 Actual (Expected) Question 30 Performance Level Evaluation # of Students Outcome: sp2013 Actual (Expected) Outcome: fa2012 Actual (Expected) related to free energy or how free energy is related to Enthalpy and Entropy, or both. Student may not know how to deduce the positive or negative signs with integers 21 26 26 9 25.6% (25%) 31.7 % (15%) 31.7 % (35%) 11.0% (25%) 19.6% (10%) 39.1 % (45%) 34.8 % (30%) 6.5% (15%) A 0 B 1 C 1 D (correct) 3 Fails to understand that catalysts do not shift the equilibrium position. Recognizes that the removal of a species from the equilibrium mixture will shift the equilibrium, but has incorrectly applied Le Chatelier’s Principle. Recognizes that the removal of a species from the equilibrium mixture will shift the equilibrium, but has incorrectly applied Le Chatelier’s Principle. Understands Le Chatelier’s Principle and also comprehends the ionization of substances in water. 20 12 9 41 24.4 % (15%) 14.6 % (15%) 11.0 % (10%) 50.0 % (60%) 30.4 % (15%) 16.3 % (15%) 17.4 % (10%) 35.9 % (60%) Question 33 Performance Level A (correct) 3 B 1 C 1 D 2 Evaluation Understands the ionization concept of acids in water as it applies to acid strength and can visually depict it with a model. # of Students Outcome: sp2013 Actual (Expected) Outcome: fa2012 Actual (Expected) Student is partially correct in applying the visual model based on strength of acids. Student is partially correct in applying the visual model based on strength of acids. Reverse order; student recognizes a pattern from the models, but does not correctly apply it to the concept of acid ionization or strength. 40 9 18 15 48.8 % (40%) 11.0 % (20%) 22.0 % (20%) 18.3 % (20%) 43.5 % (40%) 6.5 % (20%) 16.3 % (20%) 33.7 % (20%) **Graphical representations of the data are on the following page. 14 CH-152 Spring 2013 Course assessment Prepared by Moni Chauhan, David Sarno and Jun Shin 15 CH-152 Spring 2013 Course assessment Prepared by Moni Chauhan, David Sarno and Jun Shin TABLE 8B: SUMMARY OF ASSESSMENT RESULTS Desired student learning outcomes: (Copy from, Column 1,Table 6 above; include Educational Goals and/or General Education Objectives addressed) Student achievement: Describe the group achievement of each desired outcome and the knowledge and cognitive processes demonstrated. 1. Students will use the varied forms of mathematical communication: language, symbolic notation, graphs, charts, to formulate quantitative ideas and patterns. 2. Students will interpret and solve single-step and multi-step word problems. 3. Students will interpret diagrams and models as they relate to qualitative concepts and quantitative problem-solving. 4. Students will determine the order and rate law of chemical reactions by mathematical, graphical, and deductive methods. 5. Students will learn to calculate the molar solubility and solubility of sparingly soluble salts. 6. Students will demonstrate knowledge of the relationship between equilibrium and thermodynamics. 7. Students will learn and apply LeChatelier’s Principle to equilibria. 8. Students will understand the relationship between ionization and acid strength. Learning outcome 1 is associated with Curricular Question #7 on the exam tested learning outcomes #1, 2, and 4: 34.1% of students scored 3, 26.8% scored 1, and 39.1% scored 0 (22% + 17.1%). The percentage of students who chose the correct answer was slightly lower than expected. The result for choice A (0 pt) was similar to the expected outcome. For choice B (1 pt) and C (0 pt), the outcome was higher than expected. This problem tested the students’ ability to solve multi-step quantitative problems related to chemical kinetics. It requires recognition of a specific problem type, interpretation and application of tabulated data, and knowledge of discipline-specific terminology. RESULTS FROM SPRING 2013 Question #21 on the exam tested learning outcomes #2 and 5: 40.2% of students scored 3, 4.2% scored 2, 30.5% scored 1, and 24.4% scored 0. The percentage of students who chose the correct answer was much higher than expected. A score of 2 was much lower than expected. A score of 1 was approximately the same as the expected outcome, and a score of 0 was more than twice the expected outcome. This problem tested the students’ ability to solve multi-step quantitative problems related to chemical equilibrium. It requires recognition of a specific problem type, knowledge of discipline-specific terminology, and application of knowledge gained from the prior course, CH-151 (dissociation of ionic compounds). Question #27 on the exam tested learning outcomes #1, 2, and 6: 31.7% of students scored 3, 36.6% scored 1 (25.6% + 11%) and 31.7% scored 0. After careful consideration of the available answers to this question, the performance levels and the expected outcomes from fall 2012 were adjusted for spring 2013 (see Tables 7 and 8a). The result for the highest score was slightly less than the expected outcome. In total, the results for a score of 2 were either about the same or lower than expected (see table 8a). The result for a score of 1 16 CH-152 Spring 2013 Course assessment Objectives #1, 2, 5, 6, 7 in Table 5 and Gen Ed Objective #3. Learning outcome 2 is associated with Curricular Objectives #1, 2, 4, 5, 6, 7 in Table 5 and Gen Ed Objective #2, 3. Learning outcome 3 is associated with Curricular Objectives #3, 4, 6 in Table 5 and Gen Ed Objective #2 Learning outcomes 4, 5 are associated with Curricular Objectives #1, 2, 4, 5, 6, 7 in Table 5 and Gen Ed Objective #3. Learning outcomes 6, 7, 8 are associated with Curricular Objectives #1, 3, 4, 6, in Table 5 and Gen Ed Objective #2, 3. Prepared by Moni Chauhan, David Sarno and Jun Shin was about twice as high as expected. Since all of the incorrect choices could be considered partially correct (see Table 9, section A), none of them were assigned a value of 0. This problem tested the students’ ability to apply conceptual and mathematical/algebraic approaches to solving a problem that involves the relationship between equilibrium and thermodynamics. It also requires knowledge of disciplinespecific terminology. Question #30 on the exam tested learning outcome #2 and 7: 50% of students scored 3, 25.6% (14.6% + 11%) scored 1, and 24.4% scored 0. The result for the highest score was lower than the expected outcome, but still quite high. Results for a score of 1 were very close to the expected outcomes and a score of 0 was higher than expected. This problem tested the students’ knowledge of chemical equilibrium, their ability to apply LeChatelier’s Principle. It also required application of knowledge gained from the prior course, CH-151 (dissociation of ionic compounds). Question #33 on the exam tested learning outcomes #1, 3, and 8: 48.8% of students scored 3, and 18.3% scored 2. 33% (11% + 22%) scored 1. The outcome was higher than expected for a score of 3. The result for a score of 2 was slightly lower than expected. The result for one of the choices worth 1 point was lower than expected, but the other was higher than expected. None of the choices were assigned a value of 0 because they could all be justified as “partially” correct (see Table 9, Section A). This problem tested the students’ ability to solve qualitative problems in chemistry by interpreting visual models and applying them to the concept of acid strength. 17 CH-152 Spring 2013 Course assessment Prepared by Moni Chauhan, David Sarno and Jun Shin TABLE 9. EVALUATION AND RESULTING ACTION PLAN In the table below, or in a separate attachment, interpret and evaluate the assessment results, and describe the actions to be taken as a result of the assessment. In the evaluation of achievement, take into account student success in demonstrating the types of knowledge and the cognitive processes identified in the Course Objectives. A. Analysis and interpretation of assessment results: What does this show about what and how the students learned? Question #7: This question asks students to determine the order of a reaction in terms of a specific reactant. The balanced equation and an appropriate data table are provided. To solve this problem, the students must analyze the data and then perform a specific sequence of steps. The most commonly chosen incorrect answer could be indicative of two things. The simplest explanation is that the students did not recognize the type of problem or if they did, simply could not solve it. However, considering that this is a very common type of problem, another likely possibility is that the students did not read the question carefully and determined the order of the entire reaction, rather than just in terms of the particular reactant. This could account for the unexpectedly high number of students who chose this incorrect answer. Further, students who chose this response could have actually reduced the number of students who picked the correct answer. In other words, they may have known how to solve this type of problem, but did not solve the actual problem that was asked. Finally, the relatively high percentage of students who picked choice A confirms that students continue to mistake the coefficients of a chemical equation for its order with respect to reaction kinetics. Choice C was chosen least often and suggests that those students did not understand the problem or simply could not solve it. From fall 2012 to spring 2013, results improved significantly. A greater percentage of students picked the correct answer over any of the incorrect answers. Further, the number of students who chose the correct answer (order of the particular reactant) increased by approximately 13 percentage points, while there was a decrease of 11 percentage points for the most common incorrect answer (overall order of the reaction). This suggests that not only are the students able to solve this problem, but also that more of them are reading and answering the actual question being asked. There were very slight decreases in the results for the other incorrect choices. Question #21: This question asks students to determine the solubility of a slightly soluble salt. It requires students to have knowledge of chemical equilibrium and dissociation of ionic compounds. The calculation is very typical of this kind of problem and the question is very direct. The actual outcome for the correct choice was higher than our predicted outcome and suggested that many students were able to successfully dissociate the salt into its ions, properly solve for the equilibrium concentrations, and correctly apply the terminology to understand that the equilibrium concentration is equivalent to the molar solubility. For students who picked choice A, they properly dissociated the compound, but appear to have left out an algebraic step. More troubling is that nearly 30% picked choice D, which they would obtain by 18 CH-152 Spring 2013 Course assessment Prepared by Moni Chauhan, David Sarno and Jun Shin incorrectly dissociating the salt into its ions. This very common mistake shows that these students either do not fully understand how ions combine or dissociate to form neutral ionic compounds, or are still having difficulty interpreting chemical formulas and equations. From fall 2012 to spring 2013, the number of students who chose the correct answer increased by approximately 5 percentage points. The number who chose the “partial” solution (leaving out a step) dropped significantly from ~22% in the fall to ~5% in the spring. While these results indicate an overall increase in student success, this was offset by an increase of nearly 10 percentage points for students who incorrectly dissociated the compound. There was also a very slight increase in the percentage of students who chose the “worst” answer. Question #27: This question requires students to make several connections between concepts of thermodynamics (enthalpy, entropy, and free energy) and equilibrium. Since no numerical values are given, there is no calculation to be performed (and the necessary equations are not provided in the exam booklet). Instead, this problem can be solved by considering algebraic relationships among the variables; by considering the physical interpretation of the information given; or by simply memorizing a few basic relationships (often summarized on charts in textbooks). This problem was made more complicated because the four choices had to be compared to each other to answer the question. Considered separately, choice A and choice D could both give a “large” equilibrium constant (depending on the temperature), but the question asks for the “largest”. (This is why they were both assigned 2 points and equal expected outcomes.) More students picked choice A than D, but there does not seem to be any reason, except perhaps that the word “positive” used with the enthalpy and entropy was mistakenly thought to correlate to a “large” equilibrium constant, and “negative” with a “small” constant. That many more students picked the correct answer to this problem shows higherorder thinking in connecting several physical and chemical concepts with the mathematics used to describe them (or perhaps strong recall skills, as suggested above). However, most students picked the response worth one point. This could be interpreted as the students having no understanding of the problem or how to approach it. However, considering how many students chose it, the more likely explanation is that after determining that this combination of enthalpy and entropy will give a positive free energy, it was incorrectly correlated it to a large equilibrium constant (i.e. associating “positive” ΔG with “large” K). The relationship ΔG = – RTlnK shows that the opposite is true: a negative free energy is associated with a large equilibrium constant. From fall 2012 to spring 2013, the number of students who picked the “worst” choice dropped by slightly more than 7 percentage points. However, this was offset by slight drop in the percentage of students who answered correctly, as well as increases in the percent of students who picked the other incorrect choices. Question #30: This problem requires students to apply LeChatelier’s Principle by asking what kind of change would shift the equilibrium in the forward direction. Most of the students picked the correct choice, which tested two concepts. First, they had to understand how the particular ionic compound dissociates when dissolved in water and second, they had to correctly determine how it would affect the equilibrium. The two choices worth only one point each 19 CH-152 Spring 2013 Course assessment Prepared by Moni Chauhan, David Sarno and Jun Shin were picked by the fewest number of students. Both were based on the same kind of “mistake” in reasoning that would shift the reaction in the reverse direction. A surprisingly high number of students incorrectly chose “addition of a catalyst”, which they should know affects the kinetics of a reaction, but not its equilibrium position. In other words, this choice relies primarily on factual recall. From fall 2012 to spring 2013, there was significant improvement in student success on this problem. The number of students who chose the correct answer increased by 14 percentage points and there were decreases in the percentages for all of the incorrect choices. Question #33: This question uses visual models for the interpretation of acid strength based on ionization in water. It requires students to not only understand the concepts, but also to compare the diagrams and discover the patterns they contain so that they can be ranked. Most students correctly answered this question, suggesting that the diagrams clearly depicted the ionization phenomenon and the students knew greater ionization equals greater acid strength. The next largest group of students ranked them in reverse order. This frequently occurs with ranking problems and suggests that the students can discern the pattern and can rank the choices from “most to least”, but do not necessarily understand how that ranking is associated with the concept being tested. Smaller numbers of students picked seemingly random rankings, though it could be argued that in both cases, they correctly ranked two of the three acids. From fall 2012 to spring 2013, student success improved for this problem. There was not only an increase of 5.5 percentage points for the correct answer, but also a decrease of approximately 15 percentage points for the “reverse” ranking. However, the number of students who chose either of the “random” rankings also increased by an average of 5 percentage points. Changes in weighted average from fall 2012 to spring 2013: A weighted average was calculated based on the percentage of students who chose each answer and their respective point values from the rubric. This provides a simple way of looking at the overall changes in the results for each question. The correct answer is assigned 3 points and the value decreases for the incorrect answers depending on how close they bring the student to the correct solution. This can be considered as analogous to earning “partial credit”. The weighted average takes this “partial credit” into account. The closer the weighted average is to 3 points, the more students chose the correct 3 point answer or the next best 2 point answer. Question # / description 7: Kinetics 21: Solubility 27: Thermodynamics & equilibrium 30: LeChatelier’s Principle 33: Visual models and acid strength Fall 2012 weighted avg. 1.00 1.68 Spring 2013 weighted avg. 1.29 1.61 1.96 2.00 1.41 1.76 2.21 2.16 20 CH-152 Spring 2013 Course assessment Prepared by Moni Chauhan, David Sarno and Jun Shin The weighted averages show increases in the areas of kinetics, thermodynamics (very slight) and LeChatelier’s Principle. There were very slight decreases in the areas of solubility equilibria and visual models/acid strength. It is important to note that gains in the percentage of students who pick the correct answer can be offset by decreases in the percentage that choose the second-best answer. The bar graphs above clearly show the changes for each individual choice. B. Evaluation of the assessment process: What do the results suggest about how well the assignment and the assessment process worked both to help students learn and to show what they have learned? This particular assignment is given at the end of the course. Its purpose is to show what students have learned with respect to several fundamental chemical concepts that they will need to use in subsequent classes. The evaluation of the outcomes will be discussed with the members of the department to improve learning in the desired areas. This assessment will be repeated in subsequent semesters, and the collected data will be evaluated over time. In this case, the data has been assessed for two semesters in a row, first in the fall and then in the spring. Results overall seem to be better for the spring semester. This could be because many of those students have just completed CH-151 in the previous semester, so they are well prepared for this course. In contrast, students who take CH-152 in the fall may have taken CH-151 in the spring. Since more time has passed, they may be less familiar with the prerequisite material. The questions are written in very simple straightforward sentences. There is little to distract and little to misinterpret. The multiple choice answers give the correct response plus responses that anticipate the most commonly made errors. Questions #7 and #21 require a logical and very generally applicable sequence of steps to arrive at the correct answer. They also require an understanding of quantitative relationships including simple algebra, ratios, and multistep calculations. Rote memorization is only useful to a limited extent in question #27 and students are better served by understanding the physical and chemical meaning and relationships of the given terms. Question #30 relies on analytical reasoning skills and application of logic. Question #33 assesses the students’ ability to interpret simple visual models in the context of key chemical concepts. This is a critical skill in chemistry since the phenomena discussed are frequently based on the behavior of particles that cannot be seen. Thus various 2D and 3D models and interpretations are necessary. These questions were well chosen because several of them can only be answered after the student has compared all of the choices to each other. Further, some of them require higherorder thinking by connecting two or more concepts. However, because it was a multiple choice exam, when students chose the wrong answer, it was hard to tell if their choices were based on a certain type of mistake or misconception, a mistake that was then correctly applied to later steps in a problem, or random choice. This makes interpretation of the data more difficult. Future versions of this assessment may be improved by using questions whose incorrect choices can be unambiguously correlated to a single error. 21 CH-152 Spring 2013 Course assessment Prepared by Moni Chauhan, David Sarno and Jun Shin C. Resulting action plan: Based on A and B, what changes, if any, do you anticipate making? Question #7: Based on the data provided, there is only one correct way to solve this problem, and it is typically taught as a specific sequence of steps. While the results have improved from our prior assessment, we still expected a higher percentage of students to solve it correctly. Looking more closely at the results, we have realized that the simplest explanation for the most frequently chosen (incorrect) answer was that the question was simply misread. It is possible that many students determined the order of the entire reaction (2nd order), rather than the order in terms of the particular reactant (1st order). There is nothing wrong with its wording. However, when students see the same question too frequently, they may begin to ignore the details. As a result, when something has changed, it may go unnoticed. In this case, they may have correctly determined the answer, but then went on to unnecessarily perform additional steps that commonly follow. By deconstructing the steps the students perform and by asking different questions based on the same given data, it may be possible to prevent students from becoming too comfortable. This should encourage them to make sure they are answering the actual question and not the question they think they have been asked. This can be accomplished in class and also with homework assignments. Question #21: As noted in Section B, the most likely error that occurs in this type of problem is that the students incorrectly dissociate the salt into its ions. “AB2” salts are often dissociated into A+ and B2–. Those who properly dissociate it into A+2 and 2B– are typically able to solve this kind of “entry-level” solubility problem. Though the rest of the calculation may be performed correctly, if it begins with an incorrect premise, it will lead to an incorrect result. The only way that students’ understanding of this concept will improve is with exposure and practice. Dissociation can be practiced and applied at several points in CH-151 and it should be reviewed again when solubility equilibria are introduced in CH-152. Students should especially be encouraged to write complete chemical equations for every problem they encounter, both in class and on homework assignments. Question #27: This problem ties together many aspects of thermodynamics and relates it back to equilibrium. Most students will probably approach it from a purely algebraic standpoint, trying to figure out how the signs of ΔH and ΔS determine the sign of ΔG and how that is subsequently related to the equilibrium constant. To do this, they must have memorized the relevant equations (ΔG = ΔH−TΔS and ΔG = – RTlnK), which is certainly possible given sufficient practice. However, students will likely perform better on this kind of question if they are able to understand the physical significance of the magnitudes and signs of equilibrium constants (K) and thermodynamic quantities (ΔG, ΔH, ΔS). This will reduce the errors made, for example, if a student leaves out the negative sign in an equation like ΔG = – RTlnK. This particular mistake may, in fact, be the main reason for the most commonly chosen incorrect answer. In addition to thinking solely in terms of positive and negative values for ΔH and ΔS (algebraic approach), students should be encouraged to think about whether an exothermic reaction or an increase in entropy is typically “favorable” or “unfavorable” with respect to spontaneity (or more appropriately for this problem, favoring products at equilibrium). This can then be linked to the 22 CH-152 Spring 2013 Course assessment Prepared by Moni Chauhan, David Sarno and Jun Shin magnitude of the equilibrium constant. Simple relatable reactions such as combustion, splitting water, or reactions of nitrogen and oxygen in air can provide opportunities to calculate or estimate the values and then interpret the sign or magnitude. Students can then be asked if their results make sense based on their experiences. Alternatively, their experiences can be used to predict the sign or magnitude of the values before performing a calculations. Question #30: This problem requires the application of LeChatelier’s Principle to determine how the formation of a product may be increased if a reaction is at equilibrium. Most students answered correctly and relatively small percentages picked choices that would favor the reverse reaction. However, many students also said that the addition of a catalyst would increase the concentration of the product by shifting the equilibrium to the right. Students know that a catalyst will increase the reaction rate, but they often mistakenly believe that forming a product more quickly is equivalent to forming more product overall. Further, they may not know or recall that a catalyst increases the rate in both directions for a reversible reaction, so it will not affect the equilibrium position. By offering this as one of the possible answers to the question, it probably reduced the number of those who chose the correct answer. Other than being sure that students clearly understand the basic properties of catalysts, there is little more that can be done to improve the outcome on this question. More broadly, however, instruction in LeChatelier’s Principle could be improved by posing questions that involve more than the direct addition or removal of one of the species in the chemical equation. As noted in the analysis of question #7, students may get too comfortable when they always answer the same question. By posing more complex examples, such as the addition of species that interact with reactants or products in the given reaction, students will be encouraged to consider cause and effect. This would make choice B, for example, a more familiar option. This will also serve to better prepare them to understand the common-ion effect and buffers, which are both based upon LeChatelier’s Principle. Question #33: It is enormously useful to be able to visualize chemical concepts and phenomena. In this problem, the concept of acid strength and acid ionization are linked to visual models. It is notable that relatively few students ranked the diagrams out of order, while nearly half the students correctly ranked them and less than 20% put them in reverse order. This suggests that most students were able to discern the pattern. In this case, therefore, interpretation of the diagrams does not appear to be the problem. Rather, the relationship between ionization and acid strength may not be well understood. This fundamental concept can easily be overlooked in favor of performing calculations. More qualitative or descriptive questions could be asked in class to probe students’ understanding both before and after introducing more typical calculation-based problems. Our analysis and our experience indicates that students often learn how to solve specific problem types, rather than trying to understand why they are set up as they are, how to interpret their results, how to make predictions, etc. Unfortunately, such an approach results in students who cannot proceed when confronted with unfamiliar problems or scenarios. It is up to the instructors to find a balance between mechanical problem solving and actual critical thinking. Overall, for students to improve in the areas assessed, the course coordinators 23 CH-152 Spring 2013 Course assessment Prepared by Moni Chauhan, David Sarno and Jun Shin recommend practice (in and out of class) and exposure to a wider variety of problems. This should be coupled with an effort to demonstrate how the math is connected to the concepts and how concepts are connected to each other. Students should also be encouraged to reflect on their work to determine if their answer “makes sense”. This may serve to improve their confidence and help them to avoid mistakes. A synopsis of our findings, plus a bulleted action plan will be distributed to the instructors who will be urged to act on the results and recommendations. 24