Dickenson Chem5700 Assessment Fall 2014 Assessment for Chem5700 Fall 2014 Instructor: Nicholas E. Dickenson Course Description: General Biochemistry I is the first of a two semester course sequence in Biochemistry. The intent of this sequence is to provide a thorough and comprehensive survey of biochemistry for science majors (undergraduates and graduates). Chemistry 5700 focuses on bio molecular structures, enzyme mechanisms and kinetics, and metabolism. Mechanisms used in the class Weekly quizzes, lectures slides, 3 in-class critical thinking problem sets, 2 take home literaturebased homework sets, multiple in class group projects, and exams Course Learning Objectives Using the IDEA evaluation system as a guide, I have identified three main course objectives for Chem5700: 1. Gaining factual knowledge (terminology, classifications, methods, trends) 2. Learning fundamental principles, generalizations, or theories 3. Learning to apply course materials (to improve rational thinking, problem solving and decisions) I believe that the objectives outlined above are all equally important to success in this course as well as subsequent courses in the series, and ultimately in a career in Biochemistry. It is difficult to de-couple and identify examples where concepts addressed in the course do not rely on all three. Below is a list of three topic-specific learning objectives of the course and a key with the necessary IDEA learning objectives for each topic identified. A. Fundamentals of biochemistry including the connection between chemistry, math, and biology and how it pertains to a quantitative description of life. What are the building blocks of life - protein, DNA, RNA and how are they responsible for life? Specifically, how are these molecules synthesized, regulated, and most importantly, how are these processes interconnected (1,2,3). B. What are enzymes and what do they do? Specifically, how are they made, how do they function, and what are the biological and biophysical processes that define their activity and function. A quantitative understanding of enzyme kinetics is central to biochemistry and this course is often the first, and sometimes the only, exposure that undergraduate students have to this concept. We spend a significant amount of time and effort learning these concepts and how they relate to metabolic pathways and disease states (1,2,3). C. Metabolism is the summation of all the chemical processes involved in the production and maintenance of biomolecules within a living system as well as the processes by which chemical energy is made available to the organism. These complex pathways are intimately intertwined through product/reactant relationships as well as more complicated regulatory mechanisms. Chemistry 5700 focuses primarily on the catabolic mechanisms and pathways of metabolism, specifically addressing how humans makes and use energy, what forms of energy can be made and utilized, and what the by-products of such processes are and how the body secretes waste. (1,2,3). 1 Dickenson Chem5700 Assessment Fall 2014 Assessment Method 1. Course evaluations from the students identifying their perception of the importance of the identified IDEA learning objectives in Chemistry 5700. 2. Embedded questions from Biochemistry ACS exam on the final. Designed to quantify the student understanding of material found on national standardized biochemistry exams. 3. Embedded questions on the Final exam designed to assess the content specific learning objectives identified for the course (A, B, C). OUTCOMES 1) 57 students (89%) completed an IDEA evaluation at the end of the Chemistry 5700 course and among many questions, were asked to rank their progress in gaining skills in a series of course learning objectives. They ranked their progress on a scale of 1-5 with “1” and “2” being “no” and “slight” progress, respectively and “4” and “5” being “substantial” and “exceptional” progress, respectively. The responses for each of the three identified objectives were: Gaining factual knowledge (terminology, classifications, methods, trends) 2% of the students selected 1 or 2 81% of the students selected 4 or 5 Learning fundamental principles, generalizations, or theories. 2% of the students selected 1 or 2 77% of the students selected 4 or 5 Learning to apply course materials (to improve rational thinking, problem solving and decisions) 7% of the students selected 1 or 2 71% of the students selected 4 or 5 2) Questions inspired from an ACS biochemistry exam were embedded in the final exam. Test Questions: 5, 6, 41, 43, 47, 49, 50, 69, 71, 73 3) Correlation between content specific learning objectives (A, B, C) and question outcomes on the final exam. Several questions from the final exam were categorized into the three learning objective categories outlined above based on the content knowledge required to answer the question. A) Fundamentals of Biochemistry (The building blocks of life). Test Questions: 40, 42, 44, 45, 46, 48, 64, 65, 66, 75 B) Enzyme structure, function, and regulation. Test Questions: 2, 7, 10, 14, 19, 31, 34, 39, 53, 72 C) Metabolism, focusing on catabolism. Test Questions: 3, 5, 6, 18, 21, 28, 32, 35, 41, 43 2 Dickenson Chem5700 Assessment Fall 2014 100 80 60 40 20 0 Assessment of ACS Embedded Questions Percent Correct Percent Correct Class exam results. 62 students took the Chemistry 5700 final exam in the Fall of 2014 and the results for each of the categorized questions is displayed below. 100 80 60 40 20 0 100 80 60 40 20 0 40 42 44 45 46 48 64 65 66 75 Final Exam Question Number Assessment of Learning Objective B Percent Correct Percent Correct 5 6 41 43 47 49 50 69 71 73 Final Exam Question Number Assessment of Learning Objective A 2 7 10 14 19 31 34 39 53 72 Final Exam Question Number 100 80 60 40 20 0 Assessment of Learning Objective C 3 5 6 18 21 28 32 35 41 43 Final Exam Question Number Summary of Results: Final exam questions were identified that specifically addressed learning objectives for the course and the responses for these questions were evaluated for the 62 students who took the multiple choice final exam (see graphs above). The embedded test questions on the final exam had the following categorical correct response rates: o Embedded ACS Exam questions – 78% Correct o Fundamentals of Biochemistry (Learning Objective A) – 72% Correct o Enzyme structure, function, and regulation (Learning Objective B) – 66% Correct o Metabolism, focusing on catabolism (learning Objective C) – 63% Correct 3 Dickenson Chem5700 Assessment Fall 2014 The IDEA evaluation responses from the students identified the course material as being difficult (4.6/5.0), with the overall quality of the course rated 3.8/5.0 and the overall quality of the instructor 4.8/5.0. o Critiques of the course/instructor were that the tests were too long and difficult. It was also recommended that the lectures should be recorded so that they can be viewed as a means of studying. I will address both issues next year with shorter exams and recorded lectures. Changes made to improve the course in response to the collected data sets. In response to the embedded final exam questions: o The average correct response rate was 78%, suggests that the students are learning the material that they will encounter on standardized biochemistry exams. The exam averages were consistently lower than this, suggesting that the exam questions were more difficult than the embedded ACS exam questions, but I believe that is what has helped to prepare them for these standardized exam questions. o The correct response rate to questions regarding fundamentals of biochemistry (A) was 72% and increased significantly from 63% last year. Since last year, I have made a conscious effort to relate the new and more complicated topics to the fundamentals covered early in the semester. It seemed to help and I will continue with this technique. o Enzymes and enzyme catalyzed reactions (B) are extremely important concepts in biochemistry and are covered throughout the semester and are the focus of an entire chapter. The correct response rate for these questions was 66%, down from 76% last year. Last year I consciously slowed down the lecture pace for these topics and provided many in class examples of important concepts. I will take care next year to do so again and ensure that adequate time is allotted to these difficult topics. o Understanding key metabolic pathways (primarily catabolic) is one of the primary objectives of Chem5700 and is covered in the final 1/3 of the course. The correct response rate for these embedded questions was 63%, improved by 9% over last year. I adapted this year’s lecture material to describe metabolism in greater detail while covering slightly less material. I also included in-class problem sets and take home assignments that required the students to actively think through the causeeffect relationships. I will continue to evaluate the effect of covering less material more thoroughly, without removing necessary concepts. Results for the 2015 course will be collected similarly as they were above and these implemented changes will be evaluated for their effectiveness. Additional methods for improving the course will be considered based on the updated data set and will be implemented the following year. 4