Using Online Homework in Physical Chemistry Courses C. M. Woodbridge This paper was completed and submitted in partial fulfillment of the Master Teacher Program, a 2-year faculty professional development program conducted by the Center for Teaching Excellence, United States Military Academy, West Point, NY, 2009. Introduction Physical Chemistry. Those two words strike, if not terror, than at least knee-knocking fear into the hearts of most undergraduate chemistry majors. To say that Physical Chemistry is not a popular course is a gross understatement. Although, PChem it is not the capstone course in a chemistry major, it is reasonably close. This course requires intimate knowledge of a significant amount of material: Calculus, Physics, General Chemistry, Organic Chemistry, and Analytical Chemistry. This is probably the first course in the chemistry major that requires the synthesis of knowledge from a wide range of topics. Most undergraduate physical chemistry courses are a one-year survey involving four primary top ics: Thermodynamics, Kinetics, Quantum M echanics, and Statistical M echanics. While these are all rather ambitious subjects on their own, many concepts in PChem involve expanding on ideas learned in introductory chemistry. For example, in General Chemistry, students are usually required to memorize or recognize the equation used to calculate the enthalpy of reaction under standard conditions: ∆ rxn H o = ∑n ∆ i products f H io − ∑n ∆ i reac tan ts f H io <1> Comment [C1]: When they get to PChem, we explain where the equation comes from (and frequently derive it from first princip les, e.g., dH = dU + PdV + VdP for a system which only undergoes PV work 1) with the idea that students will no longer have to simply memorize this equation. Why then, does PChem inspire such terror into the heart of undergraduate and graduate students alike? There have been more studies about factors which influence student success in General Chemistry courses.2-4 Some factors which have been studied previously are math ability, background in chemistry , and, surp risingly, student attitudes. These studies5 “demonstrate that p erformance in general chemistry improves with increased formal reasoning and processing, better mathematics skills, more previous chemistry courses, better self-rating and attitudes about the course, and higher scores on placement exams.” Factors contributing to student difficulties in general chemistry 3 (as identified by the instruction staff) are presented in Figure 1. Since, in my opinion, PChem is an exp ansion on topics students encountered previously in general chemistry , I would expect that similar factors would influence success in a physical chemistry course. The only excep tion in Figure 1 would be “cultural shock” since, presumably, juniorlevel students are no longer new to the university system. Although there are relatively few students about factors influencing success in a physical chemistry courses, these few studies find a correlation between success and mathematical background.2, 6-9 One study 9 has presented perceptions of students and faculty on what causes learnin g difficulties. These perceptions are shown in Figure 2 (student perceptions) and Figure 3 (faculty p ercep tions). There seems to be good overlap between the concerns of both faculty and students. Overall, I agree with the many of the students’ views shown in Figure 2. Although PChem was a course I looked forward to in college, based on my undergraduate experience, the 2 course was based heavily on deriving equations with no accompanying discussion about the concep ts and we used handouts our instructor wrote so we didn’t have a text. I made few connections between the course material and everyday life and if there were connections between the lecture and the labs, I frequently missed them. As an instructor (TA or professor), I made a conscious effort to try and avoid perpetuating these negative experiences in my courses. 3 Learning Difficulties Staff Course Students Power relationships Cohort Number Motivation Fragmentation Study Skills Difficult Concepts Preparation Content Overload Cultural shock General Ability Prior learning Conceptions Methods Prior experience Math Lab Vocabulary Figure 1: Perceptions of student difficulties in general chemistry by the instruction staff. Adapted from Ref. 3. 4 Learning Difficulties Student Course Staff Concern re: concept usability (11%) Course is cumulative (19%) Teacher-centered instruction (44%) No motivation and interest (37%) Abstract concepts (52%) No teaching strategy (11%) Uncertainty about concepts (15%) Overload of course content (41%) Inadequate examples in lectures (26%) Course is too mathematical (33%) Inadequately prepared lectures (26%) Inconsistency in exams/lecture/lab (37%) No deep understanding (37%) Lack of everyday applicability (26%) Lack of resources (22%) Figure 2: Students’ perceptions of their difficulties in PChem. Percentages reported come from students’ responses on survey questions. Adapted from Ref. 9. I also agree with the majority of the issues identified by the faculty presented in Figure 3.9 One of the most frustrating obstacles to overcome is the lack of student motivation. Since PChem is not usually anyone’s favorite class, motivational issues are significant. Motivational issues are comp licated by the broad range of abilities of the students in a class. For example, I had a class of 7 students at Hillsdale College. The “bottom” of my class was a group of two students who scored B/B- marks on the examinations. In this class, there were some real sup erstars. It was very hard for these students to be earning Bs while the rest of their classmates 5 were earnin g As and they felt dumb. Designin g examinations and problem sets that would challen ge the top of the class without making the “bottom” of the class feel dumb or overlooked was a real ch allen ge. Learning Difficulties Student Staff Course Successful students inhibit motivation Lack of time and support Abstract concepts Lack of motivation Lack of professional development Exams promote memorization Tricks rather than understanding Overload of teaching work Teacher-centered teaching Socio-economic conditions Too many different things to do Overload of course content Differences in background Lack of resources Overlooked students No tutorials or tutors Overcrowded classes Figure 3: Faculty p erceptions of student difficulties in PChem. Adapted from Ref. 9. Both students and faculty mention the lack of resources available. While there are many PChem books to choose from, there are relatively few lab manuals (two I know of and a third that is out of p rint). Very few books come with a “course packet” that can be loaded in course management software like Blackboard but most come with an accompanying solutions manual for students and instructors. M any institutions use Atkins’ text and this comes with the most supporting resources: solutions manual (instructor and student versions), an e-book, copies of the 6 figures/tables in the text for instructors to use in lectures, and the ARIS homework system contains questions from Atkins. If you choose another book (e.g., M cQuarrie and Simon), all the figures from the text are available from the publisher to be downloaded. This is in stark contrast to the volumes of resources available for a General Chemistry text. For example, Silberberg (p ublished by M cGraw Hill) comes with a test bank, a package that can be uploaded to Blackboard, a student study guide, a student solutions manual, an instructor’s solutions manual, online homework, CDs containing animations, pre-made PowerPoint slides, downloadable figures and tables from the text, and the list just goes on. Can students and instructors manage without all these resources? Certainly. However, it would be nice to have some additional resources (besides another physical ch emistry book) to direct students to when they need help. Students also look for tricks to solve p roblems and tricks to aid in memorization of material rather than looking for a deep understanding of the material. M emorization and tricks may have worked in previous chemistry courses and they may have even gotten reasonable grades using these methods, but in PChem these methods typically stop working. Students need to have a strong understanding of the concepts as well as be able to solve problems; since there are many equations presented in a given chapter of a physical chemistry textbook, the student has to determine which one is the correct equation to use in a particular situation. Without an understanding of the underly ing concepts, students are going to consistently make silly mistakes like try ing to app ly the ideal gas law to a system in which one is asked to calculate the temp erature of ice. In other words, students have to not only know the equation but also the restrictions that come along with the equation; without this additional knowledge, all the tricks in the world aren’t going to help students. 7 Homework that incorporates both conceptual problems and mathematical problems is, I think, one of the best predictors for success in the course. “Success” may be measured in terms of a score on a graded event or student satisfaction that they are “getting it.” Additionally, I have been using Blackboard to do online homework assignments in CH480, Physical Chemistry for Life Sciences. I chose this method over more traditional pencil and paper methods since students are able to get instant feedback on their work. Granted, the feedback is limited to whether or not the answer is correct, but this is more feedback than traditional pencil-and-paper homework sets get since I allow students an unlimited number of attempts on their homework. This is the third time I have used Blackboard in this manner – p reviously, I used this method in a similar course offered during the summer at the University of Nebraska-Lincoln. Subsequent sections of this p ap er will describe the assessment methods used to determine whether online homework was beneficial in PChem as well as the results obtained for CH480 during the fall of 2008. 8 Assessment Methods The methods described below were used to assess student performance in CH480 and the success of using online homework assignments. The students’ background in general chemistry was considered. There are three possible tracks: 101/102 (regular); 151/152 (advanced); or validation. Which track a student took was comp ared to the students’ course grade in order to assess whether this had an impact on their performance in CH 480. Students’ performance on graded events were ranked as “Excellent” (> 76%), “Satisfactory ” (64-75%), or “Poor” (<64%). Graded events evaluated in this manner included the p re-lesson HW, WPRs, and the TEE. Pre-lesson HW was based largely on students’ knowledge of fundamental concepts from p re-requisite courses. A small percentage of the work (30%) was based on more sophisticated concepts outlined in the reading. WPRs and TEEs contained questions drawn from fundamental concepts as well as 480-level applications of this knowledge. In-class work was also evaluated using p eriodic one-minute papers10 and lingering questions papers (aka “muddiest point".9 Each lesson had an “earn back” opportunity where students wrote a one-minute p ap er, lingerin g question paper, or took a short quiz covering concep ts I felt they hadn’t yet mastered. Student work was ranked as “Excellent,” “Satisfactory,” or “Poor” in each case. Rankings for quizzes were based on the percentages given in the p receding p aragrap h while rankings for the papers were based on the following: Excellent = good question or good summary of the main point at the CH480 level; Satisfactory = good question or good summary of the main p oint at the CH101-102 level; Poor = question or summary indicates they missed the point of the lesson entirely. 9 Blackboard provides a wealth of statistics as well: the number of attempts a student used to comp lete an assignment, what time they began an assignment, and all their responses. While Blackboard doesn’t retain information about how long a student worked on an assignment, it is easy to get a “flavor” of this by comparing start times. 10 Results In the fall of 2008, 30 firsties were enrolled in CH480, Physical Chemistry for Life Sciences. All students were life science majors and most planned to either go to medical school or hoped to branch M edical Service Corps. In this class, 18 students took CH101-102, 6 took CH151-152, and 6 validated the general chemistry sequence. The grade distribution for each of the different tracks is p resented in Table 1 below. None of the students who validated or took advanced general chemistry received a mark below a B, which suggests that students' backgrounds do have an imp act on their success in physical chemistry. Table 1: Number of students receivin g nominally an A, B, or a C in CH480 sorted by the general chemistry sequence they took. A B C CH101-102 4 10 4 CH151-152 4 2 0 Validation 4 2 0 When comp aring students' scores on homework, WPRs and TEEs, scores which remain the same or increase from homework to WPR to TEE would suggest students learned the material or retained what they learned. Homework sets included both calculations and concep tual questions. Students were allowed to submit each homework set an unlimited number of times in Blackboard and they were given some (albeit minimal) feedback which included whether their answers were correct and some generic hints on approaching a correct answer. 11 Table 2: Comparison of course averages from homework to WPR to TEE. The events are colorcoded as follows: green = > 76%; yellow = 64-75%; red = < 64%.. The average for WPRs includes only those students who took all four WPRs. Homework WPRs Tee MC TEE Calc TEE Thermodynamics 84.80% 71.83% 64.67% 63.40% 64.03% Kinetics 82.32% 81.43% 58.33% 68.20% 63.27% Quantum 88.62% 85.41% 57.83% 75.50% 66.67% App lications 72.30% 64.85% 69.00% 66.93% TOTAL 86.29% 75.40% 71.65% 79.81% 65.54% Based on the data shown in Table 2, instead of scores increasing or remaining nearly unchanged as time went on, scores went down. The drops were not insignificant either; for example, averages dropp ed at least 10% from the homework to the WPR to the TEE in, for example, Thermodynamics. This trend, in my opinion, has more to do with the available resources for each grad ed exercise than students' knowledge actually decreasing over time. On homework, students are allowed to consult their books, their notes, their classmates, online resources posted in Blackboard, and me. In p rincip le, they have an unlimited amount of time to complete each homework set as well. On WPRs and the TEE, however, the students are under time pressure and are only allowed to use a single, double-sided exam data sheet. This type of trend is also seen in other PChem classes, as shown in Table 3. 12 Table 3 Comp arison between physical chemistry courses taught at Hillsdale College, University of Nebraska-Lincoln, and USM A. # semesters Hillsdale 10 8.2 13.1 86.1 Course Avg. 89.7 UNL 10 29.6 12.4 84.7 79.6 USMA 1 30 40 86.2 84.9 85.7 + 0.8 85 + 5 Average Avg. # students # HW sets HW Avg. Data p resented in Table 3 from Hillsdale College comprises 5 years of a 2-semester Physical Chemistry sequence taken only by Chemistry majors; the course includes a lab component. Data from UNL comp rises 4 semesters of the first semester of a year-long Physical Chemistry sequence taken by Chemistry, Biochemistry, and Chemical Engineering majors with no lab comp onent as well as 6 semesters of a one-semester Physical chemistry course geared towards Biochemistry majors. Neither UNL course has an accompanying lab. The one-semester course focuses p rimarily on Thermodynamics and Kinetics; a small unit (2 lessons) on spectroscopy was included. Courses taught at UNL and Hillsdale were conducted in a traditional lecture format. Interestingly , only the Hillsdale courses show an increase going from the homework to the WPR to the final grade. These students did p aper and pencil homework sets and had one problem set p er chapter. They did far fewer problems than either online course, which seems to suggest that dong more p roblems and including conceptual problems does not seem to improve retention of the material. In addition to comp aring course averages on various graded events, student performance on individual top ics was tracked. Three problems on the TEE came almost verbatim from 13 previous WPRs. Student performance on these problems was compared between similar problems on HW and WPRs. The results are presented in Table 4. Table 4: Percentage of students who earned at least 90% of the points on a graded event for the topic indicated. Percentages correspond to students attempting the problem and not the class as a whole. HW WPR TEE % on TEE pH of adenosine 0 17 17 63.40 Analysis of mechanism 93 13 9 68.20 Particle in a box 79 40 24 66.60 The thermodynamics question, pH of adenosine, improved dramatically between the homework and the WPR and students at retained their knowledge about this problem for the TEE. In contrast, the p ercentage of students who did well on the quantum mechanics question (particle in a box) and the kinetics question (analysis of a mechanism) dropped dramatically as time went on. The kinetics question on the HW has a very high score because of the level of the question asked was more at the 102 level rather than the 480 level. The decrease in demonstrated knowledge seems to occur no matter how the data are analyzed. 14 Discussion The answer to the question about whether online homework improves students' success on physical chemistry seems to be “no” or “very little” based on the data presented here. Why, then, do textbook comp anies continue to sp end money to develop and promote online homework systems ran gin g from simple test banks which can be added to course management systems like Blackboard or sophisticated interactive sessions like those in Aris or WebAssign? Just like athletic p ractice or practicing musical instruments, students need to drill with chemistry problems and test their fundamental knowledge. Online homework seems to me to be a reasonable alternative to the pencil and paper type of homework. The data in Table 5 seems to support this. Table 5: Comp arison of average homework and course scores in a one-semester Physical Chemistry course. # semesters Avg. # Avg. # HW HW Avg. Course Students sets Avg. Pencil & p ap er (UNL) 4 32 9.75 88.5 83.4 Blackboard (UNL) 2 31 22.5 85.6 81.4 Blackboard (USM A) 1 30 40 86.2 84.9 Blackboard Total 3 30.5 31.2 85.9 + 0.4 83.2 + 2 87 + 1 83 + 2 Overall Students who did p encil and p ap er homework had a course average which was 6% below their homework average, which suggests that their exam scores brought their grades down. In contrast, students who did Blackboard homework had course averages which were 2% lower than their homework averages so the negative impact of exams on their overall average was much less. While students’ knowledge of how to solve particle-in-a-box problems seemed to 15 decrease with time in CH480, their course grade didn’t drop very much (~3%) compared to their homework grade. Homework was only 15% of their overall grade and examinations were 55% of their overall grade. At the end of the day, it would seem the CH480 students demonstrated reasonable comp etence with the material in the course overall. So why was there no clear trend of imp rovement from homework to WPR to TEE in CH480? Homework works if students work homework properly . One study 2 shows a correlation between homework scores and success; anecdotal evidence supports this. There seem to be three main problems with homework: 1) copying and wargaming in Blackboard; 2) time management; and 3) student attitudes about PChem. These attitudes prevail whether homework is assessed online or via pencil and paper assignments. Fundamentally, everything comes down to time -- students were not convinced that homework is "physical chemistry practice" just as a daily PT routine is practice for the bi-annual PT tests and therefore students tried to get the most points for the minimum input of time. The analogy between problemsolving and athletic practice seems to be particularly apt at USMA and it is an analogy that has been around for awhile.11 The online homework assignments can be classified into four categories: calculation (C), mostly calculations (MC), mostly short answer (M SA), and short answer (SA). Calculation questions were numerical calculations where students had to enter a number. Short answer questions were either true/false, multiple choice, or fill in the blank. Course averages were 59.4%, 54.1%, 79.3%, and 90.1% for C, M C, M SA, and SA, respectively. Overall, the more calculations p resent in the homework, the more the average dropped. I would have expected M C to have a higher p ercentage than C, but there were 5-6 students who never did calculations on these M C assignments but they had to attempt a calculation to get any points on the C type 16 assignments. This data fits the "most points for the least time" model very well. Additionally, there were 5-6 students who would only submit one attempt on a homework assignment and take whatever score they earned on the assignment whether it was good or bad. There were students in this group who took this to the extreme of not going back and changing incorrect true/false answers. Copying and wargaming (i.e., goin g through all possible answers on true/false and multiple choice questions without much thought) are perceived to be time-saving devices. These behaviors will lead to individual students having high HW scores, lower WPR scores and lower scores on the TEE. Frankly copy ing can be a p roblem whether students do online homework or paper and pencil homework. While I have no p roblem with students collaborating with one another, I do caution them that copying is a poor choice since they will not be able to rely on their classmates during examination conditions. Students did comment that they didn’t like Blackboard because they didn’t know why their answers were wrong. This was p roblematic for most students as they did not start the homework assignments until after 5pm the night before it was due; at that time, I was usually home and unavailable on e-mail or for AI. To be fair, Blackboard is a passive system and doesn’t understand that the answers 0.003, .003, and 3E-3 are the same so the students have a good point. I would go back and give them credit for correct answers that Blackboard marked wrong due to a simple notation issue, I understand why it was hard for them to decide whether their answer was wrong because of notation or their answer had a more serious problem. To try and alleviate these issues, I p ut ranges on numerical answers in order to eliminate rounding errors and (most) issues with significant figures. I also tried to put cues in the text of the problem about the typ e of answer Blackboard was expecting (e.g., use scientific notation, yes/no 17 etc.). I was also very proactive about checking work as it was submitted and contacting students to let them know that their answer was wrong due to a rounding error or a mathematical error or a more serious error. Some students felt that was sufficient help but others disagreed. And yet, there were some students who thought Blackboard was OK (from HW #40 – PChem Top 5 i.e., 5 best things about PChem): • HW every lesson kinda makes you read the material • Although I initially was against use of blackboard for homework assignment, I am now a fan of the hw assignments for they prepare you for the WPRs and are a good test of reading material. • BB homework And some advice to future CH480 students (from HW #36 – Letter to the future): • Do the suggested p roblems for sure along with the assigned homeowrk [sic]…without it, you will be unsuccessful. • Also, use Blackboard to your advantage – not just for homework credit, but also as a learnin g aid …. Stay up on the homework even if you do not understand because the problems will come in handy when study ing for the big tests. • The best advice I can give to a p -chem student (under Dr. Woodbridge) is to do the homework assignments comp letely every night and try to figure out the equations BEFORE going to class. Also, read the discussion boards and the Lesson slides – they help a lot with the homework, often where Tinoco is incomprehensible. Knowing the homework problems sets y ou up for success on the problem-solving portions of the WPRs. Desp ite some students’ dislike of the online homework and some well-thought out constructive criticisms on their part, I would use it again. The usage statistics are invaluable as they allow me to determine whether a given assign ment was too long or too hard or confusing. I can tell approximately how long students sp end on a given assignment as well as how they are approaching about the assignment. Prior to the lesson, I can review the assignment summary and determine which p roblems they struggled with on and plan for this accordingly, which makes for a more student-centered lesson. I do think the students who were focused on learning the material did benefit from doing more p roblems than they would have done using paper and 18 pencil homework assignments. The students who copied other students or wargamed their way through the short answer questions are, I think, the same students who would have copied other students papers or turned in p artially comp lete submissions if we used pencil and paper homework assignments. It is my opinion that the students looking for shortcuts or to minimize their time have the wrong attitude about PChem – they are afraid of the course, not interested in the course, they lack confidence that they can do well in this difficult course, etc. One paper7 notes that faculty expectations and students’ perceptions seem to be at odds; that certainly seems to be the case here since I think online homework is great and useful while the average student disagreed. A search of the keyword “Physical Chemistry ” on the Journal of Chemical Education website returns 1492 articles from the first issue until the April 2009 issue. Of these articles, the majority are writeups of good lab exercises, good class exercises, or how best to communicate a difficult concept in PChem. There are exp lanations of why premedical students should be required to take p hysical chemistry and what content is appropriate for them12 as well as course structure and content.13 Ironically, the course structure described in reference 13 hasn’t changed much in the last 40 years. There are articles about how faculty should approach PChem so that students get the most out of the course; these approaches include Keller-type systems.14-16 Most recently , there has been a great deal of attention drawn to using M athematica17 to solve/approach physical ch emistry problems and using Process-Oriented Guided Inquiry Workshops.18-19 We the faculty are still struggling to make PChem palatable to students and yet hold them to a high standard. 19 Conclusions The use of online homework in CH480 met with mixed success. The students were not necessarily big fans of online homework but I thought it made the course more student-centered and provided a wealth of data about how students were approaching the homework. Performance on graded ev ents decreased goin g from homework to WPR to the TEE rather than increasin g but y et the drop between total homework averages and the course average is the smallest in courses like CH480 where online homework is used. Whether this drop can be attributed to the nature of the graded event (homework is unlimited resources and time with many different questions vs. examin ations have limited resources and time and often only one typ e of question on a top ic) or student attitudes toward PChem in general is unclear. Students were satisfied with the course overall. Next year, I do plan to use online homework again. I do plan to try and address the student concerns with regard to Blackboard as best I can and I plan to modify the homework so that each homework set contains questions about fundamental concepts students should have retained form p revious courses (~40%), questions about PChem-level concepts and skills students should acquire from the reading (~20%) and questions from the previous lesson which will allow them additional p ractice on key calculations. 20 References: 1. Alberty , R.A.; Silbey, R.J. Physical Chemistry, 2 nd ed.; John Wiley & Sons Inc.: New York, 1997. pp 55 – 57. 2. Hahn, K.E.; Polik, W.F. Factors Influencing Success in Physical Chemistry J. Chem. Ed. 2004 81, 567-572. 3. Kirkwood, V.; Sy mington, D. Lecturer Perceptions of Student Difficulties in a First-Year Chemistry Course. J. Chem. Ed. 1996 73, 339-343. 4. Carter, C.S.; Brickhouse N.W. What M akes Chemistry Difficult? J. Chem. Ed. 1989 66, 223-225. 5. pg. 567 in Reference 2. 6. Porile, N.T. Diagnostic Quiz to Identify Failing Students in Physical Chemistry. J Chem. Ed. 1976 53, 109-110. 7. Nicoll, G.; Francisco J.S. An Investigation of the Factors Influencing Student Performance in Physical Chemistry. J Chem Ed. 2001 78, 99-102. 8. Derrick. M. E.; Derrick F. W. Predictors of Success in Physical Chemistry. J. Chem. Ed. 2002 79, 1013-1016. 9. Sozbilir, M. What M akes Physical Chemistry Difficult? J. Chem. Ed. 2004 81, 573-578. 10. Angelo, T.A.; Cross, K.P. Classroom Assessment Techniques: A Handbook for College Teachers, 2 nd ed.; Jossey -Bass: San Francisco, 1993. 11. Sherrill, M .S. The problem method of teaching physical chemistry. J. Chem. Ed. 1929 6, 260-262 12. Randall, D.L. Premedical Physical Chemistry. J. Chem. Ed. 1932 9, 1096. 13. Amdur, I. Teaching Physical Chemistry at the M assachusetts Institute of Technology. J. Chem. Ed. 1965 42, 191 – 193. 14. Davis, L.P.; Storch, D.M.; Strawser, L.D. Physical Chemistry at USAFA: Personalized Instruction. J. Chem. Ed. 1987 64, 784-787. 15. Bent, H.A. No Easy Way: Experience with a modified-Keller physical chemistry course. J. Chem. Ed. 1974 51, 661- 664. 16. Bent, H.A. You Can’t Win. J. Chem. Ed. 1975 52, 448-450. 17. For example, Zielinski, T.J. Critical Thinking in Chemistry Using Symbolic M athematics Documents. J. Chem. Ed. 2004 81, 1533-1534. 21 18. For examp le, Hinde R.J.; Kovac, J. Student Active Learning in Physical Chemistry. J. Chem. Ed. 2001 78, 93-99. 19. Process Oriented Guided Inquiry Learning (POGIL) homepage. http://new.pogil.org/ (accessed M arch, 2009). 22