ABSTRACT
EFFICACY OF PRE-LABORATORY WORKSHEETS VERSUS
QUIZZES UTILIZING CLICKER TECHNOLOGY IN A
GENERAL CHEMISTRY COURSE
General Chemistry is a required collegiate course for many Science,
Technology, Engineering, and Mathematics (STEM) majors, yet 30-40% of
students enrolled at California State University, Fresno do not pass the course,
significantly impacting the pipeline of STEM graduates. While the laboratory
component of the class is not the only reason students fail the course, the increased
one-on-one instructional opportunities in the laboratory make it a reasonable target
for improving student success in the course. One of the current General Chemistry
laboratory requirements at California State University, Fresno is that students
complete pre-laboratory worksheets covering the experiment that they will be
performing. Problems of targeted reading and cheating limit the effectiveness of
this model. To address these issues and to emphasize the importance of prelaboratory preparation, 5-minute quizzes were developed for each experiment in
the course. The quizzes were administered using i>clicker technology allowing for
immediate student feedback. i>clicker quizzes yielded superior results to prelaboratory worksheets for the following factors: student preference, student
efficacy, and required administration and grading time. There were few instances,
however, of significant differences in student test performance between the two
pre-laboratory assessment tools. In future studies, improved metrics or larger
sample sizes may reveal differences.
Elizabeth Buchnoff
December 2012
EFFICACY OF PRE-LABORATORY WORKSHEETS VERSUS
QUIZZES UTILIZING CLICKER TECHNOLOGY IN A
GENERAL CHEMISTRY COURSE
by
Elizabeth Buchnoff
A thesis
submitted in partial
fulfillment of the requirements for the degree of
Master of Science in Chemistry
in the College of Science and Mathematics
California State University, Fresno
December 2012
APPROVED
For the Department of Chemistry:
We, the undersigned, certify that the thesis of the following student
meets the required standards of scholarship, format, and style of the
university and the student's graduate degree program for the
awarding of the master's degree.
Elizabeth Buchnoff
Thesis Author
Eric Person (Chair)
Chemistry
David Frank
Chemistry
Carol Fry Bohlin
Curriculum and Instruction
For the University Graduate Committee:
Dean, Division of Graduate Studies
AUTHORIZATION FOR REPRODUCTION
OF MASTER’S THESIS
X
I grant permission for the reproduction of this thesis in part or in
its entirety without further authorization from me, on the
condition that the person or agency requesting reproduction
absorbs the cost and provides proper acknowledgment of
authorship.
Permission to reproduce this thesis in part or in its entirety must
be obtained from me.
Signature of thesis author:
ACKNOWLEDGMENTS
This was a path I never thought I would have traveled. Opportunities tend
to fall into your lap and take you in a direction to not only better yourself, but to
help you better the lives of others. I can’t explain how much patience, persistence,
and information I’ve learned during my journey through the Master’s Program
AND Single Subject Credential Program, as well as surviving wedding planning.
First and foremost, I thank God every day for giving me this amazing
opportunity as well as the patience and will power to get through the days I
wanted to give up. Second, I would like to thank my husband, Alex Buchnoff, for
pushing me to keep typing when I didn’t want to, and encouraging me with his
famous line, “You’ll be fine. God won’t give you more than you can handle.”
Knowing how encouragement pushes me along in my studies, he picked up where
my parents left off when I moved to Fresno.
I would also like to thank my parents, Jay and Manya Michicoff, for
making me the strong-willed, determined, and God-fearing woman I am today.
Without their help, I wouldn’t have made it through my undergraduate work or my
graduate work. I also must thank my in-laws, Steven and Sasanna Buchnoff, for
putting a roof over my head to help me get in my typing mood. My friends have
also been a great source of encouragement. Thank you Laura Samarin for being
there to help me unwind during my weekend trips to LA!
I must express a great deal of gratitude to my thesis chair, Dr. Eric Person,
for being the only faculty member in the Chemistry Department available for
continuing my interest in researching chemical education. His education research
ideas never sounded too farfetched, and I am glad I was able to put one of them
v
into motion. It has been a busy two years, but I am grateful for the time I was able
to squeeze in to see if I was on the right track with my project.
I would also like to thank the other members of my thesis committee, Dr.
David Frank and Dr. Carol Fry Bohlin, for being available for me to bounce
procedure ideas or article topics to continue on with this project. In addition, I
would like to thank Dr. David Tanner for his help with the statistical aspect of this
project, as I was completely unfamiliar with running these statistical analyses.
Thanks also to the Craig School of Business front office staff members (one a
former Chem 1A student of mine) for allowing me to use their computers when I
couldn’t find any others with the software I needed during the summer.
Special thanks also goes out to Kelsey Correia and Akiteru Ikeda for their
help in developing quiz questions, collecting data, and dealing with me bugging
them each week for their pre-lab worksheets and lab scores. At the same time, I
must also thank each and every student that was involved with the study. Without
the cooperation (for the most part) of the students and the help of the TAs, this
study would never have happened.
Last but not least, I’d like to thank my teachers and professors that gave me
ideas or something to strive for. It is people like them who help encourage their
students that they can be successful in life and do anything they want if they put
their mind to it. My high school chemistry teacher, Aaron Sams, helped begin my
journey to become a crazy chemistry teacher in 2003, and I haven’t looked back
yet. Thanks, Mr. Sams!
TABLE OF CONTENTS
Page
LIST OF TABLES .................................................................................................. ix
LIST OF FIGURES ............................................................................................... xiv
INTRODUCTION AND LITERATURE REVIEW ................................................ 1
Frequent Quizzing ............................................................................................. 5
Clickers in the Classroom ................................................................................. 7
Laboratory Structure ....................................................................................... 10
Pre-Laboratory Exercises ................................................................................ 12
Scaffolding ...................................................................................................... 14
Project Aims .................................................................................................... 15
METHODOLOGY ................................................................................................. 17
Participants ...................................................................................................... 17
Assessment Tools ............................................................................................ 19
Conditions ....................................................................................................... 23
Dependent Variables ....................................................................................... 25
Data Analysis .................................................................................................. 26
RESULTS ............................................................................................................... 28
Impact of Pre-Laboratory on Student Performance ........................................ 28
Impact of Teaching Assistant Introduction Style on Student Performance .... 33
Impact of Laboratory Meeting Time on Student Performance ....................... 38
Teaching Assistant Observations .................................................................... 38
Student Observations ...................................................................................... 41
CONCLUSION ...................................................................................................... 43
Discussion ....................................................................................................... 43
vii
Page
Limitations ...................................................................................................... 49
Future Studies.................................................................................................. 51
Implications ..................................................................................................... 51
REFERENCES ....................................................................................................... 53
APPENDICES ........................................................................................................ 58
APPENDIX A: PRE-LABORATORY WORKSHEETS ...................................... 59
APPENDIX B: CLICKER QUESTIONS USED FOR EACH PRELABORATORY QUIZ ............................................................................... 72
Experiment 1: Melting Points and Mixtures ................................................... 73
Experiment 2: Mass and Volume Measurement ............................................. 74
Experiment 3: Percent Water in Hydrate ........................................................ 75
Experiment 4: Separating a Mixture, Recrystallization .................................. 76
Experiment 5: Net Ionic Equations ................................................................. 77
Experiment 6: Determining the Empirical Formula of a Compound ............. 78
Experiment 8: Alum from Scrap Aluminum................................................... 79
Experiment 9: Gasometric Analysis of Peroxide Solution ............................. 80
Experiment 12: Determining the Heat of a Reaction ...................................... 81
Experiment 13: Practice Practical ................................................................... 82
Experiment 15: Analysis for Iron in a Vitamin Pill ........................................ 83
APPENDIX C: STUDENT SURVEY ................................................................... 84
APPENDIX D: TEACHING ASSISTANT SURVEY .......................................... 86
APPENDIX E: QUESTIONS USED FOR PRE- AND POST-TEST ................... 88
APPENDIX F: MANOVA RESULTS FROM ANALYSIS OF COMPLETE
DATA SET .................................................................................................. 96
APPENDIX G: MANOVA RESULTS FROM ANALYSIS OF
DATA SET B ............................................................................................ 103
viii
Page
APPENDIX H: MANOVA RESULTS FROM ANALYSIS OF
DATA SETS A AND C ............................................................................ 110
APPENDIX I: MANOVA RESULTS FROM ANALYSIS OF
DATA SET A ............................................................................................ 117
LIST OF TABLES
Page
Table 1. Weighted Scores of Each Laboratory Grade Component in Chem 1A
Course ........................................................................................................ 1
Table 2. Frequencies and Percentages of Participant Demographic Information
(N = 135) .................................................................................................. 18
Table 3. Rubric for Grade Distribution for Grading Pre-Laboratory Questions .... 20
Table 4. Structure of Pre-Laboratory Evaluations for the Sections Used .............. 23
Table 5. Means and Standard Deviations for Pre-Laboratory Score, Percent
Error, and Laboratory Score by Pre-Laboratory Assessment for Each
Experiment ............................................................................................... 29
Table 6. Means and Standard Deviations for Pre-Laboratory Score, Percent
Error, and Laboratory Score by Pre-Laboratory Assessment for Each
Experiment Using Data Set B .................................................................. 31
Table 7. Paired Differences Between Pre-Test and Post-Test for Students
Assigned Pre-Laboratory Worksheets ..................................................... 32
Table 8. Paired Differences Between Pre-Test and Post-Test for Students
Assigned Pre-Laboratory Quizzes ........................................................... 32
Table 9. Means and Standard Deviations for Pre-Laboratory Score, Percent
Error, and Laboratory Score by Teaching Assistant for Each
Experiment ............................................................................................... 35
Table 10. Means and Standard Deviations for Pre-Laboratory Score, Percent
Error, and Laboratory Score by Teaching Assistant for Each
Experiment Using Data Sets A and C ...................................................... 37
Table 11. Means and Standard Deviations for Pre-Laboratory Score, Percent
Error, and Laboratory Score by Time for Each Experiment Using
Data Set A ................................................................................................ 39
Table 12. Student Responses To Survey Regarding Pre-Laboratory
Assessments (N = 38)............................................................................... 42
Table 13. Summary for the MANOVA Results for Pre-Laboratory Score,
Percent Error, and Laboratory Score by Pre-Laboratory Assessment
and Instructor for Experiment 1 ............................................................... 97
x
Page
Table 14. Summary for the MANOVA Results for Pre-Laboratory Score,
Percent Error, and Laboratory Score by Pre-Laboratory Assessment
and Instructor for Experiment 2 ............................................................... 97
Table 15. Summary for the MANOVA Results for Pre-Laboratory Score,
Percent Error, and Laboratory Score by Pre-Laboratory Assessment
and Instructor for Experiment 3 ............................................................... 98
Table 16. Summary for the MANOVA Results for Pre-Laboratory Score,
Percent Error, and Laboratory Score by Pre-Laboratory Assessment
and Instructor for Experiment 4 ............................................................... 98
Table 17. Summary for the MANOVA Results for Pre-Laboratory Score,
Percent Error, and Laboratory Score by Pre-Laboratory Assessment
and Instructor for Experiment 5 ............................................................... 99
Table 18. Summary for the MANOVA Results for Pre-Laboratory Score,
Percent Error, and Laboratory Score by Pre-Laboratory Assessment
and Instructor for Experiment 6 ............................................................... 99
Table 19. Summary for the MANOVA Results for Pre-Laboratory Score,
Percent Error, and Laboratory Score by Pre-Laboratory Assessment
and Instructor for Experiment 8 ............................................................. 100
Table 20. Summary for the MANOVA Results for Pre-Laboratory Score,
Percent Error, and Laboratory Score by Pre-Laboratory Assessment
and Instructor for Experiment 9 ............................................................. 100
Table 21. Summary for the MANOVA Results for Pre-Laboratory Score,
Percent Error, and Laboratory Score by Pre-Laboratory Assessment
and Instructor for Experiment 12 ........................................................... 101
Table 22. Summary for the MANOVA Results for Pre-Laboratory Score,
Percent Error, and Laboratory Score by Pre-Laboratory Assessment
and Instructor for Experiment 13 ........................................................... 101
Table 23. Summary for the MANOVA Results for Pre-Laboratory Score,
Percent Error, and Laboratory Score by Pre-Laboratory Assessment
and Instructor for Experiment 15 ........................................................... 102
Table 24. Summary for the MANOVA Results for Pre-Laboratory Score,
Percent Error, and Laboratory Score by Pre-Laboratory Assessment
for Experiment 1 .................................................................................... 104
Table 25. Summary for the MANOVA Results for Pre-Laboratory Score,
Percent Error, and Laboratory Score by Pre-Laboratory Assessment
for Experiment 2 .................................................................................... 104
xi
Page
Table 26. Summary for the MANOVA Results for Pre-Laboratory Score,
Percent Error, and Laboratory Score by Pre-Laboratory Assessment
for Experiment 3 .................................................................................... 105
Table 27. Summary for the MANOVA Results for Pre-Laboratory Score,
Percent Error, and Laboratory Score by Pre-Laboratory Assessment
for Experiment 4 .................................................................................... 105
Table 28. Summary for the MANOVA Results for Pre-Laboratory Score,
Percent Error, and Laboratory Score by Pre-Laboratory Assessment
for Experiment 5 .................................................................................... 106
Table 29. Summary for the MANOVA Results for Pre-Laboratory Score,
Percent Error, and Laboratory Score by Pre-Laboratory Assessment
for Experiment 6 .................................................................................... 106
Table 30. Summary for the MANOVA Results for Pre-Laboratory Score,
Percent Error, and Laboratory Score by Pre-Laboratory Assessment
for Experiment 8 .................................................................................... 107
Table 31.Summary for the MANOVA Results for Pre-Laboratory Score,
Percent Error, and Laboratory Score by Pre-Laboratory Assessment
for Experiment 9 .................................................................................... 107
Table 32. Summary for the MANOVA Results for Pre-Laboratory Score,
Percent Error, and Laboratory Score by Pre-Laboratory Assessment
for Experiment 12 .................................................................................. 108
Table 33. Summary for the MANOVA Results for Pre-Laboratory Score,
Percent Error, and Laboratory Score by Pre-Laboratory Assessment
for Experiment 13 .................................................................................. 108
Table 34. Summary for the MANOVA Results for Pre-Laboratory Score,
Percent Error, and Laboratory Score by Pre-Laboratory Assessment
for Experiment 15 .................................................................................. 109
Table 35. Summary for the MANOVA Results for Pre-Laboratory Score,
Percent Error, and Laboratory Score by Pre-Laboratory Assessment
and Instructor for Experiment 1 ............................................................. 111
Table 36. Summary for the MANOVA Results for Pre-Laboratory Score,
Percent Error, and Laboratory Score by Pre-Laboratory Assessment
and Instructor for Experiment 2 ............................................................. 111
Table 37. Summary for the MANOVA Results for Pre-Laboratory Score,
Percent Error, and Laboratory Score by Pre-Laboratory Assessment
and Instructor for Experiment 3 ............................................................. 112
xii
Page
Table 38. Summary for the MANOVA Results for Pre-Laboratory Score,
Percent Error, and Laboratory Score by Pre-Laboratory Assessment
and Instructor for Experiment 4 ............................................................. 112
Table 39. Summary for the MANOVA Results for Pre-Laboratory Score,
Percent Error, and Laboratory Score by Pre-Laboratory Assessment
and Instructor for Experiment 5 ............................................................. 113
Table 40. Summary for the MANOVA Results for Pre-Laboratory Score,
Percent Error, and Laboratory Score by Pre-Laboratory Assessment
and Instructor for Experiment 6 ............................................................. 113
Table 41. Summary for the MANOVA Results for Pre-Laboratory Score,
Percent Error, and Laboratory Score by Pre-Laboratory Assessment
and Instructor for Experiment 8 ............................................................. 114
Table 42. Summary for the MANOVA Results for Pre-Laboratory Score,
Percent Error, and Laboratory Score by Pre-Laboratory Assessment
and Instructor for Experiment 9 ............................................................. 114
Table 43. Summary for the MANOVA Results for Pre-Laboratory Score,
Percent Error, and Laboratory Score by Pre-Laboratory Assessment
and Instructor for Experiment 12 ........................................................... 115
Table 44. Summary for the MANOVA Results for Pre-Laboratory Score,
Percent Error, and Laboratory Score by Pre-Laboratory Assessment
and Instructor for Experiment 13 ........................................................... 115
Table 45. Summary for the MANOVA Results for Pre-Laboratory Score,
Percent Error, and Laboratory Score by Pre-Laboratory Assessment
and Instructor for Experiment 15 ........................................................... 116
Table 46. Summary for the MANOVA Results for Pre-Laboratory Score,
Percent Error, and Laboratory Score by Time for Experiment 1........... 118
Table 47. Summary for the MANOVA Results for Pre-Laboratory Score,
Percent Error, and Laboratory Score by Time for Experiment 2........... 118
Table 48. Summary for the MANOVA Results for Pre-Laboratory Score,
Percent Error, and Laboratory Score by Time for Experiment 3........... 119
Table 49. Summary for the MANOVA Results for Pre-Laboratory Score,
Percent Error, and Laboratory Score by Time for Experiment 4........... 119
Table 50. Summary for the MANOVA Results for Pre-Laboratory Score,
Percent Error, and Laboratory Score by Time for Experiment 5........... 119
Table 51. Summary for the MANOVA Results for Pre-Laboratory Score,
Percent Error, and Laboratory Score by Time for Experiment 6........... 120
xiii
Page
Table 52. Summary for the MANOVA Results for Pre-Laboratory Score,
Percent Error, and Laboratory Score by Time for Experiment 8........... 120
Table 53. Summary for the MANOVA Results for Pre-Laboratory Score,
Percent Error, and Laboratory Score by Time for Experiment 9........... 121
Table 54. Summary for the MANOVA Results for Pre-Laboratory Score,
Percent Error, and Laboratory Score by Time for Experiment 12......... 121
Table 55. Summary for the MANOVA Results for Pre-Laboratory Score,
Percent Error, and Laboratory Score by Time for Experiment 13......... 122
Table 56. Summary for the MANOVA Results for Pre-Laboratory Score,
Percent Error, and Laboratory Score by Time for Experiment 15......... 122
LIST OF FIGURES
Page
Figure 1. A screen shot of a clicker quiz question as a student will view on a
monitor in the classroom........................................................................... 8
Figure 2. Pre-test and post-test data comparing students who completed the
pre-laboratory worksheet to those who completed the pre-laboratory
quiz. ......................................................................................................... 33
INTRODUCTION AND LITERATURE REVIEW
Students who take the general chemistry course, CHEM 1A, at California
State University, Fresno (“Fresno State”) are required to register for a lecture
section as well as a laboratory section that are collectively evaluated with one
course grade. The lecture section is typically held in a large lecture hall and is
comprised of students from several laboratory sections. The laboratory portion is
smaller, ranging from 20-25 students per section. This portion is composed of two
types of activities where students perform experiments one day and participate in a
recitation section the next. While the lecture is usually taught by an experienced
instructor, the laboratories are taught primarily by chemistry graduate students,
many of whom do not have prior teaching experience and may have not covered
introductory material in 5 or more years prior to their teaching assignments. These
teaching assistants send a lab grade to the lecture instructors at the end of the
semester for inclusion in the overall course grade. The laboratory grade has four
components: laboratory practical, instructor evaluation, experiments and study
guides, and quizzes (Table 1).
Table 1. Weighted Scores of Each Laboratory Grade
Component in Chem 1A Course
Laboratory Practical
Tested on ability to use concepts learned
throughout the semester
Instructor Evaluation
Neatness, safety, participation,
punctuality, and efficiency of student
Experiments and study guides
Experimental write-ups, study guides,
activities, pre-lab worksheets
Quiz
Based on experimental technique,
concepts, problem solving skills students
used in experiments, study guides, and
activities
20%
10%
20%
50%
2
CHEM 1A is considered a gateway course for Science, Technology,
Engineering, and Math (STEM) majors. It is a degree requirement for students
majoring in chemistry, biology, earth and environmental science, physics, natural
science, food science and nutrition, environmental and occupational health,
kinesiology exercise science, enology, mechanical engineering, civil and
geomatics engineering, animal sciences, and physical therapy. Students from other
majors may also choose to take CHEM 1A to fulfill general education
requirements or admissions requirements for professional or graduate schools
including medicine, nursing, pharmacy, and dentistry. Chemistry is the central
science that is considered to be the foundation for other academic disciplines and
applications because it provides the basic laboratory skills, such as preparing
solutions, commonly used in other disciplines. The majority of students that take
CHEM 1A are freshmen, but sophomores, juniors, and occasionally seniors will
take the course to fulfill the final requirements of their major. With the large
number of lower-division undergraduates required to take the course and the
limited sections that can be offered in current facilities, it is difficult for many
students to get into the course. This has, along with the increasing tuition costs at
Fresno State, contributed to many students opting to take the course at the
community college and transfer the units over to the University.
CHEM 1A is considered a high-failure rate course at Fresno State and
many other institutions. In conversations with Dr. David Frank, the previous
CHEM 1A coordinator and author of the laboratory manual, approximately 3040% of students enrolled in the course do not pass and are not able to continue
towards their degree goals. The laboratory portion generally accounts for only
15% of the overall CHEM 1A course grade, but students must pass the laboratory
component with a D or better in order to pass the course. Performance in the
3
laboratory component of the course can also indirectly impact performance in the
lecture component of the course, as the laboratory curriculum is designed to
reinforce critical concepts and skills evaluated through homework and exams in
the lecture section as well as providing a mechanism for more personal feedback,
mentoring, and instruction.
A variety of factors contribute to poor student performance in the
laboratory sections of CHEM 1A These include (1) poor independent study skills
and background knowledge, (2) difficulty connecting concepts and skills covered
in the lecture and laboratory sections, and (3) not allocating sufficient time for out
of class reading, problems, and studying.
Student efficacy is known to be a significant contribution to difficulty in
general chemistry courses.1,2 Students who lack foundational background in
mathematics, high school chemistry, or basic study and organizational skills
typically struggle in this course. 3 These students often fall behind early in the
course due to not completing the assignments they do not understand. The
increasing expectations of independent out-of-class work in the university setting
make this problem more significant. By the time many of these students seek help,
they are already too far behind to do well in the course.
Students often have difficulty connecting concepts between the lecture and
laboratory portion of the course. The lecture portion is taught by varying
instructors that have not taught the corresponding laboratory in many years.
Instructors may not be able to use the same or similar examples of the concepts,
causing students to not be able to make the connection between concept and
laboratory results. The lecture and laboratory portion do not always coordinate
with each other concerning order and timing of topics, further causing difficulty in
understanding concepts. When performing experiments, it is assumed that students
4
have background information that they have gained from lecture and should be
able to see where the numbers in practice problems come from. If the lecture and
laboratory portions do not coincide with each other, the students will miss this
point, preventing the student from connecting practice and concept.
Students that do not allocate sufficient time to complete assignments or
study for courses are recognized to have significant barriers that decrease the
amount of success students have in CHEM 1A. This may be partially due to a
students’ busy work schedule, but it is also significantly impacted by the amount
of motivation provided. Students who are motivated to study are more capable of
becoming successful. Student motivation could be increased by improving
instructional methods used to communicate information and better clarifying
expectations to students. A logical place to start in the CHEM 1A course is the
laboratory portion since there are more opportunities for one-on-one assistance
between students and TAs.
Our country is facing a critical shortage of scientists and engineers, along
with competent science teachers. Our education system is not meeting the
demands of the employment market and therefore inhibiting economic progress
and technological development in the American economy. With the lack of
emphasis on the sciences in primary and secondary education, students do not
have sufficient exposure to interest them in the subject. The importance of math
and English language arts causes the sciences to be pushed aside.
In order to regain the public’s focus on the sciences, it is important to
encourage that they are able to learn chemistry. The laboratory component offers
significant potential for improvement aimed at increasing performance. By
increasing the amount of training available for TAs, they will be better prepared to
help students during the recitation portion of the laboratory. The small class sizes
5
allow opportunities for meaningful and personal feedback. The one-on-one
opportunities to answer questions will give students the assistance they need to
complete the assignments and connect the lecture and laboratory concepts.
This research was focused on determining whether the CHEM 1A
laboratory structure can be modified to incorporate frequent quizzing to help
scaffold student learning. Frequent quizzing using clickers was used instead of the
traditional pre-laboratory worksheets in order to determine the level of student
preparation for laboratories. The quizzes were used by the TA to determine the
level of clarification needed for students to better understand the experiments they
are to perform. The importance of preparing for the laboratories by quizzing
students was used as a motivational technique to encourage students to prepare
and come to class ready to ask questions if they do not understand the procedure
suggested in the laboratory manual.
Frequent Quizzing
Many studies have determined that the structure of courses can be used to
help engage students in the classroom, but have not been discussed much in
regards to a laboratory environment. One approach to engaging students is
incorporating frequent quizzing into the lecture format. It is a common belief
among faculty members that students study the most right before an exam.4 It is
known that regular shorter term studying is more effective than single long term
studying for long term recall and learning.4,5 One way to motivate students to
study continuously throughout the semester, instructors will incorporate frequent
quizzes to be sure they are keeping up with their studies.6,7 Frequent quizzing is a
great informative tool used to catch students’ small mistakes and misconceptions
6
before they become more solidified misconceptions or cause harm to themselves
or others.
A somewhat surprising observation of this research is that students prefer
frequent quizzing over not being quizzed.6,8 Students believe that the assignment
of frequent quizzes causes them to keep up with their studies and increases the
chance that they will do the suggested readings.5-6,8-9 This can also help reinforce
students to establish good study habits.8,9 Some explanations as to why this occurs
is that students are forced to pay more attention to the information in lecture
because they will be quizzed on it.10 Quizzes also help students use abstract
reasoning to manipulate information.10
Another positive implication of frequent quizzing in the classroom is that
students obtain a mental belief that it helps them.4,6 Students are able to correct the
misconceptions they have created from previous courses or through their
readings.5,8 By getting feedback from an incorrect question, students can rethink
their understandings regarding the concept they were tested on. Studies that have
looked at the impact of frequent quizzing on student learning and test scores do
not show any difference in achievement.4,6 The difference between students that
have not had frequent quizzes and those who have has been shown as not being
statistically significant according to some studies.11,12 The amount of learning that
occurs between the two groups is comparable. The fact that students internally
believe that the quizzes help them achieve the learning objectives in the course can
be more powerful than observing a significant difference in the scores.8 Students
can benefit greatly from the use of frequent quizzing, but there are practical
reasons that may hinder an instructor from using them in their class.
The major reasons for avoiding frequent quizzing include question
construction and incorporation as well as time constraints. Question style plays an
7
important part in how students perceive the information.13 If the instructor writes
difficult questions, students will feel discouraged because they cannot answer the
questions.10 On the other hand, if the questions are too easy, students will not feel
the need to study because they know they will get a good grade.10 Instructors must
devote time in order to create questions that are a suitable level of difficulty.6
Questions can be created prior to lecture and stored in a question bank for future
use, although class time may be consumed by the administration and grading of
the quiz. The implication of this limitation can be lowered by using online quizzes
or the use of audience response systems, or clickers.
Clickers in the Classroom
One of the challenges of implementing frequent quizzing is the devotion of
time to administering and grading quizzes. Audience response systems, or
clickers, can help to alleviate many of these challenges through automated
grading, eliminating the time needed to pass out, collect and grade quizzes.
Clickers are generally used in a large classroom setting to communicate and
exchange feedback between instructor and student. 14-19 Other functions of clickers
in the classroom are to take attendance, question and answer sessions, voting,
quizzes, tests, and group decision making.18 The types of questions instructors can
use for a clicker quiz are mostly true/false, and multiple choice questions (Figure
1). 15-16, 19 They can also be used to refocus the class or initiate discussion among
small groups.14,16,19,20-22 They are presently used in courses for nursing, business,
mathematics, and the sciences in primary, secondary, and university education.17
The i>clicker software has been standardized at Fresno State in order to be easier
to incorporate into new courses. In order to use the clicker system, the instructor
and students must have access to a base station, student remote, and software.23
8
The base and software allows instructors to use a Powerpoint® presentation to
convey questions for students to answer with remotes registered to each
student.9,12,15,21,23 The software also allows the instructor to grade the questions
efficiently, also permitting instructors to re-grade questions if needed.21
Figure 1. A screen shot of a clicker quiz question as a student will view on a
monitor in the classroom.
In a paper describing the use of clickers in large courses, Woelk states that
there are six different categories in which they can be used. The first is used to
take attendance in large courses.15,17,19,24-25 Taking attendance will causes students
to be present more often than they normally would. The next category shows that
students are prepared for the day’s lecture.25 This encourages students to do the
assigned reading prior to coming to class. The third category shows the instructor
to create an interest of the concept among the students.25 This can be done by
tapping into prior knowledge students may or may not have on the topic.19 Fourth,
students show that they have been paying attention during lecture.25 The instructor
may give the students a pop-quiz to see if, for example, students can duplicate the
9
steps required to solve a limiting reaction problem. The fifth category shows that
students understand the information that was covered.25 Students are required to
answer conceptual questions rather than factual questions. Finally, instructors can
ask students to apply the information they learned to a real-world type situation.25
Of these six categories, instructors of laboratory courses are more likely to
motivate students by using clickers to be sure students come to class and are
prepared, while encouraging them to apply the new information to the laboratory
problems.25
Clickers in the classroom can also benefit instructors when administering
quizzes because they provide immediate feedback about student understanding.
The instructor can use clicker quizzes to catch and correct student misconceptions
as well as to encourage them to attend class and learn.5,26-27 When administering
the quiz, instructors have the opportunity to display a histogram showing how
many students chose each possible response.12,15,21,28 This not only allows students
to compare their responses to the rest of the class, but can also be used to facilitate
discussion since the instructor knows where students may be struggling. The
instructor can then display the correct answer and explain why the others were
incorrect.21 At this point, instructors can modify their lecture to help explain
concepts more effectively or correct student misconceptions.
In a smaller sized class or a laboratory setting, clickers can be used to be
sure students are present and to confirm students’ preparedness for class or
experiment. Since laboratories require students to perform hands-on activities and
are at different stages from their peers, it may be difficult to incorporate questions
into the class period.17 The use of clickers in laboratories can be beneficial for the
instructor during the pre-laboratory lecture to be sure students are present and have
done the work required to prepare for the days experiment.17 The clickers can also
10
show if students have created misconceptions regarding the procedure or
background information for the experiment.12,17,21-22,24,29 Once the experiment has
been completed, clickers can be used to collect student data to initiate class
discussions about trends that are present.16-17,30 Though it may be difficult, clickers
can successfully be incorporated into any type of classroom situation.
The impact of frequent quizzing and the incorporation of clickers have been
researched and studied in regards to lecture courses rather than laboratory courses.
As previously stated, it is easier to incorporate quiz questions into a lecture format
rather than a laboratory format, although it is possible to incorporate quiz
questions into the pre-laboratory introduction as a means of testing for student
preparation. If this is done, it will help reduce the occurrences of cheating between
students since many of them work together prior to the laboratory period to
complete pre-laboratory worksheets. Students will receive rapid feedback from the
TA that corrects misconceptions and helps students recognize safety hazards
present in the laboratory.
Laboratory Structure
The lecture portion of a course gives students the background information
they need to know in order to have a general idea of what they should be
observing in the laboratory. Teaching strategies that engage students are critical to
student achievement. Teacher demonstrations and hands-on activities are useful
techniques to help students visualize the abstract concepts they learn in lecture.
Though research shows that both of these techniques help to increase a students’
understanding of concepts, the traditional laboratory structure fits in well with the
hands-on technique rather than demonstrations since students are able to
manipulate procedures.31 The traditional structure of a laboratory is adapted to
11
incorporate principles from lecture to help students gain a better understanding out
of their limited class time.
A hands-on activity requires students to perform an experiment in small
groups. The general model of a hands-on activity requires these small groups to
complete a reaction sequence by using a known solution or compound and define
the chemistry taking place.32 Once this has been done, students must use the same
reaction sequence to determine the identity of an unknown solution or
compound.32 During a hands-on experiment, students often learn how to
manipulate various laboratory skills rather than developing their concept
knowledge.32 Students can further benefit from a hands-on activity if the scenario
given is in a real-life setting.33 Real-life settings help students realize that their
results mean something other than just numbers. By having this real-life setting,
students become more excited about learning chemistry because they can see the
importance of what they are learning.
In a traditional laboratory setting, there are three phases of an experiment
that have been accepted as the norm among the science community: prelaboratory, experimental, and post-laboratory.34,35 In the pre-laboratory phase,
students must understand the procedure and conceptual background for the
experiment they will be performing. In the experimental phase, students carry out
the procedure and collect data. The post-laboratory phase is where students
analyze their results and apply what they have learned. Of these three phases, prelaboratories are the most neglected sections even though they play a critical role in
helping students prepare for the experiment.35
12
Pre-Laboratory Exercises
Studies have shown that not enough emphasis is placed on the prelaboratory phase of an experiment.35 This phase is important because students
become familiar with the procedure they will be using, as well as the conceptual
information that causes the reaction to occur and what data to collect. Prelaboratories also help students become aware of any safety hazards that may be
associated with the experiment. Currently in the CHEM 1A course at Fresno State,
students are graded on their performance on a laboratory practical, instructor
evaluation, experiments and study guides, and quizzes. The weight of each
category in the laboratory course, along with a description of what the category
consists of is shown in Table 1 (p. 1). Grades on the pre-laboratory worksheets
comprise approximately 5% of the overall laboratory grade (less than 1% of the
overall course grade). Pre-laboratories are also given a small portion of class time
and are paid very little attention to by instructors or teaching assistants (TAs). By
focusing attention on the pre-laboratory, students should increase their
understanding and performance on experiments, further increasing their scores on
assignments and exams.
As previously stated, pre-laboratory exercises are used to prepare students
for the experiment because they provide information for students to become
familiar with the procedure, safety, foundational concepts, the data they must
record, and how to analyze it.35 In personal conversations occurring between
spring 2010 and summer 2011, it has become a general consensus among
laboratory instructors that these worksheets do not adequately prepare students for
the experiments. Many students will wait until right before class to read for the
answers to the pre-laboratory worksheet questions or complete them in class by
copying the answer from their classmates. In this way, the current pre-laboratory
13
structure does not serve to help students understand the procedure or to obtain
good results.34 It is important that students access their prior knowledge to
understand the procedure and concepts before they begin so they can analyze the
information obtained from the experiment.34,36 Prior knowledge will help guide the
student to make the correct observations needed to successfully perform the
experiment.
Prior to beginning the experiment, the laboratory instructor traditionally
gives a brief 20-minute lecture regarding the procedure, safety, and purpose of the
experiment. The instructor also encourages students to become involved in the
discussion by answering questions.36 During these lectures, students who have
prepared for the experiment are more likely to answer any questions posed to
them, while the others will sit quietly.37 Students who prepare more thoroughly are
more likely to complete the experiment more efficiently and without accidents.34,37
Those who do not prepare will continually refer to their laboratory manual and
mirror other groups.37 These students are also more likely to make a mistake or
have an accident when conducting the experiment.38
Finally, instructors and students should pay more attention to prelaboratories because students gain a better understanding of the concepts they are
concentrating on. Students who prepare for a laboratory know what they should be
observing during the experiment and are not distracted by unrelated
observations.34 Students who do not prepare as well will not know what to look for
during the experiment and will be distracted by interferences. Although
interferences can be reduced by having a set procedure, it is more beneficial for
students to read through background material and decipher the procedure.34
Students who do not prepare or understand will likely not have the pre-requisite
knowledge or skills that are needed to perform the experiment.35
14
Scaffolding
Scaffolding is a theoretical concept that describes an instructional method
designed to assist students in learning new information. Scaffolding occurs when
the instructor supports students in accomplishing a task they would not be able to
complete on their own.39 The instructor models what the process should be, while
gradually allowing the student to perform the activity on their own.40
Psychologists believe that scaffolding occurs most efficiently in the Zone of
Proximal Development (ZPD).41 The ZPD is the gap between procedures they can
do on their own and what they need assistance on.39,40 This model can be related to
the student laboratory procedure.
The technique of scaffolding can be seen in the CHEM 1A lecture and
laboratory relationship. Students are given new information in lecture with which
they are unfamiliar. The ZPD gap is where the TA will model and guide students
through practice problems during the recitation portion.41 Students are then tested
on these skills in lecture since they should now be able to perform these tasks on
their own. The laboratory itself also shows scaffolding in its structure. Students are
expected to read through the laboratory manual in order to gain background and
procedural knowledge they may be unfamiliar with prior to performing the
experiment.42
The addition of pre-laboratory quizzes provides feedback to the TA
administering the quiz. In doing so, the TA can modify their pre-laboratory
introduction to the students and correct any misconceptions or misunderstandings
students may have. The TA will then model the laboratory procedure during the
pre-laboratory lecture, and allow students to practice the procedure while
receiving help from the TA. The students will then independently complete the
required calculations with the data they collect from the experiment.
15
Scaffolding can help guide students to a better understanding of new
information. Students learn unfamiliar concepts in lecture, and often times will
gain a better understanding having practiced it in their laboratory course.
Instructors and TAs can use techniques to better gauge how much their students
have learned or understood. One such technique will be considered in response to
answer the main question: Are there more effective ways for students to prepare
for a laboratory course? This study will specifically look at the implementation of
pre-laboratory quizzes as a means of scaffolding the pre-laboratory stage of an
experiment and determine if it is a more effective means of preparation than
completing a pre-laboratory worksheet.
Project Aims
The purpose of this study was to determine if the implementation of a prelaboratory quiz as an alternative to a pre-laboratory worksheet has a positive
impact on student performance. A set of quiz questions was developed to use with
i>clicker software as a means of pre-laboratory assessment in a Chemistry 1A
class. Two types of data were recorded and analyzed: objective data from student
work and subjective data from both instructors and students. The objective data
were obtained from laboratory write-up scores, experiment data, and prelaboratory activity scores. Subjective data were obtained from observations of the
laboratory sections as well as surveys given to both TAs and students. The
following research questions were identified and are addressed in the statistical
analyses:
-
Does the pre-laboratory activity type impact student preparation for
laboratory at a level that can be observed in metrics of student
performance?
16
-
Does the laboratory meeting time have an impact on student
performance?
-
Does the TA introduction and grading style have an impact on student
performance?
In performing the statistical analyses looking at student performance based
on pre-laboratory assessment type, an increase in laboratory scores is expected to
be seen as well as a decrease in student error, causing students to become more
accurate in their experimentation for courses assigned pre-laboratory quizzes is
expected. By looking at the laboratory meeting time, an increase in laboratory
scores as well as a decrease in student error is expected. Finally, the analysis of
TA introduction and grading style is expected to result in increased laboratory
scores and decreased student error for students under the instruction of a more
thorough TA. If the data prove to support this hypothesis, it would be helpful for
the Chemistry Department at Fresno State to adopt the pre-laboratory procedure
for the laboratory portion of CHEM 1A.
METHODOLOGY
Participants
The test group used in this study consisted of two laboratory sections of
CHEM 1A in the spring 2011 semester, as well as four additional laboratory
sections in the fall 2011 semester. The majority of the students that participated in
this study were biology majors, followed by chemistry and mechanical
engineering (Table 2). The academic level of the participants also varied from
freshmen to senior level students. In this study, the majority of the participants
were freshmen (53.3%), then sophomores (29.6%), juniors (13.3%), followed by
seniors (3.7%). As required by the Chemistry Department, the students were also
enrolled in the lecture portion of the CHEM 1A course, consisting of two possible
lecture instructors for spring 2011 and three possible lecture instructors for fall
2011. The assignment of students per section was not in the control of the
researcher, as the students were able to choose which section of both the lecture
and laboratory fit with their personal schedule. As a result, the lecture varied from
student to student and between laboratory sections.
The Teaching Assistants (TAs) who participated in this study had begun the
Chemistry or Forensic Science Masters of Science programs in the fall of 2010.
All three had begun teaching the CHEM 1A sections during the fall 2010 semester
and had also taken the Chemistry Laboratory Teaching Techniques course, CHEM
201, during the same semester. Prior to the spring 2011 semester, TA 1 had taught
one section of CHEM 1A as well as the general chemistry course for non-science
majors, CHEM 3A, TA 2 had taught one section of CHEM 1A, and TA 3 had
taught two sections of CHEM 1A.
18
Table 2. Frequencies and Percentages of Participant Demographic Information
(N = 135)
Variable
Planned Degree
Animal Science
Biology
Biomedical Physics
Chemistry
Civil Engineering
Criminology
Enology
Environmental Science
Food & Nutr Sci
Geology
Health Science
Kinesiology
Math
Mechanical Engineering
Natural Science
Physics
Pre-Business
Pre-Nursing
Pre-Physical Therapy
Pre-Psychology
Undeclared
Student Year
Freshman
Sophomore
Junior
Senior
n
Percent
10
48
3
24
1
1
6
3
6
2
4
2
1
10
7.4
35.6
2.2
17.8
0.7
0.7
4.4
2.2
4.4
1.5
3.0
1.5
0.7
7.4
1
2
1
1
3
5
1
0.7
1.5
0.7
0.7
2.2
3.7
0.7
72
40
18
5
53.3
29.6
13.3
3.7
19
For purposes of this study, students were assigned a five-digit code in order
to maintain confidentiality. Data are reported only in aggregate and no personal
information was collected nor used in evaluation.
Assessment Tools
The main focus of this study was to look at the impact of the pre-laboratory
assessment on student performance. In order to do this, an additional assessment
type was constructed. The pre-laboratory i>clicker quizzes were developed as a
new means of assessment, rather than the pre-laboratory worksheets that are in the
required laboratory manual. Pre-laboratory quizzes were worth the same number
of points as the worksheet to help prevent inconsistencies with grades between
experiments. The scores from the i>clicker quiz and scores for the pre-laboratory
worksheets were recorded. Other means of data collection were also explained, the
two being on-task determination as well as surveys.
Pre-Laboratory Worksheet
The pre-laboratory worksheets consisted of questions regarding background
information presented in the laboratory manual written by Dr. David Frank
(Appendix A). These questions are meant to help students understand the concepts
behind the experiment, and also include sample calculations, definitions, safety,
and procedural questions. Students were to complete the questions for a specific
experiment before class and the worksheets were collected at the beginning of the
laboratory period. The worksheets were then returned to the researcher to be
graded. Each question was graded using a 5-point scale rubric for data collection
purposes (Table 3). A missing component signified the student had quoted the
laboratory manual without explaining the logic behind their answer. A weak
component indicated that the student had answered the question by copying
20
directly from the laboratory manual instead of using their own words. The scores
for all questions were averaged to give the pre-laboratory assessment score. This
was then normalized to a score out of 100.
Table 3. Rubric for Grade Distribution for Grading PreLaboratory Questions
5
Student answered question completely and showed
understanding of the subject matter.
Complete/correct, clear, own words.
4
Student quoted the lab manual, but did not explain their
reasoning. One weak component.
3
Student was on topic, but was unclear.
One missing OR two weak components.
2
Student gave a clear answer, but was not on topic
Two missing OR three weak components.
1
Student gave a vague answer, but was not on topic
Three missing components.
0
Student left question blank
Pre-Laboratory Quiz
In order to develop a pre-laboratory quiz, a question bank was created
consisting of questions regarding information students should know in order to
perform the experiment (Appendix B). The three TAs who participated in the
study were assigned experiments and constructed two to five questions regarding
each of the following question types: concept, new terminology, procedure, safety,
and calculation or data type questions. In creating the questions, the TAs tried to
model the pre-laboratory worksheet questions, though some questions could not be
written in a multiple choice format. These questions were then combined into
three groups: procedure/safety, concept/terms, and data type questions. It took
approximately 5 minutes to put together a five-question pre-laboratory quiz from
the question bank, containing at least one question from each of the groups of
21
questions. This amount of time included choosing the question to use, formatting
the question to fit the slide, making sure the sequence of the questions fits, as well
as being sure the difficulty of all the questions is appropriate.
Since i>clicker software was used to administer the pre-laboratory quizzes,
the three TAs had to become familiar with the software. The TA that taught during
the spring 2011 semester was trained how to create the quizzes and administer
them to the class by Dr. Eric Person. At the beginning of the second semester, two
of the TAs needed the researchers assistance in using the i>clicker docking station
and program since they had never used it before. For the duration of the semester,
the two TAs were able to utilize the docking station without further assistance.
Students were required to have their own i>clicker remote since the quizzes
were administered using the i>clicker software. Students who had forgotten their
remotes were instructed to record their responses on a piece of paper and were
graded when the scores were recorded. These students lost points for not coming
to class prepared. At the beginning of the semester, students registered their
remote with their name prior to the first pre-laboratory quiz, taking approximately
five minutes to complete. On the day of a pre-laboratory quiz, students had 30
seconds to answer each question, although more time was allowed if students had
to perform calculations. After each question, the instructor had the choice of
revealing the correct answer or moving onto the next question. Once the quiz was
completed, the instructor was able to give the introduction to the day’s experiment.
When graded, the pre-laboratory quiz had the same weight as the pre-laboratory
worksheets in the overall grade for the students. This was then normalized to a
score out of 100.
22
On-Task Determination
Various approaches had been attempted to better determine how “on-task”
students were for each laboratory section. The first approach was to subjectively
grade each student on a scale from 0-10, an absence resulting in a 0 and a flawless
execution of the experiment resulting in a 10, based on the number of questions
students asked during the experiment, the nature of the questions asked, how
focused they were on the experiment, safety practices, and attendance. The second
approach was to create a check-list to grade the entire class on a scale from 1-10,
where a 1 shows that the students were not on task with the experiment and a 10
showing that the entire class performed the experiment flawlessly. The check-list
looked at the number of questions students asked during the experiment, the nature
of the questions asked, how focused students were on the experiment, safety, and
attendance. However, both of these approaches did not accurately determine how
“on-task” each student or the entire class was for each experiment since the top
priority of the TA was to teach and be available to help students perform the
experiments. The time limitation of the TAs was also a factor in not being able to
collect this data.
Survey
In order to understand the perspective of the TA as well as the student,
three main approaches were taken to collect information. The first approach taken
was for the teaching assistants to take subjective notes during each of the
experiments. In these notes, the TA would state whether or not there were any
problems and record any difficulties their students had in completing the
experiment. The TAs also took note of any comments students made regarding the
pre-laboratory assessments or the experiments.
23
The second approach was taken with the two CHEM 1A students that
participated in the study during the spring 2011 semester (Appendix C). This was
used to give an insight to how students felt about completing the two types of prelaboratory assessments. Students were also asked if the pre-laboratory assessments
could be improved to better suit their needs.
The third approach was used upon completion of the study at the end of the
fall 2011 semester (Appendix D). The TAs completed a survey to better
understand what they did differently than in previous semesters of teaching. The
TAs were also asked their opinion on which assessment they believed helped to
enhance their students’ learning experience.
Conditions
Pre-laboratory activity structures were designed to allow for comparisons
between the i>clicker quiz and paper worksheet forms of administration. This was
approached by having four sections with alternating pre-laboratory activities and
two additional sections with a consistent pre-laboratory activity (Table 4). The
conditions of three groups of data collected are explained.
Table 4. Structure of Pre-Laboratory Evaluations for
the Sections Used
Semester
Spring 2011
TTh8-11
Spring 2011
TTh 6-9
Fall 2011
MW11-2
Fall 2011
TTH 11-2
Fall 2011
MW 2-5
Fall 2011
TTH 2-5
Design
Alternating
(quiz first)
Alternating
(worksheet first)
Worksheet only
Set
A
Instructor
1
A
1
B
1
Quiz only
B
1
Alternating
(quiz first)
Alternating
(worksheet first)
C
2
C
3
24
Spring 2011 (Set A)
The two spring sections alternated between pre-laboratory quiz and the
existing pre-laboratory worksheets in the laboratory manual written by Dr. David
Frank (Appendix A). A pre-laboratory worksheet was developed for Experiment 5
since the existing worksheet requires students to predict the products of reactions.
These two sections were taught by the same instructor on the same day, but at
different times. The limitations in the course scheduling presented uncontrolled
variables in the study. Though these two courses were taught on the same days by
the same instructor, a time variable was introduced since on course was taught at
8am and the other at 6pm. The data that were collected from each student were the
pre-laboratory assessment score, identifying information from the unknown for the
experiment, and the score they received for that experiment.
Fall 2011 (Set C)
Two of the fall semester sections were chosen to alternate between prelaboratory quiz and the existing pre-laboratory manual as previously described.
These two sections were taught at the same time of day, but by two different
instructors on two different days. The limitations in the course scheduling
presented the uncontrolled variables of instructor and day. Instructor 2 met with
the class on Monday and Wednesday, and Instructor 3 met with the class on
Tuesday and Thursday. The data that was collected from each student was the prelaboratory assessment score, identifying information from the unknown for the
experiment, and the score they received for that experiment.
Fall 2011 (Set B)
The two remaining fall semester sections were assigned either the prelaboratory quiz or the pre-laboratory worksheet for the entire semester. These two
25
sections were taught by the same instructor at the same time, but on different days.
The limitations in the course scheduling presented the uncontrolled variable of
day. The section that was assigned the pre-laboratory worksheet met Mondays and
Wednesdays, and the section that was assigned the pre-laboratory quiz met
Tuesdays and Thursdays. The meeting time and day has the potential to alter the
number of lectures the student has had prior to taking the laboratory. The data that
were collected from each student were the pre-laboratory assessment score,
identifying information from the unknown for the experiment, and the score they
received for that experiment. These two groups were also given a pre-test
consisting of conceptual questions regarding the concepts they would learn
throughout the semester (Appendix E).43 The students were also given the same
test at the end of the semester as a post-test to compare student improvement
between the two pre-laboratory evaluations.
Dependent Variables
Pre-Laboratory Score
The pre-laboratory scores were obtained from either pre-laboratory quizzes
or pre-laboratory worksheets, depending on the schedule assigned to the CHEM
1A section. The pre-laboratory worksheets were graded using a five-point rubric,
and a normalized score was then calculated. The pre-laboratory quizzes were
graded by entering the correct response for each question in the i>clicker software
and allowing the total score to be calculated. These scores were scaled to a 100%
scale.
26
Percent Error
The experimental values as well as the identifying code for the unknown
each student used were collected. Using this information, the percent error was
determined for each student by calculating the percent error relative to a known
value. If a known value was unavailable, the deviation from the mean of the class
was calculated. It is expected that students that are more prepared for the
laboratory will obtain a lower percent error value. It should be noted that the
percent error values for each experiment differ from each other since each of the
experiments test different methods and concepts.
Laboratory Score
The laboratory write-ups are composed of an experimental analysis, in
which students use their experimental data to determine the identity of an
unknown, as well as a series of calculation questions similar to those they used in
their analysis. The laboratory write-ups were graded by the TA assigned to the
section. In grading, they have control over how many points each section or
question is worth since there is not a standardized grading structure throughout the
laboratory sections. For this reason, these scores were scaled to a 100% scale..
Data Analysis
Each of the three TAs collected the pre-laboratory assessment score,
identifying information from the unknown for the experiment, and the score they
received for that experiment. For purposes of this study, students were assigned a
five-digit code in order to maintain confidentiality. All data were collected and
organized in Excel® by code to ensure that a complete data set containing each of
the three scores had been collected for each experiment. The pre-laboratory
worksheet assessment was coded with the digit 0, and the pre-laboratory quiz was
27
coded with the digit 1. The students who met on Mondays and Wednesdays were
coded with the digit 1, while those that met on Tuesdays and Thursdays were
coded with the digit 2. Students that met at 8 am were coded with the digit 1, those
that met at 11 am were coded with the digit 2, those that met at 2 pm were coded
with the digit 3, and those that met at 6 pm were coded with the digit 4. In order to
remain anonymous, the TAs were coded as the digits 1, 2, and 3. Data were
reported only in aggregate and no personal information was collected nor used in
evaluation.
Once the data were organized and coded, statistical analyses were
performed for each experiment using the Statistical Package for the Social
Sciences (SPSS). Appropriate statistical comparisons, such as Multivariate
Analysis of Variance (MANOVA), were performed in an attempt to isolate any
significant differences between the pre-laboratory score, percent error, and
normalized laboratory score. The significance level used for all analyses was α =
0.05. The independent variables available for use were the day and time of the
laboratory, pre-laboratory assessment type, and TA.
The results from the pre- and post-test of the two consistent laboratory
sections in the fall 2011 semester were collected and scored in Excel®. A t-test
was performed to determine whether or not the student scores improves over the
semester based on the pre-laboratory assessment type assigned to them.
Significant results for the statistical analyses performed on the data are
reported in a table or figure format, where appropriate, to increase the clarity of
the results for each experiment that students performed.
RESULTS
The main purpose of this study was to determine if implementation of a
pre-laboratory quiz as an alternative to a pre-laboratory worksheet has a positive
impact on the level of student preparation as evaluated by student performance and
grades and using survey feedback. The impact of the scheduled laboratory time
and teaching assistant on student performance and grades were also reviewed as
controls. The results of this study are presented in this section.
Impact of Pre-Laboratory on Student Performance
General MANOVA Results
The data were collected and compiled into one spreadsheet by experiment.
A two factor MANOVA (TA x Pre-Laboratory Assessment) was performed on
each experiment using the three dependent variables: pre-laboratory assessment
score, percent error, and laboratory score. The means and standard deviations of
this initial analysis are shown in Table 5. Upon reviewing these results, nine of the
11 experiments show a significant difference in the scores between pre-laboratory
quiz and pre-laboratory worksheet (Appendix F). In the MANOVA of Experiment
1, the statistical information was not calculated since there were not enough
complete data sets to perform this calculation.
The main interest of this study was the impact of the pre-laboratory
assessment on the percent error and laboratory score as a means of measuring
student performance. Upon reviewing these two dependent variables in Table 5
there were significant differences for Experiment 9 (F1, 108= 6.94, p < 0.02) and 12
(F1, 95= 6.32, p <0.02), but no significant difference between the two prelaboratory assessments when looking at the laboratory score. It should be noted
29
that the partial eta squared values, or effect sizes, for these calculations range
between 0% and 6.3% when looking at the percent error, and between 0% and
0.9% when looking at the laboratory score.
Table 5. Means and Standard Deviations for Pre-Laboratory Score, Percent Error,
and Laboratory Score by Pre-Laboratory Assessment for Each Experiment
Pre-Laboratory Score
Experiment
1
2
3
4
5
6
8
9
12
13
15
PreLaboratory
Assessment
Worksheet
Quiz
Worksheet
Quiz
Worksheet
Quiz
Worksheet
Quiz
Worksheet
Quiz
Worksheet
Quiz
Worksheet
Quiz
Worksheet
Quiz
Worksheet
Quiz
Worksheet
Quiz
Worksheet
Quiz
M
63.32
98.26
57.59*
92.90*
60.97*
87.17*
54.76*
69.78*
54.76*
69.78*
55.50*
83.51*
60.71*
81.75*
61.24*
80.84*
53.48*
86.15*
52.29*
78.46*
60.31
66.89
SD
1.51
2.02
3.20
3.09
2.32
2.23
3.69
3.62
3.69
3.62
3.79
3.52
2.92
3.65
2.90
2.73
3.56
4.17
3.87
3.95
4.03
4.15
Percent Error
M
3.59
3.37
48.02
37.17
11.37
11.91
SD
0.79
1.06
7.28
7.04
1.79
1.72
N/A
N/A
N/A
N/A
14.98
21.93
138.14
77.04
50.30*
31.76*
26.00*
61.63*
106.89
64.15
55.99
62.35
N/A
N/A
N/A
N/A
2.84
2.64
20.72
25.90
3.66
3.44
8.88
10.41
34.58
35.34
12.42
12.80
Laboratory Score
M
82.45
90.65
78.47
81.12
83.10
81.48
80.43
70.92
80.43
70.92
76.93
72.54
75.62
81.13
81.59
73.21
72.67
84.49
77.08
87.77
76.13
67.06
* denotes p < 0.05
Upon viewing these results, it was decided that the pre-laboratory
assessment type was not a useful metric to determine the level of student
SD
1.88
2.51
2.62
2.54
1.87
1.80
3.29
3.23
3.29
3.23
2.93
2.72
2.65
3.31
2.27
2.14
2.16
2.54
2.93
3.00
3.27
3.37
30
preparation. The amount of variance between the two assessment types was too
great, mainly because they were not graded in the same manner. The prelaboratory quizzes were graded objectively using the i>clicker software and
recorded student responses were marked as simply being correct or incorrect. The
pre-laboratory worksheets were subjectively graded using a five-point rubric. For
this reason, it was decided that the pre-laboratory assessment type was not a
reliable metric and was removed.
Data Set B
In order to control the independent variable of time and instructor, a
MANOVA was performed using data set B to analyze the pre-laboratory
assessment type. It should also be noted that the day differed between the two, but
was not considered since the sections were assigned consistent pre-laboratory
assessments for the duration of the semester. The students that were assigned the
worksheet met on Mondays and Wednesdays (n = 21), while the students assigned
the quiz met on Tuesdays and Thursdays (n = 24). The students involved in data
set B were assigned either the pre-laboratory quiz or pre-laboratory worksheet for
the entire semester.
Upon reviewing the means and standard deviations of the pre-laboratory
assessment scores in Table 6, the pre-laboratory assessment type has a significant
impact on eight of the 11 experiments (Appendix G). The main interest of this
study was to look at the impact on student performance by looking at the percent
error and laboratory score. Upon reviewing these two dependent variables on
Table 6, there was only a significant difference in the laboratory score for
Experiment 1 (F1, 42= 4.68, p < 0.04), but no significant impact on the percent
error. In further analysis of Table 6, the percent error averages for the pre-
31
laboratory worksheets was lower than those of the pre-laboratory worksheets. The
averages of the laboratory scores vary from experiment to experiment as to which
pre-laboratory assessment type resulted in a higher average.
Table 6. Means and Standard Deviations for Pre-Laboratory Score, Percent Error,
and Laboratory Score by Pre-Laboratory Assessment for Each Experiment Using
Data Set B
Pre-Laboratory Score
Experiment
1
2
3
4
5
6
8
9
12
13
15
Percent Error
Laboratory Score
PreLaboratory
Assessment
M
SD
M
SD
M
SD
Worksheet
Quiz
Worksheet
Quiz
Worksheet
Quiz
Worksheet
Quiz
Worksheet
Quiz
Worksheet
Quiz
Worksheet
Quiz
Worksheet
Quiz
Worksheet
Quiz
Worksheet
Quiz
Worksheet
Quiz
65.50*
90.83*
65.63*
93.75*
55.26*
90.44*
56.00*
72.50*
51.50*
73.33*
64.71*
90.91*
64.00
75.83
66.29
79.00
56.18*
97.27*
56.53*
79.09*
61.43
66.32
4.17
3.81
4.77
3.37
3.93
3.57
5.08
46.4
7.07
6.46
3.70
3.25
4.77
4.36
7.60
6.36
5.20
4.57
5.87
5.16
6.38
5.48
N/A
N/A
8.35
34.32
5.83
13.66
N/A
N/A
N/A
N/A
11.35
13.72
N/A
N/A
32.86
36.44
33.85
45.48
58.28
78.62
20.81
23.90
N/A
N/A
11.34
8.02
3.01
2.74
N/A
N/A
N/A
N/A
2.90
2.55
N/A
N/A
6.16
5.15
11.09
9.75
40.69
35.77
3.57
3.07
85.14*
71.79*
83.43
86.97
82.49
84.33
80.48
72.68
81.27
75.38
78.12
81.73
67.27
68.94
82.83
86.25
84.62
82.30
88.82
93.27
84.86
82.12
4.56
4.16
3.46
2.45
3.14
2.85
5.02
4.58
3.81
3.48
4.94
4.34
7.40
6.76
4.95
4.14
3.87
3.40
4.15
3.65
4.98
4.27
* denotes p < 0.05
The students who were in this data set were given a pre-test and the same
questions for a post-test, consisting of conceptual questions. A paired t-test was
32
used to compare the students test results by pre-laboratory assessment. It was
assumed that the scores would increase at the end of the semester. The data show
that this was not the case. Students who were assigned either pre-laboratory
worksheets or quizzes did not increase their score on the post-test (Tables 7 and
8). The students who were assigned pre-laboratory worksheets scored significantly
worse on the test (F1, 23=2.51, p < 0.03), while those assigned the quizzes scored
lower, but not significantly lower (F1, 22 = 1.305, p=0.205). It must be mentioned
that that there were very few students whose pre- and post-test scores had
increased, but this did not impact the majority of decreased scores.
Table 7. Paired Differences Between Pre-Test and Post-Test for Students Assigned
Pre-Laboratory Worksheets
Paired Differences
Pre-Test
Post-Test
N
Mean
Std.
Deviation
24
24
39.5833
31.0606
12.8200
20.5768
t
df
Sig..
(2-tailed)
2.508
23
0.020
Table 8. Paired Differences Between Pre-Test and Post-Test for Students Assigned
Pre-Laboratory Quizzes
Paired Differences
Pre-Test
Post-Test
N
Mean
Std.
Deviation
23
23
42.4901
38.5376
13.8827
16.3240
t
1.305
N
22
0.205
In further analysis of the statistical data, it should be noted that averages for
pre-laboratory quizzes for the pre-test were higher than those for the worksheets
(Figure 2). The range of the scores was lower for the class assigned the worksheets
(9.09%-63.64%) than the quizzes (13.64%-68.18%). This general trend can also
be seen in the post-test, although one student in the class assigned worksheets did
exceptionally well in comparison to her classmates, raising the maximum score.
33
100.0
90.0
Student Score (%)
80.0
70.0
Pre-Test
60.0
50.0
Post-Test
40.0
30.0
20.0
10.0
0.0
worksheet
worksheet
Quiz
Quiz
Figure 2. Pre-test and post-test data comparing students who completed the prelaboratory worksheet to those who completed the pre-laboratory quiz.
Impact of Teaching Assistant Introduction Style on
Student Performance
The impact of the TA grading and introduction style was analyzed in this
study as well. The grading style of TAs 1 and 3 was similar because they assigned
a point value to each question, causing each experiment to have an unequal point
total. TA 2 graded largely on completeness, as all the experiments were worth the
same amount of points. The introduction style of the three TAs differed as well:
TA 1 gave the students a thorough explanation of the procedure as well as how to
perform the calculations needed, TA 3 also gave a thorough introduction but
focused on the concepts, and TA 2 did not give the students an introduction at all.
Using these guidelines, the MANOVA results for Percent Error were used to
observe the introduction style, and Laboratory Score was used to were used to
observe grading style. A summary of these results is reported below.
34
General MANOVA Results
The two factor MANOVA (TA x Pre-Laboratory Assessment) was used to
initially determine whether there were any differences between the instructors
when looking at the percent error and laboratory score (Appendix F). The means
and standard deviations of this initial analysis are shown in Table 9.
Upon reviewing the results, Experiment 3 (F2, 117 = 4.71, p < 0.02),
Experiment 6 (F2, 110= 3.50, p < 0.04), Experiment 8 (F2, 91 = 9.30, p <0.01), and
Experiment 15 (F2, 71 = 3.23, p < 0.05) show a significant difference when looking
at the percent error of each experiment. A post hoc Tukey analysis was performed
to determine where the variation lay. The results of Experiment 3 state that TA 1
and 3 are significantly different (p < 0.03). Experiment 6 results state that TA 1
and 3 are significantly different (p < 0.02). Experiment 8 results state that TA 1
and 3 are significantly different (p < 0.01) and TA 2 and 3 are significantly
different (p < 0.02). The results from the Experiment 15 post hoc test state that
there was no significant difference between the three TAs.
There was also a significant difference in the laboratory scores for
Experiment 9 (F2, 108 = 18.52, p < 0.01), Experiment 12 (F1, 95 = 25.18, p < 0.01),
Experiment 13 (F1, 81 = 5.82, p < 0.02), and Experiment 15 (F2, 71 = 7.81, p < 0.01).
A post hoc Tukey analysis was performed on Experiments 9 and 15 to determine
where the variation lies. The results of Experiment 9 state that TA 1 and 3 are
significantly different (p < 0.01), as well as TA 2 and 3 (p < 0.01 ). The results
from Experiment 15 state that TA 1 and 3 are significantly different (p < 0.01).
Post hoc analyses were not performed on Experiments 12 and 13 since only two
TAs were included in the analysis. The MANOVA results for Experiment 12 state
TA 1 and 3 were significantly different (p < 0.01). The MANOVA results for
Experiment 13 state TA 1 and 3 were significantly different (p < 0.02).
35
Table 9. Means and Standard Deviations for Pre-Laboratory Score, Percent Error,
and Laboratory Score by Teaching Assistant for Each Experiment
Pre-Laboratory Score
Teaching
Experiment Assistant
1
2
3
4
5
6
8
9
12
13
15
1
2
3
1
2
3
1
2
3
1
2
3
1
2
3
1
2
3
1
2
3
1
2
3
1
2
3
1
2
3
1
2
3
* denotes p < 0.05
Percent Error
Laboratory Score
M
SD
M
SD
M
SD
65.50*
98.26*
61.14*
75.25
N/A
N/A
74.51
84.35
62.91
58.89
71.30
60.00
58.89
71.30
60.00
74.37
48.41
80.87
74.28
76.00
60.35
72.24
56.95
82.73
68.70
N/A
55.71
65.46
N/A
52.13
63.36
59.11
68.57
2.17
2.02
2.11
2.22
N/A
N/A
1.97
3.56
3.64
3.04
5.91
5.91
3.04
5.91
5.91
3.29
5.71
5.59
3.29
4.87
4.54
2.49
4.44
4.34
2.99
N/A
5.68
3.06
N/A
6.17
3.79
5.77
6.54
5.12
3.37
2.07
42.59
N/A
N/A
8.17*
13.69*
16.53*
N/A
N/A
N/A
N/A
N/A
N/A
14.87*
15.15*
28.93*
56.68*
90.09*
226.90*
47.00
45.33
24.80
42.89
N/A
27.86
61.02
N/A
155.90
56.14*
36.48*
87.94*
1.13
1.06
1.11
5.06
N/A
N/A
1.52
2.74
2.80
N/A
N/A
N/A
N/A
N/A
N/A
2.47
4.29
4.19
23.30
34.53
32.20
3.14
5.61
5.48
7.46
N/A
14.18
27.33
N/A
55.17
11.71
17.80
20.19
85.14*
90.65*
79.76*
79.79
N/A
N/A
84.22
77.83
82.88
71.87
71.30
87.64
71.87
71.30
87.64
75.72
79.77
67.72
81.43
81.00
69.64
84.54*
80.24*
60.27*
84.01*
N/A*
61.81*
85.96*
N/A*
70.00*
80.46*
70.28*
55.18*
2.70
2.51
2.63
1.83
N/A
N/A
1.59
2.87
2.94
2.72
5.27
5.27
2.72
5.27
5.27
2.54
4.41
4.32
2.98
4.41
4.12
1.95
3.48
3.40
1.82
N/A
3.46
2.32
N/A
4.68
3.08
4.69
5.32
36
Data Sets A and C
In order to control the students’ assessment conditions, a MANOVA was
performed using data sets A (TA 1, n = 44) and C (TA 2, n = 24; TA 3, n = 22) to
analyze the impact of instructor on percent error and laboratory score (Appendix
H). It should also be noted that the day and time differed between the two, but was
not considered since the statistical data could not be calculated when added as an
independent variable. The means and standard deviations of this analysis are
shown in Table 10.
Upon reviewing the results, Experiment 3 (F2, 78 = 4.60, p <0.02),
Experiment 8 (F2, 76 = 7.60, p < 0.01), Experiment 9 (F2, 75 = 10.45, p < 0.01), and
Experiment 15 (F2, 44 = 15.61, p < 0.01) show a significant difference when
looking at the percent error of each experiment. A post hoc Tukey analysis was
performed on these experiments to determine where the variation lies. When
looking at the percent error, the results of Experiment 3 state that TA 1 and 3 are
significantly different (p < 0.02). Experiment 8 results state that TA 1 and 3 are
significantly different (p < 0.01) and TA 2 and 3 are significantly different (p <
0.03). Experiment 9 results state that TA 1 and 3 are significantly different (p <
0.01) and TA 2 and 3 are significantly different (p < 0.04). Experiment 15 results
state that TA 1 and 2 are significantly different (p < 0.01) and TA 1 and 3 are
significantly different (p < 0.03).
There was also a significant difference in the laboratory scores for
Experiment 1 (F2, 86 = 3.31, p < 0.05), Experiment 5 (F2, 86 = 3.68, p < 0.03), and
Experiment 9 (F2, 75 = 11.48, p < 0.01). A post hoc Tukey analysis was performed
on these experiments to determine where the variation lies. When looking at the
laboratory scores, the results of Experiment 1 state that TA 1 and 2 are
significantly different (p < 0.01). The results of Experiment 5 state that TA 1 and 3
37
Table 10. Means and Standard Deviations for Pre-Laboratory Score, Percent Error,
and Laboratory Score by Teaching Assistant for Each Experiment Using Data Sets
A and C
Pre-Laboratory Score
Teaching
Experiment Assistant
1
2
3
4
5
6
8
9
12
13
15
1
2
3
1
2
3
1
2
3
1
2
3
1
2
3
1
2
3
1
2
3
1
2
3
1
2
3
1
2
3
1
2
3
M
55.79a
98.26a, c
60.87c
73.93a
55.87a
68.70
76.75
84.35
63.13
59.83
51.65
85.22
55.17
71.30
60.00
69.79
48.41
80.87
70.83
76.00
60.35
71.88
57.09
82.73
56.41
75.65
50.87
54.92
N/A
41.13
60.86
52.35
55.18
SD
3.55
4.85
4.85
3.64
4.49
4.49
2.90
3.52
3.52
3.78
5.17
5.17
4.06
5.55
5.55
5.89
6.66
6.52
3.79
5.05
4.71
2.71
3.37
3.37
5.14
7.03
7.03
4.58
N/A
6.26
9.53
5.92
7.59
Percent Error
M
SD
N/A
N/A
N/A
N/A
N/A
N/A
5.86b
13.69
15.82b
N/A
N/A
N/A
N/A
N/A
N/A
18.11
15.15
28.93
46.38b
90.09c
226.90b, c
58.08b
43.27c
24.80b, c
N/A
N/A
N/A
N/A
N/A
N/A
163.27d
28.55d
87.94d
Note: Means having the same superscript denotes p < 0.05
N/A
N/A
N/A
N/A
N/A
N/A
2.23
2.70
2.70
N/A
N/A
N/A
N/A
N/A
N/A
4.40
4.98
4.87
28.07
37.42
38.90
4.11
5.10
5.10
N/A
N/A
N/A
N/A
N/A
N/A
24.37
15.15
19.42
Laboratory Score
M
69.99a
90.65a
76.51
76.01
85.00
68.39
84.97
77.83
83.26
79.14
68.70
82.80
65.40b
71.30
87.64b
70.03
79.77
67.72
81.13
81.00
69.64
84.52b
76.59c
60.27b, c
72.89
73.48
56.43
59.12
N/A
55.39
68.89
55.00
55.18
SD
3.91
5.35
5.35
2.84
3.50
3.50
2.38
2.89
2.89
3.92
5.36
5.36
4.27
5.84
5.84
3.88
4.39
4.29
3.05
4.06
3.79
2.99
3.71
3.71
4.91
6.71
6.71
5.36
N/A
7.32
9.65
6.00
7.69
38
are significantly different (p < 0.01). The results from Experiment 9 state that TA
1 and 3 are significantly different (p < 0.01), and TA 2 and 3 are significantly
different (p < 0.01).
Impact of Laboratory Meeting Time on Student
Performance
In order to determine if the time of day has an impact on student
performance, a MANOVA was performed on data set A (8 am, n = 22; 6 pm, n =
22). The day and instructor were the same for this analysis, and as previously
stated, the pre-laboratory assessment type was not included since this did not yield
reliable results. The statistics were not calculated when the pre-laboratory
assessment was included, so it was left out of this analysis.
Upon reviewing Table 11, 7 of the 11 experiments show a significant
difference in the pre-laboratory assessment scores between the time of day the
sections were held (Appendix I). It should be noted that the higher mean score was
from the pre-laboratory quizzes.
The main interest of analyzing the time was to determine if there was an
impact on the percent error and laboratory score as a means of measuring student
performance. Upon reviewing these two dependent variables on Table 11, there
were significant differences for Experiment 9 (F1, 32= 16.59, p < 0.01) and 12 (F1,
35=
7.33, p <0.02) when looking at percent error, and only Experiment 1 (F1, 41 =
4.63, p < 0.04) showed a significant difference when looking at the laboratory
score.
Teaching Assistant Observations
A total of three TAs participated in this study over the duration of two
semesters; one TA participated during the spring of 2011 and three in the fall of
39
Table 11. Means and Standard Deviations for Pre-Laboratory Score, Percent Error,
and Laboratory Score by Time for Each Experiment Using Data Set A
Pre-Laboratory Score
Experiment
Time
1
8 am
6 pm
8 am
6 pm
8 am
6 pm
8 am
6 pm
8 am
6 pm
8 am
6 pm
8 am
6 pm
8 am
6 pm
8 am
6 pm
8 am
6 pm
8 am
6 pm
2
3
4
5
6
8
9
12
13
15
* denotes p < 0.05
M
38.86*
72.73*
93.33*
54.5*
63.50*
90.00*
67.62
52.05
47.62
62.73
80.00
59.58
87.50*
54.16*
64.88*
78.89*
71.77*
47.04*
77.65*
46.18*
61.72
60.00
SD
7.12
6.96
4.92
4.16
3.88
3.66
6.35
6.21
6.20
6.06
7.53
8.96
4.07
3.64
5.01
4.72
7.98
7.35
6.59
8.20
9.85
11.02
Percent Error
M
N/A
N/A
40.21
63.13
6.65
5.06
N/A
N/A
N/A
N/A
16.51
19.72
63.99
28.78
74.89*
41.26*
82.54*
15.89*
45.42
57.30
228.68
97.86
SD
N/A
N/A
9.79
8.27
1.01
0.95
N/A
N/A
N/A
N/A
5.16
6.14
14.28
12.77
6.01
5.66
18.10
16.69
17.02
21.15
72.68
81.26
Laboratory Score
M
59.46*
80.52*
75.44
76.57
84.30
85.65
79.99
78.30
65.54
65.25
71.71
68.35
81.25
81.00
83.02
86.02
87.32
82.60
80.65
76.94
73.89
63.89
SD
7.00
6.84
4.01
3.39
2.19
2.07
6.67
6.52
6.62
6.47
2.94
3.51
3.18
2.84
2.49
2.35
3.53
3.25
1.89
2.36
5.46
6.10
40
2011. Feedback from these TAs was collected at the end of the semesters and
through informal discussions throughout the semester. The i>clicker software
utilized in this study was new to all three participating TAs, and brief training was
needed to verify that they could administer the quizzes without the assistance of
the researcher. Upon reviewing the pre-laboratory quiz scores, user errors occurred
between all three TAs. These user errors can occur by starting and stopping the
timer too quickly, resulting in not enough time for all students to answer the
question correctly. When this occurred, students had the opportunity to see the
question again and had a longer period of time to decide which choice is the
correct one. For this reason, the researcher deleted the duplicate questions when
reviewing the quiz results. In addition, the students that initially answered the
questions did not change their response, and those who did not respond were given
an opportunity to submit their answer.
The use of the i>clicker quiz impacted the amount of time required to grade
the pre-laboratory activity. A TA will usually spend approximately 30 minutes to
an hour, or 10-20 minutes of class time, to grade the pre-laboratory worksheets,
depending on the depth of the questions. The pre-laboratory quizzes can be graded
in 5 to 10 minutes. This time reduction can greatly help the TA focus on grading
other aspects of the experiment, or allow for more in-class time for students to
work on the experiment.
At the conclusion of this study, a survey was given to each TA to determine
how they checked, collected, and graded the pre-laboratory worksheets. Upon
reviewing the surveys, there is a mixed review as to which pre-laboratory
assessment was more beneficial to student learning. When asked their opinion on
which pre-laboratory assessment was more effective to student learning, one TA
stated that “the written worksheets seemed more effective because as I was
41
walking around grading them, I would have them look up a problem they got
wrong.” One of the other TAs stated that “the quizzes seemed to be more effective
because you can see where students are struggling. You can also tailor your
introduction to better explain concepts or steps of the procedure.” Other comments
that were made are to add quiz questions that required students to choose an
explanation as to why something happens, and also to have questions to verify that
students did preliminary calculations needed for some experiments.
Student Observations
Students that participated in this study during the spring 2011 semester
were given a survey asking for their opinion on which of the two methods helped
them better prepare for an experiment. A total of 38 students completed this
survey (Table 12). After reviewing the student responses, 36.8% stated they prefer
“quizzes, because you have to thoroughly read the intro to the lab instead of just
scanning for answers to the pre-lab assignment (worksheets),” 5.3% preferred the
worksheets because they “were most effective than the quizzes,” 13.2% liked
“both pre-lab worksheet and quiz” or “liked the pre-labs” in general, and 15.8%
left the question blank. The remaining 28.9% had other suggestions, such as
“discussing the next lab the class before doing the lab,” “the lab instructor to fully
explain the concepts to maximize understanding,” or “it is fine the way it is.”
From these results, the students seemed to enjoy taking the pre-laboratory quizzes
over completing the written worksheets.
Throughout the semesters, the students would express their feelings
towards the type of pre-laboratory assessment assigned to them. In the classes that
were assigned the pre-laboratory worksheet for the entire semester, students were
asking why they were not able to take the i>clicker quizzes, expressing that they
42
Table 12. Student Responses To Survey Regarding Pre-Laboratory Assessments
(N = 38)
Response
n
Percent
Prefer Quiz
14
36.8
Prefer Worksheet
2
5.3
Liked Pre-Laboratory in General
5
13.2
Unrelated Comments
11
28.9
No Comments
6
15.8
would rather do that instead of the worksheets. Another TA had said that some
students “didn’t like to bring the i>clicker to class. They liked the worksheets
better but that may’ve been because they could work in groups/pairs before class.”
The use of quizzes required students to work independently and have a broad
understanding of the experiment they were to perform on a particular day. The
written worksheets often lead to students working with others prior to beginning
class and before they are checked or turned into the TA, as has been observed by
participating TAs as well as others.
CONCLUSION
The main purpose of this pilot study was to determine if implementation of
a pre-laboratory quiz as an alternative to a pre-laboratory worksheet has a positive
impact on the level of student preparation as evaluated by student performance and
grades and using survey feedback. With the data collected, the impact of time and
instructor grading and introduction style on student performance was also
analyzed. In order to collect the required information, a total of six CHEM 1A
laboratory sections participated, ranging from 21 to 24 students per section. A
question bank was developed for each experiment performed to create the
i>clicker quizzes. Upon reviewing the results, it appears that there is no evidence
to prove or disprove the hypothesis that pre-laboratory quizzes positively increase
student performance in the laboratory. Though this was the main purpose of this
study, there were differences in Teaching Assistant (TA) workload, objectivity,
and consistency in grading; each of these is an outcome that is worth mentioning.
A discussion of the statistical analyses and outcomes follows, including
limitations in the experimental design, the implications of this study, and
suggestions for future studies performed in this non-traditional area of interest in
Chemistry.
Discussion
Impact of Pre-Laboratory Assessment
on Student Performance
The impact of the pre-laboratory assessment on student performance was
analyzed using two sets of data: the first was all of the data collected from all six
laboratory sections, and the second was using just Data Set B in an attempt to
control the independent variables of TA and time of day. Upon reviewing Data Set
44
B, there was a significant difference shown in pre-laboratory assessment scores
when looking at the pre-laboratory scores. This was to be expected since the
grading of the two pre-laboratory activities differed because pre-laboratory
quizzes were objectively graded using the i>clicker software and the quizzes were
subjectively graded using a 5-point rubric.
The results of the MANOVA for Data Set B revealed that there was no
significant difference in the percent error values based on the independent variable
of pre-laboratory assessment type. In reviewing the calculated mean values, there
are three experiments with large differences that appeared to be significantly
different. The raw data were analyzed to determine why this was not the case. In
the three experiments where this occurred, the majority of the percent error values
were very similar. The reason why the mean differs is because there were four
data entries that had much higher values, causing the mean to be higher. These
four entries were not enough to cause the two assessment types to be significantly
different.
In looking at the laboratory scores, there was only one experiment that
resulted in a significant difference in scores for Data Set B. The analysis of the
raw data showed that the scores for all of the remaining experiments were very
similar. The reason that there was only one experiment resulting in a significant
difference was that three students in the pre-laboratory quiz section earned very
low scores, while the remaining scores were slightly different. These three low
scores were enough to cause a significant difference between the pre-laboratory
quiz and worksheet sections.
The results of the pre- and post-test given to the students in Data Set B
show that students did not learn information, which was not the expected outcome.
The results show that students had essentially “un-learned” information. Upon
45
further investigation of the framework of the General Chemistry course and the
type of questions comprising the test, it was concluded that the course
unintentionally encourages students to learn how to solve problems
algorithmically or numerically rather than conceptually. Studies have shown that
students taught in this manner are unable to solve conceptual questions, but are
able to answer questions with an algorithmic approach.44-46 The results of this
analysis seem consistent with previous studies on this topic.
Impact of Teaching Assistant on
Student Performance
The impact of TAs on student performance was analyzed using two sets of
data: the first being all of the data collected from all six laboratory sections, and
the second only using Data Sets A and C since the four sections were assigned
alternating pre-laboratory assessments. Upon reviewing these two sets of data,
many similarities were observed, but the results from Data Sets A and C will be
discussed in detail. It was assumed that the laboratory scores for TA 1 and 3 will
be similar since they had thoroughly graded the laboratories by assigning a point
value to each question resulting in differing point totals for all experiments.
The percent error indicated that there were four experiments that resulted in
significant differences in the TAs. Experiment 3 showed that TA 1 and TA 3 were
significantly different since TA 3 had very high values compared to TA 1.
Experiment 8 showed that TA 1 and 3 were significantly different, as were TA 2
and 3. Further investigation of these occurrences shows that TA 3 had much
higher percent error values than the remaining two TAs. Experiment 9 resulted in
the same outcomes for the same reasons. For both experiments, TA 1 and 2 were
similar to each other. Experiment 15 resulted in TA 1 differing to both TA 2 and 3.
Upon reviewing the raw data, it was apparent that many of the students under TA
46
1 did not understand how to perform the calculations correctly, resulting in high
percent error values.
The laboratory score indicated that three of the experiments resulted in
significant differences in the TAs. When looking at Experiment 1, it should be
noted that the percent error values were not included in the statistical analysis
since these data were not collected for all students. The analysis resulted in a
significant difference between TA 1 and 2 because a few students under TA 1 did
not turn in their laboratory work. Experiment 5 resulted in a significant difference
between TA 1 and 3. TA 2 had a broad range of laboratory scores, allowing there
to not be a difference between them and the others. Experiment 9 resulted in TA 3
differing from TA 1 and 2. The students under TA 3 had higher scores than those
under the remaining TAs.
The results from these analyses do not conform to the hypothesis
previously stated. Though TA 1 and 3 had a similar grading style, the differing
laboratory scores may indicate that students did not complete the assignment
correctly or completely. It is difficult to determine where this is apparent without
seeing the students’ work. As for the percent error values, there are many
unknown factors that may significantly contribute to error, such as contamination
of reagents, changes in room temperature or humidity, or the student not allowing
their product to completely dry.
Impact of Laboratory Meeting Time
on Student Performance
When completing a MANOVA on the complete data set, the laboratory
meeting time could not be incorporated into the analysis. For this reason, Data Set
C was used for this analysis since it was assumed through the initial analysis of the
pre-laboratory assessment type that there was no significant difference between
47
worksheets and quiz performance. The independent variables of TA and Day were
also controlled using Data Set A.
The dependent variables of Percent Error and Laboratory Score were
analyzed. In this analysis, there were two experiments that had significantly
different percent error values, and only one experiment resulted in significantly
different laboratory scores. Upon analysis of the percent error means for each
experiment, the values varied between the two times. This leads to the indication
that the time of day students are present in class does not have a significant impact
on the results of an experiment. The same can be said about the laboratory scores.
Teaching Assistant Observations
One of the main advantages for TAs of giving quizzes rather than
worksheets for TAs is the reduction of the grading load. TAs have a limited
amount of time to concentrate on the teaching aspect of graduate school and
struggle to balance that with research and graduate coursework. The use of quizzes
helps to shift that time away from grading activities to providing better student
feedback on other graded assignments. The amount of time to administer the prelaboratory quizzes is approximately five minutes. The time needed for grading the
pre-laboratory worksheets can take anywhere from 30 minutes to an hour,
depending on the depth and number of questions, as well as the number of
students present in the laboratory section. The time shifts from grading to the
preparation of the quiz questions, which can be significantly lessened once a
comprehensive question bank is developed.
Once such a test bank is developed, there are many additional advantages
that make quizzes preferable. Though there may be a limitation of question writing
and the consistency of questions between laboratory sections, the use of a test
48
bank with questions of a similar difficulty can prove helpful in assigning differing
questions to the various laboratory sections. This will also help to restrict cheating
between sections, though cheating between students is the responsibility of the
TA. While there may be a lot of time involved in building a question bank, the
long term realizations that were the primary purpose are understood provided that
a question bank is completed upfront.
The final advantage established is regarding pre-laboratory lectures. TAs
have found that pre-laboratory quizzes provide feedback prior to giving the prelaboratory lectures. This benefits the students because the TA can cover the
concepts or procedure that students are struggling with by considering the class as
a whole rather than singling out a small selection of students. If the TA notices
many of the students answered with the same incorrect response, they can explain
the reason why that choice is wrong and correct any misconceptions without
identifying a particular student. When students complete the worksheet, they may
not want to ask questions or tell the TA they do not understand the instructions. As
a result, the students may not perform the experiment correctly.
Student Observations
In conversations with the students involved in this pilot study, they have
expressed more positive feelings towards the pre-laboratory quizzes than the more
traditional worksheets. Students enjoy having more time for other activities rather
than completing a worksheet for the experiment they would be working on. Since
the students are simply reading through the laboratory manual to understand the
experiment, they feel as though it is not much of a chore to complete the
assignment. Many students also expressed that the feedback they receive during
the quizzes help them understand the concepts covered in the experiment. Students
49
are also required to understand a broader portion of the material because they do
not know what they will be tested on.
From a student perspective, the advantages are geared towards a more
positive learning experience. The pre-laboratory quizzes encourage students to
actually read the background information for each of the experiments performed,
rather than skimming through the text to find the correct answer. Students have
also stated they prefer a pre-laboratory quiz over completing a written worksheet.
This is partially due to the lack of enjoyment in doing the work on the worksheet
relative to the quiz. It is to some degree to the perception that they are learning
while working on the worksheet rather than the opposite.
Limitations
Though there are several advantages to this study, limitations in the
experimental design have surfaced including: (1) the use of indirect measures of
student preparation, (2) uncontrolled variables due to lack of control in selection
of participants, (3) challenges related to technology integration, and (4) flaws in
metric structuring.
Participant groups were not controlled for lecture or laboratory instructor,
student academic ability, or student major and year. This limits conclusions of the
study as these uncontrolled variables may result in differing performance levels
between groups. Although the researcher analyzed data sets to control the
variables of day, time, or TA, the use of human subjects with varying prior
knowledge or the ability to perform experiments could mask results and
differences that would be significant with a better controlled experimental design.
This study was a preliminary attempt at integrating technology into a
laboratory setting. Significant improvements in the quiz structure and
50
implementation can potentially be made in future semesters. These improvements
may include better questions, more polished delivery due to better TA training,
better communication of expectations to students, and greater acceptance of the
format by participating students. Along these same lines, questions can be tested to
evaluate how well the measurement student preparedness works. It is possible that
once refined, many differences would have been observed in these analyses.
Each of the quantitative metrics used in the study revealed distinct flaws.
These flaws could be corrected by the standardization of each metric. Each TA
grades student work differently, placing emphasis on various aspects of the
laboratory write-up. In order to obtain more reliable experiment scores, a
standardized grading routine should be developed that would make comparison of
grade more meaningful. Within this routine, a new component should be added
where needed to allow a quantitative measure of the level of student performance
on the experiment. In addition, value collected from each student should be the
amount of deviation from the actual purity of the unknown substance used rather
than calculating the percent error. With the alteration of these two metrics, it is
conceivable that these metrics would allow observation of a significant difference
based on the pre-laboratory assessment tool.
The metrics used in this study were indirect measures of student
preparation. The lack of statistically significant differences could be due to the
quizzes not affecting student preparation, due to student preparation not affecting
student performance, or both. The lack of information on how these metrics are
affected by student preparation limits the conclusions that can be drawn regarding
how pre-laboratory quizzes affect student preparation. In order to correct this, a
more normalized means of pre-laboratory worksheet grading would be needed in
order to have less subjective grading of student responses occurring.
51
Future Studies
After conducting this study, it is recommended that pre-laboratory quizzes
be used instead of pre-laboratory worksheets based qualitative advantages on TA
time and student performance. It should be noted that no statistically significant
differences were observed in this study. In order to gain more of a significant
insight and to see a better comparison of the two pre-laboratory assessment
methods, it is suggested that the following ideas be considered for future studies
on this topic:
1. A larger sample size is needed in order to determine the full scope of
how student performance changes once the pre-laboratory type is
changed. For this, more Teaching Assistants (TAs) need to be involved
in data collection.
2. For this reason, a standardized means of laboratory introduction and
laboratory grading would need to be developed. This will ensure that
each section has the opportunity to become familiar with the same
information and each experiment is worth the same amount of points
between the sections.
3. It would also be beneficial to perform a question analysis on the quiz
questions used in this study. This could be used to determine what type
of question helps to facilitate student learning more effectively in the
laboratory.
4. Include a portion to determine whether the lecture instructor has an
impact on student performance in the laboratory.
Implications
Though this pilot study did not yield significant differences between the
pre-laboratory worksheets and pre-laboratory quizzes, the information that was
52
obtained has many implications. Many teachers and instructors develop a routine
in which they do not modify between semesters or school years. This is a
disadvantage to students since their entry level knowledge is not consistent
between semesters and years. It is important for instructors and teachers to
continually reflect on their strategies and develop different ways to reach types of
students. Although this study was conducted with college-level participants that
are held more accountable for their own learning, it is still important to have all
material available to each student.
Technology is advancing continuously and quickly. With these changes
comes the push for all teachers and instructors to utilize various forms of
technology traditionally not found in classrooms. If the inclusion of such resources
and tools is gradual and significant, it will be better accepted as a means of
instruction or assessment. It is with hope and confidence that this study does not
go without the attention of others to demonstrate techniques that can be
implemented into courses that do not traditionally use such means of instructional
direction or assessment.
REFERENCES
REFERENCES
1.
Johnstone, A. Why is science difficult to learn? Things are seldom what they
seem. J. Comp. Assist. Learn. 1991, 7(2), 75-83.
2.
Kurbanoglu, N. I.; Akin, A. The relationships between university students'
chemistry laboratory anxiety, attitudes, and self-efficacy beliefs. Aust. J.
Teach. Ed. 2010, 35(8), 48-59.
3.
Bunce, D. M.; Flens, E. A.; Neiles, K. Y. How long can students pay
attention in class? A study of student attention decline using clickers. J.
Chem. Educ. 2010, 87(12), 1438-1443.
4.
Freilich, M. B. Frequent quizzing, the final exam, and learning: is there a
correlation? J. Chem. Educ. 1989, 66(3), 219-23.
5.
Geist, J. R.; Soehren, S. E. The effect of frequent quizzes on short- and longterm academic performance. J. Dental Ed. 1997, 61(4), 339-45.
6.
Haberyan, K. A. Do weekly quizzes improve student performance on general
biology exams? American Biology Teacher 2003, 65(2), 110-14.
7.
Kee, N.; Matthews, S.; Perumalla, C. Lessons learned from an online human
physiology course. Proceedings of the International Conference on eLearning 2009, 247-254.
8.
Waite, J. Weekly quizzing really works!: Online software makes it easy.
Tech Directions 2007, 66, 6.
9.
Weinstein, S.; Wu, S. W. Readiness assessment tests vs. frequent quizzes:
student Preferences. Int. J. Teach. Learn. High. Ed. 2009, 21(2), 181-186.
10. McKenzie, G. R. Quizzes: tools or tyrants. Instructional Science 1973, 2(3),
281-293.
11. Fies, C.; Marshall, J. Classroom response systems: a review of the literature.
J. of Sci. Ed. & Tech. 2006, 15(1), 101-109.
12. Milner-Bolotin, M.; Antimirova, T.; Petrov, A. Clickers beyond the first-year
science classroom. J. Coll. Sci. Teach. 2010, 40(2), 14-18.
13. Rodrigues, S.; Taylor, N.; Cameron, M.; Syme-Smith, L.; Fortuna, C.
Questioning chemistry: the role of level, familiarity, language and taxonomy.
Sci. Ed. Inter. 2010, 21(1), 16.
55
14. Murphy, K. Using a personal response system to map cognitive efficiency
and gain insight into a proposed learning progression in preparatory
chemistry. J. Chem. Educ. 2012, 89(10), 1229-1235.
15. Graham, C. R.; Tripp, T. R.; Seawright, L.; Joeckel, G. Empowering or
compelling reluctant participators using audience response systems. Active
Learning in Higher Education 2007, 8(3), 233-258.
16. Ruder, S. M.; Straumanis, A. R. A method for writing open-ended curved
arrow notation questions for multiple-choice exams and electronic-response
systems. J. Chem. Educ.2009, 86(12), 5.
17. Sevian, H.; Robinson, W. E. Clickers promote learning in all kinds of
classes--small and large, graduate and undergraduate, lecture and lab. J. Coll.
Sci. Teach. 2011, 40(3), 14-18.
18. Patterson, B.; Kilpatrick, J.; Woebkenberg, E. Evidence for teaching practice:
The impact of clickers in a large classroom environment. Nurse Education
Today 2010, 30(7), 603-607.
19. Kenwright, K. Clickers in the classroom. TechTrends 2009, 53(1), 74-77.
20. Stowell, J. R.; Nelson, J. M. Benefits of electronic audience response systems
on student participation, learning, and emotion. Teach. Psych. 2007, 34(4),
253-258.
21. Barber, M.; Njus, D. Clicker evolution: seeking intelligent design. CBE-Life
Sciences Education 2007, 6(1), 1-8.
22. Tao, J.; Clark, J.; Gwyn, G.; Lim, D. Hand-held clickers vs. virtual clickers:
what do our students think? J. Inter. Instr. Devel. 2010, 21(4), 17-22.
23. Guse, D. M.; Zobitz, P. M. Validation of the audience response system. Brit.
J. Educ. Technol. 2011, 42(6), 985-991.
24. Kay, R.; Knaack, L. Exploring individual differences in attitudes toward
audience response systems. Canadian J. of Learn. and Tech. 2009, 35(1).
25. Woelk, K. Optimizing the use of personal response devices (clickers) in
large-enrollment introductory courses. J. Chem. Educ. 2008, 85(10), 1400.
26. Hovell, M. F.; Williams, R. L.; Semb, G. Analysis of undergraduates'
attendance at class meetings with and without grade-related contingencies: A
contrast effect. J. Ed. Res. 1979, 73(1), 50-53.
56
27. Saltman, J. Mastering introductory sociology: A teaching innovation. Teach.
Sociol. 1988, 16(3), 284-288.
28. Nee, M. J. Book review of clickers in action: Increasing student participation
in general chemistry. J. Chem. Educ. 2010, 87(8), 784-789.
29. Kay, R. H.; LeSage, A. Examining the benefits and challenges of using
audience response systems: A review of the literature. Comput. Educ. 2009,
53(3), 819-827.
30. Vital, F. Creating a positive learning environment with the use of clickers in
a high school chemistry classroom. J. Chem. Educ. 2011, 89(4), 470-473.
31. Ophardt, C. E.; Applebee, M. S.; Losey, E. N. Chemical demonstrations as
the laboratory component in nonscience majors courses. An outreachtargeted approach. J. Chem. Educ. 2005, 82(8), 1174.
32. Bruck, L. B.; Bruck, A. D.; Phelps, A. J. “Gone” into solution: assessing the
effect of hands-on activity on students’ comprehension of solubility. J.
Chem. Educ. 2009, 87(1), 107-112.
33. Grannas, A. M.; Lagalante, A. F. So these numbers really mean something?
A role playing scenario-based approach to the undergraduate instrumental
analysis laboratory. J. Chem. Educ. 2010, 87(4), 416-418.
34. Rollnick, M.; Zwane, S.; Staskun, M.; Lotz, S.; Green, G. Improving prelaboratory preparation of first year university chemistry students. Int. J. Sci.
Educ. 2001, 23(10), 1053-1071.
35. Neber, H.; Anton, M. Promoting pre-experimental activities in high-school
chemistry: Focusing on the role of students' epistemic questions. Int. J. Sci.
Educ. 2008, 30(13), 1801-1821.
36. Pogacnik, L.; Cigic, B. How to motivate students to study before they enter
the lab. J. Chem. Educ. 2006, 83(7), 1094.
37. Lyle, K. S.; Robinson, W. R. An action research report: Improving prelaboratory preparation of first-year university chemistry students. J. Chem.
Educ. 2002, 79(6), 663.
38. Fleming, F. F. No small change: Simultaneously introducing cooperative
learning and microscale experiments in an organic lab course. J. Chem. Educ.
1995, 72(8), 719.
57
39. Rosenshine, B.; Meister, C. The use of scaffolds for teaching higher-level
cognitive strategies. Educ. Leadership 1992, 49(7), 26.
40. Reigosa, C.; Jiménez-Aleixandre, M.-P. Scaffolded problem-solving in the
physics and chemistry laboratory: difficulties hindering students' assumption
of responsibility. Int. J. Sci. Educ. 2007, 29(3), 307-329.
41. Baddock, M.; Bucat, R. Effectiveness of a classroom chemistry
demonstration using the cognitive conflict strategy. Int. J. Sci. Educ. 2008,
30(8), 1115-1128.
42. Chittleborough, G. D.; Treagust, D. F.; Mocerino, M. Achieving greater
feedback and flexibility using online pre-laboratory exercises with non-major
chemistry students. J. Chem. Educ. 2007, 84(5), 884.
43. Journal of Chemical Education, O. Conceptual Questions (CQs): Chemical
Concepts Inventory. http://www.jce.divched.org/JCEDLib/QBank/collection/
CQandChP/CQs/ConceptsInventory/Concepts_Inventory.html. (accessed
Dec. 20, 2010).
44. Sawrey, B. A. Concept learning versus problem solving: Revisited. J. Chem.
Educ. 1990, 67(3), 253.
45. Pickering, M. Further studies on concept learning versus problem solving. J.
Chem. Educ. 1990, 67(3), 254.
46. Pinarbasi, T.; Canpolat, N. Students' understanding of solution chemistry
concepts. J. Chem. Educ. 2003, 80(11), 1328.
APPENDICES
APPENDIX A: PRE-LABORATORY WORKSHEETS
60
EXPERIMENT 1
Pre-Lab Assignment
Melting Points and Mixtures
Name ___________________________________ Date ______________________
1. Why must the Melt-Temp be set so that the temperature increases slowly?
2. Suppose a compound melts at 90°C. Using the heating curves for the Melt-Temp,
what is a good voltage setting to use when checking the melting point of this
substance?
3. Distinguish between a melting “range” and a melting “point.”
4. The procedure mentions a “melting range.” How do you determine the melting range?
5. Suppose you want to re-take the melting point of a sample. You note that the melted
sample has re-crystallized within the capillary tube. Can you use the same tube again
for your second determination?
61
EXPERIMENT 2
Pre-Lab Assignment
Mass and Volume Measurement
Name ___________________________________ Date ______________________
1. Define the terms “density” and “specific gravity.”
2. After reading this assignment, what procedure might you use to determine a liquid’s
density?
3. After reading this assignment, what procedure might you use to determine a metal’s
density?
4. What type of data should be plotted on the abscissa of a graph? What type of data
should be plotted on the ordinate of a graph
62
EXPERIMENT 3
Pre-Lab Assignment
Percent Water in a Hydrate
Name ___________________________________ Date ______________________
1. Writing requirement: Type a paragraph of 300 of your own words in which you
summarize the main objective, theory and procedure of this experiment.
2. Why is the empty crucible and lid heated prior to being weighed?
3. BRIEFLY outline the experimental steps in the experiment.
4. Why must the crucible be allowed to cool before weighing?
5. Why should you continuously move the Bunsen flame around the sides and bottom of
the crucible instead of just heating the bottom?
63
EXPERIMENT 4
Pre-Lab Assignment
Separating a Mixture, Recrystallization
Name ___________________________________ Date ______________________
1. Complete the following flowchart, then on a separate sheet of paper type a 300 word
summary, in your own words, that explains the rationale behind your flowchart:
what gets separated at each step, and why water is a useful solvent for this recrystallization.
Sand, salt, acetanilide
Obtain mass to 0.001 g
Mix with hot water,
then filter
Filtrate (liquid that passes
through filter)
Filtered material
Dry crystals
?
?
Filtrate
Filtered material
?
Dry Crystals
?
?
2. What are the characteristics of a solvent that are useful for recrystallization?
3. Suppose your sample was 26.2% acetanilide and the sample mass was 5.231 g.
Considering acetanilide’s cold water solubility of 0.0054 g/mL, what is the maximum
mass of acetanilide you could collect if the sample was exposed to a total of 45 mL of
cold water?
4. Give two examples of each of the following:
a) A homogeneous mixture
of two substances (e.g.:
saltwater)
b) A heterogeneous mixture
of two substances (e.g.: oil
mixed with water)
c) A heterogeneous mixture
of a pure substance (ice
floating in water)
64
Experiment 5
Pre-Lab Assignment
Name ________________________________ Date _____________________________
1. Define electrolytes.
2. What is a double replacement reaction? Give an example of this.
3. When two solutions are mixed and bubbles occur instantly, what does that tell
you? Give possible chemicals for this to occur.
4. When two solutions are mixed and heat is being produced, what does that suggest
is happening? Give possible chemicals for this to occur.
65
EXPERIMENT 6
Pre-Lab Assignment
Determining the Empirical Formula of a Compound
Name ___________________________________ Date ______________________
1. Why is it important to scour the crucible with steel wool at the start of the
experiment?
2. Why must the heated crucible be cooled to near room temperature before weighing?
3. Should the Bunsen burner flame be kept heating in one area of the crucible? Explain
why or why not.
4. If the magnesium ribbon flares up, should you stare at it? Explain.
66
EXPERIMENT 8
Pre-Lab Assignment
Name ___________________________________ Date ______________________
1. Why is the filter paper in the Buchner funnel moistened before filtration?
2. Why is it wise to scrape the printing off the aluminum can?
3. Why must suspended solids be removed from the basic solution?
4. Why is the acidic solution cooled in an ice bath?
5. Why is an ethanol-water mixture, rather than pure water, used to wash the collected
alum crystals?
6. Why should the hood fans be on when you are dissolving the aluminum?
7. Write out the steps of the procedure on a separate sheet of paper.
67
EXPERIMENT 9
Pre-Lab Assignment
Gasometric Analysis of Peroxide Solution
Name ___________________________________ Date ______________________
1. On a separate sheet of paper, outline the procedure.
2. When working with gases, what units of temperature do we use?
3. What are STP conditions?
4. Why is the leveling bulb moved so that the water level in it will be equal to the water
level in the burett..
a. Before the reaction begins
b. During the reaction
c. After the reaction is complete
5. Suppose we consider a different reaction that produces carbon dioxide gas. Would
this procedure be appropriate to use?
Why or why not? Hint: Seltzer or Club Soda tells you what about CO2 and water?
68
EXPERIMENT 12
Pre-Lab Assignment
Determining the Heat of Reaction
Name ___________________________________ Date ______________________
1. Writing requirement: Type a paragraph of 300 of your own words in which you
summarize the main objective, theory and procedure of this experiment.
2. Distinguish between heat and temperature.
3. Why must the heat capacity of the calorimeter be determined?
4. Define endothermic and exothermic processes.
5. What is the largest source of error in this experiment?
6. Write out the procedure on a separate sheet of paper.
69
EXPERIMENT 13
Pre-Lab Assignment
Redox Titration
Name ___________________________________ Date ______________________
Note: Some of these questions are derived from “Introduction to Volumetric Analysis,”
in the Appendix.
1. Define the end-point, equivalence point, and titration error of a volumetric analysis.
2. Sometimes a bubble of air gets trapped in the nozzle of a buret. How should it be
removed and why should it be removed?
3. For each mole of KMnO4 that reacts in this experiment, how many moles of
H2C2O4·H2O react?
4. Why should KMnO4 be added very slowly at the beginning of the titration?
5. What is the endpoint of this titration?
70
EXPERIMENT 15
Pre-Lab Assignment
Analysis for Iron
Name ___________________________________ Date ______________________
(Use separate sheets for all Assignments and Reports)
1. The calculation to find the milligrams of ferrous ion, Fe2+, in 325 mg (0.325 g) of
anhydrous ferrous sulfate, FeSO4, is
55.85 g Fe2+
0.325 g FeSO4 x 151.85 g FeSO = 0.120 g Fe2+ or 120 mg Fe2+
4
325 mg of which hydrate of iron (II) sulfate below contains 65 mg Fe2+? This
conforms to the label of the tablet box: 65 mg of Fe2+ for every 325 mg “iron (II)
sulfate”
a. FeSO4·4H2O
b. FeSO4·5H2O
c. FeSO4·7H2O
d. FeSO4·9H2O
2. We’ll be making up standard solutions of Fe2+ from a stock solution that is 40 ppm
(parts per million, or 1 gram solute per 1 million grams solution) in Fe2+. Let’s review
how to calculate the ppm and molarity of the diluted standard solution for Known #1,
then you can do the calculations for unknowns 2-6.
Known #1 uses 5 mL of 40 ppm Fe2+ stock, and ends up dilutint it to a new total volume
of 50 ml. Because these are very dilute solutions, you may assume that these solutions
have the density of water, where 1.0 g = 1.0 mL. Therefore 40 ppm can be considered to
be 40 g Fe2+ in 1 x 106 mL solution (Get it? 40 g per 1 million grams). So:
_ 40 g Fe2+
5 mL x 1 x 106 mL
_ 0.0002 g Fe2+
50 mL
= 0.0002 g Fe2+ ultimately is diluted to 50 mL, so:
x 1 x 106 = 4.0 ppm
To calculate the molarity of this solution, we convert
_ 40 g Fe2+
1 x 106 mL
to
_? mol Fe2+
_ 40 g Fe2+
_ 1 mol Fe2+ x 1000 mL = 7.2 x 10-5 M
=
x
1 L solution
1 x 106 mL
55.85 g Fe2+
1L
71
In a similar way, calculate the ppm and molarity of each of the other knowns, #2-6, and
summarize in the table below.
Unknown Unknown Unknown Unknown Unknown Unknown
1
2
3
4
5
6
ppm
4 ppm
Molarity 7.2 x 105
M
3. Describe Beers’ Law and Lambert’s Law (See Appendix B at the back of this lab
manual).
4. If absorbance is related to % transmittance by the formula log = A, what is the
absorbance if the % T = 50.7%?
5. Circle the choices in the parentheses: transmittance varies (exponentially or directly)
with absorptivity, path length, and concentration, while the absorbance varies
(exponentially or directly) with these parameters (see Appendix B).
6. If the spectrophotometer reading is above ____ absorbance units, it is somewhat
inaccurate. What can be done to correct this? (see Appendix B).
7. You will be graphing concentration vs. absorbance. Review the instructions on
“Preparing a Graph” in Experiment 3, Mass and Volume Measurement. Which data
will be plotted on the x-axis? On the y-axis? What are the criteria for making a
useful and accurate graph? If you need further help, read Appendix C in the back of
this lab manual.
8. A blank table has been prepared to receive your data. Enter the column headings:
Sample ID, mL stock solution, ppm Fe2+, M Fe2+, absorbance, % transmittance. In the
first column, under “Sample ID,” the row headings should be “Known 1, Known 2,
(etc.)… Known 6, Iron supplement, and Unknown.
APPENDIX B: CLICKER QUESTIONS USED FOR EACH PRELABORATORY QUIZ
73
Experiment 1: Melting Points and Mixtures
1. The main idea of this lab is to:
A. Discover the trends of the melting point of a mixture of two pure
compounds.
B. Identify an unknown.
C. Both A and B
D. Neither
E. B only
2. When determining the melting range of a compound, it is __________ okay to use
the same capillary tube.
A. always
B. never
C. sometimes
D. mostly
3. You will identify your unknown solid using ___________.
A. Melting range
B. Melting point
C. The color of the solid
D. The unknown number
4. When should you wear your goggles?
A. Never.
B. Always.
C. Only when handling chemicals.
D. Only when handling glassware.
5. You want to identify an unknown solid sample. You had determined that the
melting range for the compound to be 146.4-150.6°C. What is the most probable
identity of your unknown solid sample?
A. Phenylurea (m.p. 147°C)
B. Cholesterol (m.p. 148°C)
C. Benzilic Acid (m.p. 150°C)
D. Adipic Acid (m.p. 152°C)
74
Experiment 2: Mass and Volume Measurement
1. Identify which of the following is in the correct order specified by the lab manual
A. Determining the best glassware for measuring density →Measure density of
B. unknown solution → Measure densities of known solution
C. Measure densities of known solution → Measure density of unknown solution
→ Determining the best glassware for measuring density
D. Measure density of unknown solution → Measure densities of known solution
→ Determining the best glassware for measuring density
E. Determining the best glassware for measuring density → Measure densities of
known solution → Measure density of unknown solution
2. Define the term ‘density’
A. Ratio of the mass of a given quantity of a substance by its volume
B. Ratio of the height of a given quantity of a substance by its volume
C. Ratio of the mass of a given quantity of a substance by its area
D. None of the above
3. What are the units for density of a liquid used in this lab?
A. g/cm3
B. cm3/mL
C. g/mL
D. L/mL
4. Why is graph been prepared for this experiment?
A. To identify a trend between two variables
B. To see a bad data point caused by error in technique or method
C. All of the above
D. None of the above
5. In general, what type of data should be plotted on the horizontal axis of the graph?
A. raw chemical data
B. maximized value
C. Dependent variables
D. Independent variables
75
Experiment 3: Percent Water in Hydrate
1. What is a crystalline solid that has water molecules embedded within the
crystal lattice?
A.
B.
C.
D.
Aqueous
Hydrate
Impurity
Solution
2. What is the item in the picture called?
A. Clay Jar
B. Crucible
C. Evaporating Dish
D. Mortar and Pestle
3. Why must you not touch the crucible with your hands after it has been fired?
A. It could be hot.
B. It is ok to touch it after firing.
C. It may take away some of the weight.
D. It may add on unwanted weight.
4. Why should the crucible be slowly heated?
A. To keep the crucible from cracking.
B. To keep the hydrate from spattering.
C. To prevent the crucible from oxidizing.
D. To prevent the hydrate from being over heated.
5. Which is the correct equation for calculating the percent of water in a hydrate?
A. Divide mass of hydrate by mass of water, multiply by 100
B. Divide mass of crucible by mass of hydrate, multiply by 100
C. Divide mass of water lost by mass of hydrate, multiply by 100
D. Add mass of water lost and hydrate
76
Experiment 4: Separating a Mixture, Recrystallization
1. A ___________ is a mixture that has visibly different components, each with
different properties.
A. Acetanilide
B. Heterogeneous mixture
C. Homogeneous mixture
D. Substance
2. What is the item in the following picture?
A. Buchner funnel
B. Capillary filter
C. Cold filter
D. Gravity funnel
3. Why does the initial amount of water used to dissolve the mixture matter?
A. A reaction will occur
B. Acetanilide is slightly soluble in cool water
C. It will take too long to evaporate
D. Too little water will cause impurities in the acetanilide
4. Which method is used to separate the acetanilide from the salt in solution?
A. Decanting
B. Evaporate solvent
C. Gravity filtration
D. Vacuum filtration
5. If you collect 2.746 g sand, 1.347 g acetanilide, and 0.875 g salt from a 5.000 g
sample, what is the percent sand?
A. 17.50 %
B. 26.94%
C. 45.08%
D. 54.92%
77
Experiment 5: Net Ionic Equations
1. What is double replacement reaction?
A. The water solvent is being replaced by non-aqueous solvent
B. Procedure in which acidity is replaced with basicity.
C. Reaction in which ions ‘switch partners’
D. All of the above
2. Define an electrolyte.
A. Compound whose water solution conduct an electric current
B. A solvent that can dissolve any substances
C. A type of compound that are not dissolved in water.
D. Instrument that measures for electric current
3. When two solutions are mixed and heat is being produced, what does that suggest
is happening?
A. Neutralization between acid and base
B. Formation of weak acid
C. All of the above
D. None of the above
4. What should you do if one of the unknown solutions gets spilled in your hand?
A. Neutralize using acid
B. Use fire extinguisher
C. Wash your hand using running water immediately
D. Wipe with paper towel. None of the chemical possess serious hazard.
5. When two solutions are mixed and you don’t see anything in 5 minutes, what
should you do?
A. Add the third solution
B. Cover the top of the test tube and shake it vigorously
C. Add additional DI water (make sure DI water is added)
D. None of the above
78
Experiment 6: Determining the Empirical Formula of a
Compound
1. What should the magnesium strip look like once the reaction is complete?
A. White ash.
B. Black ash.
C. White with a red tint.
D. No change.
2. Besides safety, what is the main reason why you shouldn’t touch the crucible?
A. The crucible may be hot.
B. Oils from your hands may alter the mass.
C. You may drop the crucible and its contents.
D. You might get dust particles from the air in your sample.
3. By knowing the amount of Oxygen that combined with the Magnesium, you can
use the __________ to determine the ratio of Magnesium to Oxygen
A. Molecular mass
B. Number of atoms
C. Amount of moles of each
D. The original mass of each
4. The simplest form of a chemical formula is called the ___________.
A. Molecular formula
B. Empirical formula
C. Atomic symbol
D. Elements
5. How many moles are in 3.48 grams of MgO? (Molar mass: 40.3g/mol)
A. 0.00864 mol
B. 0.0864 mol
C. 0.864 mol
D. Not enough information
79
Experiment 8: Alum from Scrap Aluminum
1. When heating solution containing 20mL 9M H2SO4 solution, what do you need to
do?
A. Heat it fast so that you can minimize the amount of H2SO4 mist being
produced.
B. Heat it at one spot where H2SO4 solid is being seen
C. Heat it gently and thoroughly to avoid spattering
D. Heat it no more than 3min.
2. Which of the following best describes the term ‘percent yield’?
A. (amount of product experimentally obtained / maximum amount of product
theoretically possible) x 100
B. (maximum amount of product theoretically possible / amount of product
experimentally obtained) x 100
C. (Absolute error / amount of product experimentally obtained) x 100
D. (amount of product experimentally obtained / absolute error) x 100
3. Why should the hood fans be on when you are dissolving the aluminum?
A. There will be mist of KOH solution being produced, which is very irritating.
B. Considerable amount of the Aluminum will be vaporized, which will be toxic.
C. Fans are on to not over heat the solution
D. All of the above
4. What is considered as your product in this experiment?
A. Aluminum metal
B. Aluminum Hydroxide
C. Alum
D. All of the above
5. How accurate is the lab manual is telling you to be when you are weighing sample
out?
A. 0.01g
B. 0.05g
C. 0.005g
D. None of the above
80
Experiment 9: Gasometric Analysis of Peroxide
Solution
1. PV=nRT is __________.
A. Avogadro’s Law
B. Boyle’s Law
C. Charles’ Law
D. Ideal Gas Law
2. Why must the bulb be level with the water level in the burette before, during, and
after the reaction?
A. To make sure there are no leaks in the apparatus
B. So that the water doesn’t spill out of the bulb
C. So that the pressure inside the apparatus is equal to room pressure
D. To make sure that reaction occurs
3. What gas is released in the reaction in this experiment?
A. CO (g)
B. CO2 (g)
C. H2O (g)
D. O2 (g)
4. What goes in the Erlenmeyer flask?
A. H2SO4
B. H2SO4 + Peroxide
C. KMnO4
D. KMnO4 + Peroxide
5. What is the percent H2O2 of the sample if 4.56 x 10-4 moles of O2 were collected
from a 1.00 g sample?
A. 0.821%
B. 1.46%
C. 1.51%
D. 1.55 %
81
Experiment 12: Determining the Heat of a Reaction
1. Which of the two chemicals are used to determine heat of neutralization?
A. HBr and HCl
B. NaOH and HBr
C. NaOH and HCl
D. HI and HBr
2. When stirring a solution using a thermometer, what is the safety issue you will be
concerned about?
A. Thermometer may be heated up very rapidly
B. Thermometer tip is fragile, so gentle stirring is required.
C. Mercury inside the thermometer may react with salt, resulting in toxic
compound
D. None of the above.
3. Which unit should be used to determine heat?
A. Celsius Scale
B. Fahrenheit Scale
C. Joules
D. Kg
4. Which is the correct equation for calculating heat, q?
A. (Specific heat) x (volume) x (temperature change)
B. (Specific heat) x (volume) x (density)
C. (Specific heat) x (mass) x (density)
D. (Specific heat) x (mass) x (temperature change)
5. For the HEAT CAPACITY OF THE CALORIMETER section, temperature of
the mixture should be collected at:
A. The highest temperature after mixing
B. The lowest temperature after mixing
C. 2min after the two liquids are being mixed
D. 5min after the two liquids are being mixed
82
Experiment 13: Practice Practical
1. Before starting the experiment, what do you need to calculate?
A. The mass of KMnO4 stock solution needed to make a 0.2 M solution
B. The volume of KMnO4 stock solution needed to make a 0.2 M solution
C. The mass of KOH stock solution needed to make a 0.2 M solution
D. The volume of KOH stock solution needed to make a 0.2 M solution
2. When titrating into your standard oxalic acid solution, it should be heated to
___________.
A. Boiling
B. Just warm to the touch
C. 70-80°C
D. 50-60°C
3. In this lab, we will use the calculated concentration of KMnO4 to determine
____________.
A. The percent oxalic acid in an unknown sample
B. The empirical formula of oxalic acid
C. The ppm of oxalic acid in an unknown sample
D. How many atoms there are in a mole
4. What can you use to convert from mL of KMnO4 to grams of oxalic acid?
A. Molarity
B. Molar mass
C. Mole ratio
D. All of the above
E. None of the above
5. Calculate the final molarity of a 250mL solution that used 20mL of 0.2M KMnO4.
A. 0.016M KMnO4
B. 0.002M KMnO4
C. 0.03M KMnO4
D. 0.19M KMnO4
83
Experiment 15: Analysis for Iron in a Vitamin Pill
1. The wavelength of the colorimeter should be set to ____________.
A. 100 µm
B. 209 nm
C. Any wavelength, as long as you record the value
D. 508 nm
2. What is the first thing you have to do when working with spectrophotometer?
A. Allow spectrophotometer to warm up.
B. Introduce the sample.
C. Introduce the blank.
D. Record the reading without introducing anything.
3. Ratio of the intensity of light entering the medium and the intensity of light
leaving the medium is defined as:
A. Absorbance
B. Colorimetric constant
C. Transmittance
D. None of the above
4. The piece of glassware that is used to hold the sample in the spectrophotometer is
known as:
A. Spectrophotometer tube
B. 1mL volumetric flask
C. 2mL round bottom flask
D. Cuvette
5. To make a standard curve, you have to make a graph x vs y. What is x and what is
y?
A. Concentration and absorbance
B. Concentration and density
C. Absorbance and Concentration
D. Density and Concentration
APPENDIX C: STUDENT SURVEY
85
Chem 1A Survey
Please answer the following questions in regards to the pre-lab worksheets and quizzes
that were assigned to you this semester. On the back, please make any comments with
ways to help improve this study.
1. What is your grade you expect to receive in this course?
2. I enjoyed doing the pre-lab quizzes.
1.
Strongly Agree
2.
Agree
3.
Neutral
4.
Disagree
5.
Strongly Disagree
3. The pre-lab quizzes helped prepare me better for the labs.
1.
Strongly Agree
2.
Agree
3.
Neutral
4.
Disagree
5.
Strongly Disagree
4.
Disagree
5.
Strongly Disagree
4. I enjoyed doing the pre-lab worksheets.
1.
Strongly Agree
2.
Agree
3.
Neutral
5. The pre-lab worksheets helped prepare me better for the labs.
1.
Strongly Agree
2.
Agree
3.
Neutral
4.
Disagree
6. What do you think is a better way to prepare?
5.
Strongly Disagree
APPENDIX D: TEACHING ASSISTANT SURVEY
87
TA Survey
1. When you check worksheets normally (aside from this study), do you check each
question or do you check to make sure the student has something written down for
each question?
2. Which do you think is more effective to student learning? Why?
3. Which do you prefer to administer to students? Why?
4. What would you change about the quizzes?
5. What would you change about the worksheets?
6. What were some comments about worksheets/quizzes from the students?
7. Your comments about quizzes and/or worksheets.
APPENDIX E: QUESTIONS USED FOR PRE- AND POST-TEST
89
This inventory consists of 22 multiple choice questions. Carefully consider each question
and indicate the one best answer for each. Several of the questions are paired. In these
cases, the first question asks about a chemical or physical effect. The second question
then asks for the reason for the observed effect.
1. Which of the following must be the same before and after a chemical reaction?
a.
b.
c.
d.
e.
The sum of the masses of all substances involved.
The number of molecules of all substances involved.
The number of atoms of each type involved.
Both (a) and (c) must be the same.
(e) Each of the answers (a), (b), and (c) must be the same.
2. Assume a beaker of pure water has been boiling for 30 minutes. What is in the bubbles
in the boiling water?
a.
b.
c.
d.
e.
Air.
Oxygen gas and hydrogen gas.
Oxygen.
Water vapor.
Heat.
3. A glass of cold milk sometimes forms a coat of water on the outside of the glass (Often
referred to as 'sweat'). How does most of the water get there?
a. Water evaporates from the milk and condenses on the outside of the glass.
b. The glass acts like a semi-permeable membrane and allows the water to pass, but
not the milk.
c. Water vapor condenses from the air.
d. The coldness causes oxygen and hydrogen from the air combine on the glass
forming water.
4. What is the mass of the solution when 1 pound of salt is dissolved in 20 pounds of
water?
a.
b.
c.
d.
e.
19 Pounds.
20 Pounds.
Between 20 and 21 pounds.
21 pounds.
More than 21 pounds.
90
5. The diagram represents a mixture of S atoms and O2 molecules in a closed container.
Which diagram shows the results after the mixture reacts as completely as possible
according to the equation:
2S + 3O2
2SO3
6. The circle on the left shows a magnified view of a very small portion of liquid water in
a closed container.
What would the magnified view show after the water evaporates?
91
7. True or False? When a match burns, some matter is destroyed.
a. True
b. False
8. What is the reason for your answer to question 7?
a.
b.
c.
d.
e.
This chemical reaction destroys matter.
Matter is consumed by the flame.
The mass of ash is less than the match it came from.
The atoms are not destroyed, they are only rearranged.
The match weighs less after burning.
9. Heat is given off when hydrogen burns in air according to the equation
2H2 + O2
2H2O
Which of the following is responsible for the heat?
a.
b.
c.
d.
e.
Breaking hydrogen bonds gives off energy.
Breaking oxygen bonds gives off energy.
Forming hydrogen-oxygen bonds gives off energy.
Both (a) and (b) are responsible.
(a), (b), and (c) are responsible.
10. Two ice cubes are floating in water:
After the ice melts, will the water level be:
a. higher?
b. lower?
c. the same?
92
11. What is the reason for your answer to question 10?
a.
b.
c.
d.
e.
The weight of water displaced is equal to the weight of the ice.
Water is more dense in its solid form (ice).
Water molecules displace more volume than ice molecules.
The water from the ice melting changes the water level.
When ice melts, its molecules expand.
12. A 1.0-gram sample of solid iodine is placed in a tube and the tube is sealed after all of
the air is removed. The tube and the solid iodine together weigh 27.0 grams.
The tube is then heated until all of the iodine evaporates and the tube is filled with iodine
gas. Will the weight after heating be:
a.
b.
c.
d.
e.
less than 26.0 grams.
26.0 grams.
27.0 grams.
28.0 grams.
more than 28.0 grams.
13. What is the reason for your answer to question 12?
a.
b.
c.
d.
e.
A gas weighs less than a solid.
Mass is conserved.
Iodine gas is less dense than solid iodine.
Gasses rise.
Iodine gas is lighter than air.
14. What is the approximate number of carbon atoms it would take placed next to each
other to make a line that would cross this dot:
a.
b.
c.
d.
4
200
30,000,000
6.02 x 1023
93
15. Figure 1 represents a 1.0 L solution of sugar dissolved in water. The dots in the
magnification circle represent the sugar molecules. In order to simplify the diagram, the
water molecules have not been shown.
Figure 1
Which response represents the view after 1.0 L of water was added (Figure 2).
Figure 2
16. 100 mL of water at 25°C and 100 mL of alcohol at 25°C are both heated at the same
rate under identical conditions. After 3 minutes the temperature of the alcohol is 50°C.
Two minutes later the temperature of the water is 50°C. Which liquid received more heat
as it warmed to 50°C?
a.
b.
c.
d.
The water.
The alcohol.
Both received the same amount of heat.
It is impossible to tell from the information given.
17. What is the reason for your answer to question 16?
a.
b.
c.
d.
e.
Water has a higher boiling point then the alcohol.
Water takes longer to change its temperature than the alcohol.
Both increased their temperatures 25°C.
Alcohol has a lower density and vapor pressure.
Alcohol has a higher specific heat so it heats faster.
94
18. Iron combines with oxygen and water from the air to form rust. If an iron nail were
allowed to rust completely, one should find that the rust weighs:
a.
b.
c.
d.
less than the nail it came from.
the same as the nail it came from.
more than the nail it came from.
It is impossible to predict.
19. What is the reason for your answer to question 18?
a.
b.
c.
d.
e.
Rusting makes the nail lighter.
Rust contains iron and oxygen.
The nail flakes away.
The iron from the nail is destroyed.
The flaky rust weighs less than iron.
20. Salt is added to water and the mixture is stirred until no more salt dissolves. The salt
that does not dissolve is allowed to settle out. What happens to the concentration of salt
in solution if water evaporates until the volume of the solution is half the original
volume? (Assume temperature remains constant.)
The concentration
a. increases.
b. decreases.
c. stays the same.
21. What is the reason for your answer to question 20?
a.
b.
c.
d.
There is the same amount of salt in less water.
More solid salt forms.
Salt does not evaporate and is left in solution.
There is less water.
95
22. Following is a list of properties of a sample of solid sulfur:
i.
ii.
iii.
iv.
Brittle, crystalline solid.
Melting point of 113oC.
Density of 2.1 g/cm3.
Combines with oxygen to form sulfur dioxide
Which, if any, of these properties would be the same for one single atom of sulfur
obtained from the sample?
a.
b.
c.
d.
e.
i and ii only.
iii and iv only.
iv only.
All of these properties would be the same.
None of these properties would be the same.
APPENDIX F: MANOVA RESULTS FROM ANALYSIS OF
COMPLETE DATA SET
97
Table 13. Summary for the MANOVA Results for Pre-Laboratory Score, Percent
Error, and Laboratory Score by Pre-Laboratory Assessment and Instructor for
Experiment 1
Source
Dependent
Variable
PreLabScore
Accuracy
LabScore
PreLabScore
PreLab
Accuracy
LabScore
PreLabScore
Error
Accuracy
LabScore
PreLabScore
Total
Accuracy
LabScore
* denotes p < 0.05
Instructor
SS
194.48
94.94
296.42
.00
.00
.00
5718.01
1565.45
8870.13
392100.00
2440.58
476453.81
df
1
1
1
0
0
0
61
61
61
64
64
64
MS
F
194.48
94.94
296.42
.
.
.
93.74
25.66
145.41
2.08
3.70
2.04
.
.
.
Partial Eta
Squared
p
.16
.06
.16
.
.
.
.033
.057
.032
.000
.000
.000
Table 14. Summary for the MANOVA Results for Pre-Laboratory Score, Percent
Error, and Laboratory Score by Pre-Laboratory Assessment and Instructor for
Experiment 2
Source
Dependent
Variable
PreLabScore
Instructor Accuracy
LabScore
PreLabScore
PreLab
Accuracy
LabScore
PreLabScore
Error
Accuracy
LabScore
PreLabScore
Total
Accuracy
LabScore
* denotes p < 0.05
SS
.00
.00
.00
18688.59
1763.57
105.19
17169.74
89157.22
11578.61
380900.00
198849.40
394092.99
df
MS
0
.
0
.
0
.
1 18688.59
1 1763.57
1
105.19
58
296.03
58 1537.19
58
199.63
60
60
60
F
.
.
.
63.13
1.15
.53
p
.
.
.
.000*
.29
.47
Partial Eta
Squared
.000
.000
.000
.521
.019
.009
98
Table 15. Summary for the MANOVA Results for Pre-Laboratory Score, Percent
Error, and Laboratory Score by Pre-Laboratory Assessment and Instructor for
Experiment 3
Dependent
Variable
PreLabScore
Instructor Accuracy
LabScore
PreLabScore
PreLab
Accuracy
LabScore
PreLabScore
Error
Accuracy
LabScore
PreLabScore
Total
Accuracy
LabScore
* denotes p < 0.05
Source
SS
669.95
1626.23
781.40
17905.62
287.57
61.37
33782.01
20034.58
21994.97
730436.00
35810.01
845249.91
df
MS
2
334.98
2
813.11
2
390.70
1 17905.62
1
287.57
1
61.37
116
291.22
116
172.71
116
189.61
120
120
120
F
1.15
4.71
2.06
61.48
1.67
.32
Sig.
.32
.011*
.13
.000*
.20
.57
Partial Eta
Squared
.019
.075
.034
.346
.014
.003
Table 16. Summary for the MANOVA Results for Pre-Laboratory Score, Percent
Error, and Laboratory Score by Pre-Laboratory Assessment and Instructor for
Experiment 4
Dependent
Variable
PreLabScore
Instructor Accuracy
LabScore
PreLabScore
Prelab
Accuracy
LabScore
PreLabScore
Error
Accuracy
LabScore
PreLabScore
Total
Accuracy
LabScore
* denotes p < 0.05
Source
SS
1762.72
.00
3074.91
7620.15
.00
155.87
103711.98
.00
82527.89
618300.00
.00
824751.54
df
2
2
2
1
1
1
129
129
129
133
133
133
MS
881.36
.00
1537.45
7620.15
.00
155.87
803.97
.00
639.75
F
Sig.
1.10
.
2.40
9.48
.
.24
.34
.
.09
.003*
.
.62
Partial Eta
Squared
.017
.
.036
.068
.
.002
99
Table 17. Summary for the MANOVA Results for Pre-Laboratory Score, Percent
Error, and Laboratory Score by Pre-Laboratory Assessment and Instructor for
Experiment 5
Source
Dependent
Variable
PreLabScore
Accuracy
LabScore
PreLabScore
PreLab
Accuracy
LabScore
PreLabScore
Error
Accuracy
LabScore
PreLabScore
Total
Accuracy
LabScore
* denotes p < 0.05
Instructor
SS
1762.72
.00
3074.91
7620.15
.00
155.87
103711.98
.00
82527.89
618300.00
.00
824751.54
df
2
2
2
1
1
1
129
129
129
133
133
133
MS
881.36
.00
1537.45
7620.15
.00
155.87
803.97
.00
639.75
F
p
1.10
.
2.40
9.48
.
.24
.34
.
.09
.003*
.
.62
Partial Eta
Squared
.017
.
.036
.068
.
.002
Table 18. Summary for the MANOVA Results for Pre-Laboratory Score, Percent
Error, and Laboratory Score by Pre-Laboratory Assessment and Instructor for
Experiment 6
Source
Dependent
Variable
PreLabScore
Instructor Accuracy
LabScore
PreLabScore
PreLab
Accuracy
LabScore
PreLabScore
Error
Accuracy
LabScore
PreLabScore
Total
Accuracy
LabScore
* denotes p < 0.05
SS
2918.35
2833.14
1752.06
9238.17
.27
179.56
78306.04
44071.63
46688.11
683350.00
83432.79
684727.53
df
2
2
2
1
1
1
109
109
109
113
113
113
MS
1459.17
1416.57
876.03
9238.17
.27
179.56
718.40
404.33
428.33
F
2.03
3.50
2.05
12.86
.00
.42
Sig.
.14
.034*
.13
.001*
.98
.52
Partial Eta
Squared
.036
.060
.036
.106
.000
.004
100
Table 19. Summary for the MANOVA Results for Pre-Laboratory Score, Percent
Error, and Laboratory Score by Pre-Laboratory Assessment and Instructor for
Experiment 8
Source
Dependent
Variable
PreLabScore
Instructor Accuracy
LabScore
PreLabScore
PreLab
Accuracy
LabScore
PreLabScore
Error
Accuracy
LabScore
PreLabScore
Total
Accuracy
LabScore
* denotes p < 0.05
SS
1182.65
443414.86
1990.10
7674.44
2343.00
1.43
42705.52
2145795.41
35063.21
494991.84
3643051.93
616553.52
df
MS
2
591.33
2 221707.43
2
995.05
1
7674.44
1
2343.10
1
1.43
90
474.51
90 23842.17
90
389.60
94
94
94
F
1.25
9.30
2.55
16.17
.10
.00
Sig.
.29
.000*
.08
.000*
.76
.95
Partial Eta
Squared
.027
.171
.054
.152
.001
.000
Table 20. Summary for the MANOVA Results for Pre-Laboratory Score, Percent
Error, and Laboratory Score by Pre-Laboratory Assessment and Instructor for
Experiment 9
Source
Dependent
Variable
PreLabScore
Instructor Accuracy
LabScore
PreLabScore
PreLab
Accuracy
LabScore
PreLabScore
Error
Accuracy
LabScore
PreLabScore
Total
Accuracy
LabScore
* denotes p < 0.05
SS
1108.66
3923.26
9412.89
3016.58
4588.99
172.61
44319.24
70734.16
27199.73
628678.56
276061.51
730618.26
df
2
2
2
1
1
1
107
107
107
111
111
111
MS
554.33
1961.63
4706.44
3016.58
4588.99
172.61
414.20
661.07
254.20
F
1.34
2.97
18.52
7.28
6.94
.68
p
.27
.06
.000*
.008*
.010*
.41
Partial Eta
Squared
.024
.053
.257
.064
.061
.006
101
Table 21. Summary for the MANOVA Results for Pre-Laboratory Score, Percent
Error, and Laboratory Score by Pre-Laboratory Assessment and Instructor for
Experiment 12
Source
Dependent
Variable
PreLabScore
Instructor Accuracy
LabScore
PreLabScore
PreLab
Accuracy
LabScore
PreLabScore
Error
Accuracy
LabScore
PreLabScore
Total
Accuracy
LabScore
* denotes p < 0.05
SS
268.45
185.72
6323.62
23147.38
26690.16
17.24
63669.59
397018.56
23602.72
515468.92
583038.08
640360.08
df
MS
1
1
1
1
1
1
94
94
94
97
97
97
268.45
185.72
6323.62
23147.38
26690.16
17.24
677.34
4223.60
251.09
F
.40
.04
25.18
34.17
6.32
.07
p
.53
.83
.000*
.000*
.014*
.79
Partial Eta
Squared
.004
.000
.211
.267
.063
.001
Table 22. Summary for the MANOVA Results for Pre-Laboratory Score, Percent
Error, and Laboratory Score by Pre-Laboratory Assessment and Instructor for
Experiment 13
Source
Dependent
Variable
PreLabScore
Instructor Accuracy
LabScore
PreLabScore
PreLab
Accuracy
LabScore
PreLabScore
Error
Accuracy
LabScore
PreLabScore
Total
Accuracy
LabScore
* denotes p < 0.05
SS
1.17
97802.69
2039.62
11015.46
638.15
213.42
48770.41
3895719.34
28037.87
409405.00
4538909.12
605181.89
df
1
1
1
1
1
1
80
80
80
83
83
83
MS
1.17
97802.70
2039.62
11015.46
638.15
213.42
609.63
48696.49
350.47
F
.00
2.01
5.82
18.07
.01
.61
p.
.97
.16
.018*
.000*
.91
.44
Partial Eta
Squared
.000
.024
.068
.184
.000
.008
102
Table 23. Summary for the MANOVA Results for Pre-Laboratory Score, Percent
Error, and Laboratory Score by Pre-Laboratory Assessment and Instructor for
Experiment 15
Source
Dependent
Variable
PreLabScore
Instructor Accuracy
LabScore
PreLabScore
PreLab
Accuracy
LabScore
PreLabScore
Error
Accuracy
LabScore
PreLabScore
Total
Accuracy
LabScore
* denotes p < 0.05
SS
150.88
36879.69
6181.09
143.38
15621.23
95.35
41940.83
399319.88
27702.78
340364.76
670958.17
430256.45
df
2
2
2
1
1
1
70
70
70
74
74
74
MS
75.44
18439.84
3090.54
143.38
15621.23
95.35
599.16
5704.57
395.75
F
p
.13
3.23
7.81
.24
2.74
.24
.88
.045*
.001*
.63
.10
.63
Partial Eta
Squared
.004
.085
.182
.003
.038
.003
APPENDIX G: MANOVA RESULTS FROM ANALYSIS OF
DATA SET B
104
Table 24. Summary for the MANOVA Results for Pre-Laboratory Score, Percent
Error, and Laboratory Score by Pre-Laboratory Assessment for Experiment 1
Source
Dependent
Variable
PLScore
ExptScore
PLScore
Error
ExptScore
PLScore
Total
ExptScore
* denotes p < 0.05
PreLab
SS
df
MS
7001.21
1946.33
14630.33
1
1
42
7001.21
1946.33
348.34
17451.63
298452.00
286114.29
42
44
44
415.52
F
20.10
4.68
Partial Eta
Squared
p
.000*
.036*
.324
.100
Table 25. Summary for the MANOVA Results for Pre-Laboratory Score, Percent
Error, and Laboratory Score by Pre-Laboratory Assessment for Experiment 2
Source
Dependent
Variable
PLScore
Error
ExptScore
PLScore
Error
Error
ExptScore
PLScore
Total
Error
ExptScore
* denotes p < 0.05
PreLab
SS
df
MS
F
Partial Eta
Squared
p
4218.75
1
4218.75
23.22
.000*
.514
3596.43
66.59
3996.88
1
1
22
3596.43
66.59
181.68
3.49
.69
.08
.41
.137
.031
22643.19
2112.38
179075.00
22
22
24
1029.24
96.02
42044.42
178814.95
24
24
105
Table 26. Summary for the MANOVA Results for Pre-Laboratory Score, Percent
Error, and Laboratory Score by Pre-Laboratory Assessment for Experiment 3
Source
Dependent
Variable
PLScore
Error
ExptScore
PLScore
Error
Error
ExptScore
PLScore
Total
Error
ExptScore
* denotes p < 0.05
PreLab
SS
12871.14
637.42
35.52
11713.34
6899.04
7480.58
257844.00
11833.90
300337.89
df
MS
1 12871.14
1
637.42
1
35.52
40
292.83
40
172.48
40
187.01
42
42
42
F
43.95
3.70
.19
p
.000*
.06
.67
Partial Eta
Squared
.524
.085
.005
Table 27. Summary for the MANOVA Results for Pre-Laboratory Score, Percent
Error, and Laboratory Score by Pre-Laboratory Assessment for Experiment 4
Source
Dependent
Variable
PLScore
ExptScore
PLScore
Error
ExptScore
PLScore
Total
ExptScore
* denotes p < 0.05
PreLab
SS
2970.00
664.13
21674.00
21124.72
210544.00
277425.42
df
1
1
42
42
44
44
MS
2970.00
664.13
516.05
502.97
F
p
5.76
1.32
.021*
.26
Partial Eta
Squared
.121
.030
106
Table 28. Summary for the MANOVA Results for Pre-Laboratory Score, Percent
Error, and Laboratory Score by Pre-Laboratory Assessment for Experiment 5
Source
Dependent
Variable
PLScore
ExptScore
PLScore
Error
ExptScore
PLScore
Total
ExptScore
* denotes p < 0.05
PreLab
SS
df
MS
F
p
Partial Eta
Squared
5200.30
1
5200.30
5.20
.028*
.110
378.97
42038.33
1
42
378.97
1000.91
1.31
.26
.030
12192.26
224150.00
42
44
290.29
280664.46
44
Table 29. Summary for the MANOVA Results for Pre-Laboratory Score, Percent
Error, and Laboratory Score by Pre-Laboratory Assessment for Experiment 6
Source
Dependent
Variable
PLScore
PreLab
Error
ExptScore
PLScore
Error
Error
ExptScore
PLScore
Total
Error
ExptScore
* denotes p < 0.05
SS
6584.40
54.03
125.29
8605.35
5299.73
15363.15
261600.00
11632.33
266084.55
df
1
1
1
37
37
37
39
39
39
MS
6584.40
54.03
125.29
232.58
143.24
415.22
F
28.31
.38
.30
p
.000*
.54
.59
Partial Eta
Squared
.433
.010
.008
107
Table 30. Summary for the MANOVA Results for Pre-Laboratory Score, Percent
Error, and Laboratory Score by Pre-Laboratory Assessment for Experiment 8
Source
Dependent
Variable
PLScore
ExptScore
PLScore
Error
ExptScore
PLScore
Total
ExptScore
* denotes p < 0.05
PreLab
SS
df
1527.58
30.30
19135.33
46002.76
239072.00
250578.51
MS
1
1
42
42
44
44
1527.58
30.30
455.60
1095.30
F
Partial Eta
Squared
p
3.35
.03
.07
.87
.074
.001
Table 31.Summary for the MANOVA Results for Pre-Laboratory Score, Percent
Error, and Laboratory Score by Pre-Laboratory Assessment for Experiment 9
Source
Dependent
Variable
PLScore
Error
ExptScore
PLScore
Error
Error
ExptScore
PLScore
Total
Error
ExptScore
* denotes p < 0.05
PreLab
SS
df
MS
F
Partial Eta
Squared
p
1331.26
1
1331.26
1.65
.21
.049
105.91
96.34
25858.86
1
1
32
105.91
96.34
808.09
.20
.28
.66
.60
.006
.009
16994.66
10967.01
212192.00
32
32
34
531.08
342.72
58671.27
255798.82
34
34
108
Table 32. Summary for the MANOVA Results for Pre-Laboratory Score, Percent
Error, and Laboratory Score by Pre-Laboratory Assessment for Experiment 12
Source
Dependent
Variable
PLScore
PreLab
Error
ExptScore
PLScore
Error
Error
ExptScore
PLScore
Total
Error
ExptScore
* denotes p < 0.05
SS
16196.14
1296.08
51.91
17012.83
77419.46
9433.29
278825.00
142402.65
280172.36
df
MS
1 16196.14
1 1296.08
1
51.91
37
459.81
37 2092.42
37
254.95
39
39
39
F
35.22
.62
.20
p
.000*
.44
.65
Partial Eta
Squared
.488
.016
.005
Table 33. Summary for the MANOVA Results for Pre-Laboratory Score, Percent
Error, and Laboratory Score by Pre-Laboratory Assessment for Experiment 13
Source
Dependent
Variable
PLScore
PreLab
Error
ExptScore
PLScore
Error
Error
ExptScore
PLScore
Total
Error
ExptScore
* denotes p < 0.05
SS
4881.38
3968.04
189.83
21682.05
1041296.84
10844.83
213625.00
1235016.85
336364.00
df
MS
1 4881.38
1 3968.04
1
189.83
37
586.00
37 28143.16
37
293.10
39
39
39
F
p
8.33
.14
.65
.006*
.71
.43
Partial Eta
Squared
.184
.004
.017
109
Table 34. Summary for the MANOVA Results for Pre-Laboratory Score, Percent
Error, and Laboratory Score by Pre-Laboratory Assessment for Experiment 15
Dependent
Variable
PLScore
PreLab
Error
ExptScore
PLScore
Error
Error
ExptScore
PLScore
Total
Error
ExptScore
* denotes p < 0.05
Source
SS
192.53
77.04
60.73
17661.53
5533.79
10757.36
154048.00
22447.96
239700.74
df
1
1
1
31
31
31
33
33
33
MS
192.53
77.04
60.73
569.73
178.51
347.01
F
p
.34
.43
.18
.57
.52
.68
Partial Eta
Squared
.011
.014
.006
APPENDIX H: MANOVA RESULTS FROM ANALYSIS OF
DATA SETS A AND C
111
Table 35. Summary for the MANOVA Results for Pre-Laboratory Score, Percent
Error, and Laboratory Score by Pre-Laboratory Assessment and Instructor for
Experiment 1
Source
Dependent
Variable
PLScore
ExptScore
PLScore
PreLab
ExptScore
PLScore
Error
ExptScore
PLScore
Total
ExptScore
* denotes p < 0.05
Instructor
SS
12649.98
4345.32
12325.58
4766.97
45937.98
55856.18
501296.00
596354.66
df
MS
2 6324.99
2 2172.66
1 12325.58
1 4766.97
85
540.45
85
657.13
89
89
F
11.70
3.31
22.81
7.25
p
.000*
.041*
.000*
.009*
Partial Eta
Squared
.216
.072
.212
.079
Table 36. Summary for the MANOVA Results for Pre-Laboratory Score, Percent
Error, and Laboratory Score by Pre-Laboratory Assessment and Instructor for
Experiment 2
Source
Dependent
Variable
PLScore
ExptScore
PLScore
PreLab
ExptScore
PLScore
Error
ExptScore
PLScore
Total
ExptScore
* denotes p < 0.05
Instructor
SS
6311.06
3230.92
4059.18
52.18
28068.29
15907.70
389700.00
472640.43
df
2
2
1
1
67
67
71
71
MS
3155.53
1615.46
4059.18
52.18
418.93
237.43
F
p
7.53
6.80
9.69
.22
.001*
.002*
.003*
.64
Partial Eta
Squared
.184
.169
.126
.003
112
Table 37. Summary for the MANOVA Results for Pre-Laboratory Score, Percent
Error, and Laboratory Score by Pre-Laboratory Assessment and Instructor for
Experiment 3
Source
Dependent
Variable
PLScore
Instructor Error
ExptScore
PLScore
PreLab
Error
ExptScore
PLScore
Error
Error
ExptScore
PLScore
Total
Error
ExptScore
* denotes p < 0.05
SS
df
323.88
1544.24
628.02
5948.47
21.41
15.38
21599.83
12745.20
14620.18
487216.00
23976.11
559119.79
2
2
2
1
1
1
76
76
76
80
80
80
MS
161.94
772.12
314.00
5948.47
21.41
15.38
284.21
167.70
192.37
F
.57
4.60
1.63
20.93
.13
.08
p
.57
.013*
.20
.000*
.72
.78
Partial Eta
Squared
.015
.108
.041
.216
.002
.001
Table 38. Summary for the MANOVA Results for Pre-Laboratory Score, Percent
Error, and Laboratory Score by Pre-Laboratory Assessment and Instructor for
Experiment 4
Source
Dependent
Variable
PLScore
ExptScore
PLScore
PreLab
ExptScore
PLScore
Error
ExptScore
PLScore
Total
ExptScore
* denotes p < 0.05
Instructor
SS
3401.42
1124.88
2605.86
30.42
52311.04
56149.18
436311.00
591618.38
df
2
2
1
1
85
85
89
89
MS
1700.71
562.44
2605.86
30.42
615.42
660.58
F
p
2.76
.85
4.23
.05
.07
.43
.043*
.83
Partial Eta
Squared
.061
.020
.047
.001
113
Table 39. Summary for the MANOVA Results for Pre-Laboratory Score, Percent
Error, and Laboratory Score by Pre-Laboratory Assessment and Instructor for
Experiment 5
Source
Dependent
Variable
PLScore
ExptScore
PLScore
PreLab
ExptScore
PLScore
Error
ExptScore
PLScore
Total
ExptScore
* denotes p < 0.05
Instructor
SS
2509.90
5773.19
2452.45
.92
60228.19
66622.38
394150.00
544087.07
df
2
2
1
1
85
85
89
89
MS
1254.95
2886.60
2452.45
.92
708.57
783.79
Partial Eta
Squared
F
p
1.77
3.68
3.46
.00
.18
.029*
.07
.97
.040
.080
.039
.000
Table 40. Summary for the MANOVA Results for Pre-Laboratory Score, Percent
Error, and Laboratory Score by Pre-Laboratory Assessment and Instructor for
Experiment 6
Source
Dependent
Variable
PLScore
Instructor Error
ExptScore
PLScore
PreLab
Error
ExptScore
PLScore
Error
Error
ExptScore
PLScore
Total
Error
ExptScore
* denotes p < 0.05
SS
976.92
1669.10
1168.29
2932.26
72.28
79.20
68374.84
38205.06
29689.83
421750.00
71800.46
418642.99
df
2
2
2
1
1
1
70
70
70
74
74
74
MS
488.46
834.55
584.14
2932.26
72.28
79.20
976.78
545.79
424.14
F
.50
1.53
1.38
3.00
.13
.19
Partial Eta
Squared
p
.61
.22
.26
.09
.72
.67
.014
.042
.038
.041
.002
.003
114
Table 41. Summary for the MANOVA Results for Pre-Laboratory Score, Percent
Error, and Laboratory Score by Pre-Laboratory Assessment and Instructor for
Experiment 8
Source
Dependent
Variable
PLScore
Instructor Error
ExptScore
PLScore
PreLab
Error
ExptScore
PLScore
Error
Error
ExptScore
PLScore
Total
Error
ExptScore
* denotes p < 0.05
SS
3157.54
421875.95
2484.80
9880.49
11026.38
.56
49674.60
2122056.70
41836.41
428707.84
3529346.90
504322.11
df
MS
2
1578.77
2 210937.98
2
1242.40
1
9880.49
1 11026.38
1
.56
78
636.85
78 27205.86
78
536.36
82
82
82
F
2.48
7.75
2.32
15.52
.41
.00
p
.09
.001*
.11
.000*
.53
.97
Partial Eta
Squared
.060
.166
.056
.166
.005
.000
Table 42. Summary for the MANOVA Results for Pre-Laboratory Score, Percent
Error, and Laboratory Score by Pre-Laboratory Assessment and Instructor for
Experiment 9
Dependent
Variable
PLScore
Instructor Error
ExptScore
PLScore
PreLab
Error
ExptScore
PLScore
Error
Error
ExptScore
PLScore
Total
Error
ExptScore
* denotes p < 0.05
Source
SS
1140.37
13269.35
7461.01
1663.53
9574.42
76.12
21571.93
44083.70
27988.56
420086.56
217390.24
474819.44
df
2
2
2
1
1
1
75
75
75
79
79
79
MS
570.19
6634.68
3730.50
1663.53
9574.42
76.12
287.63
587.78
373.18
F
1.98
11.29
10.00
5.78
16.29
.20
p
.15
.000*
.000*
.019*
.000*
.65
Partial Eta
Squared
.050
.231
.210
.072
.178
.003
115
Table 43. Summary for the MANOVA Results for Pre-Laboratory Score, Percent
Error, and Laboratory Score by Pre-Laboratory Assessment and Instructor for
Experiment 12
Source
Dependent
Variable
PLScore
ExptScore
PLScore
PreLab
ExptScore
PLScore
Error
ExptScore
PLScore
Total
ExptScore
* denotes p < 0.05
Instructor
SS
1070.98
4002.43
5404.56
208.97
96595.46
88031.48
428695.48
514437.72
df
2
2
1
1
85
85
89
89
MS
535.49
2001.21
5404.56
208.97
1136.42
1035.66
F
p
.47
1.93
4.76
.20
.63
.15
.032*
.65
Partial Eta
Squared
.011
.043
.053
.002
Table 44. Summary for the MANOVA Results for Pre-Laboratory Score, Percent
Error, and Laboratory Score by Pre-Laboratory Assessment and Instructor for
Experiment 13
Source
Dependent
Variable
PLScore
ExptScore
PLScore
PreLab
ExptScore
PLScore
Error
ExptScore
PLScore
Total
ExptScore
* denotes p < 0.05
Instructor
SS
151.94
67.23
13106.32
1637.37
56773.02
77736.02
236564.00
299498.20
df
MS
1
151.94
1
67.23
1 13106.34
1 1637.37
63
901.16
63 1233.91
66
66
F
.17
.05
14.54
1.33
p
.68
.82
.000*
.25
Partial Eta
Squared
.003
.001
.188
.021
116
Table 45. Summary for the MANOVA Results for Pre-Laboratory Score, Percent
Error, and Laboratory Score by Pre-Laboratory Assessment and Instructor for
Experiment 15
Source
Dependent
Variable
PLScore
Instructor Error
ExptScore
PLScore
PreLab
Error
ExptScore
PLScore
Error
Error
ExptScore
PLScore
Total
Error
ExptScore
* denotes p < 0.05
SS
589.33
164804.64
1701.43
6.57
38026.63
222.22
33886.61
221725.81
34730.26
196188.76
648510.22
190555.71
df
MS
2
294.67
2 82402.32
2
850.71
1
6.57
1 38026.63
1
222.22
42
806.82
42 5279.19
42
826.91
46
46
46
F
.37
15.61
1.03
.01
7.20
.27
p
.70
.000*
.37
.93
.010*
.61
Partial Eta
Squared
.017
.426
.047
.000
.146
.006
APPENDIX I: MANOVA RESULTS FROM ANALYSIS OF
DATA SET A
118
Table 46. Summary for the MANOVA Results for Pre-Laboratory Score, Percent
Error, and Laboratory Score by Time for Experiment 1
Source
Dependent
Variable
PLScore
ExptScore
PLScore
Error
ExptScore
PLScore
Total
ExptScore
* denotes p < 0.05
Time
SS
12325.58
4766.97
43640.94
42175.60
191712.00
259044.90
df
MS
1 12325.58
1 4766.97
41 1064.41
41 1028.67
43
43
F
11.58
4.63
Partial Eta
Squared
p
.002*
.037*
.220
.102
Table 47. Summary for the MANOVA Results for Pre-Laboratory Score, Percent
Error, and Laboratory Score by Time for Experiment 2
Source
Dependent
Variable
PLScore
Time
Error
ExptScore
PLScore
Error
Error
ExptScore
PLScore
Total
Error
ExptScore
* denotes p < 0.05
SS
13179.07
df
MS
1 13179.07
4596.50
11.21
12328.57
1
1
34
4596.50
11.21
362.61
48861.54
8208.35
205425.00
34
34
36
1437.10
241.42
156804.98
216706.57
36
36
36.35
.000*
Partial Eta
Squared
.517
3.20
.05
.08
.83
.086
.001
F
p
119
Table 48. Summary for the MANOVA Results for Pre-Laboratory Score, Percent
Error, and Laboratory Score by Time for Experiment 3
Source
Dependent
Variable
PLScore
Time
Error
ExptScore
PLScore
Error
Error
ExptScore
PLScore
Total
Error
ExptScore
* denotes p < 0.05
SS
df
5948.47
21.41
15.38
7716.00
524.36
2465.11
218032.00
1692.95
248211.45
1
1
1
32
32
32
34
34
34
MS
5948.47
21.41
15.38
241.13
16.39
77.04
F
24.67
1.31
.20
Partial Eta
Squared
p
.000*
.26
.66
.435
.039
.006
Table 49. Summary for the MANOVA Results for Pre-Laboratory Score, Percent
Error, and Laboratory Score by Time for Experiment 4
Source
Dependent
Variable
PLScore
ExptScore
PLScore
Error
ExptScore
PLScore
Total
ExptScore
* denotes p < 0.05
Time
SS
df
2605.86
30.42
34763.91
38328.14
190375.00
307566.53
1
1
41
41
43
43
MS
2605.86
30.42
847.90
934.83
F
Partial Eta
Squared
p
3.07
.03
.09
.86
.070
.001
Table 50. Summary for the MANOVA Results for Pre-Laboratory Score, Percent
Error, and Laboratory Score by Time for Experiment 5
Source
Dependent
Variable
PLScore
ExptScore
PLScore
Error
ExptScore
PLScore
Total
ExptScore
* denotes p < 0.05
Time
SS
2452.45
.92
33117.32
37725.35
167300.00
221594.22
df
1
1
41
41
43
43
MS
2452.45
.92
807.74
920.13
F
3.04
.00
Partial Eta
Squared
p
.09
.98
.069
.000
120
Table 51. Summary for the MANOVA Results for Pre-Laboratory Score, Percent
Error, and Laboratory Score by Time for Experiment 6
Source
Dependent
Variable
PLScore
Error
ExptScore
PLScore
Error
Error
ExptScore
PLScore
Total
Error
ExptScore
* denotes p < 0.05
Time
SS
df
MS
F
Partial Eta
Squared
p
2932.26
1
2932.26
3.04
.09
.101
72.28
79.20
26022.92
1
1
27
72.28
79.20
963.81
.16
.54
.69
.47
.006
.020
12215.75
3979.56
177425.00
27
27
29
452.44
147.39
21513.80
147461.74
29
29
Table 52. Summary for the MANOVA Results for Pre-Laboratory Score, Percent
Error, and Laboratory Score by Time for Experiment 8
Source
Dependent
Variable
PLScore
Error
ExptScore
PLScore
Error
Error
ExptScore
PLScore
Total
Error
ExptScore
* denotes p < 0.05
Time
SS
9880.49
11026.38
.56
9005.73
110955.72
5491.00
190171.84
193018.44
242336.00
df
MS
1 9880.49
1 11026.38
1
.56
34
264.87
34 3263.40
34
161.50
36
36
36
F
37.30
3.38
.00
p
.000*
.08
.95
Partial Eta
Squared
.523
.090
.000
121
Table 53. Summary for the MANOVA Results for Pre-Laboratory Score, Percent
Error, and Laboratory Score by Time for Experiment 9
Source
Dependent
Variable
PLScore
Time
Error
ExptScore
PLScore
Error
Error
ExptScore
PLScore
Total
Error
ExptScore
* denotes p < 0.05
SS
1663.53
9574.42
76.12
12828.09
18471.14
3179.77
192190.56
138843.93
246644.44
df
1
1
1
32
32
32
34
34
34
MS
1663.53
9574.42
76.12
400.88
577.22
99.37
F
4.15
16.59
.77
p
.05
.000*
.39
Partial Eta
Squared
.115
.341
.023
Table 54. Summary for the MANOVA Results for Pre-Laboratory Score, Percent
Error, and Laboratory Score by Time for Experiment 12
Source
Dependent
Variable
PLScore
Error
ExptScore
PLScore
Error
Error
ExptScore
PLScore
Total
Error
ExptScore
* denotes p < 0.05
Time
SS
5617.45
40827.02
204.03
37885.75
194878.55
7400.90
169693.92
315749.73
273479.82
df
MS
1 5617.45
1 40827.02
1
204.03
35 1082.45
35 5567.96
35
211.45
37
37
37
F
p
5.19
7.33
.97
.029*
.010*
.33
Partial Eta
Squared
.129
.173
.027
122
Table 55. Summary for the MANOVA Results for Pre-Laboratory Score, Percent
Error, and Laboratory Score by Time for Experiment 13
Source
Dependent
Variable
PLScore
Error
ExptScore
PLScore
Error
Error
ExptScore
PLScore
Total
Error
ExptScore
* denotes p < 0.05
Time
SS
6612.20
941.32
92.30
19213.52
127959.12
1586.18
145168.00
199134.80
177281.89
df
MS
1
1
1
26
26
26
28
28
28
6612.20
941.32
92.30
738.98
4921.50
61.01
Partial Eta
Squared
F
p
8.95
.19
1.51
.006*
.67
.23
.256
.007
.055
Table 56. Summary for the MANOVA Results for Pre-Laboratory Score, Percent
Error, and Laboratory Score by Time for Experiment 15
Source
Dependent
Variable
PLScore
Time
Error
ExptScore
PLScore
Error
Error
ExptScore
PLScore
Total
Error
ExptScore
* denotes p < 0.05
SS
df
MS
6.57
38026.63
222.22
3397.97
184865.83
1043.21
36844.76
1
6.57
1 38026.63
1
222.22
7
485.42
7 26409.40
7
149.03
9
484635.16
44668.21
9
9
F
.01
1.44
1.49
Partial Eta
Squared
p
.91
.27
.26
.002
.171
.176
California State University, Fresno
Non-Exclusive Distribution License
(to make your thesis/dissertation available electronically via the library’s eCollections database)
By submitting this license, you (the author or copyright holder) grant to CSU, Fresno
Digital Scholar the non-exclusive right to reproduce, translate (as defined in the next
paragraph), and/or distribute your submission (including the abstract) worldwide in print
and electronic format and in any medium, including but not limited to audio or video.
You agree that CSU, Fresno may, without changing the content, translate the submission
to any medium or format for the purpose of preservation.
You also agree that the submission is your original work, and that you have the right to
grant the rights contained in this license. You also represent that your submission does
not, to the best of your knowledge, infringe upon anyone’s copyright.
If the submission reproduces material for which you do not hold copyright and that would
not be considered fair use outside the copyright law, you represent that you have obtained
the unrestricted permission of the copyright owner to grant CSU, Fresno the rights
required by this license, and that such third-party material is clearly identified and
acknowledged within the text or content of the submission.
If the submission is based upon work that has been sponsored or supported by an agency
or organization other than California State University, Fresno, you represent that you
have fulfilled any right of review or other obligations required by such contract or
agreement.
California State University, Fresno will clearly identify your name as the author or owner
of the submission and will not make any alteration, other than as allowed by this license,
to your submission. By typing your name and date in the fields below, you indicate
your agreement to the terms of this distribution license.
Student Name Elizabeth Buchnoff
Type full name as it appears on submission
November 7, 2012
Date