Program/Discipline/Course Assessment Report
Discipline: Biology
Course Number: BIOL 224
School/Unit: SOSC
Submitted by: Steve C. Schenk
Contributing Faculty: Eddie Burke, Jamie Campbell, Jim Collier, Will Mehm, Pamela Sandstrom, Steve C. Schenk, James Verdi, Jeff Weinert,
Dan Williams, and Beate Wone
Academic Year: 2010-2011
Complete and submit your assessment report electronically to your Academic Dean. As needed, please attach supporting documents and/or a narrative description of the assessment activities in your
program or discipline.
Program, Discipline or
Course Outcomes
In the boxes below,
summarize the outcomes
assessed in your program
or discipline during the last
year.
Outcome #1:
Students shall acquire an
understanding of the
physiological function
and anatomical structure
of the cardiovascular,
respiratory, immune,
endocrine, urinary,
reproductive and
gastrointestinal systems
including their
interrelationships.
Assessment Measures
Assessment Results
Use of Results
Effect on Program, Discipline or Course
In the boxes below,
summarize the methods
used to assess program,
discipline, or course
outcomes during the last
year.
In the boxes below, summarize
the results of your assessment
activities during the last year.
In the boxes below, summarize
how you are or how you plan to
use the results to improve student
learning.
Based on the results of this assessment, will
you revise your outcomes? If so, please
summarize how and why in the boxes below.
Students were given
one of four 4-question
quizzes at the end of
the semester consisting
of short answer
questions derived from
past assessments. The
quizzes were used to
determine common
student misconceptions
which would serve as
the basis for building a
new assessment tool.
See attached narrative.
The majority of sections
reported results each semester
yielding a minimum of 40
student responses per
question. The most notable
topic areas indicating
deficiencies were:
• Endocrinology
• Hematology
• Immunology
• Respiratory
physiology
• Urinary physiology
These areas of weakness are
largely consistent with past
assessment results.
Data generated in this report
will be used to generate a new
pre-/post-test multiple-choice
assessment tool, and will serve
as the basis for future
refinement of said tool.
1. Greater attention will be paid to
course-to-course continuity of
sequence courses, particularly
consideration of BIOL 190 in BIOL
224.
2. By identifying student
misconceptions, data from this
assessment will yield more effective
short answer questions and better
future assessment tools. This will
allow us to better identify specific
student weaknesses, enabling us to
better respond to these weaknesses.
3. The short answer question approach
to data mining will be valuable for
continued use in refining future
assessment tools.
See attached narrative for details.
The results underscored
longstanding analyses that
suggest challenges in BIOL 224
are linked to poor knowledge
retention from BIOL 190.
While this has not been
traditionally considered in the
BIOL 224 assessment process,
it will be in the future.
See attached narrative for
details.
Outcomes are poorly constructed and will
be revised for the next academic year.
Program/Discipline/Course Assessment Report
Discipline: Biology
Course Number: BIOL 224
School/Unit: SOSC
Submitted by: Steve C. Schenk
Contributing Faculty: Eddie Burke, Jamie Campbell, Jim Collier, Will Mehm, Pamela Sandstrom, Steve C. Schenk, James Verdi, Jeff Weinert,
Dan Williams, and Beate Wone
Academic Year: 2010-2011
As above
Outcome #2:
Students shall acquire
the ability to apply
analytic thinking skills in
interpreting both
qualitative and
quantitative data and
case studies.
As above
As above
For Program, Discipline or Course Assessment Reports:
I have reviewed this report:
Melissa Deadmond
Department Chair
Ted Plaggemeyer
Dean
Date: June 3, 2011
Date: June 3, 2011
JohnTuthill
Vice President of Academic Affairs and Student Services
Date August 23, 2011
As above
Program/Discipline/Course Assessment Report
Discipline: Biology
Course Number: BIOL 224
School/Unit: SOSC
Submitted by: Steve C. Schenk
Contributing Faculty: Eddie Burke, Jamie Campbell, Jim Collier, Will Mehm, Pamela Sandstrom, Steve C. Schenk, James Verdi, Jeff Weinert,
Dan Williams, and Beate Wone
Academic Year: 2010-2011
2010-11 Assessment Report for BIOL 223 and BIOL 224
Report written and data analyzed by Steve C. Schenk
Data contributed by Eddie Burke, Jamie Campbell, Jim Collier, Will Mehm, Pamela
Sandstrom, Steve C. Schenk, James Verdi, Jeff Weinert, Dan Williams, and Beate Wone
Introduction
In recent years, BIOL 223 and 224 (Human Anatomy and Physiology I & II) have been assessed using a multiple-choice tool that was
given at the beginning and the ending of the course in order to determine the growth and development of student understanding of core
content. Detailed analysis of these tests in the last two years has suggested quite strongly that the assessment tool was deficient in
ways that undermined its value in supporting and developing effective teaching of anatomy and physiology. In particular, the following
deficiencies were identified.
•
Some questions had >60% of students earning correct answers on the pre-test. This suggested that a majority of students knew the
material coming into the course (unlikely, as this was core content of the course), that the concepts being covered were very easy
(unlikely, as these questions were often on topics/concepts universally identified as challenging by instructors, or that the questions
were poorly structured and lead competent students to the right answer without knowledge of the material.
•
Some questions – particularly on topics grounded in cell biology – had poor performance tied to them on the post-test. While these
questions were often related to topics that are challenging, the questions were not structured in a way that lead to easy changes in
teaching strategies based on the wrong answers and there were multiple concerns raised independently by different instructors
about different questions in terms of their appropriateness or relevance.
It should be noted that the problematic questions cannot be attributed to any one instructor or any one teaching style. Rather, they all
seem to be borne of the same root problem: the questions were written purely from an instructor’s perspective without any sense of the
deficiencies or misconceptions our students take away from our courses.
In order to rectify this situation and generate a better assessment tool, the standard assessment tool was eliminated for this academic
year and replaced with a series of open-ended short answer questions. Sixteen questions were developed for each course to address
sixteen core concepts. Students in each section were presented with four of these questions at the end of the semester and asked to
Program/Discipline/Course Assessment Report
Discipline: Biology
Course Number: BIOL 224
School/Unit: SOSC
Submitted by: Steve C. Schenk
Contributing Faculty: Eddie Burke, Jamie Campbell, Jim Collier, Will Mehm, Pamela Sandstrom, Steve C. Schenk, James Verdi, Jeff Weinert,
Dan Williams, and Beate Wone
Academic Year: 2010-2011
respond to them. These responses were then analyzed to determine what misconceptions, errors in thinking, and misunderstandings
appeared in students who have freshly finished the course. It is important to note that this approach did NOT allow us to determine
what percentage of students successfully mastered subject A appropriately or how much their knowledge improved from the end of the
beginning to end of a semester. The goal here was to collect data that could be used to generate a new set of multiple choice
questions for future assessment tools in which the distracters will be based on those things that are genuinely confusing to students
and not any given instructor’s impressions.
In each course (i.e. BIOL 223 and BIOL 224), the sixteen questions were split into four 4-question assessment quizzes that were split
evenly among the course sections. Only post-test assessments were considered, as the focus here is on building a new assessment
tool based on misconceptions students take away from the course. In the two sections of this report below, the question categories will
be presented, followed by the sixteen questions and analysis of responses.
BIOL 224, Human Anatomy and Physiology II
Sixteen open-ended short answer questions were developed to assess student understanding of and to identify major student
misconceptions with reference to the following topics:
•
•
•
•
•
•
•
•
Endocrinology
Blood and cardiovascular physiology
Immunology
Respiratory physiology
Digestive physiology
Renal physiology
Reproduction and Development
Positive feedback
Endocrinology
1. Contrast peptide and steroid hormones in terms of transport, half-life, and receptor used.
• The vast majority could not answer this question on the post-test. Very few correct answers were seen.
• Some focused on molecular size rather than being hydrophilic or hydrophobic.
• Some had molecules backwards with steroids requiring a membrane receptor and peptides entering cells by diffusion.
4
Program/Discipline/Course Assessment Report
Discipline: Biology
Course Number: BIOL 224
School/Unit: SOSC
Submitted by: Steve C. Schenk
Contributing Faculty: Eddie Burke, Jamie Campbell, Jim Collier, Will Mehm, Pamela Sandstrom, Steve C. Schenk, James Verdi, Jeff Weinert,
Dan Williams, and Beate Wone
Academic Year: 2010-2011
•
•
Not all that understood correct relationships (e.g. steroids having longer half lives, intracellular receptors, and requiring transport
proteins) could relate this to the molecule being hydrophilic or hydrophobic.
Specific questions on specific hormone functions could be generated based on instructor input to determine on which hormones
students should have the most universal grasp.
2. Describe the relationship between the hypothalamus and the pituitary gland.
• When the neural connection to the posterior pituitary is recognized, it is not always clear that students understand hormones are
transferred through this pathway for storage and secretion. Some failed to indicate hormones are passed through this system;
others failed to recognize that hormones are secreted by the posterior pituitary and not simply stored there.
• Some suggested that hormones transported to the anterior pituitary are then released by it, rather than triggering the release of
other hormones.
• Some students recognized one but not both connections between the hypothalamus and pituitary.
Blood and cardiovascular physiology
3. Summarize the common coagulation pathway that leads to the formation of a blood clot.
• Diversity of answers but no fully correct responses.
• Many recognized the role of fibrinogen (being converted to fibrin) and could indicate names of initiation pathways, but major
clotting enzymes of the common pathway (prothrombin activator, thrombin) were not included.
• Some confused platelet plug with coagulation or discussed three stages of hemostasis.
• One referenced terms from bone repair (hard and soft callus) likely remembering the role of blood clotting in that process and
confusing the two.
• The details of this process may need to be discussed in the focus group to determine the level of detail universally agreed upon.
The lack of complete answers here suggests that many instructors may not require a detailed understanding of the common
coagulation pathway as part of their curriculum. The assessment of this topic (and related topics of blood chemistry and
hemostasis) need to be discussed in the focus group to determine the appropriate level of detail for formal assessment.
4. There are three major arterial branches emerging from the top of the aortic arch. To which body regions is blood in each of these
three branches going?
• Very few correct answers; mostly blanks or fully wrong answers.
• Some could name the three vessels in question but could not necessarily tell where blood was going.
5
Program/Discipline/Course Assessment Report
Discipline: Biology
Course Number: BIOL 224
School/Unit: SOSC
Submitted by: Steve C. Schenk
Contributing Faculty: Eddie Burke, Jamie Campbell, Jim Collier, Will Mehm, Pamela Sandstrom, Steve C. Schenk, James Verdi, Jeff Weinert,
Dan Williams, and Beate Wone
Academic Year: 2010-2011
•
•
Several suggested that the lower body was fed by one of these branches.
Several confused left and right sides of the body.
5. Focusing on the movement and distribution of ions, compare and contrast skeletal muscle action potentials (APs) with those
observed in cardiac muscle.
• Some could not describe the roles of ions (though many recognized the presence of the plateau).
• Few discussed pacemaker APs and how these are different. (At least one suggested no true RMP in cardiac muscle…true of
pacemaker cells but not contractile cells).
• The majority could describe the common role of sodium and potassium in the depolarization and repolarization phases of AP
respectively.
• A good number could tie calcium to the plateau, though very few connected simultaneous movement of calcium and potassium
as producing the plateau; some suggested simultaneous inward movement of sodium and calcium caused the plateau.
6. Summarize several ways in which sympathetic and parasympathetic stimulation of the heart and blood vessels can be used to
regulate blood pressure (BP).
• Many recognized the role of ANS stimulation in constriction and dilation of blood vessels (with several noting the differences
observed in different body regions (e.g. digestive vs. skeletal muscle)).
• Many noted HR increased. However, many seemed to suggest an increase in BP (or decrease in BP) as separate from the
change in HR rather than the product of the change in HR. In some cases, syntax errors (or flat out misunderstanding) lead to
the suggestion that the increase in HR was caused by dilation/constriction of vessels rather than the changing ANS stimulation.
• None noted the difference in effect of sympathetic and parasympathetic stimulation on SV/contractility. The term “cardiac output”
was notably absent from all answers as was any kind of reference to causes of changes in ANS stimulation (like the
baroreceptor reflex).
• Specific questions related to long-term hormonal regulation of BP could be developed based on future open-ended survey
questions.
Immunology
7. Summarize the roles of a macrophage, a helper T cells, and B cells in promoting a humoral (antibody-mediated) immune response.
• Almost no answers here that qualify as correct in context of the question.
6
Program/Discipline/Course Assessment Report
Discipline: Biology
Course Number: BIOL 224
School/Unit: SOSC
Submitted by: Steve C. Schenk
Contributing Faculty: Eddie Burke, Jamie Campbell, Jim Collier, Will Mehm, Pamela Sandstrom, Steve C. Schenk, James Verdi, Jeff Weinert,
Dan Williams, and Beate Wone
Academic Year: 2010-2011
•
•
•
Macrophages frequently cited as engaging in phagocytosis, but almost no mention of acting as a major antigen presenting cell (<
10) and no clear mentioning of use of MHC II in presentation. Target of presentation (i.e. naïve helper T cells) almost never
mentioned or mentioned incorrectly as T cells in general.
Many noted that B cells lead to antibody production. Memory cells mentioned frequently for either B or helper T cells (rarely
both) and no clear mention of B cells presenting antigen with MHC II for recognition and activation by helper T cells.
Specific questions related to innate immunity could be developed based on instructor input (largely memorized processes that
could be based on a commonly determined list of most important innate topics) and questions related to cellular immunity could
be developed based on future open-ended survey questions.
8. In terms of antibodies and antigens, what does it mean for a person to have AB+ blood?
• Very few were able to indicate that antigens are present on RBCs.
• Several suggested that to be + is to lack Rh antigen.
• Some confused antibody and antigen, suggesting that people with AB blood have the A and B antibodies.
• Some suggested both A and B antigens and antibodies were present.
• Some couched their answers entirely in terms of universal donor/recipient language and made no specific reference to antigens
or antibodies.
Respiratory physiology
9. Summarize the effects of temperature, pH, and PO2 on the delivery of oxygen to tissues in the systemic circuit.
• Poor answers all around.
• Few references to the oxygen saturation curve, left and right shifts (particularly Bohr shifts), and almost no recognition of how
these variables relate to local metabolism.
• Several confused the systemic and pulmonary circuits, making reference to the lungs and amount of oxygen carried.
• Specific questions related to gas laws could be developed based on future open-ended survey questions.
10. Describe how a change in the rate of pulmonary ventilation (breathing) could produce a decrease in plasma pH.
• Very few correct answers and many blank responses.
• Some students recognized the correct relationship between ventilation rate and pH (e.g. hypoventilation = acidosis). However,
many of these students stated this relationship backwards, many related this to oxygen levels, and only a handful (~25% of
7
Program/Discipline/Course Assessment Report
Discipline: Biology
Course Number: BIOL 224
School/Unit: SOSC
Submitted by: Steve C. Schenk
Contributing Faculty: Eddie Burke, Jamie Campbell, Jim Collier, Will Mehm, Pamela Sandstrom, Steve C. Schenk, James Verdi, Jeff Weinert,
Dan Williams, and Beate Wone
Academic Year: 2010-2011
•
those recognizing the relationship) were able to relate pH to carbon dioxide. Of these, <5 correctly connected the carbonic acidbicarbonate buffer system to carbon dioxide transport and were able to identify it as the source of the change in pH.
Specific questions related to oxygen transport and pulmonary ventilation could be developed based on future open-ended survey
questions.
Digestive physiology
11. Summarize the various ways in which the hormone cholesystokinin (CCK) influences the digestive process.
• Many blank responses on post tests.
• One suggested it lead to release of pancreatic hormones to go into the small intestine.
• One suggested it has a negative feedback effect on secretin.
• Some connected it to release of bile by the gallbladder; more (just <50%) recognized its role in inhibiting gastric motility and
secretion; very few (< 5) noted its role in triggering the release of digestive enzymes by the pancreas.
• Specific questions related to gastric cycling, deglutition, other digestive hormones, chemical digestion, and nutrient absorption
could be developed based on future open-ended survey questions.
Renal physiology
12. Describe how you expect kidney function to change in a person who is dehydrated.
• Many recognized that kidney would retain water, urine would be more concentrated, and ADH would play a role in this.
• Some suggested that ADH was produced by the kidney.
• Some suggested that baroreceptors would detect the dehydration which would be true for isotonic or hypotonic dehydration, but
not for hypertonic dehydration.
• None made specific reference to where ADH would act to change renal function.
• Several suggested the kidney would malfunction and filtration would be changed somehow, though nothing specific in terms of a
mechanism was identified.
• Specific questions related to glomerular filtration and transcellular reabsorption could be developed based on future open-ended
survey questions.
13. Summarize why the nephron loop (loop of Henle) is said to act as a countercurrent multiplier.
8
Program/Discipline/Course Assessment Report
Discipline: Biology
Course Number: BIOL 224
School/Unit: SOSC
Submitted by: Steve C. Schenk
Contributing Faculty: Eddie Burke, Jamie Campbell, Jim Collier, Will Mehm, Pamela Sandstrom, Steve C. Schenk, James Verdi, Jeff Weinert,
Dan Williams, and Beate Wone
Academic Year: 2010-2011
•
•
•
Many poor answers.
Very few mentioned the movement of ions, water, and urea in the loop of Henle. Some that did had them backwards in terms of
events of the ascending and descending limbs.
Some confused the function of the vasa recta with that of the loop of Henle.
Reproduction and Development
14. Summarize how luteinizing hormone (LH), follicle stimulating hormone (FSH), and estrogen (estradiol) promote follicle maturation
and ovulation.
• Some very good answers here; majority (just > 50%) were blank or extremely short responses.
• Some inclusion of male effects (not asked for in question).
• Most common deficiency was separating out LH from FSH in terms of specific effects.
• Second most common deficiency was failure to relate estrogen to LH/FSH surge and subsequent ovulation.
• Specific questions related to uterine cycle, gametogenesis, and male hormonall effects could be developed based on future
open-ended survey questions.
15. Assuming that an ovum is fertilized, summarize the developmental events occurring between ovulation and implantation.
• No genuinely accurate answers to this question. Many indicated they were covering it in class that day.
• Many expressed understanding of basics (i.e. where fertilization occurs, the difference between fertilization and implantation) but
there were no answers that contained proper names for stages of embryonic development. Some even referred to the egg as
becoming an “ovum” upon fertilization.
• This may reflect a general lack of emphasis on this topic by most instructors. Anecdotal conversations with several instructors
suggest that this is true and that as important as development may be in certain areas of health science, it may not be sufficiently
important as a topic in this course to merit consideration as a topic for formal assessment. A consensus on this point needs to
be reached by faculty teaching this course during a future focus group meeting.
9
Program/Discipline/Course Assessment Report
Discipline: Biology
Course Number: BIOL 224
School/Unit: SOSC
Submitted by: Steve C. Schenk
Contributing Faculty: Eddie Burke, Jamie Campbell, Jim Collier, Will Mehm, Pamela Sandstrom, Steve C. Schenk, James Verdi, Jeff Weinert,
Dan Williams, and Beate Wone
Academic Year: 2010-2011
Positive feedback
16. Identify one example of positive feedback used by the human body and explain why it is a case of positive feedback.
• Many good examples here, though almost universally the example used involved the birth process. Any questions developed
based on this should focus on other prominent positive feedback processes like blood clotting, the gastric phase of the gastric
cycle, or neural control of ventilation rather than using childbirth as an example of positive feedback.
Summary: Developing the New Assessment Tool and Modifying It in the Future
A new assessment tool will be developed over the summer of 2011 and presented to the BIOL 223/224 focus group prior to the start of
the fall 2011 semester for approval. This tool will have the following components.
(1) A bank of multiple-choice questions will be developed based on student responses to the questions asked in the 2010-11 open
response assessment.
a. These questions will all have five choices and the distracters in each question will be tied to specific misunderstandings
and misconceptions exhibited by students in the 2010-11 assessment.
b. Some additional questions may be developed in places where instructor input but not student input is needed. For
example, some topics amount to knowing definitions. An instructor-agreed upon list of terms is all that is really needed to
develop this type of question.
(2) A ten-question assessment will be developed to give each academic year.
a. Two of the questions will cover material specific to BIOL 190. As many of the most challenging topics in this two-course
sequence are grounded in cell biology, knowing how strong a grasp students have on old material is germane to
evaluating why they do or do not grasp certain concepts in BIOL 223/224 and will certainly affect the way material is
covered. Incorporating questions from BIOL 190 will provide longitudinal data for instructors of that course and also allow
instructors of BIOL 223/224 to assess how much of an impact they are having on promoting long-term retention of
knowledge and/or enhancing students understanding of core concepts.
b. The remaining eight questions will be pulled from the question bank. Questions will be selected during each academic
cycle to focus on a maximum of three systems covered in that course. This will allow a more detailed assessment of
specific topics and the development of better questions, as we will no longer need 10 – 15 questions to cover an entire
course.
10
Program/Discipline/Course Assessment Report
Discipline: Biology
Course Number: BIOL 224
School/Unit: SOSC
Submitted by: Steve C. Schenk
Contributing Faculty: Eddie Burke, Jamie Campbell, Jim Collier, Will Mehm, Pamela Sandstrom, Steve C. Schenk, James Verdi, Jeff Weinert,
Dan Williams, and Beate Wone
Academic Year: 2010-2011
(3) The 10 question assessment will be given as a pre- and post-test to assess student knowledge and understanding upon both
entering and leaving the course.
a. A threshold of 40% will be used to evaluate the effectiveness of the pre-test as a whole. If the departmental average is
consistently greater than this, the assessment will need to be reexamined for quality. (Assuming that students guess
randomly (as they should) on the 8 course specific questions and get both BIOL 190 questions correct (as they should if
they have retained the material), then averages scores should typically fall between 30 and 40%. An average greater
than 40% would indicate that at least one question is being gotten right far more often that would be expected on a pretest.
b. Veracity of individual questions will be evaluated based on the percentage of students that get them right on the pre-test.
If more than 50% of students are getting a question right on the pre-test (with the exception of the BIOL 190 questions),
there may be a problem with the question and in this case it needs to be examined.
(4) Two or three open-ended short answer questions will be used each year on the post-test.
a. These questions will be used along the lines as those in the 2010-11 assessment tool to generate more data on student
misconceptions.
b. Each academic year’s questions will need to be agreed upon by faculty during a fall focus group meeting.
c. Each section will receive only one of the selected questions. Hence, each section will do the 10 question pre-test and
post-test. Each section will then have one open-ended question to answer and only on the post test.
d. Data collected from these questions will be considered in the annual assessment report and will be used to generate new
multiple-choice questions to expand the question bank.
11