Program/Discipline/Course Assessment Report
Discipline: Biology
Course Number: BIOL 223
School/Unit: SOSC
Submitted by: Steve C. Schenk
Contributing Faculty: Eddie Burke, Jamie Campbell, Jim Collier, Will Mehm, Pamela Sandstrom, Steve C. Schenk, James Verdi, Jeff Weinert,
Dan Williams, and Beate Wone
Academic Year: 2010-2011
Complete and submit your assessment report electronically to your Academic Dean. As needed, please attach supporting documents and/or a narrative description of the assessment activities in your
program or discipline.
Program, Discipline or
Course Outcomes
In the boxes below,
summarize the outcomes
assessed in your program
or discipline during the last
year.
Outcome #1:
Cognitive knowledge of
the structure and
function and
organization of the
human body.
Assessment Measures
Assessment Results
Use of Results
Effect on Program, Discipline or Course
In the boxes below,
summarize the methods used
to assess program, discipline,
or course outcomes during
the last year.
In the boxes below, summarize
the results of your assessment
activities during the last year.
In the boxes below, summarize
how you are or how you plan to
use the results to improve
student learning.
Students were given one
of four 4-question quizzes
at the end of the semester
consisting of short answer
questions derived from
past assessments. The
quizzes were used to
determine common
student misconceptions
which would serve as the
basis for building a new
assessment tool. See
attached narrative.
The majority of sections
reported results each
semester yielding a
minimum of 40 student
responses per question. The
most notable topic areas
indicating deficiencies were:
• Histology
• Muscle contraction
• Neurophysiological
principles
• Sensory transduction
These areas of weakness are
largely consistent with past
assessment results.
Data generated in this report
1. Greater attention will be paid to
will be used to generate a new
course-to-course continuity of
pre-/post-test multiple-choice
sequence courses, particularly
assessment tool, and will
consideration of BIOL 190 in BIOL
serve as the basis for future
223.
refinement of said tool.
2. By identifying student
misconceptions, data from this
assessment will yield more effective
The results underscored
short answer questions and better
longstanding analyses that
future assessment tools. This will
suggest challenges in BIOL
allow us to better identify specific
223 are linked to poor
student weaknesses, enabling us to
knowledge retention from
better respond to these weaknesses.
BIOL 190. While this has not
been traditionally considered 3. The short answer question approach
to data mining will be valuable for
in the BIOL 223 assessment
continued use in refining future
process, it will be in the
assessment tools.
future.
See attached narrative for details.
See attached narrative for
Outcomes are poorly constructed and will
details.
be revised for the next academic year.
Based on the results of this assessment, will
you revise your outcomes? If so, please
summarize how and why in the boxes below.
Program/Discipline/Course Assessment Report
Discipline: Biology
Course Number: BIOL 223
School/Unit: SOSC
Submitted by: Steve C. Schenk
Contributing Faculty: Eddie Burke, Jamie Campbell, Jim Collier, Will Mehm, Pamela Sandstrom, Steve C. Schenk, James Verdi, Jeff Weinert,
Dan Williams, and Beate Wone
Academic Year: 2010-2011
Outcome #2:
Analytical thinking
skills.
As above
As above
As above
For Program, Discipline or Course Assessment Reports:
I have reviewed this report:
Melissa Deadmond
Department Chair
Ted Plaggemeyer
Dean
Date: June 3, 2011
Date: June 3, 2011
John Tuthill
Vice President of Academic Affairs and Student Services
Date August 23, 2011
As above
Program/Discipline/Course Assessment Report
Discipline: Biology
Course Number: BIOL 223
School/Unit: SOSC
Submitted by: Steve C. Schenk
Contributing Faculty: Eddie Burke, Jamie Campbell, Jim Collier, Will Mehm, Pamela Sandstrom, Steve C. Schenk, James Verdi, Jeff Weinert,
Dan Williams, and Beate Wone
Academic Year: 2010-2011
2010-11 Assessment Report for BIOL 223 and BIOL 224
Report written and data analyzed by Steve C. Schenk
Data contributed by Eddie Burke, Jamie Campbell, Jim Collier, Will Mehm, Pamela
Sandstrom, Steve C. Schenk, James Verdi, Jeff Weinert, Dan Williams, and Beate Wone
Introduction
In recent years, BIOL 223 and 224 (Human Anatomy and Physiology I & II) have been assessed using a multiple-choice tool that was
given at the beginning and the ending of the course in order to determine the growth and development of student understanding of core
content. Detailed analysis of these tests in the last two years has suggested quite strongly that the assessment tool was deficient in
ways that undermined its value in supporting and developing effective teaching of anatomy and physiology. In particular, the following
deficiencies were identified.
•
Some questions had >60% of students earning correct answers on the pre-test. This suggested that a majority of students knew the
material coming into the course (unlikely, as this was core content of the course), that the concepts being covered were very easy
(unlikely, as these questions were often on topics/concepts universally identified as challenging by instructors, or that the questions
were poorly structured and lead competent students to the right answer without knowledge of the material.
•
Some questions – particularly on topics grounded in cell biology – had poor performance tied to them on the post-test. While these
questions were often related to topics that are challenging, the questions were not structured in a way that lead to easy changes in
teaching strategies based on the wrong answers and there were multiple concerns raised independently by different instructors
about different questions in terms of their appropriateness or relevance.
It should be noted that the problematic questions cannot be attributed to any one instructor or any one teaching style. Rather, they all
seem to be borne of the same root problem: the questions were written purely from an instructor’s perspective without any sense of the
deficiencies or misconceptions our students take away from our courses.
In order to rectify this situation and generate a better assessment tool, the standard assessment tool was eliminated for this academic
year and replaced with a series of open-ended short answer questions. Sixteen questions were developed for each course to address
sixteen core concepts. Students in each section were presented with four of these questions at the end of the semester and asked to
respond to them. These responses were then analyzed to determine what misconceptions, errors in thinking, and misunderstandings
Program/Discipline/Course Assessment Report
Discipline: Biology
Course Number: BIOL 223
School/Unit: SOSC
Submitted by: Steve C. Schenk
Contributing Faculty: Eddie Burke, Jamie Campbell, Jim Collier, Will Mehm, Pamela Sandstrom, Steve C. Schenk, James Verdi, Jeff Weinert,
Dan Williams, and Beate Wone
Academic Year: 2010-2011
appeared in students who have freshly finished the course. It is important to note that this approach did NOT allow us to determine
what percentage of students successfully mastered subject A appropriately or how much their knowledge improved from the end of the
beginning to end of a semester. The goal here was to collect data that could be used to generate a new set of multiple choice
questions for future assessment tools in which the distracters will be based on those things that are genuinely confusing to students
and not any given instructor’s impressions.
In each course (i.e. BIOL 223 and BIOL 224), the sixteen questions were split into four 4-question assessment quizzes that were split
evenly among the course sections. Only post-test assessments were considered, as the focus here is on building a new assessment
tool based on misconceptions students take away from the course. In the two sections of this report below, the question categories will
be presented, followed by the sixteen questions and analysis of responses.
BIOL 223, Human Anatomy and Physiology I
Sixteen open-ended short answer questions were developed to assess student understanding of and to identify major student
misconceptions with reference to the following topics:
• Homeostasis, feedback, and feedback loops
• Tissue classification and membrane structure
• Anatomical directional terminology
• Protective function of the integument
• Bone matrix and bone repair
• Muscle contraction
• Fundamentals of neurophysiology
• Nervous system fundamentals
• Visual and auditory sensory transduction
For each question, major misconceptions are noted. These misconceptions will be used as the basis for developing new multiplechoice questions that will be used to construct future assessment exams that will allow a more accurate pre- and post- course view of
student understanding of central concepts.
4
Program/Discipline/Course Assessment Report
Discipline: Biology
Course Number: BIOL 223
School/Unit: SOSC
Submitted by: Steve C. Schenk
Contributing Faculty: Eddie Burke, Jamie Campbell, Jim Collier, Will Mehm, Pamela Sandstrom, Steve C. Schenk, James Verdi, Jeff Weinert,
Dan Williams, and Beate Wone
Academic Year: 2010-2011
General Misconceptions
• Ions often abbreviated incorrectly (e.g. calcium shown as not divalent or negative).
Homeostasis, feedback, and feedback loops
1. Define homeostasis in your own words and describe the structure of a homeostatic feedback loop.
• Some confuse “similar” with “same” or “unchanged” in describing homeostasis.
• Positive and negative feedback, when discussed, are typically defined accurately. However, many suggest that positive
feedback can be used in homeostasis, confusing feedback loops in general with homeostatic feedback loops in particular.
• In describing feedback loop structure: some students sometimes suggested that it had to be neural, IDing afferent and efferent
neurons as part of a generic feedback loop; none demonstrated understanding that a single organ can serve as both receptor
and integration center/integration center and effector organ.
Tissue classification and membrane structure
2. Contrast the classification of epithelial tissues with the classification of connective tissues.
• In some cases, epithelia related to “outer surface” rather than surface in general; if describing structures composed of epithelia,
glands were not consistently included.
• Students often simply listed types of each tissue class rather than discuss features of their classification.
• Cell shapes for epithelia not always correctly described (e.g. “round). One suggested that all epithelia are hexaganol (a la
surface view of a simple squamous epithelium).
• Very few made reference to the terms “protein fiber” (let alone types of fibers) and “ground substance” when addressing
connective tissue.
• A few identified connective tissues as able to “attach to…skin, or bone” but few indicated that skin contains connective tissue
and that bone is connective tissue.
• Several described tissue purely in terms of anatomical depth (i.e. epithelia = superficial, connective = deep).
3. Describe the structure of a generic serous membrane.
• Vast majority could not answer this question.
• Most partially correct answers made reference to lining body cavities and/or serous fluid secreted by epithelial cells within the
membrane.
5
Program/Discipline/Course Assessment Report
Discipline: Biology
Course Number: BIOL 223
School/Unit: SOSC
Submitted by: Steve C. Schenk
Contributing Faculty: Eddie Burke, Jamie Campbell, Jim Collier, Will Mehm, Pamela Sandstrom, Steve C. Schenk, James Verdi, Jeff Weinert,
Dan Williams, and Beate Wone
Academic Year: 2010-2011
•
Almost no references to visceral and parietal surfaces. (A couple specifically mentioned terms like “epicardium” or “pericardium.”
Anatomical directional terminology
4. Provide one example of a proximal-distal anatomical relationship, and justify your answer.
• Vast majority provided a valid example with proper justification (i.e. relation to attachment/origin point).
• Specific questions related to other anatomical terminology could be developed based on instructor suggestions…it appears that
some students may know this type of material from other courses, but even those who don’t appear to be learning this
effectively. Most appropriate questions should simply select particular general terms and test them based on common
examples.
Protective function of the integument
5. Describe how the integument serves as a barrier that helps keep microorganisms out of the body and water in the body.
• Many limited antimicrobial role of the skin to serving as a physical barrier; chemical barriers were only occasionally included and
often nonspecifically (e.g. “a special pH”). Only a few (< 5) referenced antimicrobial cells (e.g. macrophages, Langerhans cells)
within the integument.
• Water barrier typically discussed in vague terms with specific role of the epidermis not always identified. Hydrophobic lipids in
epidermis occasionally but not consistently identified.
• Several showed lack of understanding of orientation of skin with one identifying the epidermis as the “middle layer” and many
that identified specific epidermal layers with reference to water balance attributed the hydrophobic layer to either the stratum
corneum or stratum basale, or identified its location as between two non-adjacent layers (e.g. stratum granulosum and basale).
• Specific integumentary questions related to UV protection, abrasion resistance, and calcium homeostasis could be developed
based on future open-ended survey questions.
Bone matrix and bone repair
6. Identify the main components of bone matrix and the functions that can be attributed to each.
• Students were nearly as likely to identify “compact bone/spongy bone” or “osteoblasts/osteocytes/osteoclasts” when asked to
identify components of bone matrix.
• Not all that correctly related major attributes to matrix components when correctly identifying them (i.e. collagen = flexibility,
hydroxyapatite = rigidity/weight-bearing strength).
6
Program/Discipline/Course Assessment Report
Discipline: Biology
Course Number: BIOL 223
School/Unit: SOSC
Submitted by: Steve C. Schenk
Contributing Faculty: Eddie Burke, Jamie Campbell, Jim Collier, Will Mehm, Pamela Sandstrom, Steve C. Schenk, James Verdi, Jeff Weinert,
Dan Williams, and Beate Wone
Academic Year: 2010-2011
7. Summarize the process by which long bones lengthen.
• > 50% made reference to epiphyseal plate or growth plate as the location of bone lengthening; a few < 5 made reference to
zones of cartilage within the plate. Very few made any reference to the process itself (e.g. growth of cartilage, breakdown of
cartilage, replacement with bones) in any fashion. Some suggested that closure of the plate (which occurs at the end of puberty)
is the process by which elongation occurs.
• Some confused development of bone (e.g. endochondral/intramembranous ossification) with bone elongation.
• Some restricted explanation to role of osteoblasts and osteoclasts, suggesting references to increasing bone diameter or
secondary growth instead of elongation.
Muscle contraction
8. Describe how calcium and ATP are used within skeletal muscle cells during muscle contraction.
• Many related the role of calcium to release of ACh by the synaptic terminal but did not reference binding to troponin to act as a
trigger for contraction.
• Most had understanding that ATP is used in a cross-bridge cycle, though not many used that term. (Many did reference myelin.)
• Most likely points of confusion were to focus on ATP generation or to tie calcium to changes in RMP of sarcolemma by
influencing other channels.
• Specific questions on muscle metabolism, contraction types, fiber types, or controlling strength of contraction could be developed
based on future open-ended survey questions.
Fundamentals of neurophysiology
9. Describe how an action potential is generated in terms of the resting potential and movement of ions.
• Most demonstrated clear understanding of the events occurring during depolarization, repolarization, and hyperpolarization
phases of AP. However, most did not demonstrate clear understanding of trigger for AP to threshold.
o Not all described that threshold is reached from RMP through depolarization.
o Some suggested sodium was the only ion capable of getting MP to threshold.
o Few demonstrated basic understanding of generalized ion movements (i.e. + in = depol’n; + out or – in = hyperpol’n).
o One identified threshold depolarization tied to sodium-potassium ligand gates (i.e. as occurs at nicotinic cholinergic
synapses) as the general means of reaching threshold.
• Some include sodium-potassium pumps in explanation of depolarization phase of AP.
7
Program/Discipline/Course Assessment Report
Discipline: Biology
Course Number: BIOL 223
School/Unit: SOSC
Submitted by: Steve C. Schenk
Contributing Faculty: Eddie Burke, Jamie Campbell, Jim Collier, Will Mehm, Pamela Sandstrom, Steve C. Schenk, James Verdi, Jeff Weinert,
Dan Williams, and Beate Wone
Academic Year: 2010-2011
•
•
•
•
Offered synaptic release of NT (specifically ACh) as “how an action potential is generated”
Some had inaccurate syntax (e.g. “Sodium voltage gated channels open…They diffuse into the cell…”)
References to RMP not always accurate (e.g. 2 potassium out per sodium in)
Specific questions related to establishment of RMP could be developed based on future open-ended survey questions as this
was not specifically considered in this question…though some did address it an as noted, not always accurately.
10. Summarize how action potentials are propagated through an axon and how myelination affects this process.
• Majority demonstrated understanding that myelination increases the speed of AP propagation; many (but less than half) clearly
indicated the role of nodes of Ranvier and/or “leaping from node-to-node.”
• Almost none appeared able to explain how propagation occurs, with only a handful referencing depolarization of adjacent
downstream areas in an axon. Not all that referenced depolarization did so accurately, with one referencing a cleft (presumably
synaptic) and then proceeding to indicate that “There is a lot more I could explain but don’t feel like it…too many details for this
little space.”
Nervous system fundamentals
11. Contrast the parasympathetic and sympathetic divisions of the autonomic nervous system (ANS).
• Most common answer was to reference “fight or flight” and “rest and digest,” though not always correctly attributed to division
names.
• Several referenced “increased ventilation” inaccurately as result of sympathetic stimulation.
• Some accurate references to structural differences between sympathetic and parasympathetic divisions, but they were very few
in number.
• Several generally described the sympathetic as being “excitatory” and/or parasympathetic as “inhibitory.”
• No references to sympathetic only functions (e.g. thermoregulation/metabolism).
12. How do the primary motor cortex (precentral gyrus), premotor cortex, and cerebellum work together to produce voluntary movement.
• No strong answers to this question. The majority of students seemed to have no sense of the basic functions of these brain
regions, namely planning of voluntary motor activity (premotor cortex), execution of voluntary activity via APs to muscles (primary
8
Program/Discipline/Course Assessment Report
Discipline: Biology
Course Number: BIOL 223
School/Unit: SOSC
Submitted by: Steve C. Schenk
Contributing Faculty: Eddie Burke, Jamie Campbell, Jim Collier, Will Mehm, Pamela Sandstrom, Steve C. Schenk, James Verdi, Jeff Weinert,
Dan Williams, and Beate Wone
Academic Year: 2010-2011
•
motor cortex), and accounting for body position to provide coordination of motor activity (cerebellum). One student confused
“balance” with “coordination.”
While not all instructors cover the brain to the same level of detail, certain major regions (such as those addressed here) should
be covered by all instructors and other regions could be identified by instructor input. As this question and any related question
would largely amount to knowing definitions of functions for brain regions, future short answer assessments are unnecessary to
generate questions.
13. Describe the composition of spinal reflex arc and provide one example of how these reflexes can be used.
• Most had complete and correct lists of reflex arc components. The most often omitted components where typically neurons
(afferent/sensory and efferent/motor).
• Some referenced only “skin receptors” or “nociceptors” omitting other stimuli for major somatic spinal reflexes
• Examples of reflexes offered are typically descriptive but terminologically non-specific (i.e. reflex type is not named).
• Specific questions related to neural pathways/circuits (e.g. divergence and convergence) could be developed based on future
open-ended survey questions as this was not specifically considered in this question…though at least one did correctly address
divergence and relate it to conscious awareness of stimulus after reflex.
14. How would skeletal muscle and cardiac muscle each be affected if their acetylcholine receptors were blocked in such a way that
prevented acetylcholine from affecting them?
• Very few demonstrated understanding of the difference between nicotinic cholinergic and muscarinic cholinergic receptors. That
is, most suggested that both skeletal and cardiac muscle would be affected in the same fashion by blocking ACh (i.e. paralysis).
• Some incorrectly incorporated calcium and ATP into their answers, suggesting a confusion of the events occurring at the
synaptic terminal of the axon, at the postsynaptic membrane of a NMJ, and within the muscle fibers themselves.
Visual and auditory sensory transduction
15. Summarize the transduction of light to action potential occurs in the retina.
• Majority had difficulty with this question with a variety of answers.
• Several confused light conduction with AP transduction, focusing their answer on the role of the lens and refraction of light. This
suggests a disconnect on the subject of signal transduction from BIOL 190.
• Many referenced “photoreceptors” without identifying rods or cones by name.
9
Program/Discipline/Course Assessment Report
Discipline: Biology
Course Number: BIOL 223
School/Unit: SOSC
Submitted by: Steve C. Schenk
Contributing Faculty: Eddie Burke, Jamie Campbell, Jim Collier, Will Mehm, Pamela Sandstrom, Steve C. Schenk, James Verdi, Jeff Weinert,
Dan Williams, and Beate Wone
Academic Year: 2010-2011
•
•
Only around 25% made reference to the rod/bipolar cell/ganglionic cell interface with some who did describe it doing so
incorrectly (e.g. bipolar cell releasing glutamate to inhibit ganglionic cell).
Only around 33% made reference to rhodopsin and its associated shape changes that occur when exposed to light; these were
not always discussed accurately.
16. Summarize the transduction of sound to action potential occurs in the cochlea.
• As with vision, the majority had trouble with this and frequently confused transduction with conduction. (This is particularly
problematic for hearing.
• Some had a strong sense of how sound as vibration is passed through the fluids and membranes of the cochlea to ultimately
affect hair cells. None related bending of stereocilia to opening of mechanically gated channels or ion movements to produce
AP.
Summary: Developing the New Assessment Tool and Modifying It in the Future
A new assessment tool will be developed over the summer of 2011 and presented to the BIOL 223/224 focus group prior to the start of
the fall 2011 semester for approval. This tool will have the following components.
(1) A bank of multiple-choice questions will be developed based on student responses to the questions asked in the 2010-11 open
response assessment.
a. These questions will all have five choices and the distracters in each question will be tied to specific misunderstandings
and misconceptions exhibited by students in the 2010-11 assessment.
b. Some additional questions may be developed in places where instructor input but not student input is needed. For
example, some topics amount to knowing definitions. An instructor-agreed upon list of terms is all that is really needed to
develop this type of question.
(2) A ten-question assessment will be developed to give each academic year.
a. Two of the questions will cover material specific to BIOL 190. As many of the most challenging topics in this two-course
sequence are grounded in cell biology, knowing how strong a grasp students have on old material is germane to
evaluating why they do or do not grasp certain concepts in BIOL 223/224 and will certainly affect the way material is
covered. Incorporating questions from BIOL 190 will provide longitudinal data for instructors of that course and also allow
10
Program/Discipline/Course Assessment Report
Discipline: Biology
Course Number: BIOL 223
School/Unit: SOSC
Submitted by: Steve C. Schenk
Contributing Faculty: Eddie Burke, Jamie Campbell, Jim Collier, Will Mehm, Pamela Sandstrom, Steve C. Schenk, James Verdi, Jeff Weinert,
Dan Williams, and Beate Wone
Academic Year: 2010-2011
b. instructors of BIOL 223/224 to assess how much of an impact they are having on promoting long-term retention of
knowledge and/or enhancing students understanding of core concepts.
c. The remaining eight questions will be pulled from the question bank. Questions will be selected during each academic
cycle to focus on a maximum of three systems covered in that course. This will allow a more detailed assessment of
specific topics and the development of better questions, as we will no longer need 10 – 15 questions to cover an entire
course.
(3) The 10 question assessment will be given as a pre- and post-test to assess student knowledge and understanding upon both
entering and leaving the course.
a. A threshold of 40% will be used to evaluate the effectiveness of the pre-test as a whole. If the departmental average is
consistently greater than this, the assessment will need to be reexamined for quality. (Assuming that students guess
randomly (as they should) on the 8 course specific questions and get both BIOL 190 questions correct (as they should if
they have retained the material), then averages scores should typically fall between 30 and 40%. An average greater
than 40% would indicate that at least one question is being gotten right far more often that would be expected on a pretest.
b. Veracity of individual questions will be evaluated based on the percentage of students that get them right on the pre-test.
If more than 50% of students are getting a question right on the pre-test (with the exception of the BIOL 190 questions),
there may be a problem with the question and in this case it needs to be examined.
(4) Two or three open-ended short answer questions will be used each year on the post-test.
a. These questions will be used along the lines as those in the 2010-11 assessment tool to generate more data on student
misconceptions.
b. Each academic year’s questions will need to be agreed upon by faculty during a fall focus group meeting.
c. Each section will receive only one of the selected questions. Hence, each section will do the 10 question pre-test and
post-test. Each section will then have one open-ended question to answer and only on the post test.
d. Data collected from these questions will be considered in the annual assessment report and will be used to generate new
multiple-choice questions to expand the question bank.
11