The Effects of Cognitive Task Analysis-based Instruction on Students’ Achievement and
Retention in Undergraduate Biology Course
David F. Feldon, Jie Chao
Department of Curriculum, Instruction and Special Education, University of Virginia
Abstract: Poor instruction has been cited as a primary cause of attrition from STEM majors and a
major obstacle to learning for those who stay (Seymour & Hewitt, 1997). Using a double-blind
design, this study tested the hypothesis that the lack of explicit instructions in scientific inquiry
skills is a major factor in both low STEM retention and academic underperformance. This project
delivered supplemental instruction to students in a laboratory-based undergraduate biology course
that was derived either from cognitive task analyses (CTAs) conducted with expert biologists
(treatment) or was authored and delivered by an award-winning biology instructor (control).
Overall, CTA-based instruction appeared to be more beneficial than the traditional instruction for
students, especially those who were low-achieving. This differential effect of CTA-based
instruction may be attributed to an expertise reversal effect (Kalyuga, Ayres, Chandler, & Sweller,
2003).
Background
Poor instruction has been cited as a primary cause of attrition from Science, Technology, Engineering and
Mathematics (STEM) majors and a major obstacle to learning for those who stay (Seymour and Hewitt, 1997). This
research project aims to improve undergraduate retention and achievement in the biological sciences through the
reformulation of instruction in the core course sequences. The core technique employed in the instructional design
process is Cognitive Task Analysis (CTA), which is an effective tool for eliciting, analyzing, and representing expert
knowledge in a more accurate and complete manner (Clark, Feldon, van Merriënboer, Yates, & Early, 2008). CTAbased training systems that have explicitly accommodated the tacit nature of experts’ knowledge have proven to be
significantly more effective than those that have not (e.g., Merrill, 2002). Further, there is substantial evidence that
gaps in instructional content resulting from the omission of necessary steps in problem-solving procedures induce
higher levels of cognitive load in learners, which interferes with learning and can lower motivation (e.g., Kirschner,
Sweller, & Clark, 2006). To evaluate the impact of CTA-based instruction on undergraduate biology students
compared to traditional instructions by an award-winning professor, the following hypotheses were tested:
(1) CTA-based instruction reduces the rate of attrition in an introductory level biology course.
(2) CTA-based instruction leads to increased performance in scientific problem solving as measured by the quality
of biology laboratory reports.
Methodology
A double-blind experimental design was employed to test the effectiveness of CTA-based instructions compared to
traditional instruction. Supplemental instructional videos were delivered to students in a laboratory-based
undergraduate biology course (n = 1979) at a public, Tier-I university in the Southeast for five semesters from 2008
to 2010. The lectures were delivered by an award-winning biology instructor and were either written by him
(traditional) or derived from cognitive task analyses conducted with expert biologists (CTA-based). To ensure that
the treatment and control populations were equivalent in both general scientific reasoning ability and motivation,
Lawson’s Test of Scientific Reasoning (Lawson) and the Motivated Strategies for Learning Questionnaire (MSLQ)
were both administered at the beginning of the course. Neither measure found significant differences between
treatment and control samples. Dependent measures were students’ course completion records and performance on
lab reports scored with Universal Lab Rubric (Timmerman et al., in press).
Results
Overall, CTA-based instruction appeared to be more beneficial than the traditional instruction for the students,
especially those low-achieving students. Results from the first semester (Spring 2008) showed that students (n=314)
receiving traditional instruction were almost six times more likely to withdraw from the course than students
receiving CTA-based instruction (8.1% vs. 1.4% of initial enrollment). Of the students who completed the course,
those who received the CTA-based instruction demonstrated significantly higher levels of performance in the
discussion section of their written laboratory reports. Significantly higher performances were seen specifically in the
areas of analyzing data to formulate valid conclusions, considering alternative explanations, consideration for the
limitations of the experimental design and implications of the research (Feldon et al, 2010).
Results from the subsequent four semesters (Fall 2008, Spring 2009, Fall 2009, and Spring 2010) showed that
biology students who had to repeat the course due to withdrawal or failure in a previous semester receiving
traditional instruction were twice as likely as those receiving CTA-based instruction to withdraw from the course a
second time or decline to submit the major lab report assignment (61.1% vs. 31.3% of initial enrollment). Of the
students who completed the course and viewed all the instructional videos, overall differences in lab report
performance between experimental and control groups were not significant. However, significant differences were
found among subgroups of participants. For students who repeated the course during this study, CTA-based
instruction was significantly more effective than traditional instruction for some skills. These effects appeared to be
mostly contributed by female students and non-biology students.
Discussion
Most students take Biology 101 only once and earn a passing grade. However, students who struggle may withdraw
from or fail the course and choose to repeat it in a subsequent semester. Students who repeated the course were
assumed to have insufficient prior knowledge to succeed in the course during their first attempt. CTA-based
instruction is differentially effective for these struggling students and has greatest and most reliable effects on
students who were repeating the course. Further, there were limited negative effects for students who did not repeat
the course. This result may be attributed to the expertise reversal effect (Kalyuga, Ayres, Chandler, & Sweller,
2003) which describes the consequences of providing specific instructional support to learners across a wide range
of ability. Students with lower levels of prior knowledge benefit tremendously from highly explicit guidance.
However, students with more expertise can be hindered by needing to parse instruction that is too basic for them.
With regard to gender, it is possible that women in the study experience or perceive a lower level of access to
informal support mechanisms than their male counterparts. Supports could include the ability to have their
questions answered during class time or instructors’ office hours or access to networks of supportive peers who can
provide needed knowledge to bolster student understanding. Such conditions are commonly cited in studies of
differential gender outcomes in science education. It is possible that the highly explicit nature of the treatment
condition’s instruction reduces reliance on other sources of necessary information. Conversely, use of such
resources could bolster performance in the male sub-population, which would diminish observable treatment effects
and potentially invoke the expertise reversal effect where CTA-based instruction was interfering with information
gained outside the formal course structure.
References
Clark, R. E., Feldon, D. F., Van Merriënboer, J. J. G., Yates, K. A., & Early, S. (2008). Cognitive task analysis. In:
J. M. Spector, M. D. Merrill, J. J. G. van Merriënboer, & M. P. Driscoll (Eds.), Handbook of research on
educational communications and technology (3rd ed., pp. 577–593). New York: Routledge.
Feldon, D. F., Timmerman, B. C., Stowe, K. A., Showman, R. (2010). Translating expertise into effective
instruction: The impacts of Cognitive Task Analysis (CTA) on lab report quality and student retention in the
biological sciences. Journal of research in science teaching, 47(10), 1165–1185.
Kalyuga, S., Ayres, P., Chandler, P., & Sweller, J. (2003). The expertise reversal effect. Educational Psychologist,
38(1), 23-31.
Kirschner, P., Sweller, J., & Clark, R. E. (2006). Why minimally guided learning does not work: An analysis of the
failure of discovery learning, problem-based learning, experiential learning and inquiry-based learning.
Educational Psychologist, 41(2), 75–86.
Merrill, M. D. (2002). A pebble-in-the-pond model for instructional design. Performance Improvement, 41(7), 39–
44.
Paas, F. & van Merriënboer, J. (1994). Variability of worked examples and transfer of geometrical problem-solving
skills: A cognitive-load approach. Journal of Educational Psychology, 86(1), pp. 122–133.
Seymour, E., & Hewitt, N. (1997). Talking about leaving: Why undergraduates leave the sciences. Boulder, CO:
Westview.