The Effects of Cognitive Task Analysis-based Instruction on Achievement and Retention in Biology1
David F. Feldon & Jie Chao, Curry School of Education
Poor instruction has been cited as a primary cause of attrition from Science, Technology, Engineering and
Mathematics (STEM) majors and a major obstacle to learning for those who stay (Seymour and Hewitt, 1997).
This research project aims to improve undergraduate retention and achievement in the biological sciences through
the reformulation of instruction in the core course sequences. The core technique employed in the instructional
design process is Cognitive Task Analysis (CTA), which is an effective tool for eliciting, analyzing, and
representing expert knowledge in a more accurate and complete manner. A double-blind experimental design
was employed to test the effectiveness of CTA-based instruction in comparison with the traditional instruction.
Supplemental instructional videos were delivered to students in a laboratory-based undergraduate biology course
(n = 1,979) at a public, Tier-I university in the Southeast for six semesters from 2008 to 2010. The video-based
lectures were delivered by an award-winning biology instructor using a script of his own writing in the control
(traditional) condition and a script derived from cognitive task analyses conducted with expert biologists in the
treatment (CTA-based) condition. Access to the different versions of the video was provided by lab section using
random assignment. Treatment and control groups did not differ significantly on pre-treatment measures of
either motivation or scientific reasoning ability. Dependent measures used to measure the impact of the treatment
on student outcomes were students’ course attrition, performance on lab reports scored with a previously validated
rubric (inter-rater reliability > 0.8), and post-course surveys of motivation related to course topics and biology as a
major.
Overall, CTA-based instruction appeared to be more beneficial than the traditional instruction for the students,
especially those low-achieving students. Results from the first semester (Spring 2008) showed that students
(N=314) receiving traditional instruction were almost six times more likely to withdraw from the course than
students receiving CTA-based instruction (8.1% vs. 1.4% of initial enrollment). Of the students who completed
the course, those who received the CTA-based instruction demonstrated significantly higher levels of
performance in the discussion section of their written laboratory reports (effect sizes of 0.232 > Cohen’s d >
0.391). Significantly higher performances were seen specifically in the areas of analyzing data to formulate valid
conclusions, considering alternative explanations, consideration for the limitations of the experimental design
and implications of the research (Feldon et al, 2010). With the same dataset (Spring 2008), also explored were the
effects of four micro design factors of the instructions. The instructional videos were transcribed and analyzed in
terms of frequency of omission, content specificity, and knowledge representation formats (procedural or
conceptual). Compared with the traditional instructions, the CTA-based instructions had fewer omissions,
greater specificity, and more instances of procedural instruction. Despite differences in both the content and the
knowledge representation formats used within the instructional conditions, only the level of content specificity
significantly correlated positively with students’ achievement. This finding points to the importance of CTA as a
step in the instructional design process more influential than the representation format of instructional content
(Feldon & Stowe, 2009). Current and ongoing analyses of the remainder of data indicate that CTA-based
instruction is differentially effective for struggling students and has greatest and most reliable effects on students
who were repeating the course.
1
Findings reported here can be found in: Feldon, D. F., Timmerman, B. E., Stowe, K., & Showman, R. (2010).
Translating expertise into effective instruction: The impacts of cognitive task analysis (CTA) on lab report quality
and student retention in the biological sciences. Journal of Research on Science Teaching, 47(10), 1165-1185.