Three Heads are Better than One!
Undergraduate Nursing Students’ Experiences with
Collaborative Testing
Dr. Donna Martin, Assistant Professor, Faculty of Nursing;
Eunice Friesen, Associate Director Educational Innovations,
Centre for Advancement of Teaching/Learning;
Antonina De Pau, Graduate Student
Acknowledgements
• We extend our deep appreciation to the student volunteers that
participated in this study and to the staff of the Manitoba Centre
for Nursing & Health Research for their assistance and
guidance. The study was funded by the Professional
Foundations Research Fund from the Manitoba Centre for
Nursing & Health Research via the Nursing Endowment Fund,
Faculty of Nursing, University of Manitoba.
COLLABORATIVE TESTING (CT) is an formative assessment
approach that allows students to work in teams while taking an
examination to foster deeper learning/teamwork skills (Sandahl, 2009, 2010;Wiggs, 2010).
BACKGROUND:
• CT has been implemented and evaluated in higher education for >
20 years (Centrella-Nigro, 2012;) . Only recently have studies emerged in the
nursing literature (Centrella-Nigro, 2012; Durrant, Pierson, & Allen, 1985; Hickey, 2006; Heglund & Wink, 2011; Hickey, 2006;
Jones & Lishman, 2011; Lusk & Conklin, 2003; Mitchell & Melton, 2003; Molsbee, 2013; Sandahl, 2010; Wiggs, 2010).
• Because implementation of CT “has been used in a number of
different formats and designs for many years” (Jones and Lishman, 2011, p. 176), there
is great difficulty in discerning the evidence (Pandey and Kapitanoff, 2011).
• Most of the research has been descriptive in nature with some
studies using a quasi-experimental design (Sandahl, 2010).
OUTCOMES:
• Consistent outcomes of using CT include student and faculty’s
positive perceptions of the learning experience. When asked,
students consistently reported that the CT experience taught them
negotiation skills (Cortright, Collins, Rodenbaugh, & DiCarlo, 2003; Gallagher, 2009; Giuliodori, Lujan, & DiCarlo, 2008, 2009;
Haberyan & Barnett, 2010; Jones and Lishman, 2011; Lusk and Conkin, 2003; Rao, Collins, & DiCarlo, 2008).
• Inconsistent outcomes included impacts on grades, test anxiety,
critical thinking and retention of course material (Bloom, 2009; Cortright, Collins & DeCarlo,
2005; Kapitanoff, 2009; Leight et al, 2012; Meseke et al, 2009; Molsbee, 2013; Murray, 2010; Pollard & Miers, 2008; Rao & DiCarlo, 2000; Slusser &
Erickson, 2006; Woody, Woody, & Bromley, 2008; Zipp, 2007).
Rationale
A study comparing and contrasting nursing students’ experiences
with CT versus traditional testing was warranted to determine if CT
is an effective strategy for students to “learn better” and hone
teamwork skills.
Nurses are required to actively participate as a health team
member to promote patient safety (Institute of Medicine, 2010).
Nursing schools must develop models that educate nurses to
practice more effectively within health care teams (Eggertson, 2013; IOM 2010).
Guiding Frameworks
Social Interdependence Theory
Group learning enhances student success, critical thinking and deeper
learning (McWhaw 2003, as cited in Bloom, 2009).
http://www.emeraldinsight.com/fig/1350
080301003.png
Guiding Frameworks (Continued)
Cognitive Elaboration Perspective
Deep learning requires cognitive restructuring best accomplished
by explaining material to others (Whittrock, 1985).
Neuroscience - It’s biological!
Social learning requires collective problem solving challenging the
individual’s own thinking process which results in development of
areas in the brain which further enhances learning
(Caine & Caine, 2006; Ericsson, 2006; Immordino-Yang, & Damasio, 2007).
Research Questions
1.
Is there a statistically significant difference between formative
test scores in the CT versus traditional test-taking groups of
undergraduate nursing students?
2.
Does CT enhance retention of course material?
3.
What are the educational and group process experiences of
undergraduate nursing students when CT is implemented?
Research Method: Sampling Procedure
Following ERB approval,
Nursing students were recruited from 2 sections of the same
course offered by the same instructor over the same term.
Recruitment of study subjects and data collection procedures
ensured that the instructor was “blind” to the identification of
students that were/were not participating.
Method: Data Collection Procedures
1.
Informed consent obtained
2.
Demographic surveys completed
3.
Students randomly assigned into
Group A or B (Section 1), Group C
or D (Section 2)
4.
Cross over research design
Method: Data Collection Procedures (Continued)
5. Non-participating students wrote all tests
using traditional test-taking format. [test =
45 questions (MCQ and short answer)
worth 15% each].
6. Immediately prior to CT, study subjects
were randomly assigned to groups of 3.
7. CT written in one room. Non-participants
and subjects assigned to traditional testing
wrote in a separate room.
8. Each small group submitted 1answer
sheet. All students provided 2 hours to
write formative tests. All students wrote
final exam individually.
Method: Data Collection Procedures (Continued)
9. Immediately following CT, subjects
completed CT Testing Evaluation
Survey (14-items rated on a 5point Likert Scale).
10. Two audiotaped focus groups
(n=3, n=9) were conducted at the
end of the term using an interview
guide.
Research Questions
1.
Data Analysis Procedures
Is there a statistically
significant difference between
formative test scores in the
CT versus traditional testtaking (TT) groups of
undergraduate nursing
students?
1.
2.
Does CT enhance retention of
course material?
3.
3.
What are the educational and
group process experiences of
undergraduate nursing
students when CT is
implemented?
2.
Scores from demographic survey ,
CT evaluation survey, Test 1, Test 2,
final test scores uploaded into SAS
Version 9.3
Challenging questions on informative
tests were repeated in the final exam.
% of correct responses in CT, TT and
final compared (Chi-square).
Audio-tapes from focus groups
transcribed verbatim. All names and
identifying features removed. 2
research team members
independently analyzed data, then
compared codes, categories, themes
using NVivo 9.
Description of the Sample
• Convenience sample, initially 74 volunteered to participate with 4
withdrawing from the study
• n=70 (out of 110 students enrolled in 2 sections of the same course
offered by the same instructor over the same term)
• Most participants were female (60 female; 8 male; 2 unanswered)
• Most were < 30 years of age
• Almost 2/3rds of the sample was 20-24 years of age
• Almost 83% had completed 2 years of course work in the Faculty of
Nursing
• 14% had previous degree
• Participants from both sections were similar
• Overall, student population in this program is culturally diverse –
although cultural heritage was not measured by the demographic
survey, this sample was representative of the target population
Results
Impact On Grades
• Using cross-over analysis - the treatment effect (CT) was highly
significant; CT scores were on average 7.99 units higher than
traditional testing scores (5.21 – 10.77, p value <0.0001) when the test
was allocated a score out of 100.
• However, formative tests were allocated a total score of 15. Students’
overall grades increased by an average of 1.2 points/100 by
participating in CT.
• Period effect was not significant (p value = 0.9721), indicating that
students did not perform better on the 2nd test
Results Continued
Impact On Retention of Course Material
• CT, TT and final exam groups were similar in their rates of
responses to specific, challenging test questions.
• Using Chi-square, no statistically significant differences existed
between groups.*
• Based on no statistically significant difference, CT did not impact
retention of course material.
*Note: 13% of data missing
Students’ Experiences of CT:
Findings from 2 focus group interviews
Main Theme: “Three Heads are Better Than One”
I just thought 3 heads are better than 1 so chances of getting a better mark are better and
yeah, I studied more.
Students’ Experiences of CT: 5 Major Categories
Studying more – studying differently
I found I was more motivated to study for the CT
Cognitive collectivism
I feel like when others are dependent on me, I would want to be able to provide the answer.
It stuck in my head better
I feel like I learn a lot better when I discuss things or talk them out, then they stick in my
head better.
Confidence
It strengthens your knowledge coming out [of the CT] that you felt good that you knew your
material down pat because someone else knew that too
Practicing how to share knowledge/negotiate
You need to understand the materials in order to communicate and you need to be able to
communicate that. So in a way, it’s a good strategy for team working to be able to get
everybody’s input, take it all apart, and put it all back together and make sure that you have
the right answer…
Students’ Experiences of CT:
Results from CT Evaluation Surveys
Student’s Perceptions
• Most students perceived that their grades were enhanced by the
CT process.
• Approx. 79% agreed that CT helped them understand the
course material better.
Discussion
•
•
•
•
•
CT was an effective learning strategy, particularly when formative tests
were weighted appropriately and written in multiple choice format.
Differences b/w CT and traditional test-taking scores were statistically
significant supporting the previous findings from Sandahl (2010) and
Wiggs (2010).
Effective CT requires a relaxed yet professional environment with an
encouraging invigilator with sufficient time allotted for group discussion
as per Wiggs’ recommendations (2010).
A future research study using different methods to test retention of
course material is needed. For example, a study similar to Pollard and
Miers’ work in the United Kingdom (2004, 2008) is needed.
Future research about the effectiveness of CT with culturally diverse
nursing students would be beneficial.
Conclusion
Nursing education must provide ample opportunity for nursing students
to foster deeper learning/teamwork skills. CT produced statistically
significant higher test scores. Because formative tests were weighted
appropriately, students participating in CT scored an average of
1.2/100 higher than the traditional testing group in their final course
grades. In this study, there were no statistically significant differences
between CT, TT and final exam groups in rates of correct responses to
repeated challenging questions indicating that CT did not impact their
learning. This sample perceived that CT was a positive learning
experience. The main theme that emerged from the two focus group
transcripts was “three heads are better than one.” This study has
strong practical significance in that teamwork skills were facilitated.
References
Bloom, D. (2009). Collaborative test taking: Benefits for learning and retention. College Teaching. Retrieved October 20, 2010, from
http://heldrefpublications.metapress.com/index/327U1011736Q2467.pdf.
Caine, R., & Caine, G. (2006). The way we learn. Educational Leadership, 64(1), 50-54. Retrieved from
http://web.ebscohost.com.proxy2.lib.umanitoba.ca/ehost/pdfviewer/pdfviewer?sid=b6b0fe8a-8f06-4428-8976-286275d22b91%40sessionmgr14&vid=2&hid=110
Centrella-Nigro, A. (2012). Collaborative testing as posttest review. Nursing Education Perspectives, 33 (5), 340-341.
Cortright, R. N., Collins, H. L., & DiCarlo, S. E. (2005). Peer instruction enhanced meaningful learning: ability to solve novel problems. Advances in Physiology Education, 29(2), 107-11. doi:
10.1152/advan.00060.2004.
Cortright, R. N., Collins, H. L., Rodenbaugh, D. W., & DiCarlo, S. E. (2003). Student retention of course content is improved by collaborative-group testing. Advances in Physiology Education,
27(1-4), 102-8. Retrieved from http://www.ncbi.nlm.nih.gov/pubmed/12928319.
Durrant, L. K., Pierson, G., & Allen, E. M. (1985). Group testing and its effectiveness in learning selected nursing concepts. The Journal of the Royal Society for the Promotion of Health,
105(3), 107-111. doi: 10.1177/146642408510500306.
Ericsson, K.A. (2006). The influence of experience and deliberate practice on the development of superior expert performance. In K.A. Ericsson, N. Charness, P. Feltovich, and R.R. Hoffman,
R.R. (Eds.). Cambridge handbook of expertise and expert performance (pp. 685-706). Cambridge, UK: Cambridge University Press.
Giuliodori, M. J., Lujan, H. L., & DiCarlo, S. E. (2008). Collaborative group testing benefits high- and low-performing students. Advances in Physiology Education, 32(4), 274-8. doi:
10.1152/advan.00101.2007.
Haberyan, A., & Barnett, J. (2010). Collaborative testing and achievement: Are two heads really better than one? Journal of Instructional Psychology, 37, 1, 32-41.
Heglund, S., Wink, D. (2011), Impact of double testing on student knowledge in a professional issues course. Journal of Nursing Education (50) 5, 278-280.
Hickey, B. L. (2006). Lessons learned from collaborative testing. Nurse Educator, 31(2), 88-91.
References (Continued)
Immordino-Yang, M. H. and Damasio, A. (2007). We feel, therefore we learn: The relevance of affective and social neuroscience to education. Mind, Brain, and Education, 1: 3–10.
doi: 10.1111/j.1751-228X.2007.00004.x
Jones, J., Lishman, K. (2011). Collaborative posstesting in an adult health nursing course. Teaching and Learning in Nursing, 6, 176-180.
Kapitanoff, S. H. (2009). Collaborative testing: Cognitive and interpersonal processes related to enhanced test performance. Active Learning in Higher Education, 10(1), 56-70. doi:
10.1177/1469787408100195.
Leight, H., Saunders, C., Calkins, R., Withers, M. (2012). Collaborative testing improves performance but not content retention in a large-enrollment introductory biology class. CBE – Life
Sciences Education (11), 392-401.
Lusk, M., & Conklin, L. (2003). Collaborative testing to promote learning. The Journal of Nursing Education, 42(3), 121-4. Retrieved from http://www.ncbi.nlm.nih.gov/pubmed/12661712.
Meseke, C., Bovee, M., Gran, D. (2009). Impact of collaborative testing on student performance and satisfaction in a chiropractic science course. Journal of Manipulative and Physiological
Therapeutics, (32) 4, 309-314.
Mitchell, N. & Melton, S. (2003). Collaborative testing: An innovative approach to test taking. Nurse Educator, 28(2), 95-97. Retrieved February 3, 2011, from
http://ovidsp.tx.ovid.com.proxy1.lib.umanitoba.ca/sp-3.3.0b/ovidweb.cgi?&S=ECIEFPKNGIDDELDFNCCLAELBPCKPAA00&Link+Set=S.sh.15.16.19.22%7c13%7csl_10.
Molsbee, C.P. 2013.) Collaborative testing and mixed results. Teaching and Learning in Nursing (8), 22-25.
Murray, J. P. (2010). Better testing for better learning. Learning, 38(4), 148-152.
Pandey, D., Kapitanoff, S. (2011). The influence of anxiety and quality of interaction on collaborative test performance. Active Learning in Higher Education, (12), 163-174.
Pollard, K. & Miers, M. (2008). From students to professionals: Results of a longitudinal study of attitudes to pre-qualifying collaborative learning and working in health and social care in the
United Kingdom. Journal of Interprofessional Care,22(4), 399-414. Retrieved from, http://informahealthcare.com.proxy1.lib.umanitoba.ca/doi/abs/10.1080/13561820802190483
Pollard, K.C., Miers, M.E. and Gilchrist, M. (2004). Collaborative learning for collaborative working? Initial findings from a longitudinal study of health and social care students. Health & Social
Care in the Community, 12, 346–358. doi: 10.1111/j.1365-2524.2004.00504.x
References (Continued)
Rao, S. P., Collins, H. L., & DiCarlo, S. E. (2008). Collaborative testing enhances student learning. Society, 37-41.
Rao, S. P., & DiCarlo, S. E. (2000). Peer instruction improves performance on quizzes. Advances in Physiology Education,24(1), 51-55. Retrieved February 2, 2011, from
http://advan.physiology.org/cgi/content/abstract/24/1/S51.
Sandahl, S. S. (2009). Collaborative testing as a learning strategy in nursing education: A review of the literature. Nursing Education Perspectives, 30(3), 171-175.
Sandahl, S. S. (2010). Collaborative testing as a learning strategy in nursing. Nursing Education Perspectives, 31(3), 142-147.
Slusser, S. R., & Erickson, R. J. (2006). Group quizzes: An extension of the collaborative learning process. Teaching Sociology, 34(3), 249-262. doi: 10.1177/0092055X0603400304.
Wiggs, C. (2010). Collaborative testing: Assessing teamwork and critical thinking behaviors in baccalaureate nursing students. Nursing Education Today,31(3), 279 - 282.
doi:10.1016/j.nedt.2010.10.027
Woody, W. D., Woody, L. K., & Bromley, S. (2008). Anticipated group versus individual examinations: Teaching of Psychology, 35, 13-17. doi: 10.1080/00986280701818540.
Zipp, J. F. (2007). Learning by exams: The impact of two-stage cooperative tests. Teaching Sociology, 35(62), 62-76. DOI: 10.1177/0092055X0703500105