Improving scientific thinking
and experimental design skills
in undergraduate students
Angela Hodgson, North Dakota State University
Fiona Rawle, University of Toronto at Mississauga.
How did this collaboration start?



Talked at BLC
Emailed each other after BLC
Applied for and received catalytic grant
THANK YOU Jane Reece, the family of Neil Campbell,
and Pearson for your support!
Rationale – Science Process Skills

What are science process skills?
 Experimental
Design
 Graphing and visualizing data
 Statistical analysis and hypothesis testing
 Scientific communication

Why are science process skills important?
Comparison of Classes
UTM
NDSU
Class
Intro to Evolution
General Biology I
Size
2 x 380
2 x 350-400
Lab Size
24; weekly 1.5 hr lab 24; weekly 2 hr lab
Semester
Length
12 weeks
16 weeks
Learning Outcomes
UTM intervention
NDSU Intervention
1.
Explain what a hypothesis is.
1.
2.
Identify good vs bad hypotheses.
2.
Identify elements of good
experimental design.
3.
Identify independent and dependent
variables in an experimental design.
4.
Explain the importance of replication
in experimental design.
5.
Critique an experiment.
6.
Improve upon a prior experimental
design.
7.
3.
4.
5.
6.
7.
8.
Design an experiment to test a
hypothesis.
Write a good hypothesis
Identify independent and dependent
variables in an experimental design
Design an experiment to test a
hypothesis.
Record and analyze scientific data in
a spreadsheet
Perform a statistical hypothesis test
Create graph that correctly
represents scientific data
Interpret and synthesize results from
a scientific experiment
2. Case
Study Based
Tutorial
Sessions
UTM
–
Course
Design
1. “Thinking
Like a
Scientist”
Introductory
Module
6. Active
Learning
Exercises in
Lecture
Introductory
Biology Course
Re-Design
5. Scientific
Literacy
Assignment
3. Inquiry
Based Labs
4. “Science
vs.
Pseudoscienc
e” Lecture
Examples
1
.
Lecture
Intervention
Lab
Intervention
Module:
Thinking Like
a Scientist
Module:
Experimental
Design
Worksheets on
hypothesis
Testing and
experimental
design included
in every lab
Module: Critiquing
Scientific Literature
Group
Experimental
Design
Brainstorming
Session
Science vs
Pseudoscience
12.
Clicker Case Studies
Assessment
EDAT
Group EDAT
Science Process
Skills Concept
Assessment
EDAT, Final Exam
In class example - UTM

“You should take chlorophyll supplements because
your need to oxygenate your bowel”
(See “Bad Science” by Ben Goldacre)
Week #
1.
Lab Intervention
Inquiry Lab – Optimization of
ethanol production
(Yeast respiration)
Inquiry Lab: Optimization of
commercial osmotic dehydration
(Diffusion and osmosis)
Inquiry Lab: Optimize greenhouse
photosynthetic rate
(Spectrophotometry and Photosynthesis
Inquiry Lab: Does race exist?
16.
From Peggy Brickman
(ABLE proceedings Vol. 32)
(DNA electrophoresis)
Assessment
EDAT pretest
1st Lab Practical
EDAT, SPARST
2nd Lab Practical
NDSU – Example Inquiry Lab
Question
Agri-gas is a company currently producing ethanol by yeast
fermentation of a 5% sucrose solution at 20°C. The
company would like to increase the metabolic rate of the
yeast in order to maximize ethanol production. Would
changing any aspect of the current production system
increase the rate of yeast fermentation?
Materials
Incubators that can be set at 40 degrees C and 60 degrees C,
refrigerator (4 degrees C), carbohydrates- sucrose(table sugar),
glucose, brown sugar, powdered sugar, lamps with different color light
bulbs, dark cabinets, HCl (acid), base (NaOH)
Experimental Design Aptitude Test
(Sirum, 2011 Bioscience)
•
Students taking the EDAT are given the
following prompt:
•
Advertisements for a herbal product,
ginseng, claim that it promotes endurance.
To determine if the claim is fraudulent and
prior to accepting this claim, what type of
evidence would you like to see? Provide
details of an investigative design.
EDAT Scoring Rubric

1 pt. is awarded for each of the following rubric items that are
included in the student answer.
1. Recognition that an experiment can be done to test the claim
2. Identification of what variable is manipulated
3. Identification of what variable is measured
4. Description of how dependent variable is measured
5. Realization that there is one other variable that must be held
constant
6. Understanding of the placebo effect
7. Realization that there are many variables that must be held
constant
8. Understanding that the larger the sample size or number of
subjects, the better the data.
9. Understanding that the experiment needs to be repeated.
10. Awareness that one can never prove a hypothesis, that one can
never be 100% sure.
UTM: Pre and Post EDAT
1Ex
pe
2rim
In
de
en
p
t
va
3ria
D
bl
ep
e
V
ar
4ia
M
bl
ea
e
su
re
m
en
t
5C
on
tr o
6
l
-P
7O
la
th
ce
er
bo
co
ns
8ta
Sa
nt
s
m
pl
e
si
9ze
R
ep
et
10
iti
-E
on
rro
r,
lim
its
% of students including rubric item
UTM: Pre and Post Rubric Item Score
100
90
80
60
Pre-Test
70
Post-Test
50
40
30
20
10
0
NDSU: Pre and Post EDAT
N=517
NDSU: Pre and Post Rubric Item Score
N=517
Learning Outcomes
1.
2.
3.
4.
5.
6.
7.
Write a good hypothesis
Identify independent and dependent variables in an
experimental design
Design an experiment to test a hypothesis.
Record and analyze scientific data in a spreadsheet
Perform a statistical hypothesis test
Create graph that correctly represents scientific data
Interpret and synthesize results from a scientific
experiment
NDSU: Additional Rubric Items
11. Identified that a hypothesis is needed/wrote a
hypothesis.
12. Identified that a null hypothesis is needed for
hypothesis testing, or wrote a null hypothesis.
13. Stated that treatment means need to be
significantly different to reject the null hypothesis.
NDSU: Additional Pre and Post Rubric Item Scores
What we learned?
Future Directions



Combine the NDSU (inquiry labs) and UTM
(lecture modules) interventions
Use an expanded rubric for assessment.
Use the Science Process and Reasoning Skills Test
(SPARST) developed by Clarissa Dirks and Mary
Pat Wenderoth.
Acknowledgements



This project was funded by a Catalytic Grant. We
thank Jane Reece, the family of Neil Campbell,
and Pearson for this opportunity.
Clarissa Dirks and Mary Pat Wenderoth gave us
access to SPARST, which we also used as a posttest.
We also thank our students that took part in this
study, and our teaching assistants.
Science Process and Reasoning Skills
Test












Learning Outcome
Identify a hypothesis
Identify a controlling variable
Evaluate the quality of the experiment to test the hypothesis
Choose a supporting hypothesis
Analyze treatments and controlling for variables
Apply knowledge of appropriate controls
Select prediction
Evaluate experimental design to determine if the conclusion is warranted
Evaluate experimental designs to determine which will yield valid results
Compare and identify the variables for which one should control
Compare and identify the variables for which one should control