Assessing student skills in
laboratory and research
experiences.
Benjamin D. Caldwell
Department of Chemistry
Missouri Western State University, St. Joseph, MO
April 29, 2014 Experimental Biology 2014, San Diego
BMB Lab Skills

What are the expectations (minimum vs. hopes/dreams!)
for students completing BMB courses/programs

Perform basic tasks adequately vs. designing experiments

Set learning objectives

What are the expectations for students coming into an
introductory BMB course?


Pre-course survey
Determine assess strategy for achievement of objectives

Lab reports, written research/project proposals, presentations,
quizzes, exams, practical exams, etc…
First efforts

Focused on basic BMB lab skills and methods

What skills do we want to teach in our courses?

What are our objectives for program?

How in depth can we want go?

Introductory, theory, usage, proficient, mastery
WHEN SKILL IS LEARNED AND LEVEL
BMB SKILLS
Lab skills
Adapted from BMBEd
Vol. 32, No. 1, pp. 11–16, 2004
LAB TECHNIQUES

Lower Level
Courses
Theory
Usage
Proficiency
Upper Level
Theory
Usage
Competence
Review of matrix and discussion
BMB Courses
The
ory
Usa
ge
Acid/Base chemistry
Affinity techniques
Amino acid analysis
cDNA
Cell Culture-Aseptic/Sterile
Techniques
Cell Culture: Adherents vs.
suspension
Cell Culture-Bactierial/insects
Cell Culture-Mamallian
Centrifugation-high speed
Centrifugation-separation and ppt
Centrifugation-subcellular
fractionation
Centrifugation-ultraspeed
Characterization of carbohydrates
Characterization of lipids
Characterization of proteins
Chromatography-Affinity
Chromatography-ion exchange
Chromatography-size exclusion
Chromatography-HPLC
3/22/2016
Cloning
and Selection
TYPE
4
Competenc
e
Qua
ntitativ
e
Qual
itativ
e
WHEN SKILL IS LEARNED AND LEVEL
BMB SKILLS
Lower Level
Courses
Adapted from BMBEd
Vol. 32, No. 1, pp. 11–16, 2004
LAB TECHNIQUES
The
ory
Usag
e
Proficienc
y
Acid/Base chemistry
C111
C120
C120
C120
Upper Level
BMB Courses
Theor
y
Usag
e
Compete
nce
Theor
y
C321
C370
C321
C370
C321
C370
C321
C370
Affinity techniques
C370
C426
370
Amino acid analysis
C370
cDNA
Cell Culture-Aseptic/Sterile
Techniques
B215
B215
B421
TYPE
Usag
e
Compete
nce
Qua
ntitativ
e
Quali
tativ
e
X
C370
C370
X
B313
B313
X
B421
B421
B215
B421
B215
B421
B421
B390
B421
B390
B421
B390
B421
B431
X
X
B215
B390
B390
B390,
B421
B205
B215
B390
B411
B421
B390
B411
B421
B390
B411
B421
B205
B215
B390
B421
B205
B215
B390
B421
B390
B421
X
X
B390
B390
B390
B421
B390
B421
B390
B421
B390
B421
X
X
B411
B411
C370
C370
C370
C370
C370
C370
C370
C370
Characterization of proteins
C370
C370
C370
C470
C370
Chromatography-Affinity
C370
C370
B215
B215
Cell Culture: Adherents vs.
suspension
B205
B215
B205
B215
Cell Culture-Bactierial/insects
B215
B215
Cell Culture-Mamallian
Centrifugation-high speed
B215
B215
Centrifugation-separation and ppt
Centrifugation-subcellular fractionation
B215
B215
X
X
C370
C370
X
X
X
Centrifugation-ultraspeed
Characterization of carbohydrates
Characterization of lipids
X
X
X
What’s out there on lab assessment?


Experiments, procedures are available in variety of
venues
BMBEd, CBE Life Sci Ed., J. Chem Ed., …

A few lab manuals available FLABB, Boyer, Farrell/Taylor
….
Course design – project based/inquiry based research

Assessment of laboratory skills???

Our efforts


Assessment of student progress through entire lab course
Pre-course survey – assess student
comprehension/retention from earlier courses





Includes attitudinal survey + skills assessment
Midterm-assessment
Final Exam
First half of course focuses skill building
Second half – project based

Protein purification, enzyme kinetics
Pre-course skills survey
Use the following scale, circling the appropriate statement that best matches your feeling
about the following laboratory related questions.
Strongly
Disagree
1
Somewhat
Disagree
2
Neutral/
No Opinion
3
Somewhat
Agree
4
Strongly
Agree
5
1. I am able to solve basic mathematic problems that require scientific notation or exponents.
1
2
3
4
5
3. I can accurately determine the correct number of significant digits in a number.
1
2
3
4
5
79. Complete the following mathematical equation, and give your answer with the proper
number of significant digits.
a. 4.0 x 105
b. 4.00 x 105
c. 4.00367 x 105
d. 400,367
545.1x 21.3

0.029
Pre-Course Survey Sample Questions
Attitudinal Question: I am able to solve basic mathematical problems that require
scientific notation or exponents
Complete the following mathematical equation, and give your answer with the proper
number of significant digits
Strongly
Disagree
Somewhat
Agree
No
Opinion
Somewhat
Agree
Strongly
Disagree
0%
0%
0%
35%
65 %
545.1x 21.3

0.029
Pre-test: 24% Could correctly perform calculation
Mid-term: 88% Could correctly perform calculation
Attitudinal Questions: I can determine the accuracy and precision of a
micropipette from data.
Strongly
Disagree
Somewhat
Agree
No
Opinion
Somewhat
Agree
Strongly
Disagree
6%
0%
12%
23%
59 %
Pre-test: 6% Could correctly determine accuracy/precision
Mid-term: 47% could determine accuracy/precision
Pre-Lab Skills Assessment Results
100
% Correct
80
60
40
20
0
1
2
3
4
5 6 7 8
Question #
9 10 11 12
1. Density
2. Calculating Averages
3. Understand Accuracy
4. Understand Precision
5. Understand Equipment Tolerances
6. Recognize Units: Molarity
7. Recognize Solution & Dilution
Properties
8. Can Perform Dilution Calculations
9. Understand Concepts of Acidity
10. Can Perform pH Calculation from
Molarity
11. Can Calculate [H+] from pH Data
12. Understand How Buffers Maintain pH
I understand & can Identify dependent
variables
0
0
6
54
40
69
I understand the differences between
accuracy & precision
0
0
0
27
73
48
I understand basics of chromatography
0
19
21
29
31
67
I can generate a standard curve &
determine concentrations of unknowns.
0
6
15
46
33
42
I understand the need for more than 1
type of protein assay
8
13
21
44
15
56
I understand the need for more than 1
type of protein assay
8
27
23
35
6
67
13
35
27
19
6
73
8
2
10
23
56
54
I understand the basic concepts &
importance of enzyme kinetics
I understand how electrophoresis is
used to separate protein/DNA
PreLab
Post Lab
100.0
100.0
100.0
88.2
% Correct
82.4
Lab skills Assessment
Pre-Course vs. Mid-term Comparison
100.0
94.1
88.2
82.4
100.0
100.0
100.0
100.0
94.1 94.1 94.1
88.2
88.2
82.4
76.5
76.5 76.576.5
70.6
88.2
82.4
76.5
76.5 76.5 76.5
70.6
64.7
64.7
58.858.8
58.8
64.7
58.8
58.858.8
52.9
52.9
52.9
47.1
47.1
41.2
41.2
41.2
41.2
41.2
29.4
23.5
47.1
41.2
35.3
35.3 35.3
29.4
47.1
41.2
41.2
47.1
47.1
41.2
41.2
29.4
23.5
11.8
5.9
11.8
5.9
0.0
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36
Question #
Final Exam

Given: Formal report on purification of protein
characterization

Introduction to the enzyme
Materials and Methods
Data and results

No Conclusions or Discussion


Exam Question 5
What type of experiment was used to detect the presence
of the enzyme specifically in fractions collected from the gel
filtration column? What substance and physical property
were measured directly in this experiment to detect the
presence of ATPase acivity?
Learning Objectives:
• Demonstrate understanding of enzyme assays and assay
methodologies.
• Demonstrate ability to compare data from different
types of experimental procedures and data.
Exam Question 8
Were the pipettes used in the protein assays all accurate
and precise? Indicate which pipettes were accurate,
precise, both or neither. Also, indicate why any pipettes
were determined to not be accurate or precise.
Learning Objectives:
• Demonstrate ability to perform basic statistical
calculations and comparisons.
• Demonstrate understanding of the concepts of accuracy
vs. precision.
Exam Question 9
What are the Km and Vmax values for the enzyme in the
presence of MgATP and the analog CrATP. Which
compounds binds more tightly to the enzyme, MgATP or
CrATP, and explain how the data support your conclusion.
Learning Objectives:
• Demonstrate understanding of enzyme kinetics and
inhibition.
• Demonstrate ability to use data to perform biochemical
calculations
Exam Question 10
Write a conclusion paragraph for this lab report. Be sure
to comment briefly on a) the steps and procedures used to
purify the enzyme, b) methods used to specifically identify
fractions containing the protein, and c) compare the
relative kinetics of the enzyme using MgATP or CrATP
binds more tightly to the enzyme and possible mechanism
of inhibition as demonstrated by the presented data.
Learning Objectives:
• Demonstrate ability to synthesize concluding statements
based on a large body of work using different laboratory
methods and multiple forms of data representations.
• Demonstrate the ability to communicate effectively.
Final Exam Results
Student Performance on final exam questions.
Class
Q1
Q2
Q3
Q4
Q5
Q6
Q7
Q8
Q9
Bonus
Q10
Q's
Spring
92.2 76.3 92.5 75.6 66.7 64.4 68.1 80.4 56.3 81.5
2010
Spring
95.6 68.0 83.5 75.7 73.2 73.5 68.5 81.4 69.7 84.6
2011
Scores represent percentage of students receiving full credit for each question.
Average
Exam
Score (%)
70.4
79.2
65.2
82.9
Able to identify areas still needing improvement (indicated by yellow)
include:
• Interpreting graphic data (Q’s 2, 6, 9) – graphical/SDS-PAGE results
• Comprehension of enzyme assays and kinetics (Q’s 5, 9)
• Performing biochemical calculations (Q’s 7, 9)
Conclusion
- Course wide approach to assessing and tracking student
achievement of introductory biochemistry laboratory lab
skills
Future work:
Incorporate attitudinal component into final exam study,
possibly include exit interviews
Acknowledgments
Dr. Melissa Daggett, MWSU Biology Department
Merrill Schaeffer, MWSU/Boehringer Ingelheim Vetmedica
CHE 370 Students, MWSU