Creating a Coherent STEM
Gateway at Michigan State
University
A project funded by the AAU STEM
Education Initiative Project
“The first two years of college are the most critical to the
retention and recruitment of STEM majors”
- President's Council of Advisors on
Science and Technology (PCAST, 2012)
AAU STEM Initiative
AAU has launched a five-year initiative in collaboration
with our member universities to improve the quality of
undergraduate teaching and learning in science,
technology, engineering, and mathematics (STEM)
fields. This is not another study or research project on
STEM education. Instead, it is an effort based on
overwhelming existing research to influence the culture
of STEM departments at AAU universities so that faculty
members are encouraged to use student-centered,
evidence-based, active learning pedagogy in their
classes, particularly at the first-year and sophomore
levels.
https://stemedhub.org/groups/aau
Acknowledgements: the MSU AAU
team
•
•
•
•
•
•
•
•
Melanie Cooper
Joe Krajcik
Diane Ebert-May
Danny Caballero
Lynmarie Posey
Bob Geier
Sarah Jardeleza
J.T. Laverty
•
•
•
•
Sonia Underwood
Becky Matz
Cori Fata Hartley
Biological Sciences
Faculty
• Chemistry Faculty
• Physics Faculty
• CNS and Lyman-Briggs
Deans
Change model: build faculty
consensus around the aims and
rewards of reform through:
1. Developing a shared vision for gateway
course transformation in biology,
chemistry and physics
2. Developing policies and structures to
support and reward reform
Structures to reward and support
reform
• STEM Gateway Fellows
– For faculty who excel in STEM gateway courses
(modeled on the prestigious MSU Lilly Fellowships)
• STEM Alliance
– An institution-wide alliance of all entities involved in
STEM education (colleges, research centers etc.), to
facilitate communication and coordinate activities
• DBER postdoctoral fellows to assist faculty in
implementing reform efforts
A shared vision for curriculum reform:
Engage faculty in discussions to build consensus
on key issues.
– What are the core ideas in the discipline?
– What scientific practices are important?
– What cross-cutting concepts make connections
among disciplines
The result – three dimensional learning.
Disciplinary Core Idea:
 Biology
 Disciplinary
significance
 Explanatory Power
 Generative
These are examples not the full list!
 Evolution
 Cell Theory of Life
 Chemistry
 Matter is composed of
atoms
 Molecular structure
predicts macroscopic
properties
 Physics
 Force and momentum
 Waves
Scientific and Engineering Practices
The multiple ways of knowing and doing that scientists and engineers use to study the
natural world and design world.
1. Asking questions and defining
problems
2. Developing and using models
3. Planning and carrying out
investigations and design
solutions
4. Analyzing and interpreting
data
5. Using mathematics and
computational thinking
6. Constructing explanations
and designing solutions
7. Engaging in argument from
evidence
8. Obtaining, evaluating, and
communicating information
Scientific and Engineering Practices
The multiple ways of knowing and doing that scientists and engineers use to study the
natural world and design world.
1. Asking questions and defining
problems
2. Developing and using models
3. Planning and carrying out
investigations and design
solutions
4. Analyzing and interpreting
data
5. Using mathematics and
computational thinking
6. Constructing explanations
and designing solutions
7. Engaging in argument from
evidence
8. Obtaining, evaluating, and
communicating information
Crosscutting Concepts
Ideas that cut across and are important to all science
disciplines
1.
Patterns
2.
Cause and effect
3.
Scale, proportion and quantity
4.
Systems and system models
5.
Energy and matter
6.
Structure and function
7.
Stability and change
NRC Framework for Science Education 2012
Crosscutting Concepts
Ideas that cut across and are important to all science
disciplines
1.
Patterns
2.
Cause and effect
3.
Scale, proportion and quantity
4.
Systems and system models
5.
Energy and matter
6.
Structure and function
7.
Stability and change
NRC Framework for Science Education 2012
National Research Council. A Framework for K-12 Science Education: Practices, Crosscutting
Concepts, and Core Ideas. Washington, DC: The National Academies Press, 2012.
Measuring change
• Data on persistence, grades, affective domain
(motivation, attitudes, expectations) etc.
– MSU will participate in the CIC STEM Learning
Analytics Initiative.
• BUT…
– we know that grades do not necessarily equate
with learning
Our premise:
Engaging faculty to
determine the core ideas,
science practices and
cross cutting concepts
promote change
and changes in
assessment
practices
leads to changes in
classroom practice
We will measure change by describing:
• Classroom practice
– Using the Three Dimensional Learning
Observation Protocol (3D-LOP)
• Course assessments.
– Using the Three Dimensional Learning Assessment
Protocol (3D-LAP)
Assessing the Assessments
The 3D-LAP
National Research Council. A Framework for K-12 Science Education: Practices, Crosscutting
Concepts, and Core Ideas. Washington, DC: The National Academies Press, 2012.
There is "a damaging collusion between
students on the one hand and faculty on the
other –– a collusion in which students agreed to
accept bad teaching provided that they were
given bad examinations."
Quote from Peter Kennedy in “They are not
dumb, they are different” Tobias, 1990.
Clearly assessments may have
different purposes…
• To help students learn
• To measure a students understanding
• To assess a large scale program
But regardless…
“If you don’t assess what’s important,
what’s assessed becomes important”
Assessment should be to “to educate
and improve student performance, not
merely to audit it”
Wiggins, G. (1998). Educative assessment:
Designing assessments to inform and improve
student performance. San Francisco, CA: JosseyBass
We have designed and operationalized
the 3 Dimensional Learning
Assessment Protocol (3D-LAP) to help
us to:
Identify change in assessments over
time
Help faculty design 3D assessments
that provide better evidence of what
students know and can do
Part 1: Assessing 3D Learning
Scientific Practices P1 (Does the item contain a practice - yes/no)
P2 (If there is a practice, which practice is presented)
(SP)
P3 (If there is a practice, is the practice explicit/implicit)
Crosscutting
Concepts
Disciplinary Core
Ideas
CC1 (Is there a crosscutting concept (CCC) - yes/no)
CC2 (If there is a CCC, which CCC is present)
CC3 (If there is a CCC, is the CCC explicit/implicit)
DCI1 (Is there a disciplinary core idea (DCI) - yes/no)
DCI2 (If there is a DCI, which DCI is present)
DCI3 (If there is a DCI, is the DCI explicit/implicti)
Part 1: Assessing 3D Learning
Scientific Practices P1 (Does the item contain a practice - yes/no)
P2 (If there is a practice, which practice is presented)
(SP)
P3 (If there is a practice, is the practice explicit/implicit)
Crosscutting
Concepts
(CC)
Disciplinary Core
Ideas
CC1 (Is there a crosscutting concept (CC) - yes/no)
CC2 (If there is a CC, which CC is present)
CC3 (If there is a CC, is the CC explicit/implicit)
DCI1 (Is there a disciplinary core idea (DCI) - yes/no)
DCI2 (If there is a DCI, which DCI is present)
DCI3 (If there is a DCI, is the DCI explicit/implicti)
Part 1: Assessing 3D Learning
Scientific Practices P1 (Does the item contain a practice - yes/no)
P2 (If there is a practice, which practice is presented)
(SP)
P3 (If there is a practice, is the practice explicit/implicit)
Crosscutting
Concepts
(CC)
Disciplinary Core
Ideas
(DCI)
CC1 (Is there a crosscutting concept (CC) - yes/no)
CC2 (If there is a CC, which CC is present)
CC3 (If there is a CC, is the CC explicit/implicit)
DCI1 (Is there a disciplinary core idea (DCI) - yes/no)
DCI2 (If there is a DCI, which DCI is present)
DCI3 (If there is a DCI, is the DCI explicit/implicit)
Comparison of exams from a traditional and a transformed
course
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
SP
CC
DCI
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
SP
CC
DCI
Implicit SP
Explicit SP
Implicit CCC
Explicit CCC
Implicit DCI
Explicit DCI
Part 2: Question Quality
Phenomena
Does the question address a phenomenon (i.e.,
an observable event)?
Intent
What is the intent or goal of the question?
Explicit
Does the item elicit explicit evidence of student
learning that is aligned with the intent of the
question?
Learning goal
Does the question address an explicit learning
goal?
Question construction
Does the question meet acceptable practices
for valid item construction (e.g., appropriate
level of math and reading literacy, reasonable
number of choices of similar length)?
Example Chemistry Question
Dimension
Present?
Type
Practice
No
N/A
CCC
No
N/A
DCI
No
N/A
Example Physics Question
Dimension
Present?
Type
Practice
No
N/A
CCC
No
N/A
DCI
No
N/A
Sample Biology Question
Dimension
Present?
Type
Practice
No
N/A
CCC
No
N/A
DCI
No
N/A
How will we know when students
understand a phenomenon?
• “To understand a phenomenon is to
understand how it is caused”
– Strevens. M. (2013), No Understanding without
Explanation, Studies in History and Philosophy of
Science 44, 510–515
• Explanation “is the phenomenological mark of
an evolutionarily determined drive”
– Gopnik, A. (1998). Explanation as Orgasm*. Minds
and machines, 8(1), 101-118.
3DLAP Operating Definition of
Explanation
Question asks student to explain a phenomenon,
event, or observation.
• Question gives or requires student to provide the target of the
explanation
• Question requires student to reference scientific principles
and/or data
• Question requires student to provide reasoning linking
scientific principles and/or data to phenomenon, event, or
observation
Chemistry Example
When you mix acetic acid and methyl amine, this reaction occurs.
i) What type of reaction is it?
ii) Indicate what is happening at the molecular level by drawing mechanistic arrows
iii) Justify, using your knowledge of molecular structure and interactions, why this
reaction occurs
Dimension
Present?
Type
Practice
Yes
Explanation
CCC
Yes
Cause and Effect
DCI
Yes
Chemical Reactions
Is it possible to design MC questions with a practice such as
explanation?
Which is a stronger base? CH3OH or CH3NH2? Why?
 CH3NH2
 Claim
Which is a stronger base? CH3OH or CH3NH2? Why?
 CH3NH2 because N is less
electronegative than O
 Claim
 Scientific Principle
Which is a stronger base? CH3OH or CH3NH2? Why?
 CH3NH2 because N is less
electronegative than O and
therefore is better able to
donate a lone pair into a
bond with an acid.
 Claim
 Scientific Principle
 Reasoning
Mulitple Choice Exam Question
Dimension
Present?
Type
Practice
Yes (explicit)
Explanation
CCC
Yes (explicit)
Cause and Effect
DCI
Yes (explicit)
Molecular Structure and
properties
We have designed and operationalized
the 3 Dimensional Learning
Assessment Protocol (3D-LAP) to help
us to:
Identify change in assessments over
time
Help faculty design 3D assessments
that provide better evidence of what
students know and can do
Your tasks:
• Form groups of 3-4 people.
• Construct an open response exam question for a gateway course
using 3DLAP as a guide.
• Use this question to construct one or more multiple choice item(s)
• Assess two dimensions of learning:
– Crosscutting concept: Energy and matter
– Practice: Explanation
• Prepare to share your questions/describe the process with your
group.
• Choose an exemplar (or two) to share with the whole STEM Alliance
when we report out
• You have one hour!
Room assignments
• Physics and Astronomy,1400
– Danny Caballero
• Chemistry and Molecular Biology, 1415
– Melanie Cooper
• Biology, stay here
– Diane Ebert-May
• Implications for Math: 1420
– JT Laverty
Anticipated Outcomes
• Transformed STEM gateway courses – that
address core ideas and practices of the
discipline.
• Transformed teaching practices (emerge as a
consequence of changes in expectations)
• Improved learning outcomes (no inert
knowledge!)
• Cultural change emerges from shared vision