Using Technology for Content Delivery,
Formative Assessment, and Reflection
Michael S. Kirkpatrick
JMU Computer Science
Bridgewater College
Annual Pedagogy Project 2015
Using Technology for Content Delivery, Formative Assessment, and Reflection
Bridgewater College Annual Pedagogy Project 2015 • Dr. Michael S. Kirkpatrick
Workshop Objectives
At the completion of this session, participants will
demonstrate progress toward the following
objectives:
•Summarize relevant literature for active learning
•Explain how to use videos for effective content delivery
•Describe how to incorporate formative assessment into a
flipped classroom
•Identify practical strategies to encourage students’ day-today class preparation and metacognition
Using Technology for Content Delivery, Formative Assessment, and Reflection
Bridgewater College Annual Pedagogy Project 2015 • Dr. Michael S. Kirkpatrick
Agenda
• Introductions and welcome
• Active learning preassessment
• Camtasia lessons learned
• Formative assessment and PI
• Metacognition and reflection
• Discussion and exploration
Using Technology for Content Delivery, Formative Assessment, and Reflection
Bridgewater College Annual Pedagogy Project 2015 • Dr. Michael S. Kirkpatrick
“Adopting instructional practices that engage
students in the learning process is the
defining feature of active learning.”
-Michael Prince
Using Technology for Content Delivery, Formative Assessment, and Reflection
Bridgewater College Annual Pedagogy Project 2015 • Dr. Michael S. Kirkpatrick
Benefits Illustrated
Measure of performance gain
•Mechanics Diagnostic (MD)
or Force Concept Inventory
(FCI)
•62 courses (14 trad.) at
multiple institutions
•6542 students (2084)
R.R. Hake, "Interactive-engagement vs traditional methods: A six-thousand-student survey of
mechanics test data for introductory physics courses," Am. J. Phys. 66, 64- 74 (1998).
http://www.physics.indiana.edu/~sdi/ajpv3i.pdf
Using Technology for Content Delivery, Formative Assessment, and Reflection
Bridgewater College Annual Pedagogy Project 2015 • Dr. Michael S. Kirkpatrick
Collaborative and Cooperative Learning
M. Prince, “Does active learning work? A review of the research.” J. Eng. Education 93(3), 223- 241, 2004.
http://www4.ncsu.edu/unity/lockers/users/f/felder/public/Papers/Prince_AL.pdf
Using Technology for Content Delivery, Formative Assessment, and Reflection
Bridgewater College Annual Pedagogy Project 2015 • Dr. Michael S. Kirkpatrick
Problem-Based Learning
M. Prince, “Does active learning work? A review of the research.” J. Eng. Education 93(3), 223- 241, 2004.
http://www4.ncsu.edu/unity/lockers/users/f/felder/public/Papers/Prince_AL.pdf
Using Technology for Content Delivery, Formative Assessment, and Reflection
Bridgewater College Annual Pedagogy Project 2015 • Dr. Michael S. Kirkpatrick
Active Learning in STEM
Meta-analysis of 225 studies
•158 studies: average 0.47 SDs
better on CIs/exams
•67 studies: average failure rate
dropped from 33.8% to 21.8%
with active learning
S. Freeman et al., “Active learning increases student performance in science, engineering, and mathematics,”
Proceedings of the National Academy of Sciences 111(23), 8410- 8415, 2014.
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC4060654/pdf/pnas.201319030.pdf
Using Technology for Content Delivery, Formative Assessment, and Reflection
Bridgewater College Annual Pedagogy Project 2015 • Dr. Michael S. Kirkpatrick
From Content Delivery to
Formative Assessment
Using Technology for Content Delivery, Formative Assessment, and Reflection
Bridgewater College Annual Pedagogy Project 2015 • Dr. Michael S. Kirkpatrick
Using Technology for Content Delivery, Formative Assessment, and Reflection
Bridgewater College Annual Pedagogy Project 2015 • Dr. Michael S. Kirkpatrick
Using Technology for Content Delivery, Formative Assessment, and Reflection
Bridgewater College Annual Pedagogy Project 2015 • Dr. Michael S. Kirkpatrick
Using Technology for Content Delivery, Formative Assessment, and Reflection
Bridgewater College Annual Pedagogy Project 2015 • Dr. Michael S. Kirkpatrick
Using Technology for Content Delivery, Formative Assessment, and Reflection
Bridgewater College Annual Pedagogy Project 2015 • Dr. Michael S. Kirkpatrick
Lessons Learned
Be mindful of accessibility
•Never use red and green contrasts
•Create subtitles or transcripts as needed
Write a script first
•Doing so will save you time later
•Makes for easy transcript creation
Align videos with learning objectives
•Use a discussion forum for students to ask questions
•Incorporate an ungraded Moodle quiz for practice
Using Technology for Content Delivery, Formative Assessment, and Reflection
Bridgewater College Annual Pedagogy Project 2015 • Dr. Michael S. Kirkpatrick
Formative Assessment and
Peer Instruction
Using Technology for Content Delivery, Formative Assessment, and Reflection
Bridgewater College Annual Pedagogy Project 2015 • Dr. Michael S. Kirkpatrick
“Feedback is value-neutral help on worthy tasks. It
describes what the learner did and did not do in
relation to her goals. It is actionable information, and it
empowers the student to make intelligent adjustments
when she applies it to her next attempt to perform.”
-Grant Wiggins
Using Technology for Content Delivery, Formative Assessment, and Reflection
Bridgewater College Annual Pedagogy Project 2015 • Dr. Michael S. Kirkpatrick
Peer Instruction
Peer Instruction
•Created by Eric Mazur (Harvard)
•Augment class with ConcepTests
• Expose common misconceptions
• Think-vote-pair-revote pattern
•E. Mazur, Peer Instruction: A User’s Manual,
1996.http://mazur.harvard.edu/research/detailspage.php?rowid=8
Using Technology for Content Delivery, Formative Assessment, and Reflection
Bridgewater College Annual Pedagogy Project 2015 • Dr. Michael S. Kirkpatrick
Physics Question 1
Using Technology for Content Delivery, Formative Assessment, and Reflection
Bridgewater College Annual Pedagogy Project 2015 • Dr. Michael S. Kirkpatrick
Closing the Gender Gap
Cooperative learning closes the
gender gap
•Pretest scores were 10%
points higher for men
•Gap persisted with lecture
alone
•Posttest results for cooperative
classes were almost equal
PI can eliminate gender gap in physics
•T: traditional lectures
•IE: interactive lectures
•IE+: interactive assignments, lectures, tutorials
E. Mazur, “The scientific approach to teaching: Research as a basis for course design,” keynote/plenary
talk at the International Computing Education Research Conference (ICER), 2011.
http://mazur.harvard.edu/search-talks.php?function=display&rowid=1712
Using Technology for Content Delivery, Formative Assessment, and Reflection
Bridgewater College Annual Pedagogy Project 2015 • Dr. Michael S. Kirkpatrick
Closing the Gender Gap
Traditional lectures leave
women behind
•Women tend to have smaller
performance gains
Cooperative learning improves
gains for women
•...but men improve as well
E. Mazur, “The scientific approach to teaching: Research as a basis for course design,” keynote/plenary
talk at the International Computing Education Research Conference (ICER), 2011.
http://mazur.harvard.edu/search-talks.php?function=display&rowid=1712
Using Technology for Content Delivery, Formative Assessment, and Reflection
Bridgewater College Annual Pedagogy Project 2015 • Dr. Michael S. Kirkpatrick
Physics Question 2
Using Technology for Content Delivery, Formative Assessment, and Reflection
Bridgewater College Annual Pedagogy Project 2015 • Dr. Michael S. Kirkpatrick
Demos and Engagement
Performance and understanding increase with engagement
•Those who only observe sometimes learn it wrong
•Those who discuss show clearer reasoning and provide
partially correct answers
Using Technology for Content Delivery, Formative Assessment, and Reflection
Bridgewater College Annual Pedagogy Project 2015 • Dr. Michael S. Kirkpatrick
Physics Question 3
Using Technology for Content Delivery, Formative Assessment, and Reflection
Bridgewater College Annual Pedagogy Project 2015 • Dr. Michael S. Kirkpatrick
Confusion and Understanding
“Please tell us briefly what points of reading you found most
difficult or confusing.”
•“Nothing was difficult or confusing.”
•“I found the explanation inadequate. I don’t understand the
reasoning that led to the conclusion.”
Using Technology for Content Delivery, Formative Assessment, and Reflection
Bridgewater College Annual Pedagogy Project 2015 • Dr. Michael S. Kirkpatrick
Best Practices for Formative Assessment
Identify the learning gap
•Space between what students know and need to know
Bidirectional feedback
•Identify student progress and suggest corrections
Actively engage students
•Students need to assess their own understanding
Create learning progressions
•Break larger goal into subgoals
Using Technology for Content Delivery, Formative Assessment, and Reflection
Bridgewater College Annual Pedagogy Project 2015 • Dr. Michael S. Kirkpatrick
Best Practices for Formative Assessment
Ask clear questions
•Avoid ambiguity, use only one verb
Psychological safety
•Positive reinforcement, accept imperfection
Sequencing and balance
•Consider the order and type of questions, activities
Wait time
•Come ot terms with silence
Avoid pimping questions
•Do not try to establish intellectual superiority
Using Technology for Content Delivery, Formative Assessment, and Reflection
Bridgewater College Annual Pedagogy Project 2015 • Dr. Michael S. Kirkpatrick
Using Technology for Content Delivery, Formative Assessment, and Reflection
Bridgewater College Annual Pedagogy Project 2015 • Dr. Michael S. Kirkpatrick
Metacognition Space Race
Using Technology for Content Delivery, Formative Assessment, and Reflection
Bridgewater College Annual Pedagogy Project 2015 • Dr. Michael S. Kirkpatrick
Metacognitive Activities
Preassessments
•What do I already know about this topic?
Muddiest points
•What am I still confused on?
Exam corrections
•Why did I miss this question?
Documented problem solving
•What were the steps I used to solve this problem?
Learning progress journals
•How did my understanding of this concept change?
Using Technology for Content Delivery, Formative Assessment, and Reflection
Bridgewater College Annual Pedagogy Project 2015 • Dr. Michael S. Kirkpatrick
2
RSQC
Recall the most important points
•Requires student to analyze relative importance
Summarize the most important points
•Provides practice with comprehension
Construct a question you would like answered
•Encourages reflection and evaluation
Connect this material to other concepts
•Establishes scaffolding from previously learned material
Comment on your learning progress this week
•Uses metacognitive reflection to instill study habits
Using Technology for Content Delivery, Formative Assessment, and Reflection
Bridgewater College Annual Pedagogy Project 2015 • Dr. Michael S. Kirkpatrick
Tools for Metacognitive Activities
Socrative
•Good use: quick in-class polling
Qualtrics
•Good use: offline, anonymous surveys
Moodle
•Good use: graded and offline assessments
Social media
•Good use: blogs as learning journals
Piazza
•Good use: collaborative student discussions and answers
Using Technology for Content Delivery, Formative Assessment, and Reflection
Bridgewater College Annual Pedagogy Project 2015 • Dr. Michael S. Kirkpatrick
Best Practices for Metacognition
Develop metacognitive culture
•Give students freedom to be confused
•Integrate reflection into credited course work
•Model metacognitive behavior
Teach the concept and language of metacognition
•Explicit instruction over time expands skill set
Reflect the specific learning context
•Metacognition is NOT generic
Externalize mental events
•Increase accurate awareness of strengths and weaknesses
Using Technology for Content Delivery, Formative Assessment, and Reflection
Bridgewater College Annual Pedagogy Project 2015 • Dr. Michael S. Kirkpatrick
Bloom’s Taxonomy
Using Technology for Content Delivery, Formative Assessment, and Reflection
Bridgewater College Annual Pedagogy Project 2015 • Dr. Michael S. Kirkpatrick
Discussion and Exploration
Using Technology for Content Delivery, Formative Assessment, and Reflection
Bridgewater College Annual Pedagogy Project 2015 • Dr. Michael S. Kirkpatrick
Other Technology Tools
Twitter
•Good use: announcements, highlight news stories
CATME
•Good use: team formation, peer evaluation
Top Hat Monocle
•Alternative to Socrative (requires paid subscription)
Asynchronous MOOC Videos
•edX, Kahn Academy, MIT OpenCourseWare
Using Technology for Content Delivery, Formative Assessment, and Reflection
Bridgewater College Annual Pedagogy Project 2015 • Dr. Michael S. Kirkpatrick
Resources
Peer Instruction (PI)
•E. Mazur, Peer Instruction: A User’s Manual, 1996.
•http://mazur.harvard.edu/research/detailspage.php?rowid=8
Formative Assessment
•T. A. Angelo and K. P. Cross, Classroom Assessment Techniques: A Handbook for
College Teachers, 1993.
•http://wvde.state.wv.us/teach21/ExamplesofFormativeAssessment.html
http://cft.vanderbilt.edu/guides-sub-pages/cats/
Metacognition
•E. Cook, E. Kennedy, S. Y. McGuire, “Effect of Teaching Metacognitive
Learning Strategies on Performance in General Chemistry Courses,” J. Chem.
Educ., 2013, 90 (8), pp 961–967.
•http://www.lmu.edu/Assets/Centers+$!2b+Institutes/Center+for+Teaching+
Excellence/Teach+STEM+Students+How+to+Learn-Metacognition+is+the+Key!+Slides.pdf
Using Technology for Content Delivery, Formative Assessment, and Reflection
Bridgewater College Annual Pedagogy Project 2015 • Dr. Michael S. Kirkpatrick
Resources
Others
•J. D. Bransford, A. L. Brown, and R. R. Cocking, How People Learn: Brain,
Mind, Experience, and School, 2000.
•M. Weimer, Learner-Centered Teaching: Five Key Changes to Practice, 2002.
•G. Wiggins and J. McTighe, Understanding by Design, 2005.
•N. Pinchok and W. C. Brandt, “Connecting Formative Assessment Research
to Practice: An Introductory Guide for Educators,” 2009.
http://www.learningpt.org/pdfs/FormativeAssessment.pdf
•N. Chick, Metacognition. http://cft.vanderbilt.edu/guides-subpages/metacognition/
•T. Tofade, J. Elsner, S. T. Haines, “Best Practice Strategies for Effective Use
of Questions as a Teaching Tool,” Am J Pharm Educ. 2013 Sep 12; 77(7): 155.
Using Technology for Content Delivery, Formative Assessment, and Reflection
Bridgewater College Annual Pedagogy Project 2015 • Dr. Michael S. Kirkpatrick