An Introduction to Scholarly
Teaching and the Scholarship of
Teaching and Learning
!
OPID Faculty College, May 28–31, 2013
David J. Voelker
Associate Professor of Humanistic Studies and History
University of Wisconsin–Green Bay
Teaching)is)not)a)lab)science;)even)teaching)lab)science)is)not)a)lab)science!))
S3ll,)the)learner’s)mind)is)not)so)much)a)black)box—totally)inscrutable—as)it)
is)a)gray)box,)with)a)dim)view.))We)can)observe)some)of)what)goes)in;)we)can)
exercise)some)control)over)how)inputs)are)processed;)and)we)can)assess)
some)of)what)comes)out.))The)virtue)of)SoTL)is)not)that)it)gives)us)full)
command)of)the)learning)process)but)rather)that)it)heightens)our)awareness)
of)what)we)want)students)to)learn,)how)we)can)guide)their)learning)
experiences,)and)how)we)will)know)that)they)have)indeed)learned.
Contact • voelkerd@uwgb.edu
Website • http://www.thegraybox.net/ (includes shareable handouts)
© 2013 David J. Voelker (Please do not post online or distribute without permission.)
David'Voelker'•'OPID'2013'FC
WHAT$IS$THE$SCHOLARSHIP$OF$TEACHING$AND$LEARNING?
!
There!have!been!many!a,empts!to!define!the!Scholarship!of!Teaching!and!Learning!
(SoTL)!in!the!years!since!1990,!when!Ernest!Boyer!coined!the!phrase!“scholarship!of!teaching,”!
but!perhaps!the!most!pithy!definiFon!comes!from!Tony!Ciccone!(UWJMilwaukee),!former!coJ
director!of!the!Wisconsin!Teaching!Fellows!&!Scholars!program,!who!has!defined!SoTL!as!
“Scholarly!inquiry!into!student!learning![that]!advances!the!pracFce!of!teaching.”*!
!
This!definiFon!casts!SoTL!as!serious!scholarship—as!rigorous!as!our!disciplinary!
research.!!The!focus!of!SoTL,!most!of!the!Fme,!is!the!relaFonship!between!teaching!and!
learning.!!We!want!this!relaFonship!to!be!as!strong!and!posiFve!as!possible!!!SoTL!researchers!
thus!study!the!relevant!scholarly!literature,!collect!and!analyze!evidence!of!student!learning,!
and!share!their!work!in!
order!to!help!improve!
How$does$“Scholarly$Teaching”$differ$from$SoTL?
teaching!effecFveness.
Scholarly'teaching'can'be'seen'as'a'less'formal'version'of'SoTL'that'stops'
!
Although!SoTL!is!
short'of'presentaFons'and'publicaFons'(and'thus'does'not'require'IRB'
informed!by!cogniFve!
approval).''Scholarly'teachers'engage'with'the'literature'on'teaching'and'
science!and!SoTL!
learning,'idenFfy'learning'problems,'carefully'assess'student'learning,'and'
pracFFoners!someFmes!
seek'conFnually'to'improve'their'own'teaching'effecFveness.''For'a'full'
use!social!scienFfic!
discussion,'see'Gurung'and'Schwarz'(2009),'pp.'14U15.
methods,!SoTL!is!not!
reducible!to!cogniFve!or!educaFonal!psychology.!!Rather,!educators!from!across!the!academy!
(arts,!humaniFes,!social!sciences,!natural!sciences,!and!professional!studies)!apply!the!scholarly!
values!and!techniques!of!their!own!disciplines!to!the!study!of!student!learning,!borrowing!
methods!from!other!disciplines!as!needed!to!answer!their!specific!research!quesFons.!!In#the#
most#expansive#sense,#SoTL#is#a"way"of"thinking#about#how#to#improve#teaching#and#learning#
and#an#ongoing"strategy#for#doing#so,#embodied#in#an#interdisciplinary,#scholarly#discourse.!!
As!such,!SoTL!provides!a!means!of!enhancing!teaching!and!protecFng!the!integrity!of!higher!
educaFon!(especially!when!used!to!support!insFtuFonal!assessment).
*!Tony!Ciccone,!“Advancing!the!Prac5ce!of!Teaching!Through!Inquiry!into!Student!Learning”![workshop!packet],!OPID!Faculty!
College!2006,!p.!1.!!Ciccone!notes!that!this!working!defini5on!of!SoTL!was!developed!in!collabora5on!with!OPID!and!UWP
Milwaukee!colleagues.
2
David'Voelker'•'OPID'2013'FC
!
There!are!many!different!kinds!of!SoTL!projects,!but!they!usually!come!back!to!student!
learning!in!one!way!or!another.!!This!workshop!will!focus!on!helping!you!develop!a!scholarly!
teaching!or!SoTL!project!focused)on)a)learning)problem.!!You!might!consider!projects!based!on!
Pat!Hutchings’s!influenFal!taxonomy:
• What!is!the!case?!!(This!sort!of!project!seeks!to!describe!current!teaching!and!learning!
pracFce!in!order!to!gain!a!be,er!understanding!of!a!learning!problem.)
• What!works?!!(This!sort!of!project!seeks!to!evaluate!a!tradiFonal!or!innovaFve!teaching!
and!learning!pracFce!in!order!to!assess!its!effecFveness.)
• What!if?!!(This!sort!of!project!explores!the!possibiliFes!not!only!of!new!teaching!and!
learning!pracFces!but!also!of!new!teaching!and!learning!goals.)
• TheoryJBuilding!!(This!sort!of!project!seeks!to!develop!ways!of!conceptualizing!teaching!
and!learning,!ocen!within!a!parFcular!discipline.)†
These!categories!might!help!you!envision!a!project,!but!in!this!workshop!we!will!focus!less!on!
these!categories!and!more!on!defining!the!desired!learning!outcomes,!idenFfying!learning!
problems,!imagining!soluFons,!and!figuring!out!what!sort!of!evidence!you!might!collect!in!order!
to!assess!student!learning.
Workshop"Homework
For#Thursday,#May#30:
• Work!on!steps!1–4!of!the!“Developing!a!SoTL!Project”!worksheet.!!(pp.!5–6)
• Try!to!complete!at!least!2–3!of!the!“Templates!for!Analyzing!Learning!Problems.”!!(pp.!8–15)
• Reflect!briefly!and!jot!down!notes!about!any!quesFons!you!have!so!far.!!Consider:!What!
seems!difficult!about!doing!SoTL!and!scholarly!teaching?
For#Friday,#May#31:
• Work!on!steps!6–8!of!the!“Developing!a!SoTL!Project”!worksheet.!!(p.!7)
• Read!the!“Finding!Evidence!of!Learning”!table!(p.!16)!and!try!to!fill!in!the!rightJhand!column.!!
(Hint:!Use!the!numbered!list!below!the!table!and!add!some!ideas!of!your!own.!!There!is!no!
single!correct!response.)
• Reflect!briefly!and!jot!down!notes!about!any!quesFons!you!have!so!far.!!Consider:!What!else!
would!you!need!to!know!in!order!to!embark!on!a!SoTL!or!scholarly!teaching!project?
Acknowledgments
I"would"like"to"thank"the"following"people"for"sharing"ideas"related"to"this"workshop:"Angie"
Bauer,"Lendol"Calder,"Bill"Cerbin,"Nancy"Chick,"Tony"Ciccone,"La"Vonne"Cornell@Swanson,"Aeron"
Haynie,"Jennifer"Heinert,"Ruth"Homrighaus,"Regan"Gurung,"and"Ryan"MarHn.""I"also"benefited"
from"feedback"from"parHcipants"in"the"2012"Faculty"College.""Perhaps"the"best"thing"about"OPID@
style"SoTL"is"the"way"that"it"brings"together"educators"from"all"across"the"academy!
†!Taxonomy!adapted!from!Pat!Hutchings,!“Approaching!the!Scholarship!of!Teaching!and!Learning,”!in!Opening"Lines:"
Approaches"to"the"Scholarship"of"Teaching"and"Learning,"edited!by!Pat!Hutchings!(Menlo!Park,!Calif.:!Carnegie!Founda5on!for!
the!Advancement!of!Teaching,!2000),!4P6,!online!at:!hYp://5nyurl.com/7ura7fm
3
David'Voelker'•'OPID'2013'FC
BASIC$STEPS$FOR$CONDUCTING$A$SOTL$PROJECT
1. Develop!research!quesFon!about!student!learning.
—See$“Developing$a$SoTL$Project$Focused$on$a$Learning$Problem”$(p.$5)$and$“Templates$for$
Analyzing$Learning$Problems”$(pp.$8–15).
2. Begin!literature!review!(to!help!refine!quesFon).
!
—See$“Engaging$with$the$Literature$on$Teaching$and$Learning”$(p.$17–20).
3. Devise!plan!to!collect!evidence!of!student!learning.
$
—See$“Finding$Evidence$of$Learning”$(p.$16).
4. ConFnue!literature!review.
5. OpFonal:!Consult!with!possible!collaborators.
—Consider$advantages$of$comparison$groups,$addiQonal$data$collecQon,$and$methodological$
knowledge$that$come$with$collaboraQon.
6. Seek!approval!of!InsFtuFonal!Review!Board.
$
—See$“NavigaQng$the$InsQtuQonal$Review$Board$(IRB)$Process”$(p.$23).
$
—Note:$Some$projects$do$not$involve$an$intervenQon.$$(See$“What$is$the$case?”$on$p.$8.)
$
—See$the$sample$coding$rubric$below$(pp.$21–22).
7. OpFonal:!Implement!teaching/learning!“intervenFon.”
8. Collect!evidence/data.
9. Analyze!evidence!and!draw!conclusions.
10. Determine!if!project!is!complete!or!needs!another!phase.
—SoTL$projects$o[en$benefit$from$mulQple$rounds$of$data$collecQon,$with$various$tweaks$along$
the$way.
11. Make!your!results!public!
—Consider$presenQng$at$the$OPID$spring$conference,$UW$campus$teaching$conferences,$
disciplinary$conferences,$and/or$ISSOTL$to$receive$feedback$before$seeking$publicaQon.
4
David'Voelker'•'OPID'2013'FC
DEVELOPING$A$SOTL$PROJECT$FOCUSED$ON$A$LEARNING$PROBLEM:
A$STEP-BY-STEP$GUIDE
//What#are#your#learning#goals,#and#what#problems#do#they#raise?
In!How)Learning)Works,!Susan!Ambrose,!et)al.,!define!learning!as!“a!process”!that!"involves!
change!in!knowledge,!beliefs,!behaviors,!or!artudes"!and!is!"the!direct!result!of!how!
students!interpret!and!respond!to!their!experiences"!(p.!3).!!A!learning!problem!might!thus!
be!understood!as!any!barrier!to!the!learning!process.!!To!idenFfy!a!researchable!learning!
problem,!consider:!!What!fundamental!learning!are!you!trying!to!culFvate!in!your!course?!!
How!does!your!exisFng!course!design!(teaching!strategies,!assignments,!assessments,!etc.)!
already!support!this!desired!learning?!!What!learning!problems!present!a!barrier!to!student!
learning?!!Why!do!these!learning!problems!present!a!teaching!challenge?!!Make#a#short#list#
of#goals#and#problems#for#one#parGcular#course#that#you#teach.##Then#select#one#goal/
problem#set#that#seems#especially#important#and#problemaGc#for#further#exploraGon.
Learning#Goals:#Students"should"be"able"to:
Learning#Problems:
//What’s#the#nature#of#the#learning#problem#you#selected#above?
Is!the!learning!problem!specific!to!your!discipline,!or!is!it!a!general!learning!challenge?!!How!
does!the!exisFng!literature!on!teaching!and!learning!help!you!define!the!problem?!!Does!the!
problem!involve,!for!example,!disciplinary!“moves,”!a!disciplinary!“bo,leneck”!or!“threshold!
concept,”!disciplinary!big!ideas,!knowledge!structures!(mental!schema!for!organizing!
knowledge),!problemaFc!prior!knowledge,!artudinal!or!moFvaFonal!barriers,!or!challenges!
involving!learning!how!to!learn?!!How!might!these!categories!help!you!frame!the!problem?!!
(See!the!“Templates!for!Analyzing!Learning!Problems”!on!p.!8!for!help.)
5
David'Voelker'•'OPID'2013'FC
//What’s#the#scope#of#the#teaching#and#learning#challenge?
What!steps!do!you!already!take!to!culFvate!the!desired!learning!(as!opposed!to!simply!
transmirng!it)?!!How!lengthy!is!this!teaching!and!learning!process?!!Can!the!learning!be!
culFvated!and!assessed!through!a!single!assignment!or!exercise,!through!a!course!unit!of!
several!assignments!or!exercises,!or!should!you!be!thinking!about!overall!course!design?!!
Gaining!clarity!about!the!scope!of!the!learning!problem!will!help!you!figure!out!how!to!teach!
to!the!problem!and!also!when!and!how!to!assess!the!learning.
//How#can#you#best#study#the#learning#problem?
Do!you!want!to!study!the!effecFveness!of!a!current!pracFce?!!Do!you!want!to!try!to!improve!
a!current!pracFce!and!assess!the!effecFveness!of!the!change?!!Or,!do!you!want!to!study!the!
impact!of!a!new!pracFce!or!“intervenFon”?!!(If!you!think!that!you!want!to!study!a!new!
intervenFon,!consider!studying!your!current!pracFce!first!in!order!to!gather!some!benchmark!
data!and!to!deepen!your!understanding!of!the!learning!challenge.!!See!the!“What!is!the!
Case?”!template!below.)
//How#does#the#exisGng#SoTL#literature#help#you#define#your#research#quesGon?
How!does!the!relevant!scholarly!literature!help!you!restate!the!learning!problem!and!define!
what!the!desired!learning!might!look!like?!!Be!sure!to!consider!general!SoTL!research,!
disciplineJspecific!scholarship,!and!scholarship!from!related!fields.!!(See!below!for!a!guide!to!
engaging!with!the!literature!on!teaching!and!learning.!!You!will!likely!need!to!return!to!this!
step!acer!delving!into!this!literature.)
6
David'Voelker'•'OPID'2013'FC
//What#kinds#of#evidence#might#you#collect#to#assess#learning?
Asking!this!quesFon!(and!trying!new!assessment!strategies)!will!help!you!bring!your!regular!
course!assessments!into!alignment!with!your!highest!learning!prioriFes!for!your!students.
Consider!using!embedded!assessment!(addiFonal!analysis!of!regular!class!assignments!and!
exams);!qualitaFve!analysis!of!think!alouds,!concept!maps,!essays,!journals,!or!poruolios;!
crossJcourse!comparison!of!student!exam!results;!and!surveys!or!focus!groups!to!gather!
informaFon!about!student!percepFons!and!learning!pracFces.!!Depending!on!the!nature!of!
your!research!quesFon,!you!may!find!it!useful!to!collect!more!than!one!form!of!evidence!or!
data.!!Reminder:!if!you!plan!to!publish!or!present!your!findings,!you!must!secure!IRB!approval !
before!you!collect!any!data.!!(See!below!for!Fps!for!navigaFng!the!IRB!process.)
//How#will#you#contextualize#and#analyze#the#evidence#that#you#collect?
Before!you!finalize!your!project,!you!should!consider!how!you!will!make!sense!of!the!
evidence!or!data!that!you!collect.!!What!will!be!the!frame!of!reference!for!the!data?!!
PossibiliFes!include!but!are!not!limited!to:!preJtest/postJtest!research!design;!use!of!a!
comparison!or!control!group;!evaluaFon!of!student!learning!using!an!established!rubric!or!
instrument!(e.g.,!Bloom’s!taxonomy,!AACU’s!VALUE!rubrics,!or!the!ColorJBlind!Racial!
Artudes!Scale);!and!evaluaFon!of!student!learning!according!to!a!rubric!of!your!own!design!
(but!informed!by!disciplinary!standards).
//Can#you#imagine#meaningful#outcomes#for#your#project?
Have!you!situated!your!learning!goal!and!problem!within!the!context!of!a!scholarly!
discourse?!!Will!your!envisioned!approach!produce!evidence!of!significant!student!learning?!!
Will!the!evidence!that!you!have!gathered!be!persuasive!to!the!intended!audience!in!your!
field!or!in!the!SoTL!community!generally?!!Perhaps!most!importantly,!will!the!project!help!
you!improve!learning!in!your!own!course(s)?!!If!the!answer!to!these!quesFons!is!“yes,”!then!
you!are!probably!ready!to!move!forward!
7
David'Voelker'•'OPID'2013'FC
TEMPLATES$FOR$ANALYZING$LEARNING$PROBLEMS
Note:)Each)of)the)templates)below)is)structured)to)help)you)ar3culate)a)learning)goal,)a)learning)
problem,)a)teaching)interven3on)or)strategy,)and)a)means)of)collec3ng)evidence)of)learning.))I)
adapted)the)idea)from)a)general)SoTL)project)template)created)by)historian)Lendol)Calder,)who)
was)in)turn)inspired)by)the)argument)templates)of)Gerald)Graff)and)Cathy)Birkenstein.))Feel)free)
to)“hack”)these)templates)to)suit)your)needs.
What'is'the'Case?'–'p.'8
Transfer'and'ApplicaFon'of'Knowledge'–'p.'12
ProblemaFc'Prior'Knowledge'–'p.'9
Feedback'and'Learning'–'p.'13
Disciplinary'“Moves”'–'p.'10
Learning'to'Learn'–'p.'14
Threshold'Concepts'–'p.'11
Awareness,'AppreciaFon,'or'Actude'–'p.'15
What#is#the#Case?
The!most!basic!kind!of!SoTL!quesFon!does!not!involve!tesFng!out!a!new!assignment!or!
“intervenFon”!but!rather!focuses!on!a!“what!is”!quesFon—as!in,!“what’s!going!on?”!!Because!
this!kind!of!quesFon!seems!so!basic,!it!is!ocen!neglected.!!But!figuring!out!what’s!happening!
with!student!learning!can!be!a!crucial!first!step!toward!more!effecFve!course!design!and!
teaching.!!When!asking!a!“what!is”!quesFon,!you!keep!your!course!design!and!teaching!
methods!stable,!while!invesFgaFng!how!students!are!studying,!where!they!are!having!difficulty,!
etc.!!You!can!use!some!combinaFon!of!surveys,!interviews,!reflecFve!journals,!think!alouds,!and!
focused!analysis!of!submi,ed!assignments!in!order!to!formulate!a!careful!descripFon!of!how!
and!what!students!are!learning!and!where!and!why!they!are!having!difficulFes.!!The!results!of!
this!research!alone!may!be!worth!sharing,!or!they!can!be!used!to!inform!an!“intervenFon”!that!
in!turn!warrants!further!research.
Students!in!my!course!on!______________________________________________________
seem!to!have!difficulty!with!_______________________________________________________
_____________________________________________________________________________.
To!begin!to!understand!this!difficulty,!I!am!first!going!to!________________________________
_____________________________________________________________________________.
Acer!I!have!a!preliminary!understanding!of!the!problem,!I!will!do!a!second!round!of!inquiry!
using!_________________________________________________________________________!
in!order!to!delve!deeper.
NOTES:
8
David'Voelker'•'OPID'2013'FC
ProblemaGc#Prior#Knowledge
Students!do!not!enter!the!classroom!as!blank!slates.!!All!learners!bring!a!variety!of!ideas,!
assumpFons,!and!knowledge!frameworks!to!whatever!they!are!learning,!and!this!prior!
knowledge!(however!sound!or!unsound)!fundamentally!shapes!new!learning.!!EffecFve!
teaching!thus!ocen!requires!instructors!to!make!this!prior!knowledge!visible!to!students,!so!
that!they!can!selfJconsciously!evaluate!it!and!can!become!aware!of!its!strengths!and!
weaknesses.!!This!process!can!someFmes!be!frustraFng!or!even!painful!for!students,!because!it!
can!demand!that!students!rethink!cherished!ideas!and!values.!!(See!ch.!1!of!Ambrose,!et)al.,!
How)Learning)Works.)
I!want!students!to!develop!a!deep!understanding!of!_________________________________
____________________________________________________________________________.
But!many!students!come!into!my!class!with!the!mistaken!(!idea!/!assumpFon!/!belief!)!that
____________________________________________________________________________
_____________________________________.!!This!(!idea!/!assumpFon!/!belief!)!impedes!their!
learning,!because!______________________________________________________________
_____________________________________________________________________________.
I!propose!to!help!students!become!aware!of!the!strengths!and!weaknesses!of!their!prior!
knowledge!by!_________________________________________________________________
_____________________________________________________________________________.
I!will!know!that!students!are!successfully!evaluaFng!their!prior!knowledge!based!upon!the!
following!evidence:!_____________________________________________________________
_____________________________________________________________________________.
NOTES:
9
David'Voelker'•'OPID'2013'FC
Disciplinary#“Moves”
A!disciplinary!“move”!is!a!specific!skill,!procedure,!or!strategy!used!by!expert!pracFFoners!of!a!
discipline.!!For!example,!literary!scholars!pracFce!close!reading,!historians!contextualize!
primary!documents,!psychologists!manipulate!and!control!variables!(or!explore!relaFonships!
among!variables),!and!chemists!convert!to!common!units!before!making!any!other!calculaFons.!!
An!expert!pracFFoner,!of!course,!is!able!to!combine!many!such!moves!into!a!disciplinary!mode!
of!thought!and!acFon.!!If!you!are!trying!to!teach!disciplinary!thinking,!you!might!consider!
breaking!down!your!discipline’s!tools!into!a!series!of!moves!that!students!can!pracFce.!!(In!
many!disciplines,!SoTL!scholars!have!already!begun!such!work.!!See!the!“DisciplineJSpecific!
SoTL”!resources!on!the!reading!list!below!for!more!discussion!of!disciplinary!moves.)
In!my!discipline,!one!key!move!is!__________________________________________________
_____________________________________________________________________________!
_____________________________________________________________________________.!
I!currently!try!to!help!students!learn!this!move!by!____________________________________
_____________________________________________________________________________.
But!students!have!a!hard!Fme!____________________________________________________
_____________________________________________________________________________
because!______________________________________________________________________.!!
I!propose!to!improve!how!I!help!students!learn!this!move!by!____________________________
_____________________________________________________________________________.
I!will!know!that!they!are!learning!this!move!based!upon!the!following!evidence:!____________!
_____________________________________________________________________________
_____________________________________________________________________________.
NOTES:
10
David'Voelker'•'OPID'2013'FC
Threshold#Concepts
All!disciplines!have!fundamental!concepts!that!students!must!master!before!they!can!conFnue!
to!develop!disciplinary!thinking!and!skills.!!These!“threshold!concepts”!create!a!sort!of!
“bo,leneck”!through!which!students!must!pass,!but!these!ideas!are!subject!to!
misunderstanding!and!overJsimplificaFon.!!For!example,!students!in!fields!that!use!quanFtaFve!
data!must!understand!the!concept!of!staFsFcal!significance!before!they!can!carry!out!or!
evaluate!research.!!In!history,!students!must!understand!that!the!past!is!like!a!foreign!country!
where!words!and!concepts!that!seem!familiar!might!mean!something!very!different!than!than!
do!today.!!Threshold!concepts!like!these!are!not!simple!facts!that!can!be!memorized:!they!ocen!
consFtute!ways!of!thinking.!!In!many!disciplines,!SoTL!scholars!are!working!to!idenFfy!these!
threshold!concepts!and!develop!instrucFonal!strategies!for!guiding!students!through!the!
bo,leneck.!!(See!the!“DisciplineJSpecific!SoTL”!resources!on!the!reading!list!below!for!inJdepth!
discussion!of!threshold!concepts.!!The!seminal!essay!on!threshold!concepts!is!Jan!Meyer!and!
Ray!Land,!"Threshold!Concepts!and!Troublesome!Knowledge,”!cited!below!under!“FoundaFonal!
Works.”)
In!my!discipline!of!___________________________________!,!you!can’t!get!very!far!without!a!
good!understanding!of!__________________________________________________________
_____________________________________________________________________________!,
but!students!ocen!struggle!with!this!concept!because!_________________________________
_____________________________________________________________________________.
I!propose!to!guide!students!through!this!disciplinary!bo,leneck!by!_______________________
_____________________________________________________________________________.
I!will!know!that!they!are!learning!this!concept!based!upon!the!following!evidence:!__________!
_____________________________________________________________________________
_____________________________________________________________________________.
NOTES:
11
David'Voelker'•'OPID'2013'FC
Transfer#and#ApplicaGon#of#Knowledge
According!to!Grant!Wiggins!and!Jay!McTighe:!“Understanding!is!about!transfer.).).).)To!be!truly!
able![viz.,)knowledgable!and!capable]!requires!the!ability!to!transfer!what!we!have!learned!to!
new!and!someFmes!confusing!serngs.”!!In!other!words,!students!who!really!understand!
something!will!be!able!to!apply!that!understanding!appropriately!in!different!contexts.!!If!they!
cannot!do!so,!they!may!be!using!memorizaFon!and!other!strategies!to!merely!simulate!
learning.!!This!stumbling!block!to!learning!is!especially!common!in!disciplines!such!as!
mathemaFcs,!physics,!and!chemistry!where!students!must!make!use!of!formulas!to!solve!
problems.!!Students!may!seem!capable!as!long!as!they!are!told!which!formula!to!use,!but!they!
may!be!unable!to!determine!appropriate!procedures!when!they!encounter!an!unstructured,!
authenFc!problem.!!To!culFvate!deep!understanding,!then,!instructors!must!find!ways!to!help!
students!develop!their!ability!to!transfer!and!apply!what!they!know.!!(For!more!about!
“Understanding!as!transferability,”!see!Understanding)by)Design,)cited!below,!pp.!39–43.)
Students!in!my!course!on!_______________________________________________________
seem!to!have!difficulty!applying!their!knowledge!of!___________________________________
_____________________________________________________________________________
in!situaFons!where!_____________________________________________________________.
In!order!to!help!students!pracFce!applying!their!knowledge,!I!am!going!to!create!an!exercise!in!
which!they!____________________________________________________________________
_____________________________________________________________________________.
In!order!to!evaluate!their!ability!to!transfer!and!apply!their!knowledge,!I!will!use!an!assessment!
that!__________________________________________________________________________
_____________________________________________________________________________.
NOTES:
12
David'Voelker'•'OPID'2013'FC
Feedback#and#Learning
Receiving!and!responding!to!feedback!(from!various!sources)!is!an!essenFal!part!of!the!learning!
experience.!!Students!can!receive!feedback!in!many!ways.!!Although!a!simple!grade!is!a!rough!
form!of!feedback,!a!grade!explained!by!a!rubric!can!be!much!more!useful!for!helping!students!
improve!their!work.!!Students!also!receive!feedback!informally!from!instructors!and!peers.!!
Feedback!is!most!effecFve!if!students!receive!the!feedback!close!to!the!Fme!of!the!
performance!and!then!have!a!chance!to!refine!their!performance,!but!accomplishing!this!feat!is!
easier!said!than!done,!especially!in!large!classes.!!Strategies!for!providing!more!Fmely!and!
effecFve!feedback!include!making!frequent!comments!on!brief!assignments,!using!a!detailed!
rubric!to!give!feedback,!and!allowing!students!to!respond!to!feedback!through!revision!(or!even!
through!parFal!revisions).!!(For!help!with!rubrics,!see!Stevens!and!Levy,!Introduc3on)to)Rubrics,!
cited!below.!!On!the!importance!of!feedback!more!generally,!see!Ch.!5!of!Ambrose,!et)al.,)How)
Learning)Works.))On!best!pracFces!for!grading!more!generally,!see!Walvoord!and!Anderson,!
Effec3ve)Grading.)
Students!in!my!class!on!___________________________!need!feedback!in!order!to!improve!
their!ability!to!_________________________________________________________________
_____________________________________________________________________________.
In!order!to!give!more!Fmely!and!effecFve!feedback,!I!propose!to!________________________
_____________________________________________________________________________.
I!will!know!that!students!are!learning!from!the!feedback!based!upon!the!following!evidence:!
_____________________________________________________________________________
_____________________________________________________________________________.
NOTES:
13
David'Voelker'•'OPID'2013'FC
Learning#to#Learn
Different!disciplines!and!levels!of!study!demand!different!learning!tacFcs.!!Furthermore,!the!
more!advanced!the!learning,!the!more!important!it!is!for!students!to!be!selfJaware!of!their!
learning!process.!!When!students!bring!inappropriate!learning!tacFcs!to!bear!on!learning!
challenges,!the!results!can!be!frustraFng!and!unproducFve.!!In!these!situaFons,!it!is!appropriate!
and!even!necessary!for!instructors!to!guide!students!to!more!effecFve!learning!techniques.!!
(For!a!useful!discussion!of!how!students!can!become!“selfJdirected!learners,”!see!Ch.!7!of!
Ambrose,!et)al.,!How)Learning)Works.)
Students!in!my!class!on!__________________________________!!ocen!have!trouble!doing!the!
requisite!class!preparaFon!and!studying!because!_____________________________________
_____________________________________________________________________________.
In!order!to!help!students!learn!how!to!learn!in!this!class,!I!propose!to!____________________
_____________________________________________________________________________
_____________________________________________________________________________.!
I!will!know!that!they!are!approaching!the!learning!process!effecFvely!based!upon!the!following!
evidence:!_____________________________________________________________________
_____________________________________________________________________________.
NOTES:
14
David'Voelker'•'OPID'2013'FC
Awareness,#AppreciaGon,#or#AWtude
One!of!the!more!subtle!kind!of!learning!goals!involves!students!developing!a!specific!
awareness,!appreciaFon,!or!artude.!!For!example,!Professor!Dallas!Blaney!at!UW–Green!Bay!
wants!students!in!“PoliFcs!of!Developing!Areas”!to!develop!an!awareness!of!how!their!own!
lives!are!connected!to!the!lives!of!people!living!in!developing!naFons.!!(He!created!and!studied!
a!group!project!aimed!at!culFvaFng!this!awareness.)!!Even!less!tangible,!perhaps,!is!the!goal!of!
many!humaniFes!courses!to!teach!students!an!appreciaFon!of!complexity!or!ambiguity.!
If!you!want!to!study!a!learning!goal!along!these!lines,!it’s!important!to!figure!out!how!you!will!
know!when!a!student!has!developed!the!desired!appreciaFon!or!awareness.!!Some!artudes,!
such!as!those!involving!race!and!gender,!can!be!documented!using!exisFng!survey!instruments.!!
But!many!kinds!of!awareness!or!appreciaFon!may!require!you!to!develop!your!own!mode!of!
assessment.!!Before!designing!a!new!intervenFon!(exercise,!assignment,!etc.)!to!culFvate!the!
desired!awareness,!appreciaFon,!or!artude,!you!might!consider!doing!an!iniFal!assessment!to!
gauge!the!effecFveness!of!your!current!pracFces.!!Then!you!can!repeat!the!assessment!acer!
trying!a!new!instrucFonal!method.!
I!want!my!students!in!my!course!on!________________________________________________
to!develop!a!strong!(!appreciaFon!/!awareness!/!artude!)!of!____________________________
_____________________________________________________________________________.
Students!who!lack!this!(!appreciaFon!/!awareness!/!artude!)!are!likely!to!make!the!mistake!of!
_____________________________________________________________________________
________________________________.!!Students!who!possess!this!(!appreciaFon!/!awareness!/!
artude!)!demonstrate!it!through!__________________________________________________
_____________________________________________________________________________.
I!will!use!the!following!method!to!determine!to!degree!to!which!my!students!are!developing!
this!(!appreciaFon!/!awareness!/!artude!):!__________________________________________
_____________________________________________________________________________.!!
NOTES:
15
David'Voelker'•'OPID'2013'FC
FINDING$EVIDENCE$OF$LEARNING
Project$Type
(ParPal$List)
Learning$Goal
(Define$meaningful$learning.)
IntervenPon
(What$will$you$try?)
ProblemaFc'Prior' Students"will"be"able"to'
Knowledge
evaluate'how'their'prior'
knowledge'and'assumpFons'
about'the'subject'measure'up'
against'expert'knowledge.
an'assignment'that'guides'
students'to'reflect'upon'and'
evaluate'their'preconcepFons'
regarding'a'parFcular'topic
Disciplinary'
“Move”'or'
“Transfer”
Students"will"be"able"to'decide'
when'and'how'to'apply'a'
disciplinary'strategy'of'analysis'
or'problemUsolving.
a'repeaFng'inUclass'acFvity'in'
which'students'apply'a'clearly'
defined'disciplinary'way'of'
thinking'to'a'problem'or'puzzle
Threshold'
Concept
Students"will"be"able"to'apply'a'
key'disciplinary'concept'(such'
as'natural'selecFon,'staFsFcal'
significance,'or'the'greenhouse'
effect)'to'solve'an'authenFc'
disciplinary'problem.
a'course'unit'that'uFlizes'case'
studies'to'help'students'improve'
their'understanding'of'a'key'
concept
Learning'to'Learn
Students"will"be"able"to'apply'
metacogniFve'skills'(such'as'
locaFng'an'argument'and'
evidence'in'a'scholarly'essay)'
that'they'need'to'learn'
effecFvely'in'your'discipline.
an'assignment'or'learning'tool'
designed'to'help'students'
pracFce'a'parFcular'learning'skill
What'Works?'
(EvaluaFng'a'
Teaching'
Strategy)
Students"will"be"able"to'apply'a'
specific'skill'to'solve'an'
authenFc'disciplinary'problem.
a'new'teaching'strategy,'learning'
acFvity,'or'assignment
1.
2.
3.
4.
5.
6.
Possible$Evidence
(See$below$for$ideas.)
Compare'objecFve'exam'scores'of'groups'of'students'who'did'and'did'not'experience'the'intervenFon.
Compare'preU'and'postUinstrucFon'results'of'objecFve'exam'that'includes'“distractors”'based'upon'common'
misconcepFons.
Use'an'embedded'assessment'(an'assignment'or'exam'that'is'part'of'the'course)'to'evaluate'student'learning'
of'a'specific'skill,'concept,'or'body'of'knowledge.''(Note:'Generally,'this'method'requires'doing'a'more'
focused'and'detailed'analysis'of'student'work'than'rouFne'grading.)
Use'surveys'and/or'focus'groups'to'evaluate'student'studying'pracFces'(possibly'with'either'comparison'
group'or'preU'and'postUintervenFon'assessment).
Chart'development'of'student'understanding'by'analyzing'a'pair'of'concept'maps'or'short'essays'(created'by'
students'at'the'beginning'and'end'of'the'semester).
Chart'development'of'student'analyFcal'skills'by'coding'the'results'of'a'pair'of'think'alouds'(administered'at'
the'beginning'and'end'of'the'semester).
16
David'Voelker'•'OPID'2013'FC
ENGAGING$WITH$THE$LITERATURE$ON$TEACHING$AND$LEARNING
!
Doing!research!on!teaching!and!learning!is!a!scholarly!endeavor,!which!means!not!only!
gathering!and!analyzing!evidence!in!a!rigorous!fashion!but!also!studying!the!scholarly!literature!
in!the!field.!!Engaging!with!the!relevant!literature!early!and!ocen!will!help!you!develop!a!
meaningful!quesFon!and!effecFve!methodology.!!I!deliberately!use!the!phrase!“engaging!with!
the!literature”!rather!than!“conducFng!a!literature!review”!because!the!la,er!sounds!like!a!
discrete!step!than!can!be!completed!and!“checked!off”!at!the!beginning!of!a!project.
I!recommend!that!you!begin!reviewing!the!literature!while!you!are!formulaFng!and!refining!
your!research!quesFon!and!conFnue!to!explore!the!literature!as!you!define!your!research!
methods!and!analyze!the!evidence!you!collect.
!
If!you!are!just!gerng!started!doing!SoTL,!you!will!find!it!very!useful!to!read!a!few!
foundaFonal!works!on!teaching,!learning,!and!course!design.!!I!have!marked!several!books!and!
arFcles!with!an!asterisk!(*)!to!flag!them!as!essenFal!reading.!!These!works!discuss!key!
pedagogical!principles!that!you!can!use!to!help!frame!a!research!quesFon.
Teaching,!Learning,!and!Course!Design:!FoundaFonal!Works
*Ambrose,!Susan!A.,!et!al.!How)Learning)Works:)Seven)ResearchVBased)Principles)for)Smart)
Teaching.))San!Francisco:!JosseyJBass,!2010.!!Each!chapter!is!devoted!to!a!parFcular!learning!
principle!and!includes!a!set!of!specific!strategies!for!teaching!and!assessing!with!that!
principle!in!mind.
Anderson,!Lorin!W.,!et!al.,!eds.!A)Taxonomy)for)Learning,)Teaching,)and)Assessing:)A)Revision)of)
Bloom’s)Taxonomy)of)Educa3onal)Objec3ves.!!Abridged!ed.!New!York:!Longman,!2001.!!This!
revision!of!Bloom’s!classic!taxonomy!divides!learning!into!several!“process!categories”JJ
remember,!understand,!apply,!analyze,!evaluate,!and!createJJand!“knowledge!dimensions”:!
factual,!conceptual,!procedural,!and!metaJcogniFve.!!Unlike!the!original!taxonomy,!the!
revised!version!is!not!strictly!hierarchical.!!This!book!may!be!useful!for!helping!define!
learning!outcomes.
*Bain,!Ken.!What)the)Best)College)Teachers)Do.!Cambridge:!Harvard!Univ.!Press,!2004.!This!
deeply!researched!book!offers!an!analysis!not!of!teaching!“techniques”!but!of!how!
successful!college!teachers!conceive!of!their!work!as!teachers!and!structure!the!learning!
experiences!of!their!students.
Bransford,!John!D.,!Ann!L.!Brown,!and!Rodney!R.!Cocking,!eds.!How)People)Learn:)Brain,)Mind,)
Experience,)and)School,!expanded!ed.!!Washington,!D.C.:!NaFonal!Academy!Press,!2000.!!An!
accessible!introducFon!to!“the!science!of!learning.”
Cousin,!Glynis.!!“An!IntroducFon!to!Threshold!Concepts.”!Planet,!no.!17!(Dec.!2006):!4J5.!!Clear!
and!concise.!!h[p://www.gees.ac.uk/planet/p17/gc.pdf
Fink,!Dee!L.!Crea3ng)Significant)Learning)Experiences:)An)Integrated)Approach)to)Designing)
College)Courses.!!San!Francisco:!JosseyJBass,!2003.
*Meyer,!Jan,!and!Ray!Land,!"Threshold!Concepts!and!Troublesome!Knowledge:!Linkages!to!
Ways!of!Thinking!and!PracFsing!within!the!Disciplines,"!Occasional!Report!4!(May!2003).!!
h[p://3nyurl.com/cq2zxyk
*Wiggins,!Grant!and!Jay!McTighe,!Understanding)by)Design,!2nd!ed.!Upper!Saddle!River,!NJ:!
Pearson!EducaFon,!2006.!!Although!this!book!is!burdened!with!an!apparatus!of!acronyms!
and!rubrics,!its!discussion!of!understanding!and!“backward!design”!is!essenFal!reading.
17
David'Voelker'•'OPID'2013'FC
*Wirth,!Karl!R.!and!Dexter!Perkins.!“Learning!to!Learn,”!2008.!!This!essay!provides!a!concise!
overview!of!recent!models!of!learning.!!h[p://3nyurl.com/bu5r6y6
Wiske,!Martha!Stone,!ed.!Teaching)for)Understanding:)Linking)Research)with)Prac3ce.!!San!
Francisco:!JosseyJBass,!1998.
The!Scholarship!of!Teaching!and!Learning:!Research!Methods
*BishopJClark,!Cathy!and!Beth!DietzJUhler,!Engaging)in)the)Scholarship)of)Teaching)and)
Learning:)A)Guide)to)the)Process,)and)How)to)Develop)a)Project)from)Start)to)Finish.!!
Sterling,!Virginia:!Stylus,!2012.!!Very!good!introducFon!to!the!process!and!methods!of!doing!
SoTL!work.!!Includes!many!useful!examples,!worksheets,!and!citaFons.
Cerbin,!Bill.!Lesson)Study:)Using)Classroom)Inquiry)to)Improve)Teaching)and)Learning)in)Higher)
Educa3on.!Sterling,!Virginia:!Stylus,!2011.!!Great!guide!to!collaboraFve!lesson!study.
*Gurung,!Regan!A.!R.!and!Beth!Schwartz.!Op3mizing)Teaching)and)Learning:)Prac3cing)
Pedagogical)Research.!Malden,!MA:!WileyJBlackwell,!2009.!!An!excellent!introducFon!to!
both!scholarly!teaching!and!SoTL.
Huber,!Mary!Taylor!and!Pat!Hutchings.!The)Advancement)of)Learning:)Building)the)Teaching)
Commons.!!San!Francisco:!JosseyJBass,!2005.
Huber,!Mary!Taylor,!Pat!Hutchings,!and!Anthony!Ciccone.!The)Scholarship)of)Teaching)and)
Learning)Reconsidered:)Ins3tu3onal)Integra3on)and)Impact.!!San!Francisco:!JosseyJBass,!
2011.
McKinney,!Kathleen.!!Enhancing)Learning)through)the)Scholarship)of)Teaching)and)Learning:)
The)Challenges)and)Joys)of)Juggling.!!San!Francisco:!JosseyJBass,!2007.
*McKinney,!Kathleen,!ed.!!The)Scholarship)of)Teaching)and)Learning)in)and)across)the)
Disciplines.!Bloomington:!Indiana!University!Press,!2013.!!An!excellent!collecFon!of!essays!
that!address!criFcal!issues!in!SoTL!methodology.
Schwartz,!Beth!and!Regan!A.!R.!Gurung,!eds.!EvidenceVBased)Teaching)for)Higher)Educa3on.!!
American!Psychological!AssociaFon,!2012.
Research!Methods!and!Design:!General
Cresswell,!John!C.!!Qualita3ve)Inquiry)&)Research)Design:)Choosing)Among)Five)Approaches.!
3rd!ed.!Thousand!Oaks,!Calif.:!Sage!PublicaFons,!2013.
Cresswell,!John!C.!!Research)Design:)Qualita3ve,)Quan3ta3ve,)and)Mixed)Methods)Approaches.!!!
4th!ed.!!Thousand!Oaks,!Calif.:!Sage!PublicaFons,!2013.
Denzin,!Norman!K.!and!Yvonna!S.!Lincoln,!eds.!The)SAGE)Handbook)of)Qualita3ve)Research.!!4th!
ed.!Thousand!Oaks,!Calif.:!SAGE!PublicaFons,!2011.
Weiss,!Robert!S.!!Learning)From)Strangers:)The)Art)and)Method)of)Qualita3ve)Interview)Studies.!!
New!York:!Free!Press,!1995.
DisciplineJSpecific!SoTL
*Gurung,!Regan!A.!R.,!Nancy!L.!Chick,!and!Aeron!Haynie,!eds.!Exploring)Signature)Pedagogies:)
Approaches)to)Teaching)Disciplinary)Habits)of)Mind.!!Sterling,!Virginia:!Stylus!Publishing,!
2009.!Analyses!of!disciplineJspecific!pedagogies!and!related!SoTL!research.!!EssenFal!
Huber,!Mary!Taylor!and!Sherwyn!P.!Morreale,!eds.!Disciplinary)Styles)in)the)Scholarship)of)
Teaching)and)Learning:)Exploring)Common)Ground.!Washington,!D.C.:!American!AssociaFon!
for!Higher!EducaFon,!2002.
18
David'Voelker'•'OPID'2013'FC
Pace,!David,!and!Joan!Middendorf,!eds.!Decoding)the)Disciplines:)Helping)Students)Learn)
Disciplinary)Ways)of)Thinking.!!New!DirecFons!for!Teaching!and!Learning!98!(Summer!
2004).
Model!ArFcles!Useful!to!SoTL!Scholars!Regardless!of!Discipline!
Calder,!Lendol.!!“Uncoverage:!Toward!a!Signature!Pedagogy!for!the!History!Survey.”!Journal)of)
American)History!92!(March!2006):!1358–70.
Chick,!Nancy!L.,!!Holly!Hassel,!and!Aeron!Haynie.!“‘Pressing!an!Ear!against!the!Hive’:!Reading!
Literature!for!Complexity.”!Pedagogy!9!(2009):!399J422.!Excellent!example!of!how!to!apply!
disciplinary!methods!to!SoTL.
Crowe,!Alison,!Clarissa!Dirks,!and!Mary!Pat!Wenderoth.!“Biology!in!Bloom:!ImplemenFng!
Bloom’s!Taxonomy!to!Enhance!Student!Learning!in!Biology.”!CBEVVLife)Sciences)Educa3on!7!
(Winter!2008):!368J81.!!h[p://3nyurl.com/7bybscg
DeHaan,!Robert!L.!“Teaching!CreaFvity!and!InvenFve!Problem!Solving!in!Science.”!!CBEVVLife)
Sciences)Educa3on)8!(Fall!2009):!172J181.!!A!wellJstructured!analysis!of!a!teaching!and!
learning!problem,!with!a!call!for!addiFonal!research.!!h[p://3nyurl.com/86vs2nk
Sizemore,!O.!J.!and!Gary!W.!Lewandowski,!Jr.!“Learning!Might!Not!Equal!Liking:!Research!
Methods!Course!Changes!Knowledge!but!Not!Artudes.”!Teaching)of)Psychology)36!(2009):!
90J95.
Voelker,!David!J.!!“Assessing!Student!Understanding!in!Introductory!Courses:!A!Sample!
Strategy.”!History)Teacher!41!(August!2008):!505–18.!!A!good!example!of!a!meaningful!SoTL!
quesFon,!situated!within!the!relevant!literature—but!the!methods!used!to!collect!evidence!
of!student!learning!are!somewhat!rudimentary.
Other!EssenFal!PublicaFons!on!Teaching!and!Learning
Angelo,!Thomas!A.!and!K.!Patricia!Cross.!!Classroom)Assessment)Techniques:)A)Handbook)for)
College)Teachers.!!2nd!ed.!!San!Francisco:!JosseyJBass,!1993.!!This!book!describes!50!
assessment!techniques,!many!of!which!could!be!used!to!gather!evidence!of!learning!for!a!
SoTL!project.
Barkley,!Elizabeth!F.,!K.!Patricia!Cross,!and!Claire!Howell!Major.!Collabora3ve)Learning)
Techniques:)A)Handbook)for)College)Faculty.))San!Francisco:!JosseyJBass,!2005.!!This!book!
describes!and!explains!30!collaboraFve!learning!techniques!(CoLTs);!a!great!place!to!start!
for!any!SoTL!project!involving!collaboraFve!learning.
Handelsman,!Jo,!Sarah!Miller,!and!ChrisFne!Pfund.!Scien3fic)Teaching.))Madison:!Wisconsin!
Program!for!ScienFfic!Teaching,!2007.!!SoTL!for!scienFsts.
Michaelsen,!Larry!K.,!Arle,a!Bauman!Knight,!and!L.!Dee!Fink,!eds.!!TeamVBased)Learning:)A)
Transforma3ve)Use)of)Small)Groups)in)College)Teaching.!!Sterling,!Virginia:!Stylus!Publishing,!
2004.
*Nelson,!Craig!E.!"On!the!Persistence!of!Unicorns:!The!TradeJOff!between!Content!and!CriFcal!
Thinking!Revisited."!In!Bernice!A.!Pescosolido!and!Ronald!Amizade,!eds.,!The)Social)Worlds)
of)Higher)Educa3on")Handbook)for)Teaching)in)a)New)Century,!168J184.!Boston:!Pine!Forge!
Press,!1999.!!An!excellent!and!provocaFve!essay!on!students’!cogniFve!development.
Nilson,!Linda!B.!!Teaching)at)its)Best:)A)ResearchVBased)Resource)for)College)Instructors.))3rd!ed.!!
San!Francisco:!JosseyJBass,!2010.!!Perhaps!the!best!pracFcal!handbook!for!teaching.
Shulman,!Lee!S.!!“Taking!Learning!Seriously.”!Change!31:4!(July/August!1999):!10J17.!!h[p://
3nyurl.com/cfyyd75
19
David'Voelker'•'OPID'2013'FC
Stevens,!Dannelle!D.!and!Antonia!J.!Levy.!)Introduc3on)to)Rubrics:)An)Assessment)Tool)to)Save)
Grading)Time,)Convey)Effec3ve)Feedback)and)Promote)Student)Learning.!!Sterling,!Virginia:!
Stylus!Publishing,!2004.
Walvoord,!Barbara!E.!!Assessment)Clear)and)Simple:)A)Prac3cal)Guide)for)Ins3tu3ons,)
Departments,)and)General)Educa3on.!2nd!ed.!!San!Francisco:!JosseyJBass,!2010.!!A!great!
resource!for!SoTL!scholars!who!want!to!use!their!work!to!support!insFtuFonal!assessment.
Walvoord,!Barbara!E.!and!Virginia!Johnson!Anderson,!Effec3ve)Grading:)A)Tool)for)Learning)and)
Assessment.!!San!Francisco:!JosseyJBass,!1998.
AddiFonal!Resources
AssociaFon!of!American!Colleges!and!UniversiFes:!VALUE:!Valid!Assessment!of!Learning!in!
Undergraduate!EducaFon.!!ResearchJbased!rubrics!for!assessing!criFcal!thinking,!creaFve!
thinking,!etc.!!h[p://www.aacu.org/value/index.cfm
Carnegie!Mellon:!Eberly!Center!for!Teaching!and!Learning:!Principles!of!Teaching!&!Learning.!!
The!principles!of!learning!described!here!are!further!elaborated!upon!in!Ambrose,!et!al.,!
How)Learning)Works)(see!above).!)h[p://www.cmu.edu/teaching/principles/index.html
Journal)of)Chemical)Educa3on!(American!Chemical!Society).!!h[p://pubs.acs.org/journal/jceda8
Life)Sciences)Educa3on!(American!Society!for!Cell!Biology).!!Free,!fullJtext!access!to!peerJ
reviewed!SoTL!research!in!the!life!sciences—an!excellent!source!of!model!arFcles!!)h[p://
www.lifescied.org/reports/mostVread
Physics)Educa3on!(an!internaFonal!journal).!!Some!arFcles!are!available!for!free.!!h[p://
iopscience.iop.org/0031V9120/
ISSOTL:!InternaFonal!Society!for!the!Scholarship!of!Teaching!and!Learning:!SoTL!Resources.!!A!
compendium!of!online!SoTL!resources.!!h[p://www.issotl.org/SOTL.html
Siegle,!Del.!!“Likert!Scale.”!!A!handy!use!of!common!Likert!scales!(e.g.,)the!“Strongly!Agree”!to!
“Strongly!Disagree”!spectrum)!for!survey!use.!!h[p://3nyurl.com/d34wezr))!!
UWJSuperior!Library!Guide!on!SoTL.!!h[p://3nyurl.com/cf9abnf
*UWJMilwaukee,!Center!for!InstrucFonal!and!Professional!Development,!SoTL!Resources.!!This!
site,!which!will!be!relaunched!during!midJ2013,!includes!many!useful!SoTL!resources.!!Look!
for!the!“SoTL!Resources”!link!at!h[p://www4.uwm.edu/cipd/
20
David'Voelker'•'OPID'2013'FC
UNDERSTANDING$HISTORICAL$THINKING:$CODING$RUBRIC
The!goal!of!this!study,!which!is!currently!underway,!is!to!assess!the!effecFveness!of!the!overall!
design!of!my!introductory!early!American!history!course!in!helping!students!develop!their!
understanding!of!what!historians!do!and!what!it!means!to!think!historically.!!I!have!gathered!
responses!to!the!two!quesFons!below!from!my!own!students!(over!2!semesters)!and!from!a!
comparison!group!in!similar!classes!taught!by!a!two!different!instructors.!!I!am!coding!each!
response!by!marking!and!tallying!the!response!types!listed!below.!!Each!response!will!earn!
points!based!on!the!code!types!marked!with!a!plus!sign!and!will!lose!points!based!on!the!codes!
marked!with!a!minus!sign.!!In!addiFon!to!comparing!my!students!to!history!students!in!other!
courses,!I!will!also!select!several!responses!that!I!deem!“competent”!and!will!judge!the!quality!
of!the!responses!relaFve!to!these!benchmark!scores.!!(Note:!This!table!was!inspired!by!the!
coding!sheet!that!Lendol!Calder!developed!to!assess!think!alouds.)
Response Type
Examples
Code
Q1: How do historians study their subject, and what are their goals in doing so?
Questions
Observes that historians ask complex questions
about the past
Q (+)
Sources
Observes that historians use multiple primary
sources
S (+)
Perspectives
Observes that historians use sources from
different perspectives or interests
P (+)
Analysis
Observes that historians analyze primary
sources
Observes that historians contextualize primary
sources
A (+)
Interpretation
Observes that historians formulate an
interpretation of the past
I (+)
Discussing/Arguing
Observes that historians engage in discussion
and argument with other historians
Observes that historians build upon the research
of other scholars
D (+)
Public History/
Application
Observes that historians reflect on significance
of the past for understanding the present.
Observes that historians educate the public
through various means.
PH (+)
Over-Simplifying
Characterizes the study of history as a simple
act of description
States or implies that historians search for a
single definitive account of the past
OS (-)
21
David'Voelker'•'OPID'2013'FC
Response Type
Examples
Code
Q2: Give 1-2 specific examples of what historians do.
Specific Questions
Refers to specific historical questions
SQ (+)
Specific Sources
Refers to specific historical sources
SS (+)
Specific Historians
Refers to specific historians or works of
scholarship
SH (+)
Over-Simplifying
Characterizes the study of history as a simple
act of description
States or implies that historians search for a
single definitive account of the past
OS (-)
Scoring: Each student’s responses are scored using the criteria above. For each
response, I am recording the number of incidents of each of the stated response types.
(If a particular phrase could fall into more than one category, it is counted only once.)
Incidents of oversimplification (OS) are counted and deducted from the total score as
noted.
Sample tally:
Student
#001
Q
0
S
1
P
1
A
1
I
1
D
0
PH
SQ
SS
SH
OS (-)
TOTAL
0
1
2
0
-1
6
For a brief overview of my project and preliminary data analysis, see: “Studying the
Impact of Course Design on Student Learning” at http://tinyurl.com/coxwqfz.
22
David'Voelker'•'OPID'2013'FC
NAVIGATING$THE
INSTITUTIONAL$REVIEW$BOARD$(IRB)$PROCESS
If!you!plan!to!share!the!results!of!your!SoTL!research!project!(through!public!presentaFons!or!
publicaFons,!etc.),!federal!guidelines!require!you!to!secure!IRB!approval!for!doing!research!with!
human!subjects!or!parFcipants.!!(If!you!are!doing!a!preliminary!project!and!do!not!plan!to!
present!or!publish!your!data,!or!if!you!will!use!the!data!solely!for!assessment!within!your!
insFtuFon,!you!do!not!need!IRB!approval.)!!However,#if#you#do#not#get#IRB#approval#BEFORE#
your#begin#your#project,#as#a#general#rule#you#cannot#subsequently#publicize#your#findings,#no#
ma\er#how#valuable#they#may#be.##(Your!IRB!may!allow!you!to!use!general!archival!data!from!
previous!semesters!under!some!circumstances,!but!you!should!not!assume!that!this!will!be!
possible.)!!The!IRB!process!may!seem!burdensome!the!first!Fme!or!two!that!you!go!through!it,!
but!the!process!will!help!ensure!that!you!make!responsible!use!of!the!data!that!you!collect.!!In!
addiFon!to!ensuring!that!your!study!does!not!harm!students!(by!giving!some!students!an!unfair!
advantage!or!disadvantage,!for!example),!the!IRB!process!will!help!ensure!that!students!provide !
“informed!consent”!to!parFcipate!in!your!research!and!that!their!privacy!is!protected.!!
Although!there!is!an!“Exempt”!category!that!may!apply!to!some!SoTL!projects,!most!university!
IRB’s!require!SoTL!projects!to!undergo!IRB!scruFny!while!“exempFng”!them!only!from!the!
requirement!of!annual!reporFng!and!review.!!You#should#always#assume#that#your#project#
requires#IRB#approval#if#you#are#doing#research#that#you#plan#to#publicize#with#student#
parGcipants.
Here!are!several!Fps!for!navigaFng!the!IRB!approval!process:
1.!!Start!early.!!CompleFng!the!IRB!process!takes!Fme.!!If!your!project!needs!full!board!review!
(unlikely!for!SoTL!projects!but!certainly!possible),!you!will!have!wait!for!the!board!to!meet.!!
Most!boards!do!not!meet!during!the!summer,!and!some!chairs!do!not!review!proposals!during!
the!summer.!!Find!out!how!your!insFtuFonal!IRB!operates!and!plan!accordingly.!!Remember,#
you#must#receive#IRB#approval#BEFORE#you#collect#any#data#for#your#project.
2.!!Check!with!your!insFtuFon's!IRB!to!find!out!if!it!requires!human!subjects!training.!!If!so,!
complete!this!online!training!for!doing!research!with!human!subjects!before!you!begin!your!
project.!!Save!your!cerFficate!as!a!PDF!file!for!easy!sharing.!!(Some!insFtuFons!make!use!of!the!
“ProtecFng!Human!Research!ParFcipants”!cerFficaFon!offered!by!the!NIH.)
3.!!Unless!you!have!experience!with!SoTL!research,!consider!having!a!short!meeFng!with!the!
chair!of!your!insFtuFon's!IRB!to!ask!for!advice!regarding!your!proposal.!!Some!IRBs!have!special!
procedures!to!help!streamline!SoTL!work,!such!as!anonymous!informed!consent!processes.!!You!
may!also!benefit!from!asking!a!colleague!with!IRB!experience!to!review!your!first!proposal.
4.!!Keep!careful!track!of!the!data!management!procedures!that!you!agree!to!in!your!proposal.!!
Typically,!you!will!need!to!protect!the!confidenFality!or!even!anonymity!of!research!subjects.
5. !I!have!posted!a!sample!IRB!protocol!for!a!SoTL!project!at:!h[p://3nyurl.com/7o4cqfu
23
Carnegie Mellon University
From: http://www.cmu.edu/teaching/principles/learning.html
See S. Ambrose, et al., HOW LEARNING WORKS (2010) for excellent
application suggestions.
Theory and Research-based Principles of Learning
The following list presents the basic principles that underlie effective learning. These principles are distilled from
research from a variety of disciplines.
1. Students’ prior knowledge can help or hinder learning.
Students come into our courses with knowledge, beliefs, and attitudes gained in other courses and through daily
life. As students bring this knowledge to bear in our classrooms, it influences how they filter and interpret what
they are learning. If students’ prior knowledge is robust and accurate and activated at the appropriate time, it
provides a strong foundation for building new knowledge. However, when knowledge is inert, insufficient for the
task, activated inappropriately, or inaccurate, it can interfere with or impede new learning.
2. How students organize knowledge influences how they learn and apply what they know.
Students naturally make connections between pieces of knowledge. When those connections form knowledge
structures that are accurately and meaningfully organized, students are better able to retrieve and apply their
knowledge effectively and efficiently. In contrast, when knowledge is connected in inaccurate or random ways,
students can fail to retrieve or apply it appropriately.
3. Students’ motivation determines, directs, and sustains what they do to learn.
As students enter college and gain greater autonomy over what, when, and how they study and learn,
motivation plays a critical role in guiding the direction, intensity, persistence, and quality of the learning
behaviors in which they engage. When students find positive value in a learning goal or activity, expect to
successfully achieve a desired learning outcome, and perceive support from their environment, they are likely to
be strongly motivated to learn.
4. To develop mastery, students must acquire component skills, practice integrating them, and know when to apply what
they have learned.
Students must develop not only the component skills and knowledge necessary to perform complex tasks, they
must also practice combining and integrating them to develop greater fluency and automaticity. Finally, students
must learn when and how to apply the skills and knowledge they learn. As instructors, it is important that we
develop conscious awareness of these elements of mastery so as to help our students learn more effectively.
5. Goal-directed practice coupled with targeted feedback enhances the quality of students’ learning.
Learning and performance are best fostered when students engage in practice that focuses on a specific goal or
criterion, targets an appropriate level of challenge, and is of sufficient quantity and frequency to meet the
performance criteria. Practice must be coupled with feedback that explicitly communicates about some aspect(s)
of students’ performance relative to specific target criteria, provides information to help students progress in
meeting those criteria, and is given at a time and frequency that allows it to be useful.
6. Students’ current level of development interacts with the social, emotional, and intellectual climate of the course to
impact learning.
Students are not only intellectual but also social and emotional beings, and they are still developing the full
range of intellectual, social, and emotional skills. While we cannot control the developmental process, we can
shape the intellectual, social, emotional, and physical aspects of classroom climate in developmentally
appropriate ways. In fact, many studies have shown that the climate we create has implications for our students.
A negative climate may impede learning and performance, but a positive climate can energize students’ learning.
7. To become self-directed learners, students must learn to monitor and adjust their approaches to learning.
Learners may engage in a variety of metacognitive processes to monitor and control their learning—assessing
the task at hand, evaluating their own strengths and weaknesses, planning their approach, applying and
monitoring various strategies, and reflecting on the degree to which their current approach is working.
Unfortunately, students tend not to engage in these processes naturally. When students develop the skills to
engage these processes, they gain intellectual habits that not only improve their performance but also their
effectiveness as learners.
Bibliography
Anderson, J. R., Conrad, F. G., Corbett, A. T. (1989). Skill acquisition and the LISP tutor. Cognitive Science,