NovemberThoughtLeaders:
TheComingU.S.News&WorldReportRankings
ofUndergraduateTeacherEducation
Draft4‐17‐13
U.S.News&WorldReport(USN&WR)willsoonbeputtingoutitsfirstrankings
ofundergraduateteachereducationprogramsinconjunctionwiththeNational
CouncilonTeacherQuality(NCTQ).Thisproposalhasprovokedconsiderable
reactionfrombothsupportersanddetractors.Supportersargue,amongother
things,thatteachereducationisnotasstrongasitcouldbe,norastransparent,and
thatsuchareviewcanhelp.DetractorshavepointedoutthatmostofNCTQ’s
previousforaysintoteachereducationhavebeennegativetowardsteacher
education,theyhaveanagendathatdiffersinsignificantwaysfrommanyteacher
educationprograms,andtheyrelyonlimitedevidence.Mypurposeinthisbrief
pieceisnottoeithersupportorattackthesenewrankings.Rather,mypurposeisto
providecontextforbetterunderstandingwhateverrankingUNIwillreceivewhen
theycomeout.
WhyAreTheyDoingthisReview?
Oneanswer,fromNCTQ’sWebsite,is“NCTQwillpublishtheTeacherPrep
Reviewannuallytoprovideitsmaincustomers—aspiringteachersandschool
districts—withup‐to‐dateinformationonwherethebestprogramsare.”Tome,
thismaybethoughtofasthebusinesspurpose.OnthesameWebsitetheyalsonote,
Withtherighttraining,talentedandmotivatedpeoplecanbemadeintogreat
teachers.Butourteacherpreparationprogramsmustbeuptothetask.That'swhy
NCTQhasembarkedonareviewofthenation's1,400higheredteacherpreparation
programs.
Usingourwell‐honedmethodology,we'reworkingtofindtheprogramsthatare
doingthebestjobinpreparingtomorrow'seducators,thosethatneedtoimproveand
thosethatneedtoberadicallyrestructured.Ourultimategoal?Ensuringall
teachersarestudent‐readybeforetheyentertheclassroom.
Thisisthepublicpolicypurpose.Ibelievethatboththebusinessandpublicpolicy
purposesarepartofthereasonfortherankings.
WhatStandardsAretheReviewBasedOn?
NCTQhasdevelopedasetof17standardsthattheyassertrepresentthe
potentialtocreatequalityteachers(seehttp://www.nctq.org/standardsDisplay.do).
Thelistfollows.
THETALENTTEACHERSNEED
Standard1:Selectivity
Theprogramselectsteachercandidatesofstrongacademiccaliber.
Standardappliesto:Elementary,SecondaryandSpecialEducationprograms.
WHATTEACHERSSHOULDKNOW
Standard2:EarlyReading
TheprogramtrainsteachercandidatestoteachreadingasprescribedbytheCommon
CoreStateStandards.
Standardappliesto:ElementaryandSpecialEducationprograms.
Standard3:EnglishLanguageLearners
TheprogramprepareselementaryteachercandidatestoteachreadingtoEnglish‐
languagelearners.
Standardappliesto:Elementaryprograms.
Standard4:StrugglingReaders
Theprogramprepareselementaryteachercandidatestoteachreadingskillstostudents
atriskofreadingfailure.
Standardappliesto:Elementaryprograms.
Standard5:CommonCoreElementaryMathematics
TheprogrampreparesteachercandidatestosuccessfullyteachtotheCommonCore
StateStandardsforelementarymath.
Standardappliesto:ElementaryandSpecialEducationprograms.
Standard6:CommonCoreElementaryContent
Theprogramensuresthatteachercandidateshavethebroadcontentpreparation
necessarytosuccessfullyteachtotheCommonCoreStateStandards.
Standardappliesto:Elementaryprograms.
Standard7:CommonCoreMiddleSchoolContent
Theprogramensuresthatteachercandidateshavethecontentpreparationnecessaryto
successfullyteachtotheCommonCoreStateStandards.
Standardappliesto:Secondaryprograms.
Standard8:CommonCoreHighSchoolContent
Theprogramensuresthatteachercandidateshavethecontentpreparationnecessaryto
successfullyteachtotheCommonCoreStateStandards.
Standardappliesto:Secondaryprograms.
Standard9:CommonCoreContentforSpecialEducation
Theprogramensuresthatteachercandidates'contentpreparationalignswiththe
CommonCoreStateStandardsinthegradestheyarecertifiedtoteach.
Standardappliesto:SpecialEducationprograms.
WHATTEACHERSSHOULDBEABLETODO
Standard10:ClassroomManagement
Theprogramtrainsteachercandidatestosuccessfullymanageclassrooms.
Standardappliesto:ElementaryandSecondaryprograms.
Standard11:LessonPlanning
Theprogramtrainsteachercandidateshowtoplanlessons.
Standardappliesto:ElementaryandSecondaryprograms.
Standard12:AssessmentandData
Theprogramtrainsteachercandidateshowtoassesslearningandusestudent
performancedatatoinforminstruction.
Standardappliesto:ElementaryandSecondaryprograms.
Standard13:Equity
Theprogramensuresthatteachercandidatesexperienceschoolsthataresuccessful
servingstudentswhohavebeentraditionallyunderserved.
Standardappliesto:Elementary,SecondaryandSpecialEducationprograms.
Standard14:StudentTeaching
Theprogramensuresthatteachercandidateshaveastrongstudentteaching
experience.
Standardappliesto:Elementary,SecondaryandSpecialEducationprograms.
Standard15:SecondaryMethods
Theprogramrequiresteachercandidatestopracticeinstructionaltechniquesspecificto
theircontentarea.
Standardappliesto:Secondaryprograms.
Standard16:InstructionalDesignforSpecialEducation
Theprogramtrainscandidatestodesigninstructionforteachingstudentswithspecial
needs.
Standardappliesto:SpecialEducationprograms.
OUTCOMES
Standard17:Outcomes
Theprogramandinstitutioncollectandmonitordataontheirgraduates.
Standardappliesto:Elementary,SecondaryandSpecialEducationprograms.
Standard18:EvidenceofEffectiveness
Theprogram'sgraduateshaveapositiveimpactonstudentlearning.
Standardappliesto:ElementaryandSecondaryprograms.
NeededContextualization
ThecontextualizationIwishtosupplyderivesfromwhatweknowabout
measurementandevaluationasappliedtothisreview.
First,itisimportanttoconsideronwhatthereviewisbased;inthiscase,the
standardsabove.Itshouldberememberedthatnosinglereviewconsiders
everything;indeed,nosinglereviewcouldorshould.Butitisaprinciplethata
reviewcanonlyshedlightonwhatitislookingfor.Atagenerallevel,atleast,I
cannotarguewiththevalueofthislistofstandards.Itisnotpreciselythesamelist
thatUNIisrequiredtousebythestateandprofession,butitisreasonablysimilar.
Indeed,Ithinkthatmostofthestandardsrepresentthingswearealreadyrequired
todo.Itcouldbepointedoutthattheirstandardsarenotfullyvalidatedinpractice,
butneitherareanyoneelse’sintheprofession(see,forexample,Dr.Ball’s
commentsat
www.edweek.org/media/dball_why%20we%20will%20not%20boycott%20NCTQ.
pdf).
Second,goingdeeper,weneedtolookathoweachstandardisoperationalized;
thatis,whatspecificindicatorsareusedandwithwhichspecificsourcesof
evidence.Itmaybe,forexample,thattwodifferentgroupsmighthavesimilar
standardsbutjudgethemdifferently.Forexample,IdisagreewithNCQthatthe
literaturesupportsaconclusionthatsimplyacceptingstudentswithhigherGPAs
andtestscoresnecessarilymakesforbetterteachers,norisitthecaseatUNIthat
educationstudentsareatthebottom.
Third,onemightconsiderhowthejudgmentsabouteachindividualstandard
aresummarizedtocomeupwithafinalratingandthenusedtorankinstitutions.
Eachofthe17standardswillbeassessedbyexaminingsyllabiandother
documentationprovidedprimarilybytheinstitution.NCTQprovidesarubricon
theirsiteforassessment,althoughthereisonlyalimitedamountofdetail.Aswith
allratingsystems,thedevilisinthedetails.Iamunabletodeterminefromthe
NCTQwebsitehowsomeoftheindicatorswouldactuallybeassessed,andpossibly
evenmoreimportantly,howtheindividualindicatorscoreswouldbesummedto
createonefinalnumber.
Anyrankingsystemhastobeinterpretedintermsofthecomparisongroup.
Forexample,ifIwererankedastheworstsprinterinmygroup,butthatgroupwas
theU.S.Olympicsprintsquad,thenI’mstillprettyfast.Likewise,ifIhavea
classroomfullofstudentsallbetween5’9”and5’10”,beingtheshortestortallestof
thegroupdoesn’tmakemuchofadifference;itisdifferencewithoutmuch
importance.
Fourth,arethemetricsthisreviewfocusesonreallythosethatdrivegood
teacherpreparation?Therecentlycoined“McNamarafallacy”notestheproblemsin
focusingontheeasilyquantifiableattheexpenseofwhatmayultimatelybemore
important.RobertMcNamara,U.S.SecretaryofDefenseforpartoftheVietnamWar,
famouslyusedreportedenemybodycountsasevidencewewerewinningthewar.
Thehistoricalrecordsuggeststherewereothervariablesatworkthathewasnot
considering.Inthecontextofrankingcolleges,considerReedCollege’sapproach.
Since1995ReedCollegehasrefusedtoparticipateintheU.S.NewsandWorldReport
"bestcolleges"rankings....
Reeddoesparticipateinseveralotherwell‐establishedcollegeguidesthatdonot
assignnumericalrankingstoinstitutions,includingBarron's,theFiskeGuideto
Colleges,Peterson's,CollegesthatChangeLives,Newsweek'sCollegeGuide,andthe
CollegeBoard'sCollegeHandbook.Eachoftheseguidesattemptstodescribemorefully
theexperience,studentculture,andacademicenvironmentatdifferentschools.
ConsistentwithReed'snon‐participationinU.S.Newsrankings,thecollegealsodoes
notparticipateinMoneymagazine'scollege‐rankingissue.
ReedCollegehasactivelyquestionedthemethodologyandusefulnessofcollege
rankingseversincethemagazine'sbest‐collegeslistfirstappearedin1983,despitethe
factthattheissuerankedReedamongthetoptennationalliberalartscolleges.Reed's
concernintensifiedwithdisclosuresin1994bytheWallStreetJournalabout
institutionsflagrantlymanipulatingdatainordertomoveupintherankingsinU.S.
Newsandotherpopularcollegeguides.ThisledReed'sthen‐presidentStevenKoblikto
informtheeditorsofU.S.Newsthathedidn'tfindtheirprojectcredible,andthatthe
collegewouldnotbereturninganyoftheirsurveys....
Reed'spresident,ColinDiver,cautionsprospectivestudentsandparentsagainst
relyingonrankings.Rankings,hesays,aregroundedina"one‐size‐fits‐all"mentality...
.”(http://www.reed.edu/apply/news_and_articles/college_rankings.htm)
Fifth,wemightaskifthedatausedtodotherankingsisaccurate?Inthe
NCTQ/USN&WRreviewthereisnoindependentchecking.Currentlyinthenews
arereportsthatsomeuniversitieshavefudgedtheirnumberstolookbetteronthe
USN&WRrankingsofcolleges(see,forexample,
http://www.theatlanticwire.com/national/2013/02/which‐schools‐arent‐lying‐
their‐way‐higher‐us‐news‐ranking/61874/).
Sixth,evengivenaccuratedata,differentresultscanstillresultthrougha
differentmixofcriteriaandweighting.Considerhospitalrankings.Thereareat
least15differentsetofrankings,andresultsvary.Differentdata,orthesamedata
butwithdifferentcriteriaandweighting,yieldsdifferentresults.MalcolmGladwell
gaveaverythoughtfulanalysisofrankingsingeneralinarecentissueoftheAtlantic
(http://www.newyorker.com/reporting/2011/02/14/110214fa_fact_gladwell).As
anexample,heusedaCarandDrivercomparisonbetweenaCorvette,aLotus
Evora,andaPorscheCayman.TheresultswereCayman193,Corvette186,and
Evora182;soclearlytheCaymanisbest,right?Gladwellpointsout,however,that
thecomparisonusedCarandDriver’sstandardformula,thesameastheywoulduse
forcomparingfamilyminivans,eventhoughtheseareexpensivesportscars.For
example,only4%ofthefinalscorewasasubjectiveappraisalofhowthecarlooks.
Foraminivanthatmightmakesense,butforasportscar?Costisalsoonlya
relativelysmallpartofthecomparison,fineforcarenthusiasts,notsohelpfulfor
normalbuyers.Andagain,thedrivingexperiencecontainsasubstantial36%,but
forasportscarbuyershouldprobablycountforalotmore.AsGladwellpointsout,
ifonechangedtheweightingsindifferentbutequally,ormore,valuableways,the
LotusmightwinortheCorvette,basedonthesamefactualresults.Hisbottomline
onwhichisreallythebestcar,andwhatthistellsusaboutcollegerankings:
CarandDriver’sambitiontogradeeverycarintheworldaccordingtothesame
methodologywouldbefineifitlimiteditselftoasingledimension.Aheterogeneous
rankingsystemworksifitfocussesjuston,say,howmuchfunacaristodrive,orhow
good‐lookingitis,orhowbeautifullyithandles.Themagazine’sambitiontocreatea
comprehensiverankingsystem—onethatconsideredcarsalongtwenty‐one
variables,eachweightedaccordingtoasecretsaucecookedupbytheeditors—would
alsobefine,aslongasthecarsbeingcomparedweretrulysimilar.It’sonlywhenone
caristhirteenthousanddollarsmorethananotherthatjugglingtwenty‐onevariables
startstobreakdown,becauseyou’refacedwiththeimpossibletaskofdecidinghow
muchadifferenceofthatdegreeoughttomatter.Arankingcanbeheterogeneous,in
otherwords,aslongasitdoesn’ttrytobetoocomprehensive.Anditcanbe
comprehensiveaslongasitdoesn’ttrytomeasurethingsthatareheterogeneous.But
it’sanactofrealaudacitywhenarankingsystemtriestobecomprehensiveand
heterogeneous—whichisthefirstthingtokeepinmindinanyconsiderationofU.S.
News&WorldReport’sannual“BestColleges”guide.
Conclusion
IagreewiththeconclusionfromDr.Ball,DeanofEducationatMichigan,asto
whysheisnotupsetattheadventoftheserankings.
AttheUniversityofMichigan,wehavebeenredesigningourprogram,andarebusy
implementingandimprovingit.Wearesurewehavealottolearnstill,andhopethat
bystudyingourefforts,wewillbeabletokeepdevelopingandimprovingour
approaches.WealsoassumethatwewillnotdowellonalltheNCTQcriteria;we
understandthat.Insomecases,wesimplydon’tyetdocertainthingsaswellastheir
standardsspecify.Inothercases,wedonotseektodothingsinquitethewaythey
identify.(www.edweek.org/media/dball_why%20we%20will%20not%20boycott
%20NCTQ.pdf)
Thus,regardlessofwhetherUNIisratedwellorpoorly,pleaserememberthatthis
rankingisonlyonepotentialindicatorofquality,whetheryouareapotential
student,stateofficial,memberofthegeneralpublic,orotherstakeholder.Ifyouare
interestedinattendingateachereducationprogram,thenUSN&WR’srankings
couldbeoneelementofyourprocess.Butitshouldreallybeonlyonepartofyour
process.Youshouldalsovisittheinstitution,talkwithstudentsandfaculty,see
whattheyreallydo,talktograduates,examineotheravailabledata,etc.Forthoseof
usatUNI,theresultscanbepartofwhatweconsiderinthinkingaboutourquality
andareasinwhichwemightimprove.Butagain,itshouldonlybeapartofwhatwe
consider.Wehavemanyothersourcesofdata,someofwhicharedirectmeasures
insteadofmerelypaperwork.LikewiseforothersconsideringUNIandother
teachereducationprogramsinIowa—thereismoretoeachofusthancanbeshown
inthisoneparticularrankingapproach.