AssessmentofInnovativeAcademicInitiatives:
Queen’’sSupportedLearningGroupsPilotProgram
Preparedfor:
TheHigherEducationQualityCouncilofOntario
by:
JenniferMassey,SeanFieldandJeffBurrow
Queen’’sUniversity
StudentAffairsResearch&Assessment
June2011
1
Abstract
Inanefforttoenhanceengagementandretentionandimprovegradesinlargeundergraduatecourses,
SupplementalInstruction(SI)hasgrownconsiderablyoncampusesacrossNorthAmerica.Asbudgets
aresqueezedandthesizeoffirstͲyearclassesgrow,SIhasbecomeanimportantcomponentinthe
deliveryofundergraduateeducation.CriticalexaminationoftheimpactsofSIislimited,however,anda
deeperunderstandingofthestrengthsandweaknessesofthisapproachisurgentlyneeded,givenits
growingprevalence.ThispapercriticallyassessestheimpactofoneapproachtoSI,SupportedLearning
Groups(SLGs),atQueen’’sUniversity.Ourfindingsindicateseveralfactorsinfluencethelikelihood
studentswillattendSLGsessions,includinggender,yearofstudy,previousacademicperformance,and
identifyingasadomesticstudent.HoldingSLGsessionsinresidences,weargue,isalsoalikely
determinantofSLGparticipation.AlthoughourfindingssuggestthattheSLGprogramatQueen’’s
Universityhasshownsignsofpositive,althoughmixed,successduringitspilotyears,weassertthat
furtherresearchremainstobedonetounderstandmorefullytheroleofthisprograminsupporting
studentsuccess.TheevidencepresentedsuggestsSupportedLearningGroupsplayanimportantrolein
reinforcingacademicbestpracticesandprovidingguidedstudytimeforstudentsatacademicrisk.They
areshowntobeaneffectiveadditiontotraditionalacademicresourcessuchasseminarsandlectures.
Keywords:Supportedlearninggroups,supplementalinstruction,peereducation,assessment,student
affairs
2
Acknowledgements
Theauthorsofthereportwishtoexpresstheirsincereappreciationandgratitudetothemany
individualswhohelpedthroughoutthevariousstagesoftheresearchproject.Thisprojectand
thisreportwouldnothavebeenpossiblewithouttheircontributions.
First,wewouldliketothankElspethChristie,CoordinatorofLearningStrategiesatQueen’’s
UniversityforintroducingtheSLGprogramtoQueen’’sandinvitingustomeasurethe
outcomes.LizParsonsCoordinator,SupportedLearningGroupsforherassistancein
coordinatingthecollectionofparticipationdataeachweek.
WewouldalsoliketothankDrJasonLaker(formerAssociateViceͲPrincipalandDeanof
StudentAffairs),andtheDepartmentofBiologyandPsychologyforalloftheirsupportand
advicethroughoutthedurationoftheproject.
WewishtoacknowledgethecontributionoftheGeneralResearchEthicsBoardatQueen’’s
University.Werecognizethecomplexityofthistypeofstudy,andarethereforeverygrateful
forthediligentandconstructiveadvicethatallowedustocapturetheinformationneededin
thisstudyinaethicallyresponsibleway.
ThankyoutoChrisConway,DirectoroftheOfficeofInstitutionalResearchandPlanningat
Queen’’sUniversityforassistingwiththeextractionofdatafromthestudentrecordssystem.
AsincerenoteofappreciationisextendedtoLinaDiGenova,ManagerofStudentAffairs
AssessmentatMcGillUniversitywhoprovidedathoughtful,criticalreviewofanearlydraftof
thisreport.Yourcomments,suggestions,andcritiqueswereveryuseful.
Finally,wewouldliketoexpressagenuinethankstoallofthestudentswhoparticipatedinthis
researchprojectbyansweringquestionsinthesurveysandparticipatinginfocusgroups.This
researchwouldhavebeenimpossiblewithouttheinformationyouprovided.
3
Introduction
Inrecentyears,Supplementalinstruction(SI)hasgrownconsiderablyonNorthAmericancampusesin
anefforttoenhancestudentengagementandretentioninlargeundergraduatecourses,aswellas
improvegrades.TheestablishmentofSLGprogramsatuniversitiesacrossCanadahasbeenfuelledby
findingsthatsuggeststudentswhoparticipateinSLGsexperiencegreateracademicsuccessthan
studentswhodonotparticipate(McInnis,2001;Tinto,2002;Yorke&Thomas,2003;Peat,Dalzeil,&
Grant,2001).Inparticular,previousstudieshavefoundstudentswhoparticipateinSLGsessionsearn
highergradesthannonͲparticipants(Arendale1997;MartinandArendale1994).Asbudgetsare
squeezedandfirstͲyearclasssizesincrease,SIhasbecomeanimportantcomponentofthedeliveryof
undergraduateeducation.CriticalexaminationoftheimpactsofSIislimited,however,andadeeper
understandingofthestrengthsandweaknessesofthisapproachtothedeliveryofundergraduate
educationisurgentlyneededgiventhegrowinguseofSIacrossNorthAmerica.
ThisreportcriticallyassessestheimpactofoneapproachtoSI,SupportedLearningGroups
(SLGs),anditsimpactatQueen’’sUniversity.WearguewhileSLGsareincreasinglypresentedasacostͲ
efficient‘‘solution’’tothegrowthinthenumberoflargeundergraduatecourses,theimpactofthis
programatpostͲsecondaryinstitutionsremainsinstitutionallyspecific.AtQueen’’s,wefindwhenthe
endogenousimpactofdemographicdifferencesandentrancegradesiscontrolledfor,SLGparticipation
hasamixedandlargelystatisticallyinsignificantimpactonstudents’’finalgrades.Wealsofind,however,
thatSLGsplayanimportantroleinenhancingstudent’’sacademicengagement.We,therefore,assert
SLGsareaneffectiveadditiontotraditionalacademicresourcesatQueen’’s,suchasseminarsand
lectures,byreinforcingacademicbestpractices,however,theircontribution,weargue,remains
supplemental.
WecallforuniversitiesandcollegestoconsidercarefullythereasonsforimplementingSLGson
theircampusesandtoadoptanempiricallyinformed,outcomesͲbasedapproach.Wealsocallfora
4
rigorousassessmentoftheseprograms,whichconsiderstheinstitutionalembeddednessofindividual
collegesanduniversitiesforthepurposeofunderstandingthedistinctandnuancedroletheseprograms
playinstudentsuccessatdifferentpostͲsecondaryinstitutions.
Thereportbeginswithacriticalreviewofpreviousresearchintotheimpactsofsupplemental
instruction,payingparticularattentiontothelimitedresearchonsupportedlearninggroupsinCanada.
Thethirdsectionofthereportoutlinestheempiricalframeworkthatguidedthisstudy.Theremainder
ofthereportdiscussesthefindingsofouranalysisandtheimplicationsofthesefindings.
AReviewofSupplementalInstruction
TheSupportedLearningGroupsmodelofSupplementalInstructionwasoriginallydevelopedatthe
UniversityofMissouriKansasCity(UMKC)inthe1970s.Itwasdevelopedinresponsetoanalarming
attritionrateintheirhealthsciencesprogram,particularlyamongstmarginalizedstudents(Blanc&
Martin1994).WhilevarioussupportservicesexistedattheUMKCLearningCentre,somestudentsfelt
theresourcesavailableweretoobroadandnotapplicabletothecourseswithwhichtheywere
struggling(Blanc,DeBuhr,&Martin,1983).Dr.DeannaMartin,fromtheUMKCCentreforAcademic
Development,conductedareviewofexistingretentionprogramsatUKMCandnotedhighͲriskcourses
wereoftenprerequisitesforseveralothercourses,andsuccessinthesecoursesrequiredskillsetsthat
firstͲyearstudentshadrarelymastered(Blancetal.,1983).Identifyingthis,sheproposedan
interventiontoprovideadditionalacademicsupporttostudentsinhighͲriskcourses.Theintervention
utilizedapeerͲledacademicmodelinwhichupperͲyearundergraduatestudents,whohadpreviously
achievedagradeof80%orhigherinthecourse,ledacourseͲspecificstudysession(Blancetal.,1983).
UpperͲyearstudentswhowereselectedasSLGleadersreceivedtrainingintheprinciplesofSI,aswellas
handsͲontraininginleadingpeerͲbasedgrouplearning.Thesestudentsattendedlecturesforthe
correspondinghighͲriskcourseand,withthesupportofprofessionalstaffandfaculty,designedstudy
5
sessionstoreviewrecentcoursematerialandintroducestudentstoarangeofstudyskillsand
strategies.Acentralgoalofthesesessionswastohelpstudentsgainconfidencewiththecourse
material.Thiswasachievedbyhelpingstudentslearntodeveloptheanswerstotheirownquestionsin
theSLGsessions.Studentsweretaughtavarietyofstudyskillsandapproachestolearningandwere
encouragedtoworkwithotherstudentsintheclasstosolveproblems.
Sincethen,thenumberofSIprogramsoncampusesacrossNorthAmericaandinternationally
hasgrownsubstantially.Itisestimatedthatover500collegesanduniversitiesintheUnitedStates,as
wellasgrowingnumberofpostsecondaryinstitutionsinCanada,theUnitedKingdom,Australia,and
elsewhere,haveadoptedSIprograms(BlancandMartin1994).Thepotentialofboostingstudentgrades
andretentionisnodoubtappealingtoanypostͲsecondaryinstitution.TheadoptionofSIprograms,
however,hasnotbeenuniformacrosscampuses,andevidencefromcollegesanduniversitiesthathave
adoptedSImodelssuggeststheimpactofSI,likemanyinterventions,mayvarybetweeninstitutionsand
programs.Inthissection,wereviewsomeofthesefindings.Thisreviewisbynomeansexhaustive;the
intentistoidentifythemesinthisliterature,differencesbetweenSIprograms,andthesomeofthe
weaknessesofpreviousSIresearch.
Sinceitsinceptionin1973,theCenterforAcademicDevelopmentatUMKChasledthewayin
compiling,analyzing,andmonitoringtheimpactofSIprogramsintheUnitedStates(Fayowski&
MacMillan2008;Ramirez1997).UsingaquasiͲexperimentaldesignanddataonthousandsofstudents
fromSIprogramsacrosstheUnitedStatesoverseveralyears,therehasbeenaplethoraofwork
showingSIparticipantsperformbetterthannonͲSIparticipantsacrossinstitutionsanddisciplines
(Arendale1997;Martin&Arendale1994;Congos&Schoeps1993;Martin&Arendale1992;Martin&
Blanc1991;Wolfe1987;Blancetal.1983).Overall,thisresearchhasshownSIprogramshaveapositive
impactonaparticipant’’smeangrades,retentionrate,andlearningskilldevelopment.SIhas
6
subsequentlybeenrecognizedtwicebytheUSDepartmentofEducationasanexemplaryprogram
(Fayowski&MacMillan2008).
ThefindingsfromthesestudieshaveprovidedaspringboardforthedevelopmentofSI
programsatuniversitiesandcollegesacrossCanadaandtheUnitedStates.Theyhavealsogenerated
additionalresearchontheimpactSIprograms.Someofthisresearchhasfollowedtheleadofprevious
studiesusingdatafromacrossinstitutions,whileotherresearchhasbeeninstitutionspecific.Etter,
Burmeister,andEldertookametaͲanalysisapproachtoexaminingtheimpactofSIonstudentsenrolled
inintroductoryaccountingwhentheyuseddatafromtheUMKCCentreforAcademicDevelopment
(2000).Theirdatawerecompiledfrom132coursesofferedby21institutionsacrosstheUSbetween
1986and1995.TheauthorsfoundSIimprovedaparticipant’’sexperiencesinintroductoryaccounting,as
wellasimprovedtheirperformanceandretention(Etteretal.2000).
RathPeterfund,Xenos,BaylissandCarnal,bycontrast,tookaninstitutionalspecificapproachby
measuringtheimpactofSIonstudentsenrolledinIntroductoryBiologyatSanFranciscoStateUniversity
(2007).Usingdatafrom1,526students,collectedbetween1999and2005,theyfoundSIparticipants,
onaverage,earnedhighergrades,andweremorelikelytoearna““CͲ““gradeorhigherinthecourse
overall.Theauthorsfoundstudentsfrommarginalizedbackgroundsbenefitedmostfromparticipatingin
SI.SIparticipantsfrommarginalizedbackgrounds,theyfound,weremorelikelytoearnhigheraverage
gradesinIntroductoryBiologyandgraduatefromtheUniversity.
LoviscekandCloutier’’sexaminationoftheimpactofSIonstudentsenrolledinintroductory
economicsattheUniversityofWisconsinͲParksideyieldedsimilarresults(1987).Usingdatacollected
from81studentsand2Ͳstageregressionanalysis,theauthorsfoundSIhadastatisticallysignificantand
positiveimpactonacademicperformance.Theyarguedpreviousregressionestimatesunderestimated
theimpactofSIonstudentsuccess,andthatSIsessionsweremoreeffectiveatenhancinglearningand
economicliteracy,thanpassivestatemandatedpolicies.
7
ResearchfromCanadaontheimpactofSIprogramshasbeenlimited.LikeRathetal.(2007)and
Loviscek&Cloutier’’s(1997)work,muchofthisresearchhasbeeninstitutionalspecific.
Fayowski&MacMillan(2008),forexample,examinedtheimpactofSIonstudentsenrolledin
introductorycalculusattheUniversityofNorthernBritishColumbia.Toconducttheiranalysis,the
authorsuseddatacollectedfromapproximately1,250studentsbetween2001and2004.UsingANCOVA
andregressionanalyses,tocontrolforstudents’’priorGPAandgender,theyfoundSIparticipants,on
average,earnedtwolettergradeshigherthannonͲparticipants.
FindingsfromCarletonUniversitycorrespondwiththeseresults.Usingdatafrom4,942
students,collectedinthe2007Ͳ2008schoolyear,Miles,PolovinaͲVukovic,LittlejohnandMarini,
comparedSIparticipantswithnonͲSIparticipantsacross30coursesinseveralfaculties(2010).The
authorsfoundSIparticipantsachievedhigherfinalgradesthannonͲparticipantsaftercontrollingfor
students’’admissionaverages.Basedonattendancefrequency,theauthorsfoundSIparticipantsearned
finalgradesuptoawholelettergradehigherthannonͲparticipants.
EvidencefromtheUniversityofGuelphandUniversityoftheFraserValleysimilarlysuggestSI
hasapositiveimpactonstudentperformance.FindingsfromtheUniversityofGuelph,whichwasoneof
thefirstuniversitiestoestablishanSIprograminCanada,indicateSIparticipantsearnfinalgrades2.50
to5.50percentagepointshigherthannonͲparticipants(Wilson,2005).Comparatively,SLGparticipants
atUFVhavebeenfoundtoachievehighergradesthannonͲparticipants,andlesslikelythannonͲ
participantstowithdrawornotcompleteSLGassociatedcourses(UFV,2010). Overall,evidenceonthebenefitsofSI,andSLGsinparticular,onstudentperformanceand
retentionissignificant.Indeed,inadditiontotheshortͲtermcoursespecificimpactofSIprograms,
evidencealsosuggestssuchprogramshavealongͲtermpositiveimpactonparticipants(Ogden,
Thompson,Russell,&Simons,2003;Ramirez1997).TherearethreemainweaknessesincurrentSI
literature,however,andthesehavereceivedonlymixedattentiontodate.
8
Thefirstoftheseweaknessesistheimpactofendogeneity(selfͲselectionbias).InSI,the
problemofendogeneityariseswhenresearcherstrytodistinguishtheimpactofSIparticipationfrom
thestudent’’sunderlyingabilityandmotivationforacademicperformance.AlthoughMartinand
Arendale(1992)havepreviouslyfoundstudentmotivation(afactorinselfselectionbias)insufficiently
accountsfordifferencesinSIoutcomes,thereisagrowingconsensusthatselfͲselectionbiasinthisarea
ofresearchdoesindeedmatter.SelfͲselectionbiasconcernsarenotuniquetoSIresearch.Indeed,
researchthatseekstodrawcasualrelationsthroughempiricalanalysisisconsistentlychallengedbythis
issue.IntheSIliterature,severalauthorshaveattemptedtocontrolforselfͲselectionbiasthrough
variousmeans.LoviscekandCloutier,forexample,useaHeckmantwoͲstageregressionmodelfor
estimatingtheinfluenceofSIparticipationonastudent’’sacademicperformance(1997).Otherauthors,
bycontrast,haveoptedtouseANCOVAs(Fayowski&MacMillan2008;Kochenouretal.1997;Mileset
al.,2010),whichcomparetheoutcomesoftwoormoregroupswhiletakingintoaccounttheinfluence
ofoneormorecovariates.Whilethedevelopmentofmoresophisticatedstatisticaltechniquesto
controlfortheimpactofendogeneitycontinue,attentiontoitsresultantimpactremainmixed(seefor
exampleMahdi,2006;Ogdenetal.,2003;Rathetal.,2007).
ThesecondissueassociatedwithSIresearchisthelackofattentionpaidtoinstitutionaland
programdiversity.TheresultspresentedbyEtteretal.(2000),forexample,revealdifferencesinSI
participationrates,andoutcomesvarybetweenpublicandprivateaswellaslargeandsmallpostͲ
secondaryinstitutions.Whileonlydescriptive,thesedatasuggesttheimpactofSIvariesbetween
institutionsandprograms.Thisisnotanewobservation.Previousauthorshaveraisedquestionsabout
howsystematicdifferencesinprogramspecification,administration,andparticipantcompositionhave
affectedSIoutcomes(Burmeister,Kenny,&Nice,1996).Yet,noknownstudiesinCanadaorelsewhere,
havesystematicallyreviewedhowprogramandstudentdiversitymayaffectsuccessfulSI
implementationandparticipantsuccess.
9
WhileitiscommonforauthorsexaminingtheimpactofSIatspecificinstitutionstoidentifythe
institutionanddescribethestructureoftheSIprogram(therearemanyvariationsoftheSImodel),
therehasbeensurprisinglylittlefocusonwhatmakestheseinstitutions,andconsequentlytheirSI
programs,unique.AlthoughpostͲsecondaryinstitutionshavemuchincommon,theyarehistorically,
geographically,culturally,socially,andnationallyembedded(Amin,1999).Consequently,largeurban
andprivateuniversitiesarelikelytohavedifferentfaculties,enrolments,courseofferings,classsizes,
andresourcesthansmallertownpublicuniversitieslocatedinmoreruralareas.Similarly,olderpostͲ
secondaryinstitutionsmayhave,overtime,developeddifferentstudentͲcentredresourcesandsupport
systemsthannewlyestablisheduniversities,anduniversitiesinCanadaarelikelytodiffersomewhat
fromuniversitiesintheUSduetosystemichistoricalinstitutionaldifferences.Furthermore,some
universitiesmayattractahigherproportionofhighachievingacademicallyresilientstudents,byvirtue
oftheuniversity’’sreputationorstudent’’sprioraccesstoeducationalenrichment,whileother
universitiesmaynot.Consequently,thediversityofpostͲsecondaryinstitutiontypesandtheirlocal
institutionalembeddednessmaynotbeconducivetodirectlycomparingSIprogramoutcomesacross
institutionsthatarecharacteristicallydistinct(i.e.publicvs.private,largevs.small,newvs.old,or
geographicallydispersed).Thesedifferencesmayhelpexplaindeviationsinmeasuredprogram
dynamicsandoutcomesthatlargeͲscalemetaanalyticalapproachestendtoconcealatthesubstantive
riskofovergeneralizingprogramsuccess(seeforexampleBurmeisteretal.,1996;Etteretal.2000;
Kochenouretal.,1997;Schwartz1992).
Finally,therehasbeenlittlecriticaldiscussiononthechangingroleofSIprogramsatpostͲ
secondaryinstitutions.AccordingtoBlancandMartin(1994),theimpetusforSIwasthedesireto
significantlyreduceattritionratesamongstmarginalizedstudentsinhealthrelatedprogramsintheUS
byprovidingpeerͲorientedacademicsupportinhistoricallydifficultcourses(i.e.courseswithattrition
ratesofapproximately30%).ItssubsequentadoptionbyuniversitiesandcollegesacrosstheUSand
10
internationallyisnodoubtaconsequenceoftheprogram’’swidespreadsuccess,andthissuccessshould
becelebrated.TherecentimpetusfortheexpansionofSIprogramsacrossandwithininstitutions,
however,seemstohavesomewhatdivergedformitsoriginalintent.Forexample,whilestudent
attritionhasalwayshadanassociatedmonetarycostforpostͲsecondaryinstitutions,Ramirez(1997:3)
notesan““eraoffiscalconstraints””waspartiallytheimpetusfortheprioritizationofasuccessfulSI
program.Similarly,LoviscekandCloutier(1997:75)argue““sincetheSIprogramisstaffedlargelyby
undergraduatestudents,itmaybeacostͲeffectiveoptionthatsmallerundergraduateinstitutionsmay
wanttoconsider””(emphasisadded).Finally,Kochenouretal.(1997)havearguedSIismorecost
effectivepercontacthour,thaneithertutoringorcoursebasedskilldevelopment(averaging$4USper
hourin1994Ͳ95,lesssomeadministrativecosts),whichisalsoduetoitsdependenceonlowerpaid(or
unpaid)undergraduatestudentSIleaders.
ForpostͲsecondaryinstitutions,theallureofSI’’spurportedcosteffectivenessisobviousamid
successivewavesoffundingcutbacksandtheassociatedneoliberalreformsinhighereducationthat
haveoccurredoverthepast30years(atleastinCanada).Forstudents,ontheotherhand,SIprograms
areanexcellentopportunity(forbothleadersandparticipants)tobuildskillsandenrichtheiracademic
experience.TheutilizationofSIasacosteffectivesubstituteforinstructionandguidancebyprofessors
andtraineduniversityprofessionals,ratherthanasasupplementtotheseresources,however,isa
disturbingtrendthatappearstoruncountertoSI’’sstatedobjectivesofenhancingretentionand
performance.Indeed,therelationshipbetweenneoliberalreformsinhighereducation,fiscal
constraints,andthegrowthofSIprogramsremainscriticallyunexplored.
WiththesestrengthsandweaknessesoftheSIresearchinmind,theremainderofthisreportis
dedicatedtocriticallyexaminingthepilotimplementationoftheSLGprogram,aformofSI,atQueen’’s
University.Thisassessmentfocusesspecificallyonthestudentparticipantimpactofthepilotprogram
usingavailabledata.Basedonresource,time,andspaceconstraints,thisreportdoesnotdirectly
11
respondtoallofthegapsintheliteratureoutlinedabove.Rather,itspecificallyaddressestheneedto
controlfortheendogenousimpactofselfͲselectionbias,andinterpretsthefindingsthatfollowas
uniquetotheinstitutionalembeddednessofQueen’’sUniversityandtheSIprogramatQueen’’s.Whilea
systemicreviewoftherelationshipbetweenhighereducation,fiscalconstraints,andthegrowthofSI
programsisoutsidethescopeofthispaper,thisrelationshipisconsideredinrelationtoourfindingsin
ourconcludingremarks.
SupplementalInstructionatQueen’’sUniversity
Queen’’sUniversityisaresearchͲintensivemidͲsizedpostsecondaryinstitutionlocatedhalfwaybetween
TorontoandMontréalinKingston,Ontario.Establishedin1841,Queen’’sisoneoftheoldest
postsecondaryinstitutionsinCanada,andoffersawiderangeofprofessional,undergraduate,and
graduateprogramsintheareasofscience,thearts,thesocialsciences,medicine,business,law,and
education.Followingitsrichacademictraditionandahistoryofstudentsuccess,Queen’’sisconstantly
seekinginnovativewaystohelpstudentsreachtheiracademicpotential.Overthepastfiveyears,
Queen’’shassignificantlyexpandeditsrangeofinnovativeacademicsupportservices,includingthe
expansionofonlineresources.AnimportantpartofQueens’’portfolioofinnovativeacademicsupport
servicesisSupportedLearningGroups(SLGs),aformofSupplementalInstruction(SI).
Queen’’sUniversityinitiallypiloteditsSLGprogramduringthe2008Ͳ2009academicyearin
Biology102andBiology103.ThispilotwassubsequentlyextendedtoincludePsychology100in2009Ͳ
10.WhileplanningtheexpansionoftheSLGprogram,Queen’’sstaffconsultedandconductedsitevisits
togainadeeperunderstandingofhowotheruniversitiesmanagedandoperatedSLGprograms.These
visitsprovidedadditionalinformationonSLGprogramimplementation,SLGsessionandcurriculum
development,peerͲleaderrecruitment,supportandtrainingandSLGpromotionandmarketing.
12
ArecruitmentstrategyforallSLGpeerleaderswaslaunchedinApril2009.Toqualify,SLG
leadershadto:(1)beacurrentPeerLearningAssistant(PLA);(2)havesuccessfullycompletedthefirstͲ
yearBiology/PsychologycoursesthatwouldbethefocusofSLGprogramatQueen’’s;(3)showinterest
intheSLGprogram;and(4)demonstrateexcellentfacilitationskills.Whentherecruitingprocesswas
completed,thePLAsparticipatedinapreͲservicetrainingsessionfacilitatedbytheSLGCoordinatorand
theCoordinatorofLearningStrategiesOutreachbeforeclassesbeganinSeptember.Thistraining
sessionequippedPLAleaderswiththetoolstheyneededtofacilitateanSLGsession.Duringthetraining,
PLAleaderswererequiredtoleadamockͲSLGsessionwheretheylearnedhowtoreͲdirectquestions
backtothestudents,reducingthetendencyofparticipantstoviewthePLAasanexpertandto
encouragegroupledproblemsolving.Thetrainingalsocoveredprogramlogistics,aswellas
administrativeissues(roombookings,attendance,etc.).
AtQueen’’s,twoteamsofPLAsrotatedonabiͲweeklyschedulesostudentsintheSLGswere
exposedtomultiplePLAsinsession.Sessionswereheldonaweeklybasisandwereprimarilycomposed
offirstͲyearstudents.IntheSLGsessions,PLAsmodeledpersonallearningstrategiesandfacilitated
activitiesthataidedstudentsinunderstandingcoursematerialanddevelopingtheirownlearning
strategies.ThePLAswerealsoresponsiblefortakingattendanceeachweek1.
TheCoordinatorwasresponsibleforcreatingweeklystudyguidesforsessions.ThesecourseͲ
basedguidescontainedactivitiesthatstudentsworkedthroughcollaborativelytohelpthemunderstand
andreinforcelecturematerialfromthepreviousweek.FeedbackfromtheinitialSLGpilotprogram
indicatedstudentsfelttherewasnotenoughemphasisonthecoursecontentoraclearlinkbetween
coursematerialandlearningstrategies.Consequently,the2009Ͳ2010studyguidesbalancedcourse
contentwithlearningstrategydevelopment.
1
Itshouldbenotedthatattendancewastakenatthesesessionsforthepurposesoftheresearchproject.No
academiccreditwasgivenforattendanceandinformationaboutwhichstudentsattendedtheSLGwasnever
sharedwiththecourseinstructors.
13
TheCoordinatorwasalsoresponsibleforpromotingtheSLGprogram.Beforeclassesbeganin
September,ResidenceLifestaffreceiveddetailedinformationabouttheprogram,whichtheyusedto
helppromoteSLGstostudents.Theprogramalsocollaboratedwiththefacultymembersteaching
Psychology100,Biology102,andBiology103.InstructorspostedlinkstotheSLGschedulesontheir
coursewebsites,andtheSLGCoordinatormadeabriefinformationpresentationduringthefirstweek
ofclasses.
SLGsessionsatQueen’’swereheldinstudentresidences.Theyformedpartofthebroader
residenceeducationalprogrammingmodel,whichisbasedonthephilosophythatstudents’’firstͲyear
experienceinresidencehasapowerfuleffectontheirattitudesandapproachestolearning(Pascarella
&Terenzini2005;Trotter&Robert2006;Upcraft,Gardner,&Barefoot2005).Additionally,previous
researchhasfoundstudentslivinginresidencehavegreatercriticalthinkingskillsthanfirstͲyear
studentslivingoffͲcampus(Kuh,Douglas,Lund,&RaminͲGyurnek1994;Pascarellaetal.,1993)andit
hasbeenfoundtobeanidealenvironmentfordevelopingandconductingsmallgroupwork(Tinto,
2002;Yorke&Thomas,2003).
ToguidetheassessmentoftheSLGprogramatQueen’’s,thisreportinvestigatesfivekeyresearch
questions:
1. Whatfactorsinfluencestudents’’likelihoodtoparticipateinSLGsessions?
2. TowhatextentdoesparticipationinSLGsleadtoincreasedacademicsuccessinacourse?
3. TowhatextentdotheSLGsincreasecoursematerialretention?
4. TowhatextentdotheSLGsincreaseengagementwiththecoursematerial?
5. TowhatextentdotheSLGsessionsenhancestudyskills?
14
DataandMethods
Thisstudyadoptedamixedmethodsapproach.Quantitativedatawerecompiledfromstudentsurveys,
studentrecords,andSLGattendancefilescollectedduringthe2009Ͳ2010academicyear.Qualitative
datawerecollectedthroughfocusgroupsconductedattheendofthe2009Ͳ2010academicyear.
DataCollection
Atthebeginningofthefallsemester,allstudentsregisteredinBiology102andPsychology100
wereinvitedtoparticipateinapreͲcoursesurveyadministeredduringthefirstclassofthesemester.
Thissurveysoughttogatheraninitialunderstandingofstudents’’awarenessandcomfortlevelwith
variouslearningstrategies.Studentawarenesswasgaugedbyyes/noquestions,whilecomfortlevels
wereassessedusinga5ͲpointLikertscale.Thesurveyaskedparticipantstoprovidetheirstudent
identificationnumbersotheirresponsescouldbematchedwithdatacollectedinthepostͲSLGsurvey.
Attheendofthewintersemester,allstudentsregisteredinBiology103andPsychology100
receivedaninvitationtoparticipateinthepostͲcoursesurvey.Thissurveywasadministeredonline,due
toconcernsaboutlowinͲclassresponseratesduringthelastweekofthesemester.Studentsreceived
aneͲmailcontainingaletterofinformationandauniqueURLlinkingthemtothesurvey.ThepostͲ
coursesurveyincludedthesamequestionsonlearningstrategiesasthepreͲcoursesurvey.Italso
includedquestionsfromtheClassroomSurveyofStudentEngagement(CLASSE)instrument,whichis
basedontheNSSEsurvey2.
RegularSLGparticipants(thosewhohadattendedatleast6ofthe22sessionsofferedeach
semester)wereinvitedtoparticipateinafocusgroupattheendofthe2009Ͳ2010academicyear.They
weresentapersonalizedemailinvitingthemtoparticipateandwereofferedamealanda$15giftcard
2
WhiletheNSSEisintendedtomeasurelevelsofengagementattheinstitutionallevel,theCLASSEsurveyinstead
focusesonthefrequencyandpracticesofengagementactivitiesataclasslevel(UniversityofAlabama,2010).
15
fortheirtime.SeveralreminderinvitationsweresentoutoverathreeͲweekperiodand,ultimately,
threefocusgroupswithsevenparticipantseachwereheld.Thepurposeofthefocusgroupswasto
capturestudents’’reflectionsandexperiencesintheprogram.Thefocusgroupsgatheredinformation
regardingstudents’’motivationsforparticipatingintheSLGprogram,aswellasinformationonwhat
skillsandstrategiesstudentslearnedfromattendingsessions.Thefocusgroupswereheldinthe
residencehalls,wereconductedbyatrainedfocusgroupfacilitator,andwereaudioͲrecorded.
DataAnalysis
QuantitativedatawereanalyzedusingStataandSPSS.Linearregressionwithrobuststandard
errorswasusedtoestimatetheimpactofcovariatesontheSLGparticipation.Propensityscore
matching(PSM)wasalsoadoptedforthepurposeofexaminingtheimpactattendingSLGsessionshad
onastudent’’sfinalgrades,studyskilldevelopment,andacademicengagement3.Inhighereducation
research,typicallyinvolvingprogramandcoursebasedinterventions,PSMisusedtoidentifytheimpact
ofparticipationwhileadjustingforfactorsthatinfluenceselfͲselectionintothesesameprograms
(Padgett,Salisbury,An,&Pascarella,2010).
Covariatesfortheseanalyseswerechosenbasedonavailabledata.Themorecovariatesusedin
themodel,thestrongerthemodel,andideallyresearcherstrytoinclude,andthereforetocontrolfor,a
rangeofdemographicandothercharacteristicsintheregressionandPSManalyses.Inthisstudy,the
covariatesusedwereclearedbytheQueen’’sUniversityGeneralResearchEthicsBoardaspartofthe
ethicsapplication.Allcovariateswerejudgedtobeindependentandnotendogenouslydetermined.
3
Thepropensityscoreswerecalculatedusingaprobitmodel(thedefaultestimator)andmatchedusing
thenearestneighbourmatchingalgorithm3.TheunderlyingconditionsofPSMarejudgedtobemet.
Thesetofparsimoniouscovariatesincludedintheestimationwerejudgedtobevalidmatchingcriteria,
and,onlyobservationsinthetreatmentandcontrolgroupsthatsharedacommonregionofsupport
wereusedintheanalysis.Matchingwasconductedwithreplacement.
16
Thecovariatesusedinthelinearregressionanalysesarelistedintheregressionresultsfoundin
Table3.Inthetestingphaseoftheanalysis,someinitialcovariatesweredroppedduetoalackof
observationsandcollinearitywithothercovariates.Thecovariatesusedintheregressionanalyses
include:gender,fullͲtime/partͲtimestudentstatus,identifyingasaninternationalstudent,yearofstudy,
averageentrancegrade,andotherSLGattendance(i.e.attending,orhavingattended,SLGsessionsin
Psychology100orBiology102atQueen’’sUniversity).Averageentrancegradeswereincludedasa
covariateforthepurposeofcontrollingforselfͲselectionbiasassociatedwithprioracademic
achievement,similartothatofotherstudies.
InthePSManalysesonfinalgradesandacademicengagement,SLGparticipantsandnonͲSLG
participantswerelikewisematchedaccordingtogender,fullͲtime/partͲtimestudentstatus,identifying
asaninternationalstudent,yearofstudy,averageentrancegrade4,andotherSLGattendance.Inthe
PSManalysesonstudents’’confidenceusingstudystrategiesSLGparticipantsandnonͲSLGparticipants
wereadditionallymatchedontheirpreͲcoursesurveyLikertscoresmeasuringtheirconfidencewith
studyskills.PreͲcourseLikertscoreswereincludedforthepurposeofaccountingforstudents’’
confidenceusingtheseparticularstudyskillspriortoattendingSLGsessions.
Varianceinflationfactor(VIF)scoresweregeneratedforallcovariatesincludedinthemodelsto
detectandestimatetheinfluenceofmulticollinearity,whichcanskewthemodelresults(seefor
exampleGreene2008;Tabachnick&Fidell,2007).WhileseveralacceptableVIFlimitshavebeen
proposedbypreviousauthors(seeforexampleO'Brien2007),alimitoffourwasadoptedforthe
purposesofthisreport,suggestingthatatthelimitthestandarderrorassociatedwithaparticular
covariatewouldbedoublewhatitwouldotherwisebeifitwerecompletelyorthogonal(Greene,2008;
O'Brien2007;Tabachnick&Fidell2007).NoVIFscoreswerefoundtoexceed2.01andmostwerebelow
4
Averageentrancegradeswereavailablefor44outof70internationalstudents.
17
1.33,meaningthestandarderrorsforthesecovariateswerehigherthantheywouldhavebeenifthe
covariateswerecompletelyorthogonal,butwellwithinconservativeVIFlimits.
Focusgrouprecordingsweretranscribedandpreparedforanalysis.Eachfocusgrouptranscript
wasreadinitsentiretybeforecodingbegan.Then,eachwasreadasecondandthird,andwhere
necessary,afourthtime,andinitialcodesweremadethroughout.Codesfromeachtranscriptwerethen
collatedintoaseparatelist,andparticularattentionwaspaidtocodesthatoverlappedorwere
duplicated.Thetranscriptswerereviewedafinaltimewiththissetofrevisedcodes.
Results
Thissectionofthereportaddressesthefiveresearchquestionsunderpinningtheassessmentof
theQueen’’sSLGprogram.Foreaseofreading,Table1providesasummaryoftheresearchquestions,
methods,andkeyfindings.
[INSERTTABLE1––Summaryoftheresearchquestions,methods,andkeyfindings]
ResearchQuestion1:FactorsinfluencingSLGParticipation
DemographicinformationforallstudentsenrolledinPsychology100(n=1885),Biology102(n=1051)
andBiology103(n=951),aswellasthedemographicinformationfortheSLGparticipants(n=125,n=
92,andn=63respectively)canbefoundinTable2.ForthepurposeofthisanalysiswedefineSLG
participantsasthosestudentswhohaveattendedoneormoresession.ThedatainTable2indicate
studentsinthethreetargetcoursesarepredominatelydomestic,fullͲtime,firstyearwomenenrolledin
theFacultyofArtsandScience.Thesefigurestranslateintoa6.6%participationrateinPsychology100,
a8.8%participationrateinBiology102,anda6.6%participationrateinBiology103.Thesedataalso
indicateSLGparticipantsare,onaverage,observationallysimilartostudentsinPsychology100,Biology
18
102andBiology103,withtheexceptionofgender.AccordingtothedatainTable2,proportionately
morewomenattendedSLGsessionsthanmen,whichcorrespondswiththefindingsofotherauthors
(Loviscek&Cloutier,1997).
[INSERTTABLE2––DescriptiveCharacteristics]
TheresultsoftheregressionanalysesareinTables3.Thefirsttwomodelswerefoundtohaver2
statisticsof0.302and0.356,respectively,suggestingthesemodelsaccountedforapproximatelyone
thirdofthevarianceinthedata.Thethirdmodelwasfoundtohaveanr2statisticof0.056(shownin
Table3).Allthreemodelswerefoundtobestatisticallysignificant,suggestingtheyfitthedatabut,as
expected,covariatesonlyaccountforpartofthevariance,leavingtheremaindertobecapturedbythe
residual.
Table3showsthecoefficientsforgender,yearofstudy,andidentifyingasaninternational
studentwerenegativeacrossallthreemodels.Inregardtogender,thissuggestsmenwerelesslikelyto
attendSLGsessionsthanwomen.Theseresults,however,wereonlystatisticallysignificantfor
Psychology100andBiology103.Thesameistrueforthecoefficientsassociatedwithidentifyingasan
internationalstudent.Whileallthreecoefficientswerenegative,indicatinginternationalstudentswere
lesslikelytoattendsessionsthandomesticstudents,onlythecoefficientsforPsychology100and
Biology103werestatisticallysignificant.Bycontrast,whileallthreecoefficientswerenegativeforthe
yearofstudycoefficients,indicatingthatfirst(andsecond)yearstudentsweremorelikelytoattend
sessionsthanupperͲyearstudents(asexpected),onlythecoefficientsforPsychology100andBiology
102werestatisticallysignificant.
Thecoefficientsassociatedwithstudents’’entranceaveragewerepositiveacrossallthree
models,butstatisticallysignificantforonlyBiology102.ThissuggeststhereislikelyamildselfͲselection
19
biasassociatedwithformeracademicachievement,butthisbiasmaynotbeasstrongastheselfͲ
selectionbiasassociatedwithothercovariates.AttendingSLGsessionsforothercourses,forexample,
wasfoundtohaveapositiveandstatisticallysignificanteffectonthelikelihoodofstudentsattending
SLGsessionsinallthreemodels.WhileattendingPsychology100sessionswasfoundtohaveapositive
andsignificantimpactonBiology103SLGattendance,Biology102attendancewasnot.Registration
statusaseitherafullͲtimeorpartͲtimestudent,bycontrast,wasfoundtohavenostatistically
significanteffectinanyofthemodels.
[INSERTTABLE3ͲLinearRegressionResultsforSLGAttendance]
Analysisoffocusgroupdataprovidessomeinsightintowhystudentschoosetoparticipatein
SLGsessions.AllfocusgroupparticipantsnotedtheybelievedattendingSLGsessionswouldhelpthem
improvetheirgrades,andthatthisinfluencedtheirdecisiontoattendsessions.Onestudent,for
example,posited““IguessIjustdecidedtoparticipatetotryandimprovemygrades””.Otherstudents
reportedlyattendedsessionstoenhancetheirunderstandingofcoursecontent.Onestudentsaid,for
example,SLGswere““areallygoodopportunityjusttoreviewthingsandtojust,like,gooveritonemore
time””.Comparatively,otherstudentsexplainedtheylikedthesmallgroupformat.Notallstudents,
however,cametotheSLGswithasimilarlystrongunderstandingofthecoursematerial.Bycontrast,
somestudentsreportedlyattendedsessionsinordertokeeppacewiththecoursematerial.One
studentexplained““thecoursematerial,theyweregoingthroughitreallyfast[inclass]……andIneeded
someextrahelp””,whileanothersimilarlyattestedthat““Ithink[SLGs]helpedmetogetcaughtupwith
thingssoIcouldunderstandtheweek’’slesson””.
Overall,theresultsofthesemodelslendseveralkeyinsightsintowhatfactorsinfluenceSLG
participationatQueen’’sUniversity.First,theseresultssuggesttheremaybeagenderdynamic
20
influencingstudentparticipationinSLGs,asevidencedbythenegativeandstatisticallysignificant
coefficients.Inaddition,theseresultssuggestsocialandotherbarriersmaypreventsomestudents,
particularlyinternationalstudents,fromparticipatinginSLGsessions.Second,theseresultssuggest
attendingSLGsessionsforonecoursemayhaveaspillovereffectandincreasethelikelihoodofstudents
attendingSLGsessionsforothercourses.Focusgroupdatasuggeststhisisaconsequenceofheightened
studentawarenessofthebenefitsofSIamongSLGattendees.ParticipantsreportedattendingSLGsin
onecourse(i.e.Biology)becausetheyfoundittobebeneficialinanothercourse(i.e.Psychology).
Finally,theseresultssuggestthefactorsthatinfluenceparticipationinSLGsmayvaryslightlyfrom
coursetocoursewithinthesameinstitution.WhilemanyofthefactorsthatdriveSIparticipationmay
becommon,theremaybenuanceddifferencesinthestrengthofparticipationcovariatesbetween
scienceandsocialsciencecoursesandcourseswithmorethanonepart(i.e.Biology102andBiology
103).Furtherresearchisneeded,however,tounderstandthesenuanceddifferencesandwhatbarriers
mayexistforstudentswhowishtoattendSLGsessionsbutdonot.
ResearchQuestion2:SLGParticipationandAcademicPerformance
Table4comparesSLGattendancefrequencywithstudents’’averageuniversityentrancegradesandtheir
averagefinalgradesineachcourse.ItshowsmostSLGparticipantsattended1to3sessionsandfew
studentsattended8ormoresessions.Nospecificobservablepatternsinentranceorfinalgrades
emergefromthistable.
[INSERTTABLE4––ComparisonofSLGAttendance,AverageFinalGrades,andAverageEntranceGrades]
ThePSMresultsinTable5showmixedresultsregardingtherelationshipbetweenSLG
attendanceandfinalgrades,afteraccountingfordemographicdifferences,otherSLGsession
21
attendance,andentranceaverages.WhiletheresultsinthetophalfofTable5showattendingSLG
sessionshasapositiveeffectonstudents’’finalgrades,fewresultswerestatisticallysignificant.Infact,
onlyattendingtwoSLGsessionsinBiology102wasfoundtohaveapositivestatisticallysignificantresult
onstudents’’finalgrades5.Theseresults,however,werelikelyinfluencebyalownumberof
observations.
WhenparticipantsandnonͲparticipantswerecomparedbasedonhavingattended““atleast””x
numberofsessions,forthepurposeofincludingmoreobservations,morestatisticallysignificantresults
emerged,butonlyamongststudentswhoattendedfewersessions.Studentswhoattendedatleastone
SLGsessioninPsychology100,andoneortwoSLGsessionsinBiology102,forexample,hadsignificantly
highergradesthanstudentswhoattendednoSLGsessionsinthesecourses.Bycontrast,fewstatistically
significantdifferencesinfinalgradesemergedbetweennonͲparticipantsandSLGparticipantswho
attendedmultiplesessions,withtheexceptionofBiology103.Infact,inBiology103studentswho
attendedatleasttwoorthreeSLGsessionshadsignificantlylowergradesthanthosewhodidnotattend
SLGsessions.
[INSERTTABLE5––PSMResultsforImpactofSLGAttendanceonFinalGrades]
Focusgroupdatasupportthequantitativefindings.Whilestudentscitedgradeimprovementas
theprimaryreasonforattendingSLGsessions,manycommentedthesesessionsdidnothavethe
desiredeffectontheiracademicperformance.Onestudent,forexample,attestedthatattendingSLG
sessionsmadeher““[feel]moreconfident.ButIdon’’tthinkthat[theSLG]mademedowellthough””.
Overall,theseresultssuggestthatwhilestudnetsmightinitiallybemotivatedtoattendSLGsessionsto
enhancetheirgradesthebenefitstheyderrivefromSLGattendancemayaccrueelsewhere
5
Theresultsof5PSManalysesinthetophalfofTable5wereomittedduetostatisticallyinsignificant
probitmodelsonwhichmatchingwasbased,likelyduetotherelativelylownumberofobservations.
22
Theseresultsshouldbeinterpretedcarefullyduetotherelativelylownumberofobservations
available,especiallyafterparticipantsandnonͲparticipantswerematchedontheirrelativelikeness
(commonregionofsupport).Nonetheless,thesePSMresultssuggesttheimpactofSLGattendanceon
student’’sfinalgradesmaynotbeclearͲcutwhenpriorSLGattendance,entrancegrades,and
demographiccharacteristicsarecontrolledfor.Moreresearchisneededtofullyunderstandtheimpact
ofSIatQueen’’sonstudents’’performanceinspecificcourses.
ResearchQuestion3:SLGParticipationandAttrition
Table6comparestrendsincoursecompletionrates.Itshows,onaverage,Biology102and103have
completionratesofabout90%whereas,thecompletionrateinPsychology100isapproximately83%.
Table6alsoindicatesaveragecompletionratesinthesecourseshaveremainedrelativelystable
betweenthe2006Ͳ2007schoolyearand2009Ͳ2010schoolyear,withtheexceptionofBiology103.
Table6indicatesthatBiology103sawamarkedincreaseinitscompletionratein2008Ͳ2009.
[INSERTTABLE6––TrendsinCourseCompletionRates]
Table7showsthegradedistributionanddropratesofSLGparticipantsandnonͲSLGparticipantsin
Psychology100,Biology102andBiology103.Itcomparestheproportionofstudentsthatearnedgrades
above80%,aswellasthepercentagewhoearnedgradesbelow60%,inthe““D””and““F””graderanges.
Thesignificanceofgradesinthe““D””and““F””graderangesinintroductorycoursesisthattheyarealikely
indicatorofsubsequentattrition,becausethesecoursesprovideprerequisiteskillsandknowledge
requiredinlaterclasses.AccordingtothedatainTable7,theproportionofparticipantsandnonͲ
participantswhoearnedgradesbelow50%(an““F””grade)werecomparableacrossallthreecourses.In
Biology103,thepercentageofparticipantsandnonͲparticipantswhoearnedgradesinthe50Ͳ59%
range(a““D””grade)werealsocomparable.InPsychology100andBiology102,however,thedatashow
23
SLGparticipantsperformedbetterthannonͲparticipants,earningproportionallyfewergradesinthe50Ͳ
59%range.WhiletherelativelylownumberofSLGparticipantsearninggradesbelowthe50%and50Ͳ
59%rangesispositive(n=7andn=18,respectively,acrossallthreecourses),thelownumberof
observationsmakestatisticaltestingdifficult,particularlyusingPSM.Propensityscorematchinganalyses
comparingparticipantsandnonͲparticipantsintheserangesproducedstatisticallyunreliableresults
and,therefore,wereomitted.
[INSERTTABLE7ͲGradeDistributionofParticipantsandNonͲParticipants]
Table7alsoindicatestheproportionofstudentswhodroppedPsychology100,Biology102and
Biology103wasloweramongstSLGparticipantsthannonͲSLGparticipants.InBiology102,noSLG
participantswerefoundtodropthecoursecomparedto7%ofstudentsinthenonͲSLGgroup.Whilethe
relativelylownumberofstudentswhodroppedcoursesamongstSLGparticipants(n=5acrossallthree
courses)isalsopositive,thelownumberofobservations,similarly,makestatisticaltestingdifficult,
particularlyusingPSM.Duetostatisticalunreliabilityoftheunderlyingprobitmodelsandtheresultant
PSMresults,thesefigureswerealsoomitted.
Altogether,thedatainTables6and7indicatethatinthe2009Ͳ2010academicyear33%of
studentsinPsychology100earnedaDgrade,anFgradeorwithdrewfromthecourse.InBiology102
andBiology103thepercentageofstudentswhoearnedaDgrade,anFgradeorwithdrewfromthe
coursewas18%and9%,respectively.Whileitislikely,itisnotclearfromthesedatawhetherSLG
attendanceispositivelyassociatedwithlowerlevelsofattritionatQueen’’sUniversity.
24
ResearchQuestion4:SLGParticipationandEngagement
Table8summarizesdatafromseveralCLASSEsurveyquestionsrelatedtoacademicengagement
includedinthepostͲcoursesurvey6.ThedataindicateSLGparticipantsmorefrequentlyaskedquestions
inclass,includeddiverseperspectivesinassignmentsanddiscussions,drewonawidevarietyofideas
andconcepts,andmorefrequentlydiscussedcourseconceptsoutsideofclassthannonͲSLG
participants.Additionally,nonͲSLGparticipantsreportedmorefrequentlycomingtoclasswithouthaving
completedassignedreadingsandassignments,andmorefrequentlymissingclassaltogether.
[INSERTTABLE8ͲChangesinFactorsRelatedtoStudentEngagement]
ResultsfromthePSManalyses,inTable9,coincidewiththesedata.WhentheimpactofSLGattendence
isstatisiticallyseparatedbycourse,anddemographicandothercovariatesarecontrolledfor,the
findingsindicateSLGparticipantsinPsychology100weresignificantlymorelikelytoaskquestionsin
class,drawonawidevarietyofideasandconcepts,anddiscusscourseconceptsoutsideofclass.The
resultsalsoindicateSLGparticipantsinPsychology100werelesslikelytomissclassandlesslikelyto
cometoclasswithoutcompletingassignedreadingsandassignments.Similarly,Biology102participants
werefoundtobesignificantlymorelikelytoaskquestionsinclass,contributetoclassdiscussion,include
adiversearrayofperspectivesintheirassignments,anddiscusscourseconceptsoutsideofclass.
Comparatively,Biology103SLGparticipantsweresignificantlymorelikelytoincludeadiversearrayof
perspectivesintheirassignmentsbutlesslikelytocontributetoclassdiscussion.Overall,theseresults
suggestthatSLGparticipantsweresignificantlymoreacademicallyengagedthannonͲSLGparticipants,
onaverage.
6
ThetablesummarizesstudentresponsestoseveralfourͲpointscalequestions,where1isequalto““None””and4
isequalto““Fiveormore””.
25
[INSERTTABLE9ͲPSMResultsforImpactofSLGAttendanceonStudentEngagement]
DatafromthefocusgroupssupportsthequantitativefindingsontherelationshipbetweenSLG
sessionsandstudentengagement.Onestudentcommented,forexample,““inthelectures,theprofjust
goesoverthings.Whereas[inSLGs]youhaveactualquestions.Soitmakesyouactuallythinkabout
exactlywhathappened,what’’sgoingon””.Anotherstudentsimilarlyremarked““Ipersonallylearnby
explainingthingstootherpeopleortalkingaboutthings,andthat’’swhatwedidintheSLGs.Alotofthe
timewediscussedwithinourgroups,amorehandsͲonapproachtolearning””.
Focusgroupdatasuggeststhisheightenedengagementinthecoursemay,atleastinpart,bea
consequenceofpeeracademicsupportnetworksformedthroughSLGs.Oneparticipantnoted,for
example,““everyoneisreallyfriendlysoyoustarttalkingtopeople.Butit’’salsogoodtotalktoother
peopleaboutthecoursematerialandit’’sgoodtogetdifferentperspectives……butit’’ssomethingthat
makesyoufeelpartofsomethingandshowsthatpeopleactuallycare””.Anotherstudentechoedthese
sentimentssaying““youseethepeoplewhoattendthesegroupsandifyouseethemaroundcampuslike
yousay,‘‘Hello’’.Andyougettoknowthemthroughthesegroupsandyouhaveastudybuddyinsome
way””.Theseresults,consequently,suggestSLGsessionsatQueen’’sincreaseacademicengagement,
encourageselfͲdirectedgrouporientedlearning,andincreasestudents’’confidencewithcoursematerial
ResearchQuestion5:SLGParticipationandStudySkills
Table10showschangesinstudents’’confidenceworkingwithsixcommonlearningstrategiespracticed
inSLGsessions,basedondatafromthepreͲcourseandpostͲcoursesurveys7.InPsychology100,both
SLGparticipantsandnonͲSLGparticipantswerefoundtohaveimprovedtheirconfidencewiththesesix
studyskillsduringthesemester.WhileSLGparticipantsinBiology102and103similarlyreportedan
7
ThetablesummarizesstudentresponsestoseveralfiveͲpointLikertscalequestionswhere1is““Very
Unconfident””and5is““VeryConfident””.
26
increaseinconfidencewiththesestudyskillsbetweenthepreandpostͲcoursesurveys(withthe
exceptionofstudyingoutloud)nonͲSLGparticipantsinBiologyreportedadecreaseinconfidenceusing
thesesamestudyskills.
[INSERTTABLE10––ChangesinStudentConfidenceinUsingLearningStrategies]
WhenparticipantsandnonͲparticipantswerecomparedusingPSM,SLGparticipantsin
Psychology100reportedbeingsignificantlymoreconfidentusingmindmapsthannonͲSLGparticipants,
butsignificantlylessconfidentusingtheCornellmethodanddevelopingtestquestions.Bycontrast,
Biology102SLGparticipantsreportedbeingsignificantlymoreconfidentthannonͲSLGparticipants
developingstudyschedulesthannonͲparticipants,butsignificantlylessconfidentusingmnemonic
devices.Biology103SLGparticipants,comparatively,reportedbeingsignificantlymoreconfidentthan
nonͲSLGparticipantsusingtheCornellmethodandmindmaps.ThePSMmatchingresults,however,
mustbeinterpretedwithcautionduetothelownumberofobservations.
[INSERTTABLE11ͲPSMResultsforImpactofSLGAttendanceonLearningSkillsConfidence]
Overall,whilethedatainTable10indicatesSLGparticipantswere,onaverage,moreconfident
thannonͲSLGparticipantswiththesestudyskills,thePSMresultsweremoremixed.Whentheeffectof
SLGparticipationinBiology102and103wasseparatedandseveraldemographicandothervariables
controlledforusingPSM,SLGparticipantswerefoundtobesignificantlymoreconfidentthannonͲSLG
participantsusingsomestudyskills,whilesignificantlylessconfidentusingothers.Theseresultssuggest
althoughSLGparticipantsarelikelytohaveincreasedtheirconfidencewithsomestudystrategiesthis
increaseisnotlikelyuniformacrossstudyskillsorbetweencourses.
27
Thesemixedresultsmaysuggestapositivespillovereffectisoccurring.Whilepositive,thisspill
overeffectmakesstatisticalcomparisonsbetweenSLGparticipantsandnonͲSLGparticipantsdifficult.
TheseresultsmayalsobeattributedtononͲparticipantsattendingsessionsonlearningstrategy
developmenthostedbytheLearningStrategiesDepartment,whicharenotaccountedforinthis
comparison.Similarly,theseresultsmayalsoreflectlearningskilldevelopmentacquiredbyparticipants
andnonͲparticipantsbeforecomingtoQueen’’sUniversity.
DatafromthefocusgroupsindicatethestudystrategieslearnedinSLGsessionswereimportant
tostudents’’academicdevelopment.Onestudentsaid,forexample,““intermsoftransferableskills,it
wasaftertheSLGthatIstartedmakingmindmapsalotmore””.Anotherstudent,bycomparison,
remarkedthatinsession““[we]woulddraworwritethingsoncueͲcards.Thathelpedmewith
memorizationinothercourses””.
Timemanagementwasthestudyskillmostfrequentlycitedbyfocusgroupparticipants.Some
students,forexample,usedSLGsessionsaspartoftheiracademictimemanagementstrategy.One
studentattested““[I]haveahardtimeplanningtimemanagement.SoifIcome[totheSLGs]everyweek
thenIamactuallystuckheretodoworkfortwohours””.Anotherparticipantsimilarlyexplained““I’’d
come[totheSLG]andactuallydothework,butwhenIwouldn’’tgo,Iwouldn’’tfollowupwithit””.Other
students,bycontrast,reportedapplyingthetimemanagementstrategieslearnedintheSLGsession,
suchasthe50/10rule,tomakemoreeffectiveuseoftheirindependentstudytime.
EmpiricalLimitations
Whiletheseresultspresentedinthisreportareimportant,theyshouldbeinterpretedcarefullyduethe
relativelylownumberofSLGparticipantswithwhichtocomparenonͲSLGparticipants.Thesmall
numberofparticipantsfromwhichtodrawdataisasystemicstatisticalproblemassociatedwith
assessingtargetedacademicprogramsthatattractrelativelylownumbersofstudents.Theseresults
28
shouldalsobeinterpretedcarefullybecause,likeothercomparativestatisticaltechniques,theymaybe
confoundedbyspillovereffectsassociatedwithknowledgetransferbetweenparticipantsandnonͲ
participants.WhileknowledgespilloverisapositiveoutcomeofSIinitiatives,itcanmakestatistical
comparisonsdifficult.
ImplicationsandConclusion
Theresultspresentedintheprecedingsectionshedvariousshadesoflightonthefactorsthatinfluence
SLGattendanceandtheresultantacademicoutcomesatQueen’’sUniversity.Theresultsofouranalysis
indicategenderandidentifyingasaninternationalstudentmayinfluencethelikelihoodstudentswill
attendSLGsessions.Whileouranalysispresentsnospecificexplanationforthesefindings,theresults
suggestinvisiblebarriersmaybepreventingsomestudentswhowishtoattendSLGsessionsfrom
attending.OurresultsalsosuggeststudentswithrelativelyhighentranceaveragesmayselfͲselectto
participateinSLGsessions,particularlyinBiology102,andthatpriorSLGattendancelikelyaffects
subsequentSLGparticipationinothercourses.Theseresults,therefore,indicatethatdirectgrade
comparisonsofparticipantsandnonͲparticipantsmaybebiasedwithoutaccountingforthesefactors.
WhenselfͲselectionbiaswasaccountedforusingpropensityscorematching,theresultsindicate
therelationshipbetweenSLGparticipationandfinalgradesmaynotbeclearcut.Inmanycases,the
differencebetweenaparticipant’’sandanonͲparticipant’’sfinalgradeswerefoundtobestatistically
insignificant.Wherefinalgradedifferenceswerestatisticallysignificant,thesignsofthecoefficients
weremixed,suggestingthatinsomecasesSLGparticipantsperformedbetterthannonͲparticipants
whileinothercasestheyperformedmorepoorly.
ThedataonstudentattritionindicatesalowerproportionofSLGparticipantsinPsychology100
andBiology102earnedgradesbetween50Ͳ59%.Theproportionofstudentswhoearnedgradesbelow
50%werecomparableacrossallthreecourses.ThedataalsoindicateproportionallyfewerSLG
29
participantsdroppedtheassociatedcourses,particularlyPsychology100.Whilealownumberof
observationsamongstSLGparticipantspreventedempiricaltestingandcontrollingforselfͲselection
bias,theseresultssuggestSLGattendancelikelyhasapositiveimpactonloweringstudentattrition.
Changesinrelativestudyskilldevelopmentwerealsomixed.Thedatasuggesttherearefew
discernibledifferencesbetweenparticipantsandnonͲparticipantsinstudyskillconfidence.Thedata
indicatewhileSLGparticipantsincreasedtheirconfidenceusingthesixstudyskillsunderexamination,
mostobservablepostͲcourseLikertscaledifferencesbetweenparticipantsandnonͲparticipantsare
minor.Altogether,thesedataandtheresultsofthePSManalysessuggestalthoughSLGparticipantsare
likelytohaveincreasedtheirconfidencewithsomestudystrategiesthisincreaseisnotuniformacross
studyskillsorbetweencourses.ThelackofdiscernibledifferencesbetweenSLGparticipantsandnonͲ
SLGparticipantsmaysuggestapositivespillovereffectisoccurring.
Bycontrast,theresultsofthisanalysisindicatethatSLGslikelyhaveapositiveeffecton
academicengagement.Onaverage,SLGparticipantsweremorelikelytoaskquestionsinclass,included
diverseperspectivesinassignmentsanddiscussions,drawonawidevarietyofideasandconcepts,and
discusscourseconceptsoutsideofclassthannonͲSLGparticipants.SLGparticipantsinPschology100
werealsosignificantlylesslikelytocometoclasswithouthavingcompletedassignedreadingsand
assignments,andlesslikelytomissclassaltogether.
Thereislittledoubttheresultsofthisanalysiswereinfluencedbyinstitutionaluniquenessof
Queen’’sUniversityanditsvariantontheSIprogram.Asoneoftheoldestpostsecondaryinstitutionsin
Canada,Queen’’sdrawsundergraduatestudentsinternationally,butalsofromfamilieswitharich
alumnusbackground.ItsrelativelyrurallocationbetweenTorontoandMontreal,itsmediumsize,and
itsselectionofundergraduateprogramsalsolikelyhadaneffectonwhatstudentsselectedtoattend
Queen’’s,thediversityofitsundergraduatepopulation,andtheimpactofSI.
30
Hostingsessionsinresidencealsolikelyhadaneffectontheseresults.Whiletherearemany
benefitstorunningsessionsinresidence,residencemayinviteadversesocialdynamics(suchas
personalorgroupconflicts)thatspilloverfromresidenceand,subsequently,affecttheparticipationand
engagementofsomestudentswhowishtoparticipateinsessions.Additionally,hostingsessionsin
residencelikelyfacilitatedknowledgespilloverbetweenSLGparticipantsandnonͲSLGparticipants.
Whilethisisapositiveoutcome,thisspillovereffectmakesstatisticalcomparisonsbetweenparticipants
andnonͲparticipantsdifficultandlikelycomplicatedtheresultsofthisanalysis.
Overall,theresultspresentedinthisreportprovideoneofthefirstglimpsesintotheimpactof
theSLGprogramatQueen’’sUniversity.WhilethispaperfindstheSLGprogramatQueen’’shasshown
signsofpositive,althoughmixed,successduringitspilotyears,muchresearchremainstobedoneon
theroleofthisprogramasitcontinuestoevolveandgrowintheyearstocome.Whatiscertainfrom
theseresults,however,isthatSupplementalInstruction,andSupportedLearningGroupsatQueen’’sin
particular,areanunlikelysubstitutefortraditionalacademicresourcessuchaslabs,seminarsand
lecturesthatareledbytrainedacademicprofessionals.Theevidencepresentedinthisreportsuggests
SupportedLearningGroupscanplayanimportantsupplementaryroletotheseresourcesbyreinforcing
academicbestpracticesandbyprovidingguidedstudytimeforstudents,especiallythoseatacademic
risk.Inparticular,theresultsofthisreportunderlinetheimportanceofSLGsinincreasingacademic
engagement.
However,atleasttwosignificantquestionsremainunanswered.First,whiletheSLGprogramis
opentoallstudents,itappearsstudentsmostatͲrisk,orthosewouldcouldpotentiallybenefitfromthe
sessionsmostdidnotattendveryoften,orinsomecasesatall.Furtherresearchisneededtoexplore
strategiestoincreaseparticipationamongstudentswithlowengagement,thosewhoaremostatriskof
notcompletingthecourse,failingorreceivingagradetoolowtoallowprogressionintoupperyears
needs.Secondly,anissuestronglyrelatedtothefirstpoint,isthequestionofinstitutionalexpectations.
31
AsSIincreasesacrossCanada,understandingvariationsininstitutionalexpectationsofprogramslike
SLGiscriticaltounpackingtherelationshipbetweenneoliberalreformsinhighereducation,fiscal
constraints,andthegrowthofSLGprograms.TheexpectationsassociatedwithSLSprogramsarebroad,
complexandexpanding.Understandingtheseexpectationsandthecontextwithinwhichtheseare
constructed,aswellaswhatoutcomesarerealisticandreasonableisurgentlyneededtoenable
institutionsfordetermineappropriateusesfortheprogramandbenchmarksofsuccess.
AlthoughaplethoraofresearchindicatestheimplementationofSIprogramscanhavepositive
dramaticeffectsonstudentperformanceandretention,wearguetheseresultsmayvarygreatly
betweeninstitutions,andindeedbetweencourses.TheexpectationthatSIcanbeappliedwithuniform
resultsislikelyunrealistic,andmaybepartiallyattributedtometaanalyticalapproachesthatconceal
institutionaldifferences,aswellasearlyempiricalworkthatlackedofattentiontoproblemsassociated
withendogeneity.HeightenedexpectationsforSI,evidencesuggests,mayalsobeattributabletothe
financialmotivationsofpostͲsecondaryinstitutionsseekingcosteffectivemeansofboostingstudent
performance.SIprogramscanbeanimportantadditiontotraditionalacademicresources.However,SI
programscannotbeusedasasubstitutefortraditionalresourcesandarenotaonestopsolutionto
issuesofpooracademicperformanceandretention.Bycontrast,theyshouldbeviewedasanadditional
resourceforstudents,andexpectationsfortheirperformanceshouldbeinstitutionspecific.
32
References
Amin,A.(1999).AnInstitutionalistPerspectiveonRegionalEconomicDevelopment. International
JournalofUrbanandRegionalResearch23(2),p.365Ͳ378
Arendale,D.R.(1997).SupplementalInstruction:ReviewofresearchconcerningtheeffectivenessofSI
fromtheUniversityofMissouriͲKansasCityandotherinstitutionsfromacrosstheUnitedStates.
NationalCenterforSupplementalInstruction,UniversityofMissouriͲKansasCity.AccessedMay
2011,availableat:
http://www.eric.ed.gov/ERICWebPortal/search/detailmini.jsp?_nfpb=true&_&ERICExtSearch_Se
archValue_0=ED457797&ERICExtSearch_SearchType_0=no&accno=ED457797
Astin,A.(1993).WhatMattersinCollege?FourCriticalYearsRevisited,SanFrancisco,Cal.:JosseyͲBass.
BaxterMagolda,M.andJ.(1998).DevelopingselfͲauthorshipinyoungadultlife.JournalofCollege
StudentDevelopment,39(2),p.143Ͳ156.
BaxterMagolda,M.B.,&Buckley,J.(1997,March).ConstructiveͲdevelopmentalpedagogy:
Linking
knowledgeconstructionandstudents’’epistemologicaldevelopment. Paperpresentedattheannual
meetingoftheAmericanEducationalResearch Association,Chicago,IL.
Blanc,R.L,DeBuhr,L.,&,andD.Martin(1983).Breakingtheattritioncycle.JournalofHigher
Education,54(1),p.80Ͳ90.
Blanc,R.,&Martin,D.C.(1994).SupplementalInstruction:IncreasingStudentPerformanceand
PersistenceinDifficultAcademicCourses.Academic Medicine,69(6).
Burmeister,S,P.Kenny,&Nice,D.(1996).AnalysisoftheeffectivenessofSIsessionsfor
college
algebra,calculus,andstatistics.InResearchinCollegiateMathematicalEducationII,J.Kaput,A.H.
Schoenfeld,andE.Dubinskyeds.,AmericanMathematicalSociety,RhodeIsland,pp145Ͳ154.
Carlsmith,K.M.,andL.J.Cooper.(2002).Apersuasiveexampleofcollaborativelearning.Teachingof
Psychology,29(2),132Ͳ135.
Chickering,A.W.,andZ.F.Gamson,Z.F.(Eds.).(1991).Applyingthesevenprinciplesforgoodpracticein
undergraduateeducation.SanFrancisco:JosseyͲBass.
Coleman,J.S.,(1988).Socialcapitalinthecreationofhumancapital.AmericanJournalofSociology,94,
95––120.
Congos,D.H.andN.Schoeps.(1993).Doessupplementalinstructionreallyworkandwhatisitanyway?
StudiesinHigherEducation,18(2),165Ͳ176.
Etter,E.R.,S.L.Burmeister,andR.J.Elder.(2000).Improvingstudentperformanceandretentionvia
supplementalinstruction.JournalofAccountingEducation,18(4),355Ͳ368.
33
Evans,N.J.,D.S.Forney,andF.GuidoͲDiBrito.(1998).Studentdevelopmentincollege:Theory,research,
andpractice.SanFrancisco:JosseyͲBass.
Fayowski,V,andP.D.MacMillan.(2008).AnevaluationoftheSupplementalInstructionprogrammein
afirstyearcalculuscourse.InternationalJournalofMathematicalEducationinScienceand
Technology,39(7),843Ͳ855.
Greene,W.H.(2008).EconometricAnalysis.NewJersey,PrenticeHill.
Hersberger,J.(2003)Aqualitativeapproachtoexamininginformationtransferviasocialnetworks
amonghomelesspopulations.NewReviewofInformationBehaviorResearch4(1),p.95Ͳ108.
Kochenour,E.O.,D.S.Jolley,J.G.Kaup,D.L.Patrick,K.D.Roach,L.A.Wenzler.(1997).Supplemental
Instruction:AneffectivecomponentofStudentAffairsprogramming.JournalofCollegeStudent
Development,38(6),577Ͳ586.
Loviscek,A.L.andCloutier.N.R.(1997).Supplementalinstructionandtheenhancementofstudent
performanceineconomicprinciples.AmericanEconomist,41(2),70Ͳ76.
Kuh,G.D.,K.B.Douglas,J.P.Lund,andJ.RaminͲGyurnek.(1994).Studentlearningoutsidetheclassroom:
Transcendingartificialboundaries.ASHEͲERICHigherEducationReport,8.Washington,DC:The
GeorgeWashingtonUniversity,GraduateSchoolofEducationandHumanDevelopment.
Mahdi,A.E.(2004).PeerͲSupportedlearninggroups:Acollaborativeapproachtosupportingstudents
learninginengineeringandtechnology.TransactionsonAdvancesinEngineeringEducation.
AccessedMay2011,availableat:
http://74.125.155.132/scholar?q=cache:URBYD08y6D8J:scholar.google.com/&hl=en&as_sdt=0,5
&as_vis=1
Martin,D.C.,&Arendale,D.R.(1990).Supplementalinstruction:Improvingstudentperformance,
increasingstudentpersistence.KansasCity,MO:UniversityofMissouri.
Martin,D.C.,andD.R.Arendale.(1992).SupplementalinstructionimprovingfirstͲyearstudentsuccessin
highriskcourses(2nded.MonographNo.7).Columbia,SC:NationalResourceCentreforthe
FreshmenYearExperience,UniversityofSouthCarolina.
Martin,D.C.,andD.R.Arendale.(1994).SupplementalInstruction:Reviewofresearchconcerningthe
effectivenessofSIfromtheUniversityofMissouriͲKansasCityandotherinstitutionsfromacross
theUnitedStates.NationalCenterforSupplementalInstruction,UniversityofMissouriͲKansas
City.AccessedMay2011,availableat:
http://www.eric.ed.gov/ERICWebPortal/search/detailmini.jsp?_nfpb=true&_&ERICExtSearch_Se
archValue_0=ED370502&ERICExtSearch_SearchType_0=no&accno=ED370502
Massey,D.S.,C.Z.Charles,G.F.Lundy,andM.J.Fischer.(2003).Thesourceoftheriver.Princeton
UniversityPress:Princeton,NewJersey.
34
McInnis,C.(2001).Signsofdisengagement?:ThechangingundergraduateexperienceinAustralian
universities.Inauguralprofessoriallecture.Melbourne:FacultyofEducation,Universityof
Melbourne.AccessedMay2011,availableat:
http://www.eric.ed.gov/ERICWebPortal/search/detailmini.jsp?_nfpb=true&_&ERICExtSearch_Se
archValue_0=ED466720&ERICExtSearch_SearchType_0=no&accno=ED466720
Miles,C.A.,D.PolovinaͲVukovic,D.Littlejohn,andA.Marini.(2010)TheeffectivenessofthepeerͲ
assistedstudysessions(PASS)programinenhancingstudentacademicsuccessatCarleton
University.AccessedJuly2010,availableat:
http://www.heqco.ca/SiteCollectionDocuments/PASS%20ENG.pdf
Martin,D,,andD.Arendale.(1994).SupplementalInstruction:Reviewofresearchconcerningthe
effectivenessofSIfromtheUniversityofMissouriͲKansasCityandotherinstitutionsfromacross
theUnitedStates.NationalCenterforSupplementalInstruction,UniversityofMissouriͲKansas
City.AccessedMay2011,availableat:
http://www.eric.ed.gov/ERICWebPortal/search/detailmini.jsp?_nfpb=true&_&ERICExtSearch_Se
archValue_0=ED370502&ERICExtSearch_SearchType_0=no&accno=ED370502
O'Brien,R.M.(2007).Acautionregardingrulesofthumbforvarianceinflationfactors.Qualityand
Quantity41(5),673Ͳ690.
Ogden,P.,D.Thompson,A.Russell,andC.Simons.(2003).SupplementalInstruction:ShortͲandLongͲ
termImpact.JournalofDevelopmentEducation,26(3),2Ͳ8.
Padgett,R.D.,Salisbury,M.H.,An,B.P.,&Pascarella,E.T.(2010).Required,practical,orunnecessary?
Anexaminationanddemonstrationofpropensityscorematchingusinglongitudinalsecondary
data.NewDirectionsforInstitutionalResearch,AssessmnentSupplement,29––42.Pascarella,E.
T.,Bohr,L.,Nora,A.,Zusman,B.,Inman,P.,&Desler,M.(1993).Cognitiveimpactsoflivingon
campusversuscommutingtocollege.JournalofCollegeStudentDevelopment,34,216Ͳ220.
Pascarella,E.T.,&Terenzini,P.T.(2005)Howcollegeeffectstudents,volume2:Athirddecadeof
research.SanFranciscoCA:JosseyͲBass.
Peat,M.,J.Dalziel,andA.M.Grant.(2001)Enhancingthefirstyearstudentexperiencebyfacilitatingthe
developmentofpeernetworksthroughaoneͲdayworkshop.HigherEducationResearch&
Development,20(2),199Ͳ215.
Ramirez,G.M.(1997).SupplementalInstruction:Thelongtermimpact.JournalofDevelopmental
Education,21(1),2Ͳ10.
Rath,K.A.,A.R.Peterfreund,S.P.Xenos,F.Bayliss,andN.Carnal.(2007).SupplementalInstructionin
IntroductoryBiologyI:Enhancingtheperformanceandretentionofunderrepresentedminority
students.LifeSciencesEducation.6(3),203Ͳ216.
Schwartz,M.(1992).Studysessionsandhighergrades:Questioningthecausallink.CollegeStudent
Journal26(3),292Ͳ299.
35
Stone,M.E.,andG.Jacobs.(2006).Supplementalinstruction:Newvisionsforempoweringstudent
learning.SanFrancisco,CA:JosseyͲBass.
Tabachnick,B.G.,&Fidell,L.S.(2007).Usingmultivariatestatistics.PearsonEducation:Toronto
Tinto,V.(1993).Leavingcollege:Rethinkingthecausesandcuresofstudentattrition.(2nded.).Chicago:
TheUniversityofChicagoPress.
Tinto,V.(2002).EstablishingConditionsforStudentSuccess:LessonsLearnedintheUnitedStates.
Paperpresentedatthe11thAnnualConferenceoftheEuropeanAccessNetwork,Monash
University,Prato,Italy.AccessedJune2011,Availableat:http://faculty.soe.syr.edu/vinto/
Trotter,E.,andC.A.Roberts.(2006).Enhancingtheearlystudentexperience.HigherEducationResearch
andDevelopment,25(4),371––386.
UniversityoftheFraserValley(2010).Supportedlearninggroups.AccessedMay2010
http://www.ufv.ca/SLG/About_SLG.htm
Upcraft,M.L.,J.N.Gardner,andB.O.Barefoot.(2005).ChallengingandSupportingtheFirstͲYear
Student:AHandbookforImprovingtheFirstYearofCollege.SanFrancisco:JosseyͲBass.
Visor,J.N.,J.J.Johnson,andL.N.Cole.(1992).Therelationshipofsupplementalinstructiontoaffect.
JournalofDevelopmentalEducation,16(2),12Ͳ18.
Wilcox,P.,S.Winn,andM.FyvieͲGauld.(2005)‘‘Itwasnothingtodowiththeuniversity,itwasjustthe
people’’:theroleofsocialsupportinthefirstyearexperienceofuniversity,StudiesinHigher
Education,30(6),707––722.
Wilson,M.(2005).SupplementalinstructionintheCanadiancontext.JournalofStudentCentered
Learning,2(2),109Ͳ119.
Whipple,W.R.(1987).Collaborativelearning.AAHEbulletin,40(2),3Ͳ7.
Wolfe,R.E.(1987).TheSupplementalInstructionprogram:Developinglearningandthinkingskills.
JournalofReading,31(3),228Ͳ232
Yorke,M.,andL.Thomas.(2003)ImprovingtheretentionofstudentsfromlowersocioͲeconomic
groups,JournalofHigherEducationPolicyandManagement,25(1),63––74.
36
Table1:Summaryoftheresearchquestions,datasourceandtypeofanalysisandkeyfindingsfromeachsectionofanalysis
ResearchQuestion
1.Whatfactorsinfluence
students’’likelihoodto
participateinSLGsessions?
DataSource
ͲOfficeoftheRegistrar
ͲSLGAttendance
ͲFocusGroups
2.Towhatextentdoes
participationinSLGsleadto
increasedacademicsuccessin
acourse?
3.TowhatextentdotheSLGs
increasecourseretention?
MainMethodof
Analysis
MainFindings
ͲLinearRegression
ͲFocusGroup
TranscriptAnalysis
ͲFactorsthatinfluenceparticipationinSLGsessionsatQueen’’sincludegender,
havingattendedanSLGsessionforanothercourse,identifyingasaninternational
student,andentrancegrades.
ͲPropensityscore
Matching
ͲSLGsessionsmayhaveapositiveinfluenceonstudent’’sacademicperformance
butPSMresultsindicatethatSLGattendancecannotbeclearlylinkedwith
academicsuccessatQueen’’s.
ͲMultivariateAnalysis
ͲRelativelyfewSLGparticipantswerefoundtodroptargetcourses.Whileitis
likely,itisnotclearfromthesedatawhetherSLGattendanceispositively
associatedwithlowerlevelsofattritionatQueen’’s.DandFgraderateswere
comparablebetweenSLGparticipantsandnonͲparticipants,withtheexceptionof
Biology102whereSLGparticipantsearnedproportionallyfewerDgrades.
StatisticaldifferencesbetweenparticipantsandnonͲparticipantsindroprates,D
gradesandFgraderatescouldnotbedetectedduetolowobservationnumbers.
ͲOfficeoftheRegistrar
ͲSLGAttendance
4.TowhatextentdotheSLGs
increaseengagementwiththe
coursematerial?
ͲFocusGroups
ͲPostͲCourseSurveys
ͲPropensityscore
Matching
ͲFocusGroup
Transcript&Text
Analysis
5.TowhatextentdotheSLG
sessionsenhancestudyskills?
ͲFocusGroups
ͲPreandPostͲCourse
Surveys
ͲPropensityScore
Matching
ͲFocusGroup
TranscriptAnalysis
37
Ͳ.TheseresultssuggestthatSLGsessionslikelyincreaseacademicengagement,
encourageselfͲdirectedgrouporientedlearning,andincreasestudents’’
confidencewithcoursematerial.Overall,SLGparticipantsmoresignificantlymore
likeytoaskquestionsinclass,includeddiverseperspectivesinassignmentsand
discussions,drewonawidvarietyofideasandconcepts,andmorefrequently
discussedcourseconceptsoutsideofclassthannonͲSLGparticipants.Onaverage,
SLGparticipantswerealsomorelikelytoattendclassandcompleteassigned
readingsandassignments.
ͲChangesinrelativestudyskilldevelopmentweremixed.Dataandanalyses
suggestthattherearefewdiscernibledifferencesbetweenparticipantsandnonͲ
participantsinstudyskillconfidence.AlthoughSLGparticipantsarelikelytohave
increasedtheirconfidencewithsomestudystrategiesthisincreaseisnotuniform
acrossstudyskillsorbetweencourses.
Table2:DescriptiveCharacteristics
Gender
Women
Men
YearofStudy
1stYear
2ndYear
3rdYear
4thYear
Faculty
Arts&Science
Business
Engineering
Nursing
International
Yes
No
Registration
FullͲtime
PartͲtime
Psychology100
Count
%
AllStudents
Biology102
Count
%
1364
521
1538
185
101
61
1733
43
27
82
72%
28%
82%
10%
5%
3%
92%
2%
1%
4%
684
367
948
48
27
28
1013
8
29
1
65%
35%
90%
5%
3%
3%
96%
1%
3%
0%
625
326
906
25
14
6
939
4
7
1
66%
34%
95%
3%
2%
1%
99%
0%
1%
0%
57
1828
1734
151
3%
97%
92%
8%
27
1024
1013
38
3%
97%
96%
4%
19
932
932
19
2%
98%
98%
2%
Biology103
Count
%
38
Psychology100
Count
%
98
27
123
1
1
0
119
4
2
0
3
122
124
1
78%
22%
98%
1%
1%
0%
95%
3%
2%
0%
2%
98%
99%
1%
SLGParticipants
Biology102
Count
%
69
23
91
1
0
0
90
2
0
0
2
90
89
3
75%
25%
99%
1%
0%
97%
3%
0%
0%
3%
97%
96%
4%
Biology103
Count
%
53
10
60
2
1
0
60
1
0
2
1
62
62
1
84%
16%
95%
3%
2%
0%
95%
2%
0%
3%
2%
98%
98%
2%
Table3:LinearRegressionResultsforSLGAttendance
PsychologySLG
Coeff.
Gender(Men=1)
EntranceAverage
Full/PartTime(Full=1)
YearofStudy
InternationalStudent(Yes=1)
PsychologySLGAttendance
Biology102SLGAttendance
Constant
Ͳ0.075
0.001
0.033
Ͳ0.047
Ͳ0.097
No.Observations
F
Prob>F
RSqrd
RootMSE
Std.Err.
1.030
0.082
1710
11.670
0.000
0.302
Biology102SLG
t
Coeff.
0.045 Ͳ1.660 *
0.005 0.190 0.048 0.700 0.014 Ͳ3.460 ***
0.058 Ͳ1.670 *
0.225 4.570 *** 0.432 0.190 (6,1703)
0.911 Ͳ0.016
0.007
Ͳ0.056
Ͳ0.041
Ͳ0.032
0.416
Ͳ0.360
Std.Err.
Biology103SLG
t
0.048
0.004
0.127
0.014
0.075
0.062
Ͳ0.330
1.650
Ͳ0.440
Ͳ2.920
Ͳ0.430
6.700
0.370 Ͳ0.97
(6,948)
*
***
***
Std.Err.
t
Ͳ0.172
0.000
Ͳ0.196
Ͳ0.026
Ͳ0.156
0.127
0.008
0.404
0.041
0.007
0.277
0.028
0.076
0.062
0.065
0.623
Ͳ4.180
0.020
Ͳ0.710
Ͳ0.930
Ͳ2.050
2.030
0.120
0.650
***
**
**
(7,869)
995
12.450
0.000
0.356 877
3.490
0.001
0.059
0.679 0.736 *indicatesstatisticalsignificanceat10%,**at5%and***at1%.
39
Coeff.
Table4:ComparisonofSLGAttendance,AverageFinalGrades,andAverageEntranceGrades
Numberof
Sessions
Attended
0
1
2
3
4
5
6
7
8
9
morethan9
Timein
Session
(Hours)
0
1.5
3
4.5
6
7.5
9
10.5
12
13.5
>15
Psychology100
EntranceAvg.
Grades
87.79
87.88
88.48
89.19
90.58
88.03
88.95
89.63
86.57
86.30
89.68
Biology102
Avg.FinalGrade
No.
Students
1592
59
22
9
6
6
4
3
3
1
4
Grades
71.80
74.51
74.22
70.50
74.17
76.50
83.00
81.00
74.00
60.00
82.25
EntranceAvg.
No.
Students
1441
61
23
8
6
6
4
4
3
1
4
Grades
89.04
91.27
91.21
89.29
84.85
88.25
90.07
n/a
89.33
94.50
n/a
40
No.
Students
873
41
18
10
2
4
3
0
3
1
0
Biology103
Avg.FinalGrade
Grades
72.39
76.69
79.74
75.90
67.50
85.50
74.75
n/a
78.00
83.00
n/a
No.
Students
887
45
23
10
2
4
4
0
3
1
0
EntranceAvg.
Grades
89.32
90.33
89.49
87.43
90.90
86.80
n/a
88.50
n/a
91.90
n/a
No.
Students
816
24
17
10
5
2
0
1
0
2
0
Avg.FinalGrade
Grades
75.01
77.38
72.41
73.20
69.40
64.00
76.00
71.00
n/a
78.50
n/a
No.
Students
836
24
17
10
5
2
1
1
0
2
0
Table5:PSMResultsforImpactofSLGAttendanceonFinalGrades
SLGSession
Attendance
Hours
AttendedOneSession
1.5Hours
AttendedTwo
Sessions
3Hours
AttendedThree
Sessions
4.5Hours
AttendedFour
Sessions
6Hours
Course
Psychology100
Biology102
Biology103
Psychology100
Biology102
Biology103
Psychology100
Biology102
Biology103
Psychology100
Biology102
Biology103
AttendedAtLeast
OneSession
1.5Hours+
AttendedAtLeast
TwoSessions
3Hours+
AttendedAtLeast
ThreeSessions
4.5Hours+
AttendedAtLeast
FourSessions
6Hours+
AttendedAtLeastFive
Sessions
7.5Hours+
Psychology100
Biology102
Biology103
Psychology100
Biology102
Biology103
Psychology100
Biology102
Biology103
Psychology100
Biology102
Biology103
Psychology100
Biology102
Biology103
Participants
No.Treat
57
41
23
21
17
16
7
9
10
5
NonͲParticip.
No.Control
331
147
69
157
82
64
40
36
18
65
FinalGradeATT
(Difference)
1.360
1.700
Ͳ
3.432
7.274
Ͳ
Ͳ
Ͳ
Ͳ5.460
Ͳ
2
4
Ͳ4.333
Std.Err.
t
2.243
1.953
Ͳ
3.425
2.597
Ͳ
Ͳ
Ͳ
2.739
Ͳ
0.606
0.871
Ͳ
1.002
2.801
Ͳ
Ͳ
Ͳ
Ͳ1.993
Ͳ
5.406
Ͳ0.802
***
1
1
1
*
1
4
20
Ͳ
Ͳ
Ͳ
1
114
81
58
57
41
36
35
23
20
28
13
10
22
11
3
499
214
178
298
133
112
222
54
50
184
15
33
113
11
13
3.130
4.032
Ͳ1.953
2.517
4.640
Ͳ2.993
3.855
3.668
Ͳ4.342
3.393
0.846
Ͳ4.253
2.422
2.767
Ͳ3.850
1.799
1.572
1.415
2.458
2.213
1.684
3.067
3.553
2.215
3.319
4.87
3.217
3.661
5.301
5.859
1.740
2.564
Ͳ1.380
1.024
2.097
Ͳ1.777
1.257
1.032
Ͳ1.960
1.022
0.174
Ͳ1.322
0.662
0.522
Ͳ0.657
*
**
**
*
*
*indicatesstatisticalsignificanceat10%,**at5%and***at1%.Note1:Theprobitmodelfromwhichthepropensityscoreswerecalculatedwerefoundtobe
statisticallyinsignificant,likelyduetotherelativelylownumberofobservations.Theseresultswere,therefore,omitted.
41
Table6:TrendsinCourseCompletionRates
Course
Term
OverallCourseAverage
Biology102
Fall
2006/07
Status
No.
Completed
DroppedWithPenalty
DroppedWithoutPenalty
OverallCourseAverage
Biology103
Winter
Fall&
Winter
94.9
977
94.2
8
0.8
16
1.7
19
77
8
31
3.3
41
79 75 DroppedWithPenalty
12
1.4
84
10
81.6
%
72.9
981
93.3
1.8
6
0.6
4
64
6.1
73.4 75
87.6
885
92.6
898
94.4
13
1.5
12
1.3
13
1.4
93
10.9
59
6.2
40
4.2
72.1 1,248
No.
72.6 883
DroppedWithoutPenalty
DroppedWithPenalty
2009/10
%
91.2
750
DroppedWithoutPenalty
No.
880
88.5
Completed
2008/09
%
73.9 741
OverallCourseAverage
Psychology
100
No.
76.6 Completed
2007/08
%
72.9 1,378
83.7
71.5 1,522
82.8
72.1
1,563
82.8
89
5.8
122
7.4
147
8
138
7.3
192
12.6
146
8.9
169
9.2
187
9.9
42
Table7:GradeDistributionofParticipantsandNonͲParticipants
Psychology100
NonͲParticipants
FinalMean
GradeRange
No.
%
Dropped
Lessthan50%
50Ͳ59%
60Ͳ69%
70Ͳ79%
80Ͳ89%
317
64
209
355
331
329
Above90%
153
Biology102
SLGParticipants
Mean
Grade%
No.
%
18%
4%
12%
20%
19%
19%
n/a
38.266
55.033
64.823
73.991
83.824
4
6
12
20
27
38
9%
94.366
18
NonͲParticipants
Mean
Grade%
No.
%
3%
5%
10%
16%
22%
30%
n/a
42.333
55.5
64.1
73.111
83.711
70
14
102
227
279
208
14%
95.944
57
Biology103
SLGParticipants
Mean
Grade%
No.
%
7%
1%
11%
24%
29%
22%
n/a
41.643
55.069
64.93
73.943
83.62
0
1
5
17
24
27
6%
92.035
18
43
NonͲParticipants
Mean
Grade%
No.
%
0%
1%
5%
18%
26%
29%
n/a
45
55.4
64.53
74.542
83.778
52
13
21
156
346
270
20%
92.611
30
SLGParticipants
Mean
Grade%
No.
%
Mean
Grade%
6%
1%
2%
18%
39%
30%
n/a
30.692
55.524
65.968
74.182
83.085
1
0
1
16
29
16
2%
0%
2%
25%
46%
25%
n/a
n/a
59
66.375
74.103
83.063
3%
91.767
0
0%
n/a
Table8:ChangesinFactorsRelatedtoStudentEngagement
Psychology100
SLGParticipants
NonͲParticipants
Obs Mean
Std.
Dev.
Obs
Mean
Std.
Dev.
Biology102/103
SLGParticipants
NonͲParticipants
Mean Obs Mean
Diff.
Std.
Dev.
Obs Mean
Std.
Dev.
Mean
Diff.
AskedQuestionsDuringClass
39
1.36
0.74
288
1.33
0.70
0.03
48
1.48
0.80
169
1.27
0.61
0.21
ContributedtoClassDiscussion
38
1.23
0.54
284
1.39
0.74
Ͳ0.16
42
1.17
0.44
163
1.25
0.62
Ͳ0.08
Cametoclasswithoutcompeting
readingsorassignments
36
3.11
0.95
276
3.27
1.03
Ͳ0.16
46
2.87
1.26
165
3.02
1.12
Ͳ0.15
20
1.85
1.09
175
1.73
1.05
0.12
42
1.90
0.98
141
1.64
0.78
0.27
Puttogetherideasorconceptsfrom
differentcourseswhencompleting
assignmentsorduringclassdiscussions
26
2.27
1.12
208
1.95
1.06
0.32
49
2.41
1.04
170
2.23
0.92
0.18
Discussedideasfromyourclasswith
othersoutsideofclass(students,family
members,coworkers,etc.)
39
3.33
0.96
291
3.07
1.10
0.27
49
2.92
1.10
170
2.64
1.01
0.28
Numberoftimesabsentfromthisclass
thissemester
39
1.97 0.96 298 2.352 1.04
Ͳ0.38
49
2.29 0.9789 171
2.67 1.03
Ͳ0.39
Includeddiverseperspectives(different
races,religions,genders,political
beliefs,etc.)inclassdiscussionsor
writingassignments
44
45
Table9:PSMResultsforImpactofSLGAttendanceonStudentEngagement
Participa
ntsNo.
Treat
Psychology100
NonͲ
LikertScale
Std.
Part.No.
ATT
Err.
Control (Difference)
Biology102
NonͲ
LikertScale
Participants
Part.No.
ATT
No.Treat
Control (Difference)
t
Std.
Err.
Biology103
NonͲ
LikertScale
Participants
Part.No.
ATT
No.Treat
Control (Difference)
t
Std.
Err.
t
AskedQuestions
DuringClass
35
100
0.23
0.08
3.08
***
21
49
0.40
0.11
3.63
***
24
46
0.10
0.10
ContributedtoClass
Discussion
34
99
Ͳ0.05
0.06
Ͳ0.75
16
48
0.25
0.08
3.02
***
21
44
Ͳ0.20
0.05 Ͳ4.22 ***
Cametoclasswithout
competingreadingsor
assignments
32
97
Ͳ0.21
0.10
Ͳ2.08
**
19
47
Ͳ0.07
0.17
Ͳ0.40
24
44
0.07
0.20
0.36
18
59
Ͳ0.03
0.12
Ͳ0.24
18
38
0.59
0.13
4.58
21
39
0.27
0.14
1.86
22
67
0.39
0.12
3.22
***
22
50
0.21
0.15
1.35
24
46
Ͳ0.03
0.17 Ͳ0.17
35
98
0.18
0.11
1.72
*
22
50
0.32
0.16
2.04
24
46
Ͳ0.13
0.18 Ͳ0.71
35
100
Ͳ0.45
0.11
Ͳ4.19
***
22
50
0.03
0.15
0.19
24
46
Ͳ0.05
0.16 Ͳ0.28
Includeddiverse
perspectives(different
races,religions,
genders,political
beliefs,etc.)inclass
discussionsorwriting
assignments
Puttogetherideasor
conceptsfrom
differentcourseswhen
completing
assignmentsorduring
classdiscussions
Discussedideasfrom
yourclasswithothers
outsideofclass
(students,family
members,coworkers,
etc.)
Numberoftimes
absentfromthisclass
thissemester
*indicatesstatisticalsignificanceat10%,**at5%and***at1%.
46
***
**
1.04
*
Table10:ChangesinStudentConfidenceinUsingLearningStrategies
Psychology100
SLGParticipants
PreͲtest
Obs Mean
Biology102/103
NonͲSLGParticipants
PostͲTest
PreͲtest
SLGParticipants
PostͲTest
PreͲtest
NonͲSLGParticipants
PostͲTest
PreͲtest
PostͲTest
Std. Obs Mean Std. Mean Obs Mean Std. Obs Mean Std. Mean Obs Mean Std. Obs Mean Std. Mean Obs Mean Std. Obs Mean Std. Mean
Dev.
Dev. Diff.
Dev.
Dev. Diff.
Dev.
Dev. Diff.
Dev.
Dev. Diff.
Cornell
method
78
2.82
0.98
32
3.19
1.15
0.37
758
2.93
1.06
223
3.10
1.33
0.18
68
2.94
1.02
41
3.17
1.26
0.23
403
3.00
1.10
131
2.92
1.32
Ͳ0.08
Study
schedules
93
2.85
1.22
37
3.81
1.02
0.96
978
2.92
1.11
261
3.65
1.08
0.73
95
3.74
0.88
44
3.84
0.91
0.10
556
3.62
0.91
155
3.55
1.12
Ͳ0.07
Mindmaps
90
2.90
1.04
37
3.41
0.98
0.51
919
2.95
1.02
254
3.17
1.11
0.21
91
2.88
1.04
44
3.43
1.19
0.55
519
3.00
0.96
142
2.79
1.13
Ͳ0.21
Mnemonic
devices
78
2.94
1.06
30
3.10
1.18
0.16
849
2.95
1.09
236
3.15
1.17
0.20
80
3.16
1.11
42
3.33
1.16
0.17
471
3.23
1.02
139
3.04
1.19
Ͳ0.19
Developing
test
questions
97
3.07
1.16
37
3.16
1.28
0.09
996
2.95
1.08
269
3.60
1.07
0.64
97
3.58
1.07
44
3.59
1.13
0.01
562
3.56
0.99
154
3.36
1.20
Ͳ0.20
Studying
outloud
99
2.92
1.42
36
3.86
1.20
0.94
999
2.90
1.35
275
3.94
1.12
1.04
92
4.01
1.07
45
3.93
1.10
Ͳ0.08
566
3.87
0.93
155
3.75
1.15
Ͳ0.13
47
Table11:PSMResultsforImpactofSLGAttendanceonLearningSkillsConfidence
Cornellmethod
Psychology100
NonͲ
LikertScale
Participants
Particip.
ATT
No.Treat
No.Control (Difference)
Std.
Err.
Bio102
Participants
No.Treat
t
18
17
Ͳ0.076
25
22
30
19
0.227
0.716
Ͳ
***
2.960
0.173 1.314
0.202 3.544 ***
19
21
Ͳ0.241
0.246
Ͳ0.98
Developingtest
questions
26
28
Ͳ0.429
0.21
Ͳ
2.038
Studyingout
loud
25
27
0.034
0.226 0.151
Studyschedules
Mindmaps
Mnemonic
devices
Biology102
NonͲ
LikertScale
Particip.
ATT
No.Control (Difference)
Std.
Err.
t
13
9
Ͳ0.516
18
18
19
15
0.451
0.373
Ͳ
1.139
0.224 2.013 *
0.311 1.198
17
12
Ͳ0.86
0.278
**
18
16
0.20
19
15
0.136
0.257
*indicatesstatisticalsignificanceat10%,**at5%and***at1%.
48
Bio103
Participants
No.Treat
Biology103
NonͲ
LikertScale
Particip.
ATT
No.Control (Difference)
Std.
Err.
t
11
8
0.773
0.426
1.816 *
18
19
13
9
0.056
0.997
0.18
0.438
0.308
2.274 **
15
11
Ͳ0.08
0.26
Ͳ
0.321
0.288 0.692
19
12
0.20
0.305
0.668
0.195 0.697
18
19
0.249
0.268
0.928
0.453
Ͳ
***
3.104