the value and effectiveness of project management simulation as a

THEVALUEANDEFFECTIVENESSOFPROJECTMANAGEMENT SIMULATIONASACAPSTONEACTIVITYINANEXECUTIVEMBA EDUCATIONALPROGRAM by JamesMichaelSzot Athesis submittedforthedegreeof DoctorofPhilosophyinStrategy,Programme&ProjectManagement SkemaBusinessSchool,Lille March2013 PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE ii THEVALUEANDEFFECTIVENESSOFPROJECTMANAGEMENTSIMULATIONASA CAPSTONEACTIVITYINANEXECUTIVEMBAEDUCATIONALPROGRAMS Athesissubmitted by JamesMichaelSzot toSkemaBusinessSchool inpartialfulfillmentoftherequirementsforthedegreeof DoctorofPhilosophy inthesubjectof Strategy,Programme&ProjectManagement ThisdissertationhasbeenacceptedforthefacultyofSkemaBusinessSchool PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE iii Certificate Icertifythattheworkinthisthesishasnotpreviouslybeensubmittedfora degreenorhasitbeensubmittedforadegreeexceptasfullyacknowledgedinthetext.I alsocertifythatthethesishasbeenwrittenbyme.AnyhelpthatIhavereceivedinmy researchworkandthepreparationofthethesisitselfhasbeenacknowledged.In addition,Icertifythatallinformationandliteratureusedareindicatedinthethesis. JamesM.Szot PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE iv Acknowledgement Anundertakingofthisenormityisonlypossiblewiththesupportand encouragementofothers.Attheriskofomittingsomeonewhodeservesmention,I’d liketothankseveralgroupsofpeopleforsharingtheirknowledgeandinspiration. Firstthanksgototheexecutiveeducationprojectmanagementgraduate studentsatTheUniversityofTexasatDallaswhoparticipatedinthisstudy;JamesJoiner ofTheUniversityofTexasatDallasforencouragingmetostartthejourney;Dr. ChristopheBredillet,thenofESCLille,whowelcomedmeintotheprogram;thePhD FacultyatSkemaBusinessSchool,especiallyDr.RalphMuellerandDr.PhilippeRuizfor theirexcellentcoursescoveringresearchmethods,qualitativeandmixedmethods research,andquantitativemethodsandresearchtechniques;Dr.FrankAnbari,Clinical ProfessorofProjectManagementatDrexelUniversity,forhisguidanceandadvocacy;Dr. RodneyTurnerforhisleadershipduringmylateryearsintheprogram;andexaminers Dr.LynnCrawfordandDr.YoungHoonKwakwhoprovidedvaluablesuggestionsfor editingthefinalversion. ClosertohomeandmoreremovedintimeI’dliketothankDr.SuzanneStoutand Dr.JohnWiorkowskiformyinitialtutelageinresearchmethodsandappliedstatistics duringanearliergraduatedegreeprogramatTheUniversityofTexasatDallas.Lessons learnedfromthemwhettedmyappetiteforresearchandsetthestageforthisendeavor. Noneofthiswouldhavebeenachievablehaditnotbeenformyparents,Joseph andGenevieveSzotwhobroughtmeintothisworld,instilledaworkethic,andmade suretheysavedenoughmoneyformyundergraduatetuitionsoIcouldbethefirstinour familytoearnacollegedegree. Andlast,butcertainlynotleast,manythanksgotomywife,Patricia,forliving withonemoredistractioninourwonderfullifejourneytogetherasIworkedonthis researchorjettedofftoFranceforanotherseminar. Allshortcomingsinthisresearchareattributabletomealone. PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE v Table of Contents Certificate ................................................................................................................................... iii Acknowledgement ..................................................................................................................... iv List of Illustrations .................................................................................................................... viii List of Tables ............................................................................................................................... ix List of Acronyms and Abbreviations ........................................................................................... xi Abstract .................................................................................................................................... xiii Introduction ................................................................................................................................ 1 Background ............................................................................................................................. 2 Research Aim ........................................................................................................................ 15 SimProject – the simulation under study ............................................................................. 16 Approach and Methodology ................................................................................................ 17 Hypotheses Tested ............................................................................................................... 19 Organization of this Study .................................................................................................... 20 Literature Review ..................................................................................................................... 22 Historical Perspective ........................................................................................................... 22 Search Methodology ............................................................................................................ 23 Taxonomy ............................................................................................................................. 25 Definitions ............................................................................................................................ 27 Experiential Learning ............................................................................................................ 31 Simulation Games as Experiential Learning Activities (ELA) ................................................ 33 Simulation Game Effectiveness Research ............................................................................ 55 Project Management Simulation Gaming Research ............................................................ 70 Summary ............................................................................................................................ 108 Methodology .......................................................................................................................... 111 Research Paradigm ............................................................................................................. 111 PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE vi Research Approach ............................................................................................................ 113 Research Design ................................................................................................................. 114 Survey Design ..................................................................................................................... 115 Pilot Study .......................................................................................................................... 119 Participants ......................................................................................................................... 121 Procedure ........................................................................................................................... 123 Data Processing and Analysis ............................................................................................. 126 Ethical Considerations ........................................................................................................ 129 Summary ............................................................................................................................ 129 Results .................................................................................................................................... 131 Initial data analysis ............................................................................................................. 131 Verification of assumption of parametric data .................................................................. 135 Scale Reliability ................................................................................................................... 142 Participant demographics .................................................................................................. 145 Descriptive statistics ........................................................................................................... 155 Hypothesis testing .............................................................................................................. 155 Qualitative Results.............................................................................................................. 160 Course Evaluation Survey Quantitative Data ..................................................................... 166 Summary of Results ............................................................................................................ 167 Discussion ............................................................................................................................... 169 Experiential Learning and the Project Management Simulation ....................................... 169 Discussion of Findings ........................................................................................................ 170 Limitations of this research ................................................................................................ 179 Summary ............................................................................................................................ 180 Conclusion .............................................................................................................................. 182 Contribution to Knowledge ................................................................................................ 183 PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE vii Implications ........................................................................................................................ 184 Opportunities for further study ......................................................................................... 185 Appendix A Literature Search Journal Sources .................................................................... 187 Appendix B Presimulation Team Assignment ...................................................................... 190 Appendix C Presimulation Survey ........................................................................................ 209 Appendix D Postsimulation Survey ...................................................................................... 217 Appendix E Project Simulation Final Presentation .............................................................. 227 Appendix F SimProject – About the Simulation .................................................................. 228 SimProject, an Engaging Experience .................................................................................. 228 Playing the game. ............................................................................................................... 229 Appendix G Descriptive Statistics for Variable Components ............................................... 232 References .............................................................................................................................. 239 PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE viii List of Illustrations Figure1–UniversitieswithGAC‐accreditedprogramsbyregion................................................5 Figure2‐LiteratureReviewTaxonomy................................................................................................26 Figure3‐ResearchApproach.................................................................................................................113 Figure4‐Pretest‐posttestdesign..........................................................................................................114 Figure5‐Researchmodel........................................................................................................................115 Figure6‐Teamgendermix.....................................................................................................................146 Figure7‐Teamtechnical/non‐technicalbackgroundmix.........................................................146 Figure8‐Contacthoursofpriorprojectmanagementtrainingoreducation....................147 Figure9‐Priortraining/educationbyteam.....................................................................................148 Figure10‐Projectexperience................................................................................................................149 Figure11‐Professionalexperience.....................................................................................................149 Figure12‐Worldregionofstudentorigin........................................................................................150 Figure13‐Worldregionoforiginbyteam.......................................................................................151 Figure14‐Industrybycohortgroup...................................................................................................152 Figure15‐Industrybyteam...................................................................................................................152 Figure16‐Jobtitlebycohortgroup....................................................................................................153 Figure17‐Jobtitlebyteam.....................................................................................................................153 Figure18‐Annualincomebycohortgroup......................................................................................154 Figure19‐Annualincomebyteam......................................................................................................154 Figure20‐Comparisonofaverageknowledgeratings................................................................171 Figure21‐Comparisonofaverageconfidenceratings................................................................172 Figure22‐Comparisonofpre‐andpost‐simulationattitudesonsimulation....................176 PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE ix List of Tables Table1‐Projectmanagementcompetenceitems.............................................................................86 Table2‐Teamexperienceassessmentitems......................................................................................86 Table3‐Attitudevariablequestions...................................................................................................116 Table4‐Problem‐solvingquestions....................................................................................................116 Table5‐Presimulationdemographicvariables..............................................................................116 Table6‐Postsimulationdemographicvariables............................................................................117 Table7‐Postsimulationopen‐endedquestions.............................................................................117 Table8‐RelevantProjectManagementCorePhaseexitsurveyquestions.........................118 Table9‐Pilottestscalereliability........................................................................................................120 Table10‐Simulationparticipants........................................................................................................122 Table11‐Datareviewsummary...........................................................................................................134 Table12‐Presimulationquestionresponseratebyvariable...................................................134 Table13‐Postsimulationquestionresponseratebyvariable.................................................135 Table14‐CombinedGroupsBandDdatasettestsofnormality.............................................136 Table15‐Testsofnormalitysplitbycohortgroup.......................................................................137 Table16–Cohortgroupmeanscores.................................................................................................139 Table17‐Resultsofindependentsamplest‐testbetweencohortgroups..........................140 Table18‐ScaleK1‐ReliabilityStatistics...........................................................................................142 Table19‐ScaleK1‐presimulationknowledgeItem‐TotalStatistics....................................143 Table20‐Scalereliability‐GroupsBANDD(OC2010ANDOC2011)..................................143 Table21‐RevisedScaleReliability‐GroupsBANDD(OC2010ANDOC2011)...............145 Table22‐Pairedsamplesstatistics......................................................................................................156 Table23‐Pairedsamplescorrelations...............................................................................................157 Table24–Pairedsampletestresults..................................................................................................158 Table25‐Courseevaluationratings....................................................................................................167 Table26‐Summaryofresults................................................................................................................168 Table27–SimProjectattitudequestions...........................................................................................177 Table28‐Descriptivestatisticsforpresimulationknowledgevariable(K1)....................232 Table29‐Descriptivestatisticsforpresimulationconfidencevariable(C1).....................233 Table30–Descriptivestatisticsforpresimulationteamexperience(T1)..........................233 Table31‐Descriptivestatisticsforpresimulationteamexperience(TP1)........................234 PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE x Table32‐Descriptivestatisticsforpresimulationgenericteamattitude(Tg1)...............234 Table33‐Descriptivestatisticsforpresimulationgenericsimulationattitude(SG1)...234 Table34‐Descriptivestatisticsforpresimulationprojectsimulationattitude(SP1)....235 Table35‐Descriptivestatisticsforpresimulationtechnicalknowledgeapplication(N1, E1,P1)...............................................................................................................................................................235 Table36‐Descriptivestatisticsforpostsimulationknowledgeperception(K2).............236 Table37‐Descriptivestatisticsforpostsimulationconfidenceperception(C2).............236 Table38‐Descriptivestatisticsforpostsimulationteamexperience(T2).........................237 Table39‐Descriptivestatisticsforpostsimulationteamexperience(TP2)......................237 Table40‐Descriptivestatisticsforpostsimulationgenericteamworkattitude(SG2)..237 Table41‐Descriptivestatisticsforpostsimulationgenericsimulationattitude(SG2).238 Table42‐Descriptivestatisticsforpostsimulationprojectsimulationattitude(SP2)..238 Table43‐Descriptivestatisticsforpostsimulationtechnicalknowledgeapplication(N2, E2,P2)...............................................................................................................................................................238 PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE xi List of Acronyms and Abbreviations AACSB AACSBInternational‐TheAssociationtoAdvanceCollegiateSchoolsof Business(formerlytheAmericanAssociationofCollegiateSchoolsof Business) ABSEL AssociationforBusinessSimulationandExperientialLearning AMA AmericanManagementAssociation ANOVA AnalysisofVariance ELA ExperientialLearningActivity EMBA ExecutiveMasterofBusinessAdministration GAC GlobalAccreditationCenterforProjectManagementEducationPrograms (sponsoredbytheProjectManagementInstituteandsometimesreferred toasthePMI‐GAC) GUI GraphicalUserInterface ISAGA InternationalSimulationandGamingAssociation JASAG JapaneseAssociationofSimulationandGaming MBA MasterofBusinessAdministration MMR MixedMethodResearch MS MasterofScience NASAGA NorthAmericanSimulationandGamingAssociation PMBOK®GuideTheProjectManagementInstituteStandard:AGuidetotheProject ManagementBodyofKnowledge PMI PMI‐GAC ProjectManagementInstitute ProjectManagementInstitutesponsoredGlobalAccreditationCenterfor ProjectManagementEducationPrograms PMP ProjectManagementProfessionalcredentialawardedbytheProject ManagementInstitute PMT ProjectManagementTrainer,asimulatordevelopedbyDr.Avraham Shtub PTB ProjectTeamBuilder,asimulatordevelopedbyDr.AvrahamShtubbased onexperiencewithPMT SACS SouthernAssociationofCollegesandSchools SAGSET SocietyforAcademicGamingandSimulationinEducationandTraining PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE SCH SemesterCreditHour(nominallyequatesto15hoursofcontacttimeand another30hoursofindependentstudy) SLO StudentLearningObjective SPSS Asoftwareprogramforstatisticalanalysisoriginallystandingfor StatisticalPackagefortheSocialSciences xii PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE xiii Abstract Moststudentsrespondfavorablywhenaskedabouttheuseofasimulationgame inabusinessschoolcourse.However,havingenjoyedthegamedoesn’tnecessarily meanitwasagoodinvestmentofthestudent’stimeandtuitiondollars.Thisresearch reviewsthecurrentstateofmeasuringtheeffectivenessofsimulationgameexperiential learningandbuildsonseveraltechniquesfoundintheliteraturetoexaminetheuseof theSimProjectsimulationgameinoneuniversity’sprojectmanagementgraduate program.Thisresearchisimportantbecauseincreasingdemandforqualifiedproject managershasresultedinsubstantialgrowthinthenumberofacademicproject managementdegreeprogramsandincreasedemphasisonassuranceoflearningby academicaccreditingbodiesandgovernmentalentitiesrequiresexaminationofthe effectivenessofmethodsusedtotrainandeducateprojectmanagers. Theliteraturereviewdiscussesthreerelevantbodiesofknowledge:learning theory,simulationgameapplicationandsimulationgameeffectiveness.Mostliterature onbusinesssimulationgameeffectivenessexplorestheuseofmarketing,strategyor totalenterprisesimulationgames;relativelyfewarticlesreportresearchontheuseof projectmanagementsimulationgames.Thescarcityofresearchontheuseofproject managementsimulationgamesasapedagogicaltoolidentifiesagapinman’sknowledge. Usingmixedmethodtechniquesandaunitofanalysisoftheindividual,this researchtakesapostpositivismapproachtowardmeasuringthechangeinstudent perceptions,attitudesandabilitiesresultingfromparticipationinaprojectmanagement simulationgameandexploreswhattheylearnedfromtheexperience.Resultsare comparedwithanotherresearcher’sfindingstopositgeneralizabilityofconclusions. Thisresearchfoundfavorableattitudesoverallandsignificantincreasesin studentperceptionsofknowledgeandtheabilitytoapplythatknowledge,significant increasesinattitudestowardstheirteamandteamwork,andnosignificantincreaseina favorableattitudetowardssimulationgames.Thoughnotaspecificlearningobjective,a surprisingfindingwasnoincreaseintheabilitytodevelopandanalyzeschedule networkdiagramsortointerpretearnedvalueperformancegraphs. TheuseoftheSimProjectgamewasfoundtobeavaluablecomponentofthe curriculumandaneffectiveuseofclasstime.Recommendationsincludeusing PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE xiv assignmentsorinterimbriefingsonanyconceptstheeducatorwantsreinforcedduring theexperienceandtheuseofformalself‐assessmenttechniquestoenhancelearning. PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 1 Introduction Increasingdemandforqualifiedprojectmanagersandtheresultingexponential growthinthenumberofprofessionalprojectmanagercertificationsanduniversity projectmanagementdegreeprogramscombinedwithincreasedemphasisonassurance oflearningbyacademicaccreditingbodiesandgovernmentalentitiesaffirmthe importanceofexaminingtheeffectivenessofmethodsusedtotrainandeducateproject managers. Simulationgamesareoftenusedinundergraduateandgraduatemarketingand strategicmanagementcoursesandarealsofoundinsomeprojectmanagementcourses. Thepurposeofthisresearchistoinvestigatetheuseofaparticularcomputer‐based projectmanagementsimulationgameasacapstoneexperientialactivityfollowinga seriesofprojectmanagementcoursesinanexecutiveeducationMasterofBusiness Administration(MBA)cohortprogram,oftenreferredtoasanExecutiveMBAorEMBA withanemphasisinprojectmanagement.Usingasimulationinthismannercanrequire uptohalfthecontacthoursofa3semester‐credit‐hour(SCH)courseandonecan’thelp butwonderwhetherornotthisisaneffectiveuseoftimewithajustifiablereturnon tuitiondollarinvestment. Withoriginsfromwargamingboardgamesdatingbackto3000BC,theonsetof businessgamingintheUnitedStatesiscreditedtotheAmericanManagement Associationinthemid‐1950s(Wolfe,1993).While“asaneducationaltool,business simulationgameshavegrownconsiderablyinuseduringthepast40yearsandhave movedfrombeingasupplementalexerciseinbusinesscoursestoacentralmodeof businessinstruction”(Faria,Hutchinson,Wellington,&Gold,2009),furtherresearchon itseffectivenessisneededas“theempiricalresearchontheinstructionaleffectivenessof gamesisfragmented,filledwithill‐definedterms,andplaguedwithmethodological flaws”(Hays,2005).Further,“studiesontheeducationalmeritsofsimulationsoftenare measuredontheaffectivedomain,notthecognitivedomaintheypurporttomeasure”(P. H.Anderson&Lawton,2009) Thisstudyreviewstheliteratureontheuseandevaluationofsimulationgames asexperientiallearningactivitiesinmanagementcurriculaandusesseveralprevious studiesasthebasistoexplorethevalueandeffectivenessofoneparticulardelivery PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 2 modelusingaparticularprojectmanagementsimulationgameasanexperiential learningactivity. Theoverarchingaimofthisresearchistodeterminehowstudentsperceivetheir participationinaprojectmanagementsimulationgameandwhetherornotthisisa valuedexperience.Mixedmethodresearch(MMR)techniquesareusedtoevaluate studentperceptionsandanalysisabilitybeforeandafterexperiencingthesimulation gameandtoexplorewhatstudentsbelievetheylearnedfromtheexperience.Although theuseofMMRtechniquesisoftenunderpinnedbytheparadigmofpragmatism,the philosophyofthisstudyispostpositivismasresultsarecomparedwiththefindingsof anotherresearcherusingadifferentsimulationgameinadifferentacademicsettingto positgeneralizability. Thischapterdiscussestherelevanceofthisresearchinthecontextofthe growingneedfordevelopmentofcompetentprojectmanagers,theneedforrelevant projectmanagementeducationandthegapinman’sknowledgerelatedtoassessingthe valueandeffectivenessofprojectmanagementsimulationgamesinaneducational environment;describestheapproachtousingtheprojectmanagementsimulationgame intheprogramunderstudyandexpandstheresearchaimintoresearchquestions; introducesthesimulationgameusedinthestudy;discussestheapproachand methodologybuiltonpriorstudiesofotherresearchers;statesthehypothesestestedin thequantitativeaspectofthisstudy;anddescribestheorganizationofthisstudy. Background Wecareaboutprojectmanagementeducationbecausethereisagrowingneedto developcompetentprojectmanagers.Thisisevidencedbytheexponentialgrowthof membershipintheProjectManagementInstituteanditscertificationofProject ManagementProfessionals(PMP®)andbythegrowthinthenumberofacademicproject managementdegreeprograms. The need for developing competent project managers.AccordingtoPMI®,the ProjectManagementInstitute,(2010a),theneedforcompetentprojectmanagersis immense: InthePersianGulfandChinaSearegionsalone–whereentirecitiesarebeing built,seeminglyovernight–ashortageof6millionskilledprojectprofessionals isexpectedby2013. PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 3 PMI(ProjectManagementInstitute,2010a)furthersuggeststhereisatrainingand educationgap: Ofthe20millionpeopleparticipatinginprojectsworldwide,justonemillion haveprofessionallyrecognizedformaltrainingonhowbesttoexecutethose projects. Inadditiontothetrainingandeducationgap,PMI(ProjectManagementInstitute, 2010b)claimsthereisanincreasingtalentgapduetoexperiencedprojectmanagers beingclosetoretirementinadvancedmarketsandashortageoftrainedorexperienced projectmanagersinmanyemergingeconomies. Ifweaccepttheimpliedpremisethatprojectsaremorelikelytobesuccessfulif theyareledbypeopletrainedandeducatedinprojectmanagement,takentogether thesetwoclaimssuggesttheneedforprogramstotrainandeducateprojectmanagers. Thetrainingandeducationprovidersappeartoberespondingtothisneed.Wehave movedfromthebeliefthatprojectmanagement“hasfailedtocapturetheimaginationof academics”(Maylor,2001)to“interestineducationandtraininginprojectmanagement hasbeengrowingatanextremelyrapidpace”(Anbari,2010).Anbaribelievesthis interestistheresultof“growingrecognitionoftheimportantcontributionsofproject managementtoanorganizationalcompetitivepositioninthemarketplace,individual careerprogress,economicandsocietaldevelopment,andremediesforshortcomingsof projectoutputsandoutcomes.”Hisconclusionsappeartobesubstantiatedbythe membership,certification,andaccreditationgrowthoftheProjectManagement Institute. Growth of project management as a professional discipline.Recognitionofproject managementasaprofessionaldisciplineisevidencedbytheexponentialgrowthofthe ProjectManagementInstitute(PMI),itscertificationofProjectManagement Professionals(PMP®),anditsaccreditationofuniversityacademicprogramsinproject managementbythePMIGlobalAccreditationCenterforProjectManagement(PMI‐GAC). Foundedin1969byworkingprojectmanagers,PMIstates“ourprimarygoalisto advancethepractice,science,andprofessionofprojectmanagementthroughoutthe worldinaconscientiousandproactivemannersothatorganizationseverywherewill PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 4 embrace,value,andutilizeprojectmanagementandthenattributetheirsuccessestoit” (ProjectManagementInstitute,2010d). PMImembershipgrewslowlyafteritsfoundingand26yearslaterin1995,Iwas membernumber59,000+andcertifiedPMPnumber5,300+.Morerapidgrowthbegan inthelate1990s.Duringthesucceeding16yearsfrom1995to2012,PMIgrewtonearly 396,000activememberswithover500,000activePMPcredentialsasofNovember30, 2012.Whileittookover25yearstoaward5,000PMPcredentials,PMIcertifiesover 5,000insomemonths(ProjectManagementInstitute,2013b). CandidatesforthePMPcredentialwithafour‐yearcollegedegreemust documentatleast4,500hoursofprofessionalprojectmanagementexperienceleading anddirectingprojecttasks,receiveaminimumof35contacthoursofformalproject managementeducation,anddemonstratetheirabilitytoapplyprojectmanagement conceptsandexperiencebypassingascenario‐basedexamination(ProjectManagement Institute,2011). Whiletherearemanyprovidersofferingexam‐prepcoursestofulfillthe35 contact‐houreducationrequirement,somepractitionersofprojectmanagementprefer togobeyondmeetingtheminimumrequirementandinsteadattendanacademic programleadingtoanacademiccertificateordegree.Whensearchingforan appropriateprogram,manyincludereviewofPMI’swebsitelistingofaccreditedproject managementdegreeprogramsaspartoftheirsearch. Accreditation of project management academic degree programs.PMIestablished itsGlobalAccreditationCenter(GAC)in2001as“anindependentacademicaccreditation bodytoadvanceexcellenceinprojectmanagementeducationworldwidethrough collaborationwithandsupportofacademicinstitutions,andthroughaccreditationof academicprograms”(ProjectManagementInstitute,2009).TheGACreportsthatit found2bachelor’sleveland9master’slevelacademicprojectmanagementprograms activein1994,primarilyinthefieldofconstructionmanagement(ProjectManagement Institute,2010c).In2009thisincreasedtoafindingof640degreeprogramsofferedby 456worldwideinstitutionsinmultiplefieldsofstudy. Figure1depictsthegrowthinthenumberofuniversitieswithGAC‐accredited programsinprojectmanagementbyregion(NA=NorthAmerica,EMEA=Europe‐Middle East‐Asia,AP=AsiaPacific,LA=LatinAmerica).AsofOctober31,2012,therewere40 PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 5 universitiesin13countrieswith86GAC‐accrediteddegreeprograms(Project ManagementInstitute,2012). FIGURE1–UNIVERSITIESWITHGAC‐ACCREDITEDPROGRAMSBYREGION 45 40 35 30 25 20 15 10 5 0 LA AP EMEA 2012 2011 2010 2009 2008 2007 2006 2005 2004 2003 2002 NA Despitethisgrowthinthenumberofacademicprogramsrelatedtoproject management,PMIisconcernedtherearenotenoughacademicandtradeinstitute programsinprojectmanagementtobridgethegap.Theyalsopointoutthatdeliveringa successfulprojectmanagementeducationalprogramrequiresacarefulselectionof instructors,curriculum,andstandards;combinedwithafirmcommitmentandquality resources(ProjectManagementInstitute,2010b). Projectmanagementeducationprogramsaretypicallyfoundintechnology schoolsanduniversitycollegesofengineeringandmanagement(business).This researchconsidersevaluationofoneaspectofeducationalquality,theeffectivenessof usingaprojectsimulationgameasaformofexperientiallearninginoneuniversity’s graduatemanagementprogram. The value of management education.Althoughtherelevanceofmanagement educationprogramsisoftencriticizedinthepopularpressbyrespectedacademicslike JeffreyPfefferandHenryMintzberg(Starkey&Tiratsoo,2007),Yeaple(2006)found MBAprogramsremainpopularbecausetheyprovidevalue:“Asurveyof3,771MBA graduatesfindsthateightyearsaftergraduation,93%agreetheirgraduatemanagement educationwasworththetimeandinvestment.” SimilartothegrowthcurvesofPMImembershipandcredentialedPMPsisthe growthofmaster’sdegreesinbusinessawardedbyU.S.universities.Theseincreased fromabout3,000inthelate1950’stonearly130,000intheearly2000’s;withMBA’s PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 6 nowaccountingfor25%ofthemaster’sdegreesawardedintheUnitedStates(Dennis& Smith,2006).DennisandSmithnote“themanager’scareernowhastheattributesofa profession”andentryintothecareerdependsonvalidationofasetofcredentials,the MBAdegreebeing“theentryticketorunioncardforemployeesonthemanagement track.” Withover500,000activeProjectManagementProfessionals(PMPs)credentialed byPMI,projectmanagersmayfindtheyneedanadvanceddegreeinproject managementoranMBAtofurtherdeveloptheirknowledge,skillsandabilitiesandto differentiatethemselvesfromthosePMPswhoseonlyprojectmanagementtrainingor educationwasa35contact‐hourPMPexampreparationbootcamp. Thechallengeforeducationalinstitutionsistogobeyondteachingthe fundamentalsandtoprovidealearningexperiencethatpreparesstudentstoactually manageprojects,leadprojectteams,anddealwithavarietyofethicaldilemmas.The aimofthisresearchistodeterminewhetheraparticularprojectmanagementsimulation gamehelpsdothisinagraduatelevelmanagementdegreeprogramdesignedspecifically forworkingprofessionals. InsharpcontrasttothecriticismsofferedbyPfefferandFong(2002)and Mintzberg(2004),Yeaple(2006)notedthatpart‐timeprogramsprovidedhigher financialgainstotheparticipantsthanfull‐timeprogramsatthesameschoolsbecause theydidn’tinvolvetheopportunitycostoflostincomewhileattendingschool.Inan informalsurveyofPMI‐GACaccreditedprojectmanagementmaster’sdegreeprograms,I observedthatmanyarepart‐timeandtargetedatworkingadults.Inmyconversations withpart‐timeEMBAstudents,Ifindthemtobeveryprotectiveoftheirtimeandto expecttheircourseworktoberelevantandapplicable.Theeducationalvalidityand theirperceivedreturnoninvestmentfromplayingasimulationgameisofconsiderable importance,especiallyifthesimulationexperienceisusedasacapstoneandisoneof theirlastexperiencesintheprojectmanagementphaseofaprogram. Yeaple(2006)claimsthatgoingtobusinessschooldoesnotaddvalue,it multipliesit–fortalentedandmotivatedstudentswhoapplythetoolslearnedwithskill andenthusiasm.Heconcludes“thevalueofagivenschool’sMBAdegreeisthereforethe productofitsincomingstudentqualitytimesthequalityofitseducationalprocess.”This studyexploresthequalityofoneaspectoftheeducationalprocessforanEMBAprogram withanemphasisinprojectmanagement:theuseofasimulationgameasanexperiential PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 7 learningcapstoneactivityattheconclusionofaseriesofprojectmanagementcore courses. Experiential learning and business games.Experientiallearningactivities(ELA) andsimulationgamesareoftenfoundinMBAprograms.Therelevanceandimportance ofexperientiallearningwasmadebyKolb(1984)inhisoftencitedseminalbookon experienceasthesourceoflearninganddevelopment.Aformofexperientiallearning, simulationgamesemphasizingtheapplicationoftoolsareidealfordevelopingproject managers“providedtheyhighlightthemodusoperandiandproblemsolvingmethodsof projectmanagers”(Hutcheson,1984).Simulationsinbusinesseducationhelpbridgethe gapbetweenthetraditionaleducationalparadigmfocusedonthetransmissionof knowledgeandthedisciplineswheretheapplicationofknowledgeintheperformanceof realtasksisneeded(Larréché,1987).Overtheyears,“simulationshavebeenusedwith increasingfrequencyinthedevelopmentofmanagementtalent”(Thornton&Cleveland, 1990). Fariaetal(2009)reviewedallthearticlespublishedinSimulationandGaming fromthefirstissuepublishedinMarch1970throughtheSeptember2008issueand foundthat304outof1,115fullarticles(27%)coveredsomeaspectofbusiness simulationgameeducationandlearning.Theyidentifiedninecentralthemesastowhy educatorsusebusinesssimulationgames,tabulatedtheseinorderoffrequency mentionedbydecade,andnotedthatthetopfivetopicsappearedinthetopfiveofeach decade.Thesewere: experiencegainedthroughbusinessgames thestrategyaspectsofbusinessgames thedecision‐makingexperiencegainedthroughbusinessgames thelearningoutcomesprovidedbybusinessgames theteamworkexperienceprovidedthroughbusinessgames(Fariaetal.,2009) Theynotedthatifoneassumedacorrelationbetweenthearticlesappearingin SimulationandGamingandthereasonswhyeducatorswereusingbusinessgames,the reasonshave“remainedremarkablythesameduringthepast40years”(Fariaetal., 2009).Theyalsofoundthat“inthe2000’sexperienceasanarticletopicjumpedfrom thirdplacetofirst,andlearningobjectivesandoutcomesmovedfromfifthplaceto third.”Fariaetal.attributethisshifttothetrendofaccreditingorganizations,suchas AACSBInternational‐TheAssociationtoAdvanceCollegiateSchoolsofBusiness PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 8 (AACSB),toemphasizelearningoutcomemeasurement.ThePMI‐GACsimilarlyrequires anoutcomes‐basedassessmentoflearning(ProjectManagementInstitute,2009).One opportunityforfurtherresearchistodeterminewhatlearningoutcomesareintendedby educatorsusingprojectmanagementsimulationsasexperientiallearningactivitiesin theirdegreeprograms. Whenaskedwhyasimulationgamewasoriginallyincludedinthecurriculum understudy,thedegreeprogramfoundingdirectorsaid“forstudentstopulltheir learningtogetherintoanintegratedandfunactivity.”Whilesuchanobjectivemightbe viewedwithskepticismbyanaccreditationsitevisitor,itsuggestsblindacceptanceof thevalueofsimulationgamesinacademicprograms.Thecurrentsyllabuscallsforthe studentstodemonstratetheirabilitytoworkasateamtoplanandexecuteasimulated project. Non‐specificlearningobjectivesandqualitativeassessmentofthisformof experientiallearningmaybethenorm.Fariaetal(2009)observethat: Debriefinghasgrowntremendouslyasatopicofinterestinsimulationresearch inthepastdecade.Thediscussionofthelearningintentofbusinesssimulation exercisescoupledwithfeedbackfromthestudentsastowhattheyhave experiencedandlearnedhasalwaysbeenacentralpartofbusinesssimulation gamingresearchthroughthedecades. AnotedcriticoftraditionalMBAprograms,HenryMintzberg(2004)arguesthat “conventionalMBAprogramstrainthewrongpeopleinthewrongwayswiththewrong consequences”andexecutive(EMBA)programs“takemoreexperiencedpeopleonapart timebasis,andthendomuchofthesamething....traintherightpeopleinthewrong waysandwiththewrongconsequences.Thatisbecausetheymostlyfailtousethe experiencethesepeoplehave.”However,peopledorelyontheirexperienceswhen playingasimulationgame,andwhileMintzbergarguesthattheartificialexperience providedbycomputersimulationsandroleplayingareatthelowendofthe authenticity‐methodscale,hedoesacknowledge“thebusinessgamecanhavean appropriateroleinthebusinessschool”asaneffectivemethodforillustratingconcepts andforstudentstolearnhowtoapplytheseconcepts.Hisconclusionisbusinessgames areacapstoneforwhattheMBAteaches,buthisconcernisthetypicalMBAdoesnot PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 9 teachmanagement.Theintentofthisstudyisnottotakeissuewithhisconclusions,but todeterminewhethertheuseofaparticularprojectmanagementsimulationgameasa capstoneexperientiallearningactivityinagraduatemanagementprogramreturns benefitscommensuratewiththetimespentplayingit. Assurance of learning.Collegiateandprofessionalaccreditingagenciessuchas theSouthernAssociationofCollegesandSchools(SACS)CommissiononColleges,AACSB andGACallhavestandardsrelatedtoassuranceoflearning.Whiletheydonotspecify howlearningistobeassured,theyrequireevidenceoflearningassessment,attainment, andcontinuousimprovementforinitialaccreditationandonarecurringbasisthereafter. SACS(2009)isaregionalaccreditingbodyfocusingontheoverallinstitutionand referstoitsU.S.SecretaryofEducationrecognition“underTitleIVofthe1998Higher EducationAmendmentsandotherfederalprograms”initsFederalmandaterequirement dealingwithstudentachievementtoaffirm“theinstitutionevaluatessuccesswith respecttostudentachievement”and“theinstitution’scurriculumisdirectlyrelatedand appropriatetothepurposeandgoalsoftheinstitutionandthediplomas,certificates,or degreesawarded.”UnderitsInstitutionalEffectivenessstandarditrequires“the institutionidentifiesexpectedoutcomes,assessestheextenttowhichitachievesthese outcomes,andprovidesevidenceofimprovementbasedonanalysisofresultsin… educationalprograms,toincludelearningoutcomes.” AACSB(2007)focusesondegreeprogramsandisinterestedinprogram‐level learninggoalsthatarebroaderinscopethanindividualcoursegoals,noting“each specializedmaster’sprogramwillneedauniquesetof4‐10learninggoals;however, somemaybethesameacrossallsuchprograms.”Thesebroadlearninggoalsshould addressbothgeneralknowledgeandskillssuchascommunicationsandethical reasoningaswellasknowledgeandskillsthatdirectly“relatetomanagementtasksthat formthebusinessfoundationofdegreerequirements.”Thesegoalsarerestatedasone ormorelearningobjectiveswithmeasurableattributes.Forexample,theLearningGoal of“ourgraduateswillbeeffectivecommunicators”mayhaveasoneofitsobjectives, “studentswilldemonstrateanacceptablelevelofwrittencommunicationskills.”An exampleAssessmentMeasureis,“anessayisrequiredforadmissionwhichisevaluated forwritingcompetencies.” SimilartoSACSandAACSB,PMI(2010c)states“theprogramshallhaveclearly statedlearningoutcomes…thataredirectlyrelatedtothestatedmissionandobjectives PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 10 [oftheprogram].”Butitgoesfurtherandnotonlydefinesalistof14topicsthatwould “normallybeexpectedtobepartoftheprogramorprerequisiteknowledge”butalso lists5areasoffocuscontainingatotalof39learningoutcomesthatprogramsapplying foraccreditationmustaddressinitsself‐evaluationreport.AkeycomponentoftheGAC accreditationprocessisdescribingandbeingauditedonhowtheuniversityassures studentsarepreparedtotakeresponsibilityforthetasksdescribedinthelearning outcomes. Sincesimulationgamesareexperientiallearningactivitiesattemptingtomodel realityandmayrequireasignificantnumberofcoursestudyhourstoimplement,one wouldexpectthattheywouldhelpsatisfytherequirementforassuranceoflearning. Researchdemonstratingthelearningeffectivenessofprojectsimulationgamescanhelp justifytheiruseaspedagogicaltooltopreparestudentstotakeresponsibilityforthe tasksdescribedinlearningoutcomes. Simulation games as experiential learning activities.Theuseofbusinessgames andsimulationsasatoolforprovidingexperientiallearningisnotnew.Thefirst practicalbusinessgameintheUnitedStateswasintroducedbytheAmerican ManagementAssociation(AMA)overahalfcenturyagoin1956(Biggs,1990;Dillman& Cook,1969;Meier,Newell,&Pazer,1969;Taylor&Walford,1978).AMA’sTop ManagementSimulationwasimmediatelyrecognized“asanimportantnewapproachto jobinduction”and“almostovernight,gamingbecameapopulartrainingactivityfor universitiesworkinginthisfield”(Taylor&Walford,1978).FariaandWellington (2004)foundthatthepercentageofrespondingAACSBmemberschoolsusingatleast onebusinesssimulationgrewfrom71.1%in1962to97.5%in1998.Although,usage rateswerehighestinstrategicmanagement/businesspolicyandmarketingcourses, usagewasalsoreportedforthecategoriesoffinance,management,accounting,and otherbusiness. Faria(1998)foundthat,onaverage,25.1%ofthecourse’sgradewasbasedon thesimulationgameand23.8%ofclasstimewasdevotedtoplayingit.Thiscontrasts with25%ofthegradeand40%oftheclasstimefortheprogramstudiedbymy research.Fariaalsofoundthatfacultyusingbusinesssimulationgamesratedthem highestinperceivedteachingeffectiveness,overlectures,cases,andtextbooks;but facultywhohadneverusedabusinesssimulationgamerankedthemlowest.Thiswould suggestthatnotmuchhaschangedsinceBiggs(1990)wrote: PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 11 Giventherapidgrowthonemightassumethattheeducationalmeritsofbusiness gamesarewellestablished.Thefactis,however,thattheireducationalmerits havebeensubjecttoconsiderabledebate.Therearestudieswhichindicatethat otherformsofpedagogyarejustaseffectiveormoreeffectivethanbusiness games,whileotherstudiesfindthereversetobetrue. Whilethecomparisonoftheuseofbusinessgamesoraparticularbusinessgame versusotherpedagogiesisaninterestingopportunityforfurtherresearch,theaimof thisresearchislimitedtoexploringwhetheraparticularsimulationgameascurrently deployedisagooduseoftimeinanexistingacademicprogram. Project management simulation games.Usingsimulationtodevelopproject managersisalsonotnew.DillmanandCook(1969)describethedevelopmentand evaluationofanon‐computer‐basedprojectmanagementsimulationthatwasusedto developresearchanddevelopmentprojectmanagers.Itgrewoutofaneed“foramore realisticandbroaderbasedexercise”thanwasbeingusedinanexistingtraining program.Thedevelopersbelieved“increasedrealismandgreaterinvolvementby participantswouldbeobtainedbytheuseofasimulatedsituationwhichdemanded morecomplexskills,somedegreeofroleplaying,andincreasedopportunityfordecision making.”Theirsimulationrequiredabouthalfofaone‐weektrainingprogram,roughly equivalenttohalfofthecontacttimefoundinatypical3semestercredithour(SCH) collegecourse.Theyfound“overallpositivereactionstowardtheuseofsimulationin thetrainingprogram”andconcluded“simulationappearstobeaverypromisingtoolin thetrainingofR&Dprojectmanagers.”Theseconclusionswerebasedonparticipant feedbackratherthanmeasurementofperformancegainsversusspecificlearning objectives. Inaninterestingsidenote,Dr.Cook(1976)lateropinedinanAcademyof ManagementJournalarticlethathumanrelationsandcommunicationswere underrepresentedinasurveyheconductedonthe“currentstatusofproject managementinstructioninAmericancollegesanduniversities.”Bothofthesetopicsare importantaspectsintheplayingoftheprojectmanagementsimulationgameunder study. PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 12 However,morethantwentyyearslater,KeysandBiggs(1990)makenomention ofprojectmanagementsimulationgamesintheirchapterreviewingcomputerized businessgamesavailablefromwell‐knownpublishersintheABSELGuidetoBusiness GamingandExperientialLearning(J.W.Gentry,1990a).Discussedaretotalenterprise businessgamesandfunctionalbusinessgamesforaccounting‐finance,marketing, personnel‐humanresourcesandproduction/operations.Afurthersearchofthe literaturerevealedveryfewarticlesrelatingspecificallytoprojectmanagement simulationgames. SimProject,thesimulationusedinthisstudy,firstbecamewidelyavailablefroma well‐knownpublisherasanoptionalbundlewithGrayandLarson’ssecondedition textbook,ProjectManagement:TheManagerialProcess(Gray&Larson,2003),remained therethroughthefourthedition(Gray&Larson,2008)andwasdroppedfromthefifth edition(Larson&Gray,2011)afterthesimulationauthorscancelledtheircontractwith thepublisher(thesimulationisnowdirectmarketedbytheauthors).Anappendix suggestinguseofthesimulationthroughoutthecourseofstudyappearedinthesecond throughfourtheditionsofthetextbookandofferedexercisestohelpintegratethe simulation“intothe‘fabric’oftheclass”to“bringhometheapplicationofthetoolsand techniquesofprojectmanagement“(Gray&Larson,2008). Severalarticleswerefoundexaminingtheuseofprojectmanagementsimulation gamesinacademicprograms.Thesevariedinmethodsfromqualitativestudiesrelying onpostsimulationstudentcommentsandinstructorobservations(S.Al‐Jibouri, Mawdesley,Scott,&Gribble,2005;S.H.Al‐Jibouri&Mawdesley,2001;Collofello,2000; L.S.Cook&Olson,2006;J.M.Cooper,2011;Dantas,Barros,&Werner,2004;Dillman& Cook,1969)toquantitativestudiesfollowingamorerigorousresearchmethodology (Davidovitch,Parush,&Shtub,2006;Davidovitch,Shtub,&Parush,2007;Davidovitch, Parush,&Shtub,2008;Davidovitch,Parush,&Shtub,2009;Davidovitch,Parush,& Shtub,2010;McCreery,2003;Pfahl,2004).ThesearediscussedintheLiteratureReview chapter. Oneevaluationtechnique,theuseofstudentperceptiongainsasthebasisfor measuringeffectiveness,wasusedbyMcCreery(2003)inhisinvestigationintotheuse ofacomputer‐basedprojectmanagementsimulationgameinagraduateproject managementcourse.McCreeryfound“thesimulationexerciseimprovesparticipant knowledgelevelsaswellastheabilityofparticipantstoapplythatknowledge.” PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 13 McCreery’sresearchwasconductedtoexploretheacademicuseofasimulation originallydesignedforuseincorporateenvironmentsbyafor‐profittrainingcompany. Buzzetto‐MoreandMitchell(2009)similarlyassessedtheeffectivenessofa businesssimulationandfound“studentsoverwhelminglyfeltthesimulationhelped themunderstandtheapplicationofkeyconceptsandlearnthedecisionmakingprocess thatoccursinprofessionalbusinesspractice.” Ahn(2008)investigatedtheuseofabusinessgameinanentrepreneurship courseandfoundapositivecorrelationbetweenstudentperceptionsofthelearning experienceandtheirgameperformancescores. ThisresearchstudybuildsonMcCreery’s(2003)approachtomeasuretheself‐ reportedgainofknowledgeandabilitytoapplythatknowledgeasaresultofthe simulationandexploresitscorrelationwithincreasedabilitytodevelopandanalyze simpleschedulesandinterpretprojectearnedvaluedata.Italsoincludesanadaptation ofBuzzetto‐MoreandMitchell’s(2009)andAhn’s(2008)questionnairestoassess studentexpectationsandsatisfactionwiththesimulationgameexperience. Why this research is important.Usingasimulationgameaspedagogycanrequire manycontacthoursinacourse,henceitisimportanttoaffirmalearningbenefit commensuratewiththetimeinvested.However,assessingtheeffectivenessofa simulationgameasalearningtooliscomplicatedanddifficult.Gameresultsand learningareafunctionofmanyvariablesincludingthegamemodel,participant intelligenceandpersonality,instructorrole,gameadministrationprocess,numberof periodsofplay,and,forteamsimulations,teamsizeandinteraction(Greenlaw& Wyman,1973). Earlyresearchcomparedsimulationgamingversusotherlearningmodes. GreenlawandWyman(1973)comparedlearningtomeetcourseobjectivesusinggames versusothermodesofteachingsuchascasestudiesandconsideredbothmultifunctional generalbusinessgamessimulatingtopmanagementdecision‐makingandmono‐ functionalgamesemphasizingonlyonebusinessfunction.Theyobservedthat,although theauthorsoftheresearchtheyreviewedwereveryenthusiasticabouttheirfindings, “verylittle‘hard’researchhasbeendoneongaming–especiallyconcerningwhat playerslearntomeetcourseobjectives,whichwasourprimeinterest.”Reasonsfound forthisincludedthedifficultyindesigningavalidresearchmethodologywithmany “soft”and“hard”variablestocontendwithandthebeliefthatgamesmaybeusedto PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 14 teach“intangible”conceptsandprovide“awareness”ratherthanteachspecificfactsand relationships. Despitetheconcernoverwhetherlearningreallyoccurswhileplayinga simulationgame,Waggener(1979)foundthatstudentspreferexperientiallearning techniquesoverthetraditionaltextbookapproachandprefersimulationsovercase studiesasaformofexperientiallearning. GreenlawandWyman(1973)furtherfoundthat“strong‘learning’ingamesmay notnecessarilybereflectedbygoodgamingperformance”and“converselyabsenceof learningmaynotalwaysbereflectedby‘poor’performance.”Somethingotherthan simulationresultsmustbemeasuredtodetermineeffectivenessandlearning.In contrasttotheconclusionsfoundinsomeoftheresearchtheyreviewed,Greenlawand Wymanconcluded“theeffortandexpenditureswhichhavethusfarbeeninvestedin developingbusinessgameshavenotbeenjustifiedbytheknowledgeofspecificallywhat gamesteach,ifanything.”Ratherthancreatingnewsimulationgames,theysuggested moreemphasisonresearchingexistinggamesisneeded.Thisstudymakesa contributioninthatregard. Taylor(1978)laterobservedthat“presentinformationconcerningthelearning impactofsimulationisfragmentedandbasedmoreonhunchandgeneralimpression, thanonsystematicvalidatedresearchstudy.”However,hefoundnostudiesclaiming thatsimulationwasanyworsethananyothertechniqueinteachingfactualmaterial.He alsosuggestedthatmanyusersofsimulationswouldnotwishtoevaluateitslearning possibilitiesseparatelyfromtheotherlearningmodesinagiventeachingunit.Rather “theywouldarguethatthesimulationactsasastimulustosubsequentlearningandthat thisspin‐offinterestcanbeproperlyconsideredaspartofthebenefitofthetechnique, eventhoughitmaybedevelopedthroughmoretraditionalmethodsoflearning.” TwelveyearslaterKeysandWolfe(1990)“found60fairlyrigorousstudiesthat provideevidencetobusinessgames’generalyetproblematiceducationalefficacy”and observed: Asequivocalasthesefindingsare,manyoftheclaimsandcounterclaimsforthe teachingpowerofbusinessgamesrestonanecdotalmaterialorinadequateor poorlyimplementedresearchdesigns.Theseresearchdefectshavecloudedthe PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 15 businessgamingliteratureandhavehamperedthecreationofacumulative streamofresearch. Followingtheirreviewoftheliteratureontheeducationalvalueofmanagement games,KeysandWolfe(1990)observe“managementgameshavebeenfoundtobe generallyeffectiveinthestrategicmanagementtypecourse”and“someusefulresearch isemergingintheothermanagementareasthoughnotasextensiveasinstrategic management.”Theyconclude,“Thereisagreatneedforcontinuityofresearchinthe managementgamingarea,andforfreshnewapproachestoresearchdesignwheregaps areapparent.”Myresearchhasfoundlittlewrittenspecificallyabouttheuseofproject managementsimulationgamesineducationandisacontributiontothebodyofresearch inthis“othermanagementarea”andprovides“somecontinuityofresearch”tothe projectmanagementsimulationgameresearchreportedbyMcCreery(2003). Research Aim Thegapinman’sknowledgeisthevalueandeffectivenessofusingthePintoand Parente(2003)SimProjectprojectmanagementsimulationgameasacapstoneactivity inagraduateEMBAprogramwithanemphasisinprojectmanagement.Thepurposeof thisstudyistodetermineiftheuseofthissimulationprovidesvaluetothestudentand isaneffectiveuseofclasstime.Theresultswillbeofinteresttoprojectmanagement educatorsusingorcontemplatingtheuseofaprojectmanagementsimulationgamein theircurriculum. IntheEMBAprogrambeingstudied,thesimulationisusedasacapstoneactivity attheconclusionofa21SCHprojectmanagementcorecurriculum.Theapproachused introducesthescopeofthesimulatedprojectalongwithsomeinitialplanningestimates intheformofateamassignmentduringthefourthofsixsequentialprojectmanagement coursesandtheprojectissimulatedasamajorcomponentofthesixthsequentialproject managementcourse. Duringthefourthsequentialcourse,studentteamsfirstdevelopabaseline scheduleandbudgetbasedongiveninformation,thendevelopastaffingmanagement planandrevisedscheduleandbudgetbasedontheiranalysisofavailableresources(see AppendixBforteamassignmentinstructions).Mostofthisworkisdoneashomework outsidetheclassroom. PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 16 Duringthesixthsequentialcourse,sixteenhoursofcontacttime(fortypercentof thecontacthoursforthecourse)areusedtoexecutethesimulatedprojectanddiscuss theresults.Duringexecution,theteamsattempttoobtaintheresourcesidentifiedin theirstaffingmanagementplan,assignthemtotasks,monitorperformance,and implementchangesasneededinanattempttocompletetheprojecton‐timeandwithin budget.Thisisasubstantialportionofthecontacttimeforthesixthcourseanditis importanttoknowwhetherornotthisactivityisavaluablelearningexperience. Theresearchquestionsforthisstudyare:  Dostudentself‐assessmentsofprojectmanagementknowledgeand confidenceintheirabilitytoapplythatknowledgeincreasefollowingthe simulationexperience?Further,areMcCreey’s(2003)resultsrepeatable inadifferentcontextwithadifferentsimulationgame?  Howdostudentopinionsregardingtheirteamexperienceintheprogram andgroupworkingeneralchangeasaresultofthissimulation experience?  Howdostudentopinionsregardingtheuseofsimulationsasalearning toolchangeasaresultofthissimulationexperience?  Arestudentsbetterabletodevelopandinterpretschedulenetwork diagramsandanalyzeearnedvaluedataasaresultofthesimulation experience?  Whatdostudentsfindvaluableabouttheexperience? Accordingly,theunitofanalysisforthisstudyistheindividualstudent. Sinceincludingasimulationgameaspartofacoursecurriculumcanrequirea substantialamountoftimeandadditionalcost,thisresearchshouldbeusefultoanyone consideringtheuseofaprojectmanagementsimulationgameasalearningtoolin professionaltrainingaswellasinacademiceducationprograms.Thisstudyis consideredexploratoryandidentifiesopportunitiesforcontinuedresearch. SimProject – the simulation under study TheprogramunderstudyusesSimProject,developedbyDr.JeffreyPintoandDr. DianeParenteofThePennsylvaniaStateUniversity,forthesimulationgameexperiential learningactivity.Thissimulationisusedbytheprogrambecauseofitsmentioninthe prefaceandappendixofaprogramtextbook(Gray&Larson,2008)andbecauseofits PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 17 successfulapplicationaspartofprofessionaldevelopmenttrainingprogram implementedforacorporatecustomerpriortotheonsetofthisstudy. Accordingtothedevelopers,SimProjectprovides“virtual‘first‐hand’experience inmanagingprojects”(SimProfessionals,2009): Computersimulationsencourageteamdevelopment,collaboration,global thinking,andapredilectiontoconsidertheramificationsofdecisionsandtheir effectonthebottomline–inotherwords,manyoftheskillsthatareusefulto projectmanagersandteammembersinbusiness.Thepurposeofthissimulation istotietogethermanyofthesalientchallengesofprojectmanagementinorder togivestudentsthedeepestpossibleunderstandingofthecomplexitiesinvolved inundertakingaproject.Thegoalofthesimulationwillbetohavestudents manageaprojectfrominitiationtocompletion.Withinthisframeworkthe studentwillneedtoemployanddevelopskillspertinenttopersonnelselection andtraining,motivation,conflictmanagement,andstakeholdermanagement. Studentswillberequiredtouseplanningandschedulingtechniques,suchas workbreakdownstructures,PERT/CPM,scopedevelopment,andriskanalysis (SimProfessionals,2009). Thissimulationgameistypicallyplayedbystudentteamswhodevelopaplan involvingfourtypesofdecisions:resourcehiringandrelease,resourcetrainingto improveexpectedperformance,managerialactionstoinfluenceresourceperformance, andassignmentofresourcestoactivities(Pinto&Parente,2003).Thestudentteams analyzethegiveninformationforacommonproject,competeagainsteachotherfor acquisitionofresourcesfromacommonresourcepool,assigntheseresourcestothe simulatedproject’stasks,andreceivefeedbackonthesimulatedprojectteam’s performance.Thesimulatorprovidesactualcostandtaskdurationinformation followingeachsimulationroundandrankseachteam’sperformanceinfourcategories: Cost,Time,Functionality,andStakeholderSatisfaction. Approach and Methodology Thisstudyusesmixedmethodresearch(MMR)techniqueswithanemphasison buildingonthequantitativeresearchofMcCreery(2003),Ahn(2008),andBuzzetto‐ More&Mitchell(2009)tomeasurestudentperceptionsofvalueandeffectivenessofthe PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 18 simulationexperiencerelatedtoknowledge,confidence,teamworkandthepedagogical useofsimulationgames.Thequalitativeaspectofthisstudyassesseswhatstudents claimtheylearnedfromtheexperience.Togethertheresultsanswertheoverarching researchquestionofhowstudentsperceivetheirexperienceintheprojectmanagement simulationgameandwhattheybelievetheylearnedfromit.AlthoughMMRresearchis mostoftenassociatedwiththeparadigmofpragmatism(Tashakkori&Teddlie,2010b); thisresearchtakesasingleparadigmstanceunderpinnedbypostpositivismowingtoits comparisonwiththeresultsofMcCreery(2003)andsuggestionofgeneralizability. McCreery(2003)foundthatusingateam‐basedprojectsimulationaspartofa graduatelevelcourseinprojectmanagement“improvesparticipantknowledgelevelsas wellastheabilityofparticipantstoapplythatknowledge.”Ahn(2008)foundapositive correlationbetweenstudents’simulationgameperformanceandtheirperceptionofthe learningexperience.Buzzetto‐MoreandMitchell(2009)foundthatusingacapstone simulation“enhance[d]theoveralleducationalandpersonaldevelopmentexperiences ofminoritystudentsenrolledinhighereducationbusinessprograms.” McCreery(2003)usedalongitudinalapproachofpre‐andpost‐simulationself‐ assessmentwithLikert‐typeratingscalestodetermineifthereweresignificantchanges inself‐perceptionsofprojectmanagementknowledgeandabilitytoapplythis knowledgeovertheperiodofexecutingtheprojectsimulationandpostsimulation questionsregardingtheteamexperience.Ahn(2008)andBuzzetto‐MoreandMitchell (2009)usedpostsimulationinstrumentstoassessperceptionsontheuseofsimulation gamesasalearningtool. Thequantitativeapproachusedinthisresearchexpandsonaspectsof McCreery’s(2003)longitudinalself‐assessmentapproachbyaddinglongitudinal adaptationsofMcCreery’steamworkquestionsandAhn’s(2008)andBuzzetto‐Moreand Mitchell’s(2009)simulationperceptionquestions,andbyaddingpre‐andpost‐ simulationquestionsassessingstudentknowledgeandabilitytoperformproject schedulecriticalpathanalysisandinterpretearnedvaluedata. Thequalitativeapproachexaminespostsimulationquestionnaireresponses regardingthesimulationexperienceandprogramfeedbacksurveysissuedattheendof theProjectManagementCorePhaseandaftergraduation.Responsesarecategorizedto identifyemergentthemesdescribingwhattheylearnedfromtheexperience. PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 19 Twoon‐campuscohortsofEMBAwithanemphasisinprojectmanagement studentsaretheprimaryfocusofthisstudy.Datawerealsocollectedandanalyzedfor twogroupsofsimilaradultlearners.Thefirstgroupparticipatedinacorporate professionaldevelopment(non‐academic)programwithcontentsimilartotheEMBA projectmanagementcorecoursesbutwithouttherigorofacademicassessment.This groupservedasapilotfortheresearchmethodology.Theothergroupparticipatedina singleprojectmanagementoverviewcourseinanotherprogramanditsdatawillbeused infutureresearchstudies.Allgroupscompletedthesamepresimulationpreparatory exerciseandencounteredasimilarsimulationexperience. Hypotheses Tested Thequantitativeapproachtestssevenhypothesesclaimingparticipantswill perceiveordemonstrateasignificantincreaseintheirperceptions,attitudesand abilitiesafterparticipatingintheprojectmanagementsimulationgameexperiential learningactivity. Hypotheses1and2explorewhetherMcCreey’s(2003)findingofsignificant gainsinprojectmanagementknowledgeandabilitytoapplythatknowledgeare repeatablewithadifferentsimulationinadifferentsetting. H1: Participantswillassesstheirprojectmanagementknowledgelevelhigher aftercompletingthesimulationgame. H2: Participantswillassesstheirabilitytoapplyprojectmanagement knowledgehigheraftercompletingthesimulationgame. Hypotheses3and4explorewhethertherearesignificantdifferencesin participants’opinionsoftheirsimulationteamandofteamprocessesingeneralafter experiencingthesimulation. H3: Participantswillreportahigheropinionoftheirteamexperienceafter completingthesimulationgame. H4: Participantswillreportahigheropinionofgroupprocessesingeneralafter completingthesimulationgame. PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 20 ExpandingontheworkofAhn(2008)andBuzzetto‐MoreandMitchell(2009), Hypotheses5and6explorewhethersignificantgainsinopinionontheuseof simulationsaslearningactivitiesandtheprojectmanagementsimulationinparticular arefound. H5: Participantswillreportahigheropinionontheuseofsimulationsasa learningtoolaftercompletingthesimulationgame. H6: Participantswillreportahigheropinionofthisspecificproject managementsimulationaftercompletingthesimulationgame. Hypothesis7exploreswhethersignificantgainsintheabilitytosolvetypical projectmanagementcriticalpathandearnedvaluemanagementanalysisproblemsare found. H7: Participantswillbebetterabletosolveprojectscheduleandearnedvalue analysisproblemsaftercompletingthesimulationgame. Organization of this Study Thischapterdemonstratestherelevanceofthisresearchbydiscussingthe growingneedforcompetentprojectmanagementprofessionals,theneedforrelevant projectmanagementeducation,andthegapinman’sknowledgerelatedtomeasuring thevalueandeffectivenessofprojectmanagementsimulationgamesinaneducational environment.Fiveresearchquestionsareidentified,andfollowinganintroductionof thesimulationgameunderstudyandtheresearchapproach,thesevenhypotheses associatedwiththequantitativeresearchquestionsarelisted. TheLiteratureReviewchapterdescribesthesearchmethodology,providesa historicalperspectiveonkeydefinitions,discussesthetheoreticalunderpinningof experientiallearningandtheuseofsimulationgamesasexperientiallearningactivities, exploresbusinesssimulationgameandprojectmanagementsimulationgameresearch toidentifypracticesrelatingtoeffectivenessevaluation,andconcludesadditional researchisneeded. TheMethodologychapterdiscussesthephilosophicalapproachtakenand providesdetailsontheresearchapproach,researchdesign,surveydesign,pilotstudy, PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 21 participantselection,anddataprocessingandanalysis.Itconcludeswithastatementof ethicalconsiderations. TheResultschapterreportstheresultsofacheckofthedataforcompleteness andoutliersandthetreatmentusedformissingdata,validatestheassumptionof parametricdata,describesthediversityofthestudentparticipants,analyzesthedata basedonthehypothesesunderstudy,summarizesthequalitativefindings,andreports additionalquantitativedatafromacourseevaluationsurvey. TheDiscussionchapterdiscussestheimplicationsandlimitationsofthese findings. TheConclusionschaptersummarizesthisresearchandidentifiesopportunities forcontinuedstudy. PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 22 Literature Review Thischapterprovidesahistoricalperspectiveontheuseofsimulationgamesas experientiallearningactivitiesinacademicprograms,explorestheresearchtoidentify thepracticesrelatingtoeffectivedeliveryandevaluationofsimulationgamesas experientiallearningactivitiesinanacademicenvironment,andidentifiesaneedfor additionalresearch.Followingabriefhistoricalperspectiveanddescriptionofmy searchmethodology,Iprovideareviewoftherelevantliteraturebeginningwitha definitionoftermsfollowedbysectionsdiscussingexperientiallearningtheory,theuse ofsimulationgamesasexperientiallearningactivities,anoverviewofbusiness simulationgamingresearchandadetailedlookatprojectmanagementsimulation gamingresearch.Thechapterconcludeswithadiscussionofthegapinman’s knowledge. Historical Perspective Althoughthevalueofplayandgamesineducationandtrainingcanbedatedback tothewritingsofPlato,mostpublishedworkoneducationalsimulationandgaming datesfromthe1960s(Megarry,1978).Megarryobservesthatthetechniquesdescribed inthepublicationsshereviewedfrom1968‐1977arederivednotfrompriortheoretical writings,butratherfromthebusinessmanagementtrainingdevelopedin1956bya researchgroupoftheAmericanManagementAssociationcitingtheinfluenceofthe ancienttraditionofmilitarygaming. Amorerecentreviewofthepast40yearsofdevelopmentsinbusiness simulationgamingbyFariaetal.(2009)confirmedthisancestryanddiscussedthe evolutionofbusinessgamingfrommanuallyscoredsimulationgamesoflimiteddecision andfeedbackcomplexitytosimulationgameswithincreasinglycomplexdecisionsand moredetailedfeedbackhostedonserversaccessedviatheWorldWideWeb.The desirabilityofthistrendtointernethostedsimulationgamesissupportedbythe researchofAshleigh,Ojiako,ChipuluandWang(2012)whofoundprojectmanagement studentsdesire“ablendoflearningthatresidesontheintersectionof‘transferable skills’and‘e‐learningenvironments.’”Despitethispreferencefortheuseoftechnology andpedagogicalinnovation,“thefundamentalreasonsastowhyeducatorsusebusiness simulationgameshavenotchangedmuchduringthepast40years”(Fariaetal.,2009). Fariaetal.(2009)furthernote,“asgameshavebecomemorecomplexbecauseof PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 23 advancesincomputingpower,theneedforgroupdiscussionanddecisionmakingto understandandmanagethiscomplexityhasbecomegreater.” IncontrasttoKeys(1977)earlyfindingthattheinstructorshouldplaya significantroleinguidingthegamelearning,thisneedforgroupprocess/teamwork“has transferredfarmoreofthelearningresponsibilityofbusinessgamestothegame participantswhilemakingthegameslessdependentonactiveinstructoroperationand manipulation”(Fariaetal.,2009).Asaresultofthishands‐offrolefortheinstructor,the postsimulationgamedebriefdiscussionhasbecomeagrowingresearchinterestduring the2000’s.Thisincreasedinterestislikelytheresultofoutcome‐basedlearning measuremandatesofaccreditingbodiesandatendencyforthesimulationgameto becomethecenterpieceofthebusinesscourse(Fariaetal.,2009). ConsistentwiththisbackgroundandalamentbyAndersonandLawton(2009) thatmuchoftheresearchontheeffectivenessofusingbusinesssimulationsas experientiallearningactivitieshasbeenbasedonmeasuringparticipantorinstructor perceptionsoflearningratherthanonactual,direct,objectiveevidenceoflearning,my literaturesearchfoundaneedforadditionalresearchonevaluatingtheeffectivenessof businesssimulationsingeneralandprojectmanagementsimulationsascapstone activitiesinparticular.Ifoundnogenerallyacceptedprocessforevaluating effectivenessofbusinesssimulationgames,verylittleonprojectmanagementsimulation games,andconfirmedAndersonandLawton’sobservationthateffectivenessmeasures tendtorelyonsubjectiveassessmentofparticipantperceptionsandsatisfactionwiththe experience.Whilemostagreedirectmeasurementoflearningispreferred,thisisoften difficultbecauselearningobjectivesarenotalwaysclear,teamdynamicsareoften involved,andlearningcanoccurwhilelosingthegame(Greenlaw&Wyman,1973). Search Methodology Aninitialsearchoftheliteratureusingkeywords“managementeducation, projectmanagement,experientiallearning,businesssimulation,simulationgames, effectiveness,MBA,andassessment”resultedinafindingofseveralhundredarticles, Ph.D.dissertationsandbookstoreviewforrelevance.Primarysearchengineswerea universitylibraryhostedManagementPOWERSEARCHwhichsimultaneouslysearches AcademicSearchComplete,BusinessSourceComplete,EconLit,andRegionalBusiness News;EducationPOWERSEARCHwhichsimultaneouslysearchesAcademicSearch Complete,EducationResearchComplete(ERIC),andProfessionalDevelopment PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 24 Collection;SCOPUS;GoogleScholar;ProQuestfordissertationsandtheses;andthe university’scatalogkeywordsearch.Relatedarticleswerefoundinover60journals (listedinAppendixA;boldfontindicatescitedreference)indicatingbroadinterestinthe useofsimulationgamesasapedagogicaltoolforexperientiallearning.Onearticleby McCreery(2003)intheInternationalJournalofProjectManagement,“Assessingthe valueofaprojectmanagementsimulationtrainingexercise,”provedparticularly valuableasitprovidedafoundationformyresearchdesign. ThejournalSimulationandGaming:AnInternationalJournalofTheory,Design andResearch(oftencitedasSimulation&Gaming)wasidentifiedastheleadingacademic journalcontainingarticlesdiscussingtheuseofsimulationgamesinanacademic environment.Theprimarysponsorofthisbi‐monthlypublication,theAssociationfor BusinessSimulationandExperientialLearning,alsopublishesitsannualconference proceedingsinDevelopmentsinBusinessSimulation&ExperientialLearning(ABSEL, 2011).Oneoftheirstatedmaingoals“toaugmenttechniquesusedfortheassessmentof educationandthedevelopmentoflearningtheory”iscongruentwiththeintentofstudy. PursuitofanotherABSELgoal,“tofacilitatecommunicationonaglobalscale amongspecialistsdesigningandusingbusinesssimulationsandexperiential methodologies,”isevidencedbytheirlistedtiestootherorganizationswithaninterest insimulationandgaming(ABSEL,2011).“TheseincludetheInternationalSimulation andGamingAssociation(ISAGA),theJapaneseAssociationofSimulationandGaming (JASAG),andtheNorthAmericanSimulationandGamingAssociation(NASAGA).”ISAGA andNASAGAareco‐authorsofSimulationandGaming.Anotherorganization,theSociety forAcademicGamingandSimulationinEducationandTraining(SAGSET),published PerspectivesonAcademicGamingandSimulation¸theproceedingsoftheirannual conferences,andajournal,Simulation/GamesforLearning;butthesedonotappeartobe inwidecirculationasaninitialreviewoftheirwebsite(SAGSET,2011)referred inquiriesforpasteditions,ifavailable,tooneofitsmembersandthecurrentwebsite (SAGSET,2013)onlylistsoneessayandnoneofthesejournalsaspublications. Foundedin1974,ASBEListheleadingsocietyencouragingacademicresearchin theuseofsimulationgamesinacademicbusinessmanagementprogramsandthe contentsofSimulationandGamingissueswerealsoreviewedmanuallyforappropriate articlesthatmayhavebeenmissedbysearchengines.ABSELpublishedGuideto BusinessGamingandExperientialLearningin1990toprovide“thedefinitive,hands‐on PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 25 guideforbusinesseducatorswishingtousegamesandexperientialexercisesto maximumeffect”(J.W.Gentry,1990a).Thisseminalpublicationincludedchaptersby nineteenexpertcontributorsandintroducesABSELasanorganizationandsimulation gamingasanexperientiallearningactivity;discussesgamedevelopment,futuretrends, non‐gameexperientialexercises;andconcludeswiththreechaptersdiscussingthe evaluationofexperientiallearning.Worksbytheseauthorsandotherscitedbythem werealsoexploredforcontentrelevanttothisstudy. ABSELconferenceproceedingsarepublishedinDevelopmentsinBusiness SimulationandExperientialLearninganditspredecessor,DevelopmentsinBusiness SimulationandExperientialExercises,thesecondmostcitedpublicationsinthisstudy. Taxonomy AsdepictedinFigure2,theliteratureusedinthisstudyisgroupedintothree broadcategories:LearningTheory,SimulationGameApplicationandSimulationGame Effectiveness.LearningTheoryencompassesthetheoreticalunderpinningofKolb’s (1984)seminalfindingsonexperientiallearning,Mayer’s(2002)discussionofrote versusmeaningfullearningandKrathwohl’s(2002)descriptionoftherevisionto Bloom’sTaxonomyofEducationalObjectives.SimulationGameApplicationincludesthe argumentsforandagainsttheuseofsimulationgamesasexperientiallearningactivities ineducationalprogramsandguidelinesfortheiradministration.SimulationGame EffectivenessisdividedintoBusinessManagement,wheremostoftheresearchtakes placeusingtotalenterprise,strategyormarketingmanagementsimulationgames,and ProjectManagement,whereverylittleresearchhasbeendoneandeverystudyfoundis cited. TheBusinessManagementresearchcanbefurthercategorizedintoarticles discussingevaluationmethodsandtheiradequacy,theuseofsimulationgamesto achievecourseobjectives,earlyexamplesofrigorouslearningresearchjustifying simulationgamesasapedagogicaltool,proposedframeworksforfutureresearchand examplesofstudiesattemptingtomeasuresomeaspectofthebenefitofincludinga simulationgameinanacademiccourseofstudy.Assessmentoflearningisfoundtobea complicatedandcontinuingtopicofcontroversyamongresearchersleadingtocallsfor additionalresearch. PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 26 Muchoftheprojectmanagementresearchlacksrigorinmethodologyand/or reporting.NotableexceptionsarethearticlesbyDavidovitchetal.(2006;2007;2008; 2009;2010),McCreery(2003)andPfahl(2004). Mystudybuildsonthisknowledgeandmakesacontributiontowardsclosingthe knowledgegaprelatedtotheeffectiveuseofprojectmanagementsimulationgamesin academicprograms. FIGURE2‐LITERATUREREVIEWTAXONOMY Project Management Capstone Simulation Effectiveness Learning Theory Simulation Game Application Experiential Learning Kolb, 1984 Usage and History Faria et al., 2009; Faria & Wellington, 2004 Meaningful Learning Mayer, 2002 Promoting Learning Gentry, 1990; Hertel & Millis, 2002; Jones, 1987 Revised Bloom’s Taxonomy Krathwolh, 2002 Debriefing Lederman, 1984; Thatcher, 1990; Thiagarajan, 1992 Student Assessment Anderson & Lawton, 1988, 1992, 2007; Salas et al, 2009 Simulation Game Effectiveness Business Management Project Management Evaluation Gosenpud, 1990; Gosen(pud) & Washbush, 2004, 2010 Qualitative Post Perceptions Collofello, 2000 Dillman & Cook, 1969 Relation to Course Objectives Greenlaw & Wyman, 1973 Rigorous Learning Research Keys, 1977 Multi‐source Framework Stainton, Johnson & Borodzicz, 2010 Methodology Examples Qualitative Green, 2004 Quantitative Posttest Anderson, 2005; Ardobor & Daneshfar, 2006; Baglione & Tucci, 2010; Buzzetto‐More & Mitchell, 2009 Mixed Method Posttest Graziano, 2003; Lainema & Lainema, 2007 Quantitative Pretest‐Posttest Gamlath, 2009; Klein, 1980, 1984; Seethamraju, 2011; Smalt, 1999; Wellington et al., 2012; Williams & Williams, 2011 Quantitative Post Simulation Score Martin, 2000 Quantitative Post Perceptions Dantas, Barros & Werner, 2004 Mixed Method Post Al‐Jibouri & Mawdesley, 2001; Al‐Jibouri et al., 2005; Cooper, 2011 Quantitative PrePost Perception Cook and Olson, 2006 McCreery, 2003 Quantitative PrePost Simulation Score Davidovitch et al., 2006, 2007, 2008, 2009, 2010 Mixed Method PrePost Pfahl, 2004 PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 27 Definitions Althoughsimulationshavebeenusedineducationforover50years,“thereare nogenerallyaccepteddefinitionsofaneducationalsimulationoritsmanyvariations” (Hertel&Millis,2002).Thissectionintroducesseveralofthedefinitionsfoundinthe literatureandconcludestheearlydefinitionshavestoodthetestoftime. Experiential Learning.Whilecautioning“whatthestudenttakesawayfroma particularexperienceisoftenidiosyncratictohis/herperceptionsoftheexperience,and issomewhatoutsidethecontroloftheinstructor,”Gentry(1990b)discussesseveral definitionsand“criticalcomponentstoexperientiallearning”andconcludes: Experientiallearningisparticipative,interactive,andapplied.Itallowscontact withtheenvironment,andexposurestoprocessesthatarehighlyvariableand uncertain.Itinvolvesthewholeperson;learningtakesplaceontheaffectiveand behavioraldimensionsaswellasonthecognitivedimension.Theexperience needstobestructuredtosomedegree;relevantlearningobjectivesneedtobe specifiedandtheconductoftheexperienceneedstobemonitored.Students needtoevaluatetheexperienceinlightoftheoryandinlightoftheirown feelings.Andprocessfeedbackneedstobeprovidedtothestudent(andpossibly supersede)theoutcomefeedbackreceivedbythestudent. Gentry(1990b)developedthisdescriptionbyconsideringthepriorworkofan AACSBtaskforceexploringappliedandexperientiallearningcurriculumdevelopment andprior“definitionalworkbyHoover(1974)atthefirstABSELconference”and“ina subsequentpaper(HooverandWhitehead1975,p.25),”combinedwith“theoverall experientiallearningtaskstructureproposedbyWolfeandByrne(1975).”Gentrynoted, “Thisprocess‐orientedapproachissomewhatsimilarinnaturetothoseproposedby Kolb(1984)andLewin(1951),”seminalauthorsonthetopicofexperientiallearning. Thesecharacteristicsofexperientiallearningarealsocongruentwiththeconcept ofmeaningfullearningdescribedbyMayer(2002)inhiscomparisonofthreelearning outcomes(nolearning,rotelearningandmeaningfullearning)andtheirrelationshipto thecognitiveprocessesoftheRevisedBloom’sTaxonomy.Inthiscomparison,rote learnerscanrecallinformationbutareunabletotransferthisknowledgetonew PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 28 situationswhereasmeaningfullearnerscannotonlyrecallinformation,theycanapply thisknowledgetonewproblemsanddifferentlearningsituations.AccordingtoMayer: Meaningfullearningoccurswhenstudentsbuildtheknowledgeandcognitive processesneededforsuccessfulproblemsolving.Problemsolvinginvolves devisingawayofachievingagoalthatonehasneverpreviouslyachieved;thatis, figuringouthowtochangeasituationfromitsgivenstateintoagoalstate (Mayer,1992).Twomajorcomponentsinproblemsolvingare(a)problem representation,and(b)problemsolution,inwhichastudentdevisesandcarries outaplanforsolvingaproblem(Mayer,1992). Mayer(2002)differentiatesrotelearninganditsassociationwithRemember,the simplestcognitivecategoryintheRevisedBloom’sTaxonomy,frommeaningfullearning anditsrelationshipwiththefiveothercognitivecategoriesthatare“increasinglyrelated totransfer(Understand,Apply,Analyze,Evaluate,andCreate).”Remember,describedas “retrievingrelevantknowledgefromlong‐termmemory”categorizesonlytwocognitive processesinBloom’srevisedTaxonomy(RecognizingandRecalling),whereastheother fivecategoriescontain17cognitiveprocesses(Understand:Interpreting,Exemplifying, Classifying,Summarizing,Inferring,Comparing,Explaining;Apply:Executing. Implementing;Analyze:Differentiating,Organizing,Attributing;Evaluate:Checking, Critiquing;Create:Generating,Planning,Producing)(Krathwohl,2002).Allofthesemay berequiredwhenparticipatinginanexperientiallearningactivitysuchasabusiness managementorprojectmanagementsimulationgame. ThisstudyacknowledgesthecongruenceofGentry’s(1990b)descriptionwith theseminalworkofKolb(1984),Mayer’s(2002)descriptionofMeaningfulLearningand theRevisedBloom’sTaxonomyasdescribedbyKrathwohl(2002),andconsiders experientiallearningtobeaninstructor‐facilitatedlearningexperiencewherethe studentslearnbyreflectingontheiractiveinvolvementinanexperientialactivity requiringtheuseofanalyticalskillstoconceptualizetheexperienceanddecisionmaking andproblemsolvingskillstointeractwithandapplyrelevantconceptsduringthe activity.ThisdefinitionalsoalignswithGentry’s(1997)latershortdefinition: “Experientiallearningistheaffective,cognitive,andbehavioralchangeinastudent pursuanttosomestructuralexperiencedesignedtofacilitatethesechanges.” PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 29 Game.Heyman(1975),anearlyauthoronclassroomuseofsimulationgames, definesagameas“acontestinwhichpeopleagreetoabidebyasetofrules”and comments“theveryword‘game’meansfuntostudents–andmostotherpeople–which maybeachangefrommanyschoolactivities,onereasonwhygamesarepopularin classrooms.” HertelandMillis(2002)distinguishsimulationsfromgamesbynoting“games ofteninvolveelementsoffantasyandmake‐belief...[with]rulesofplay[that]...are likelytobefixedandrigid,unlikethemorefluidandoftenspontaneousguidelinesof [educational]simulations”which“typicallyplacestudentsintrue‐to‐liferoles.” Forthepurposesofthisresearch,Heyman’s(1975)definitionofgameis sufficient. Simulation.Asimulationis“animitationorsimplificationofsomeaspectof reality”(Heyman,1975).Heymanadds“amoreelaboratedefinitionis:Asimulationis anactivitywhoserulestendtogenerateinthetotalbehavioroftheparticipantsamodel ofsomerealworldprocess.” HertelandMillis(2002)prefernottoengagein“semanticdebates”onthe “variouspermutationsofsimulations,games,roleplaying,andotherinteractive pedagogies”andsuggestthat“educationsimulationstypicallyplacestudentsintrue‐to‐ liferoles,andalthoughthesimulationactivitiesare‘realworld,’modificationsoccurfor learningpurposes.” Forthepurposesofthisresearch,Hyman’s(1975)formerdefinition,“an imitationorsimplificationofsomeaspectofreality”issufficientasthenotionofrulesin themoreelaboratedefinitioncomeintoplaywhenconsideringthedefinitionfora simulationgame.Asimulationcanbealiveroleplayoraninteractivecomputermodel. Simulation game.Combininghisdefinitionsforsimulationandgame,Heyman (1975)definesasimulationgameas:“Anactivitythatcombinesthecharacteristicof both‘game’and‘simulation;’itimitatessomepartofrealityandisacontext.Simulation games,thereforearebothenjoyableandeducational,withtheemphasisonlearning” (Heyman,1975). HertelandMillis(2002)prefertoavoidcombiningtermsinto“hyphenated horrors”andadvise“theusetowhichateacherputsateachingtechniqueismore importantthanwhatthetechniqueiscalled. PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 30 Thetermssimulationandgameareoftenusedinterchangeablyinresearch articlesandmaybefoundwithadescriptivemodifiersuchasbusiness,marketing, strategicmanagement,totalenterprise,orprojectmanagement. Forthepurposeofthisresearchthetermsimulationgamewillbeusedtoreferto anexperientiallearningactivitywithrulesofplaythatsimulatesaspectsofthereal worldtoachieveeducationallearningobjectives. Inthecaseofgraduatebusinessmanagementeducation,simulationgamesare oftencomputerizedtoprovideadynamicandrobustlearningenvironment.Biggs (1990)describescomputerizedbusinessgamesasonesinwhich“gameplayers (participants,students)assumetheroleofdecision‐makersinorganizations.”Headds: Frequently,thecomplexityofthegameissuchthattheparticipantsaregrouped intoteamsofthreeormoremembers.Theteams’decisionareasmaycoverthe totalfirmorafunctionalunitofafirm,dependingonthefocusofthesimulation. Theplayersareprovidedwithaplayer’smanualwhichpresentsthe“rulesofthe game,”describestheenvironment,andgivesastartingpointforthefirm.The startingpointisusuallythesameforeachfirmintheindustry.Theparticipants submitasetofdecisionsfortheirfirmtothegameadministrator(theinstructor ortrainerorhis/herdesignee).Eachsetofdecisionsusuallyrepresentsa quarteroftheyearorayearoftheoperationofthefirm.Thegames administrator,usingthecomputer,processesthedecisionsandreturnsthe resultstotheparticipants.Theparticipants,giventheircurrentsituation, prepareanothersetofdecisionswhicharethenprocessedbythegame administrator.Thefactthatparticipantsmakedecisionsforanumberof decisionperiodsforcesthemtolivewiththeconsequencesoftheirprevious decisions. Theoutput(results)receivedbyparticipantsgenerallyconsistsofatleast abalancesheetandanincomestatement.Frequentlyatleastonepageof supplementaloutputisprovidedeachfirmandinsomeinstancesagreatmany pagesareprovided. Thisdescriptionisrelevantandeasilyadaptedfortheprojectmanagement simulationunderstudy.Ratherthansimulatetheoperationofthetotalfirmora PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 31 functionalunitofthefirm,thedecisionareascoveraprojectbeingundertakenbythe firm.Eachteamhasthesamestartingpoint,i.e.,thesameprojectandobjectiveswitha resourcepoolcommontoallteams.Eachsetofdecisionsrepresentsaphaseofthe projectratherthanacalendarquarteroryearandteamsmustlivewiththe consequencesofdecisionsmadeduringearlierphases.Theoutputconsistsofschedule, costandresourceefficiencyresultsforeachworkperiod. Experiential Learning WritingintheforewordofDavidKolb’sseminaltextonexperientiallearning, notedcontemporaryleadershipscholarWarrenBenniscreditsKolbwithbeingthefirst toconclusivelydemonstrate“thatlearningisasocialprocessbasedoncarefully cultivatedexperiencewhichchallengeseverypreceptandconceptofwhatnowadays passesfor‘teaching’”(Kolb,1984).Kolbdrawson“theintellectualoriginsof experientiallearningfromtheworksofJohnDewey,KurtLewin,andJohnPiaget”to describe“theprocessofexperientiallearningandproposesamodeloftheunderlying structureofthelearningprocessbasedonresearchinpsychology,philosophy,and physiology.”Hecreditsthe“greatRussiancognitivetheoristL.S.Vygotsky,thatlearning fromexperienceistheprocesswherebyhumandevelopmentoccurs”(Kolb,1984). Kolb(1984)regardsDewey,LewinandPiaget“astheforemostintellectual ancestorsofexperientiallearningtheory”andidentifiesthecommoncharacteristicsof theirlearningmodelstodescribethenatureofexperientiallearningas“theprocess wherebyknowledgeiscreatedthroughthetransformationofexperience.”His descriptionemphasizes“theprocessofadaptationandlearningasopposedtocontentor outcomes...thatknowledgeisatransformationprocess,beingcontinuouslycreatedand recreated,notanindependententitytobeacquiredortransmitted”andthat“learning transformsexperienceinbothitsobjectiveandsubjectiveforms”(Kolb,1984). Basedonhisresearchofindividualityinlearningandtheconceptsoflearning styles,Kolb(1984)proposedatwo‐dimensionalperspectiveoflearningstylewitha learningspacedefinedbythedimensionsabstract/concreteandactive/reflectivebut cautionedthatthismodelonlyrepresentedqualitativedifferencesinelementary learningorientations,adding“tofullyappreciateaperson’sapproachtolearning,we needtounderstandhisorherpositiononathirddimension,thatofdevelopment.”Asit relatestolearninganddevelopmentinhighereducation,Kolbfound“anyeducational program...canbeviewedashavingdegreesoforientationtowardeachofthefour PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 32 learningmodesintheexperientiallearningmodel,labeledasaffective,perceptual, symbolic,andbehavioral,toconnotetheoverallclimatetheycreateandtheparticular learningskillormodetheyrequire.” Intheaffectivelycomplexlearningenvironment,learnersarelikelytoexperience whatitistobeaprofessionalinthefieldunderstudyby“engaginginactivitiesthat simulateormirrorwhattheydoasgraduates,ortheyareencouragedtoreflectuponan experiencetogeneratetheseinsightsorfeelingsaboutthemselves(Kolb,1984).” Perceptivelycomplexlearningenvironmentsrequireconsideringatopicfrom differentperspectivesandindifferentways.“Ifataskisbeingdoneoraproblemis beingsolved,theemphasisismoreonhowitgetsdone,theprocess,thanonthesolution. ...Learnersarethusfreetoexploreothers’ideas,opinions,andreactionsinorderto determinetheirownperspective”(Kolb,1984).Kolbstatestheteacher’srolehereisto emphasizeinquiryandreflectiononresultsasaguidetofutureactionsratherthanto evaluatebasedontheattainmentofthecorrectsolution. Symbolicallycomplexlearningenvironmentsinvolve“tryingtosolvea[n abstract]problemforwhichthereisusuallyarightanswerorabestsolution”(Kolb, 1984).Thelearnerisguidedandconstrainedbyrulesandtheteacheristheaccepted expert,timekeeper,taskmaster,andscheduleenforcer.“Successismeasuredagainstthe rightorbestsolution,expertopinion,orotherwiserigidcriteriaimposedbytheteacher oracceptedinthefieldofstudy”(Kolb,1984).Kolbfoundthatlearnerspreferringthis learningenvironmenttendtoprefertheoryreadingsandthinkingaloneandthatgroup exercisesandsimulationshindertheirabilitytolearn. Behaviorallycomplexlearningenvironmentsemphasizeapplyingknowledgeor skillstoapracticalproblemthatneednothaveacorrectorbestanswer;“butitdoes havetobesomethingthelearnercanrelateto,value,andfeelsomeintrinsicsatisfaction fromhavingsolved”(Kolb,1984).Thiscouldbeareal‐lifeproblem,case,orsimulation withafocusoncompletingthetaskwherethelearnerisresponsiblefordecidingona courseofactionandmanaginghisorhertimewithintheconstraintsofpossible checkpointsandadeadline. Theuseofasimulationgameasanexperientiallearningactivityinan educationalprogramhasastrongorientationtotheaffective,perceptualandbehavioral learningmodeswherestudentslearnbydoingtasksrelatedtoworkintheirprofessional field,oftenonteamswheretheyalsolearnfromeachother. PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 33 Simulation Games as Experiential Learning Activities (ELA) Theuseofbusinesssimulationgameshasgrownsubstantiallyinacademic programsand“reachedtherelativepointofsaturationinvariousAmericanbusiness courseapplications”(Wolfe,1993).Durablelearningispromotedacrossallphasesof thesimulationexperiencebystudentsstructuringtheinformationtheyreceiveandact onintoknowledge(Zantow,Knowlton,&Sharp,2005).“Withrespectto[student] attitudes,thereappearstobelittletorisk,andmuchtogain,fromintegratingbusiness simulationsintobusinessschoolprograms”(P.H.Anderson&Lawton,2009).Although therehavebeendramatictechnologicalchangesenhancingmultipledimensions(listed byFaria,etal.,2009,as“realism,accessibility,compatibility,flexibilityandscale, simplicityofuse,decisionsupportsystemsandcommunication”),thebasicexperiential learningprocessforapplyingeducationalsimulationgameshasn’tchangedmuchover thedecadesfromtheonedescribedbyTaylorandWalford(1978): 1.Participantstakeonroleswhicharerepresentationsofrolesintherealworld, andthenmakedecisionsinresponsetotheirassessmentofthesettinginwhich theyfindthemselves. 2.Theyexperiencesimulatedconsequenceswhichrelatetotheirdecisionsand performances. 3.Theymonitortheresultsoftheiractionsandreflectontherelationship betweentheirowndecisionsandtheresultantconsequences. Interrelationshipsbetweenalargenumberoffactorscanbedisplayed, visiblymanipulatedandadjusted.Asthesituationdevelopssonewstrategies needtobeformulatedandadopted. Thisdescriptionofthelearningprocessisverysimilartotheonedescribedby Fariaetal.(2009)over30yearslater: Asvehiclesforinstruction,businesssimulationsremainaspowerfultodayas theywerewhenfirstintroduced.Theyallowfordynamicbusinessdecision makingwhereplayersformastrategyandthencarryoutaseriesofdecisionsto implementthestrategy.Gameparticipantsreceivefeedbackthatdemonstrates theconsequencesoftheirdecisions,andtheparticipantsareabletoevaluate PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 34 theirstrategiesand,ifnecessary,reformulatetheirstrategies.Theexperience gainedfromtherepeatediterationsofdecisionperiodsprovidesdirectfeedback toplayers,fromwhichtheyareabletolearn. Whathaschangedisthecomplexityofthegamesresultingfromadvancesin computingpowerandtheavailabilityoftheinternet.Teamworkremainsimportant,but nowlessofaresearchinterestthandebriefing,as“theneedforgroupdiscussion...to understandandmanagethiscomplexityhasbecomegreater”(Fariaetal.,2009).Faria etal.note“thewhyofbusinessgameusagehasremainedremarkablythesameduring thepast40years”andalthoughtheresearchinterestin debriefinghasgrowntremendouslyinthepastdecade....thelearningintentof businesssimulationexercisescoupledwithfeedbackfromthestudentsasto whattheyhaveexperiencedandlearnedhasalwaysbeenacentralpartof businesssimulationgamingresearchthroughthedecades. Thissectionsummarizestheseminalargumentsforandagainstusingsimulation gamesasexperientiallearningactivitiesandreviewsguidelinesforadministeringa businesssimulationgameinanacademicenvironment. The case for and against simulations.Theuseofsimulationsasanexperiential teachingtechniquegainedpopularitybecausetheymadetheeducationalprocessmore effectivebyrelatingacademicactivitiestotherealworld(Heyman,1975).Simulations weave“substance‐specificinformationintoreal‐lifeproblemsinmeaningfulwaysthat studentscanunderstand”(Hertel&Millis,2002).Awell‐designedsimulationgamecan simultaneouslyteachtheoryandprovidepracticalapplicationpractice(Salas,Wildman, &Piccolo,2009).Assuch,theyaresuperiortootherlearningstrategiesfordeveloping complexcompetencies.Simulationsbasedonmodelsthatarecarefullyconstructedand realisticcanprovidestudentswithalaboratoryenvironmentthatcompressestimeand facilitatesexperimentingundercontrolledconditions(Meieretal.,1969).While acknowledgingthat“simulationscancreativelyfocusonexperimentation,prediction, andevaluation(Cunningham,1984),”HertelandMillis(2002)stressthe“general educationalgoalsof(a)transferofknowledge,(b)skilldevelopment,and(c)the applicationofbothknowledgeandskills.” PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 35 Educationalsimulationscanalsohelpbridgethegapsbetweendisciplines. Educationsimulationsprovideuniquestructuresforintegratingsubstantive principlesaswellasdoctrinesandskills.Thiscapabilitymakesthemparticularly usefulforcapstonecourses[emphasisadded]thatattempttobringtogetherthe essentiallearningobjectivesofmultiplecourseswithinanacademicor professionaldiscipline....simulationsshouldbeconsideredasacentral structurearoundwhichacapstonecourseevolves.(Hertel&Millis,2002) Anattractiveaspectofsimulationsisthatthey: createasocialenvironmentinwhichparticipantsbecomeinvolvedwith whateverresourcestheyhave.“Individualization”and“startingfromwhereyou are”happennaturallyinsimulations.Becauseparticipantswilljumpinattheir ownlevels,theselectionofanappropriatesimulationisnotasdifficultasthe selectionofappropriatebooks,films,orothereducationalmedia.(Heyman, 1975) Advantages.Studentsparticipateinsimulationgamesonafairlyequalbasis withoutregardforpasteducationandachievements.Thefunaspectofsimulation gamesmakesthemexcitingandmotivating.Withtheemphasisonaction,studentsare drawnintoanofteninterdisciplinaryanddynamicscenarioandgivetheirfullattention tomaking“almostreal”decisionsastheyplayaroleand“sometimesexcitedly[become] involvedwithothersinnegotiating,persuading,andresistingpersuasion”(Heyman, 1975).Thedynamicnatureofsimulationgameshelpsstudentsbridgethegaptoreality bymovingbeyondtheapplicationofconventionalwisdomandcookbooksolutions. “Issuesmustbetreatedontheirownmerits,alternativestrategiesmustbedevisedand attempted,resultsobservedandconclusionsdrawn,onthebasisofdirectexperience” (Taylor&Walford,1978).Theintegrativenatureofalarge‐scalesimulationgamemay providestudentswithagreaterclarityofthesystemsaspectsofacomplextopicthanis achievablefromthe“divideandconquerapproach...usedinmostlecture‐based courses”(J.W.Gentry,1990b). PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 36 Studentsreportaheightenedinterestinlearningaftercompletingasimulation game.Althoughthiseffectisdifficulttomeasure,manyeducatorsbelievethisis sufficientreasonfortheircontinueduseanddevelopment(Taylor&Walford,1978). Heyman(1975)notes: Thewheelinganddealingthatsimulationsrequiregivestudentsintensive practiceinverbalandwrittencommunication.Thiskindofcommunicationin simulationsdoesnotcausetheapprehensioninherentin“beingcalledon,”orin makingapresentation.Thisdiffusingofpublicperformanceisagreathelpfor theshyorquietstudent,whateverhisorherskills....Almostallsimulationscan berepeatedwiththesameparticipantsandtheywilllearnsomethingneweach time.Theknowledgegainedfromthepreviousplay(pluschanceaspectsofthe game)willmakethesimulationadifferent,yetstillproductive,experiencewhen itisusedagain. Learningfromsimulationsoccursatdiverselevels.The“highflyersare motivatedtoprogresstoevengreaterheightswithoutadverseeffectonthelessgifted who,inturn,learnfromtheirpeers”(Taylor&Walford,1978). Anotheradvantage(thatsomeeducatorsmayviewasathreat)isremovingthe student‐educatorpolarizationandchangingtheroleoftheeducatorfrom“sageonstage” tothatoffacilitator,andperhapsinterpreter,duringtheconductofsimulationgames. Theself‐monitoringaspectofmanysimulationgamesmakestheeducator’sroleto inspire,stimulate,motivate,andhelpstudentslearn;ratherthantoteach,direct,order andjudgeasstudentsmakedecisions,observetheconsequences,andmakefurther decisionsbasedontheirevaluationoftheconsequences(Heyman,1975;Taylor& Walford,1978). Simulationgamescanalsobeusedasdiagnostictoolsinconjunctionwithother instrumentsandexperientiallearningactivities.Forexample,studentswith3‐5yearsof workexperienceandanundergraduatebusinessdegreefromanAACSB‐accredited institutionattendingaone‐yearMBAprogramatBabsonCollege,participateina3‐day businesssimulationaspartofa2‐weekprogram‐beginningresidencydesignedto provideabaselineoftheirfunctionalskillsandidentifyspecificareasforimprovement. PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 37 Otheractivitiesduringthistwoweekresidencyincludeapersonalcareerassessment exercise,team‐buildingexercises,andtwogroupprojects(Schlesinger,1996). Disadvantages.Asidefrompotentiallimitedavailabilityandcost,somemayview theextratimerequiredtobecomefamiliarwiththegameandtointegrateitintothe curriculumasnotjustifiedbythebenefits.Anotherargumentagainstissomestudents maynotfindthemrelevant.Otherargumentsaresimulationstaketimetoconduct,may createdisruptivenoise,maynotbeeasytostageduetofacilitylimitations,andmaybe difficulttoaccuratelygrade.Animportantconcernisthat,whileemotionalinvolvement ofstudentsisgoodfromamotivationalperspective,emotionsmaygetoutofcontroland requiretheeducatortoinitiateamoderatingintervention.(Heyman,1975;Taylor& Walford,1978;Wolfe,1990). Views of simulation game users, former users, and never users.Supportingthese earlyobservationsbyHeyman(1975)andTaylorandWolford(1978),Fariaand Wellington(2004)surveyedbusinessschoolfacultymembersandfound,consistentwith Faria’s(1998)earlierresearch,that“across1,085respondents...30.6%werecurrent businessgameusers,17.1%wereformergameusers,and52.3%wereneverusers.”Of thecurrentusers,theyfoundadisproportionatelyhighnumberoffullandassociate professors.Theysuggestthattheadditionaltimerequiredtosuccessfullyincorporatea simulationintoacoursemaybeviewedasnon‐justifiablebyyoungerfacultymembers whoareundersignificantpressuretopublishorperish.Anotherreasongivenis youngerfacultymembersarelessfamiliarwithsimulationgamesandlesslikelytohave receivedpromotionalmaterialsfromsimulationgamesuppliers.Inthisstudy,Fariaand Wellington(2004)found: Thetopthreeperceivedadvantagesofsimulationgamesforstudentsarethe sameforcurrentandformergameusers–simulationsprovideexperientialor active,participativelearning;simulationsintegratethedifferentfunctional businessareas;andsimulationsallowfortheoryapplication.Never‐usersreport thetopgameadvantagesasprovidingexperientiallearning,realism,and allowingfortheoryapplication.Withrespecttoadvantagesforinstructors... simulationsprovideinteractive/dynamicexercisesandtheyallowfortheory application....Otherimportantsimulationgameadvantages[are]...simulations interestandmotivatestudents,gamesintegratethefirm’sfunctionalareas,[and] PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 38 simulationsprovideameasureofstudentcomprehensionandsubjectmatter understanding. Theprimarylearningobjectivegivenbycurrentusersforselectingabusiness simulationgamewastogivestudentsdecision‐makingexperience(Faria&Wellington, 2004);otherobjectivesincluded“integratingtheorywithpractice,introducingstudents tobusinessplanning,andgivingstudentsexperiencewithgroupdecisionmakingand teamwork.”FariaandWellingtonfoundthat75%ofallgameusersstronglyfeltthey achievedthelearningobjectivesusingsimulationgames.Never‐userscitedtherequired preparationtime,poorfitofknownsimulationgameswiththecourse,lackof informationaboutavailablesimulationgamesandpreferenceforanalternatepedagogy asreasonsfornotusingsimulationgamesintheircourses. Theoverallimpressioniseducatorsbelievetheadvantagesoutweighthe disadvantagesandsimulationsareavalue‐addedexperientiallearningactivity(Faria& Wellington,2004;J.W.Gentry,1990a;Hertel&Millis,2002;Heyman,1975;Jones,1987; Taylor&Walford,1978;Wolfe,1997).However,Wolfe(1990)cautions: Theuseofgamesbyinstructorsmaybeself‐servingtotheextentthatcertain instructorsmadethecoursesmoreinterestingtothemselves,madethemfeel theywereaccomplishingsomething,anditrelievedthemofhavingtodeliver lectures. Administration guidelines.Whatstudentsactuallylearnfromthesimulation experienceisafactoroftheirperceptionsoftheexperience,somethingtheinstructor cannotcontrol;however,theinstructordoeshavecontrolovertheexperiential environmentandthepedagogyusedtocreatethatexperienceandisresponsiblefor providingahigh‐qualityexperience(J.W.Gentry,1990b).Otherthantheimportanceof thepostsimulationgamedebriefdiscussionbeinghighlightedinconjunctionwiththe emergenceofcomputer‐basedsimulationgames,theguidelinesforusingasimulation gameasanexperientiallearningactivitydonotappeartohavechangedmuchsincethe earlytextsadvocatingtheiruse(Fariaetal.,2009).Accordingly,thissectionincludes someofthisearlyworkinthediscussionofthesimulationenvironmentandadvicefor directing,debriefingandgradingsimulationgameexperientiallearningactivities. PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 39 The simulation environment.Studentsengagedinasimulationareactivelearners whoshouldself‐discovertheconceptsandprinciplesbuiltintothesimulationmodeland experiencedduringtheactivity(Dukes&Seidner,1978).Intheiranthologydescribinga varietyofsimulationgames,DukesandSeidnersuggestthatoneofthebestwaysto insurelearningoccursasaresultofparticipatinginasimulationgameistodesignthe game“insuchawayastomakethedesiredlearningrequisitetoplayingthegame,andto provideforatleastsemicontrolledreinforcementforappropriateplayerbehavior.”This designshouldtakethepurposeofthegameintoconsideration.Ifcognitiveskill developmentisthegoal,acomputer‐basedsimulationand/orrigidsetofgamerules maybeappropriate;however,interpersonalexchangesmaybemoreappropriateifthe objectivesofthesimulationgamefallwithintheaffectivedomain(Dukes&Seidner, 1978). Jones(1987)concursinhishandbookontheuseofsimulationsbyteachersand trainers:“Intheactionpartofthesimulationthereisnoteacher....Asimulationmustbe anon‐taughtevent...ifitistaughtthenitisnotasimulation.”Further,teachersmust resistthetemptationtohelptheparticipantssucceed.Inasimulationgame,asinreal life,studentswilllearnfromtheirmistakesandinthecaseofasimulationgame, “mistakesareinevitableanddesirable...thegreaterthedisasterthegreaterthe learning”(Jones,1987).Jonesemphasizesthatforlearningtooccurit’simportantfor thestudentstobeperformingfunctionalrolesinthesimulationandforthemtohave sufficientinformationtoperformasaprofessionalinthatrole. Learningdoesnotnecessarilyneedtooccurduringtheexecutionofthe simulationandfrequentlyoccursaftertheeventduringthedebriefdiscussion.The postsimulationdebriefprovidestimeforadiscussionaboutwhathappened.However, stagingthesimulationgamewithdiscreetworkperiodsfollowedbyreflectiontimeprior toenteringthenextroundofdecisionshelpsenhancelearning.“Learningfrom experiencemustallowtimeforreflectiononthatexperience,andtheopportunitytotry again.Instantenlightenmentisnomoreanessentialfeatureofsimulationsthanitisof lifeoutsidetheclassroom”(Jones,1987). Learning objectives.UseofthebroadlearningobjectivesfoundbyFariaand Wellington(2004)maybecriticizedbythoselookingforspecific,measurableoutcomes becausetheyaregeneralinnatureanddifficulttoassess;however,“identifyingand PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 40 specifyingoutcomesofexperientiallearningisparticularlyproblematic”(Gosenpud, 1990).Gosenpudexplains: Usuallytheteacherordesignerofanexercisedefineswhatistobelearned. However,thelearneroftenlearnsthingsnotintendedbythedesigner,andoften thisunintendedlearningismorevaluablebecauseitisrelevanttothelearner. Theproblemforassessingexperientiallearningisthatevaluation,definedbythe designer,maymisstherealworthoftheexperientialexperiencebecausewhatis valuabletothelearnerisdefinedbythelearnerandmayhavenothingtodowith thedesigner’sintention. Inaddition,inexperientiallearningintendedoutcomesareoftenvague sincethefocusonexperientiallearningisusuallyonverycomplex,abstract phenomena. Jones(1987)agreesbroadlearningobjectivesmaybesufficientandiscriticalof the“Americanapproachtoeducation,whichtendstobebasedonstep‐by‐steplearning aimedatresultsthatcanbequantifiedbymeansofobjective(andfrequent)testing.” JonesdoesnotbelieveitisessentialforasimulationgameELAtohaveclearlydefined educationalobjectives:“Someofthemostfamousandeffectivesimulationsare educationallyambiguous,andtheprocessisusuallyfarmoreimportantthantheend product.”HertelandMillis(2002)agree:“Education—particularlyhighereducation— shouldbedesignedonthebasisofapplicationofknowledge,interactionwithideasand people,experience,feedback,andreflection.Simulationsarestructuredpreciselyon thesepremises.”Thepostgamedebriefdiscussionthereforeisacriticalelementofthe overallexperienceandservestoclarifywhatoccurredduringthesimulationgameand toassesswhatwaslearnedwithregardstothesevagueorself‐discoveredobjectives. Theimportanceoftheeducator’sroleinleadingthisdiscussionandtheneedforstrong discussion‐leadingskillsshouldnotbeunderestimated(Dukes&Seidner,1978). AndersonandLawton(2003)observethatmostsimulationgamesareusedafter studentsshouldhavelearnedthefundamentalconceptsofthediscipline,eitherearlierin thecurrentcourse;orinthecaseofacapstonecourse,inanearliercourse.Inbothof thesecases,thelearningobjectivesemphasizetheapplicationofthecourseconceptsina mannerthatistransferabletotherealworld.However,theyhavesuccessfullyuseda PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 41 simulationgameatthebeginningofanintroductoryundergraduatecoursewith alternateobjectives“toprovidestudentswithevidencethattheylackknowledgeoftools requiredofaneffectivemarketingmanager”and“tomotivatestudentstoacquirethe toolstheyneedtoimprovetheirperformance.” Directing a simulation game.Onceasimulationgamehasbeencarefullyselected tofulfilloneormorecourselearningobjectives,it’simportanttosharethelearning objective(s)withthestudentsandtoinformthemofhowtheirindividualperformance willbeevaluated.Aprereadoftheplayer’smanualcanbeassuredbyconductingashort quizpriortobeginningthegame.Researchhasshownthatstudentswholearnthe simulationbyreadingthemanualorfromtheirpeersperceivesignificantlymore transferorlearningasaresultofthesimulationexperiencethaniftheinstructor explainstheoperationofthesimulationtothem(B.W.Mayer,Dale,Fraccastoro,&Moss, 2011).Priortostarting,ashortdiscussioncanbeheldtoreviewthesimulationandits rules,clarifyconfusingoroftenmisunderstoodaspectsofitsoperation,andanswer proceduralquestions;ifthesimulationgamewillbeplayedbyteams,theidealnumber formostgamesappearstobefromthreetofive(Fritzche&Cotter,1990).Apractice periodconductedbeforeofficialplaybeginscanresultinstudentsreportingtheywere moreeffectiveandthesimulationgameexperiencewasmorevaluablethanifnopreplay practicewasallowed(Snow,Gehlen,&Green,2002).Heyman(1975)offersthisbasic ruleandguidancefordirectingthesimulation: Saynomorethanthefewwordsnecessary:beforethegame,tostartit;during thegame,tokeepitrunning;afterthegame,tokeepthediscussiongoing.The simulationitselftellstheplayerswhattheyneedtoknowinorderforthegame toworkeffectively.Runthesimulation,notthestudents....Youarenotthereto teach...thesimulationdoestheteaching.Youwillseeitduringthegame,and laterinthepostgamediscussion,whenthestudentswillrevealwhattheyhave learned.Anattempttoteachwillonlyinterferewiththesimulation....Agame director,unlikeateacher,doesn’tsayhowtobehaveinagame,nordoesheor shecommentonstudents’decisionsinorafterthesimulation. HertelandMillis(2002)agree:“Simulationsarenotteachingeventsinthe normalcontextofaninstructor‐studentrelationship.Inmostcases,onlybyremoving PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 42 theinstructorfromthedirectactionwillstudentsfullyassumetheirrolesandtake chargeofthescenarioaction.”Somewhatcountertothisadvice,FritzscheandCotter (1990)suggestitmaybeappropriatefortheeducatortoemphasizecertainprinciples thoughtheuseofsupplementalcasesorexercises;however,thismaybeintendedto occurbetweensimulationruns,notduringthem,andshouldnotconstitutecoaching. Debriefing.Thepostsimulationdebriefisanimportanttooltohelpassure learning(Fariaetal.,2009;Hertel&Millis,2002;Heyman,1975;Peters&Vissers,2004). Itcanbeconductedindividually,witheachteam,ortogetherwithalltheteams.The educatorlistensforindicationsoflearningandcanuseaseriesofpreplannedquestions relatedtothelearningobjectivestofacilitatethisdiscussion(Pate&Parker,1973). Conductingitwithalltheparticipantstogetherofferstheadvantageoflearningfrom eachotherratherthanfromamorelimitedperspective(Fritzche&Cotter,1990).When asimulationgameisusedasaneducationalexperientiallearningactivity,thedebrief discussionoftenfocusesonthestudents’performancesinthegame.Thejointdiscussion approachcanhelpmakethelimitationsofaparticularapproachsalienttothestudents, butit’salsoimportanttodiscusstheconnectionbetweenthegameandtherealworld (Peters&Vissers,2004). Scholarship gap.Crookall(2010)expressesconcernthatproperdebriefingisnot beingdoneasmuchasitshouldbe,“despiteanentireS&G[Simulation&Gaming] symposiumin1992beingdevotedtothetopic,guesteditedbyoneofdebriefing’smost articulateproponents,LindaLederman.”CrookallandFariaetal.(Fariaetal.,2009) believethescholarshipgapcreatingthisconcernischangingintherightdirection,buta searchofSimulation&Gamingshowsmostrecentarticlesmentioningdebriefingcitethe importanceofit,butdonotoffernotnewscholarshiponthetopic. Despitethegenerallyrecognizedassumptionthatthedebriefprocessisa primarilyfacilitatoroflearningfromsimulationgames,Lederman(1992)andPetersand Vissers(2004)foundlittlewrittenonthetheoreticalandconceptualunderpinningofthe debriefingprocess,andwhattheydidfindcamefromoutsidethedomainofsimulation gaming.Thisistroublingbecause: Feedbackiscriticalforproperlearningtotakeplaceafteranexperience.The studentshouldnotbeallowedtoconcludewhatwaslearnedwithoutreceiving feedback;thereistoomuchevidencethathumanbeingsdonotdothisproperly. PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 43 Thedebriefingsessioniscrucial[emphasisadded].Studentsneedtoarticulate theirperceptionofwhatwaslearned,andtheinstructorneedstoputthingsinto abroaderperspective.(J.W.Gentry,1990a) Learning process.Thedebriefingsessionshouldbeconsideredalearningprocess inandofitselfandtheroleoftheeducatoristoleadthestudentsthroughapurposeful discussionoftheirexperienceplayingthesimulationgame(Lederman,1984;Lederman, 1992).Itshouldbetreatedasadistincteventandseparatedfromtheplayingofthe simulationgame.Thiscanbedonebytakingashortbreakifbothoccurduringthesame classperiod.Thedebriefshoulddiscussthestudents’personalfeelingsaboutthe experience(e.g.,elation,embarrassment,anger),therolesplayedandwhathappened duringthesimulation,andwhattheylearnedfromtheexperience,includingwhatcanbe tiedtotheirrealworld(Hertel&Millis,2002). Twoassumptionsare(1)theexperienceaffectedthestudentsinameaningful way,and(2)processingofthatexperience,usuallyintheformofadiscussion,isneeded “toprovideinsightintothatexperienceanditsimpact”(Lederman,1992). Lederman(1984)describesthepostsimulationgamelearningprocessas: Astructured,guidedmethodforbringingmeaningtotheexperienceandfor learningfromthatmeaning.Itinvolvestalkingabouttheexperiences,analyzing them,evaluatingthem,andintegratingthemintoone’scognitiveandconscious database.Itisthepartoftheprocessinwhichthestudentsreflectuponthe experiences,andtheimplicationsofthoseexperiences,fortheworldexternalto theclassroom.Itisthepartoftheprocessinwhichwhatisbeingcreatedfor studentsisanewwayofseeingtheworld,ofperceivingitandmakingsenseofit. Theinstructorguides;studentscreateforthemselves,basedoninsights,and fromthisrelatewhattheyareseeingnowtowhattheykneworthoughtbefore. Lederman(1984)advisesthat“knowledgethatistheproductofexperienceis highlysubjective;itistheproductoftheinteractionbetweentheindividualandthe experience”and“ratherthanconsideringthe[postsimulationdiscussion]processa simpledebriefing,itisbetterdepictedastheCognitiveAssimilationofExperience PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 44 (Lederman,1983),orthatpartofthemethodologyinwhichwhatthestudenthas experiencedisintegratedintosomecognitivecomprehension.” Procedures and design.Debriefingsaregenerallyconductedasguideddiscussions withproceduresbasedonclasssizeandavailabletime.Withasmallnumberof students,everyonecanparticipateinthediscussion.Withlargergroups,volunteerscan besolicitedwithcaregiventonotallowafewpeopletodominatethediscussion.For shortsimulations,atleast25%ofthetotaltimeallottedtothesimulationplusdebrief maybeneededforthediscussion.Writtenreflectionontheexperiencemayalsobe requiredandthiscantaketheformofajournalwithentriesaftereachoftheactivity periodsinmultiphasesimulations(Hertel&Millis,2002). Regardlessofitsfrequency,writtenreflectionshelpstudentsintegratetheir simulationexperiencewiththeirpriorknowledgeandexperiencesandforce themtoorganizetheirthoughtsandemotions,enablingmoreproductive participationinagroupdiscussion.Instructorsrequirethesewrittenreflections formanyreasons:tocapturedetailsofwhathappenedandstudentrationalefor them,toallowstudentstoshareanyemotionsengenderedbythesimulation,and togivestudentsanopportunitytonotepersonalissuesorquestionsthatthey mightnotmentioninagroupsetting.(Hertel&Millis,2002) Thesewrittenreflectionscantaketheformofself‐diagnosticlearninglogswhich arereferredtoduringthedebriefdiscussionandcanalsobereviewedbytheeducatoras partofthegradingprocess(Hertel&Millis,2002).Thelogreportsanswerstoquestions suchas: (a)Whichcourseconceptsorprincipleswereusefultoyouinworkingthe currentproblemorissue?Wheredidyoulearnthemfrom?—fromindependent research?fromyourtextbook?fromotherstudents?fromthesimulationcoach? fromoutsideexperts?(b)Whichcourseconceptorprinciplethatyoupreviously learneddidthisnewconceptorprinciplebuildon?(c)Ifyouexperienced difficultyorwereunabletoworktheproblemorissue,whatinformationor knowledgewouldhaveenhancedyourabilitytoworkit?Wherecouldyouhave obtainedthisknowledge?(Hertel&Millis,2002) PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 45 Severalmodelsorframeworksareavailabletoassisteducatorswithdebrief discussiondesign(Hertel&Millis,2002;Heyman,1975;Lederman,1984;Lederman, 1992;Thatcher,1990;Thiagarajan,1992);thesearediscussedbelow. Heyman(1975)offeredsomesimpleadviceforfacilitatingthepostgame debriefingdiscussion:  Letitflow,letitgo.Atthestart,therewillprobablybetenminutesor morewhenitwillrangeallovertheexperience;allowthistohappen. Thesharingoffeelings–andreactiontoothers’feelingsisvery important....  Ifitisnecessary...tostartthediscussion,asktheplayerswhatkindsof feelingstheyexperiencedduringthesimulation...  Thepostgamediscussioncancovermanyquestionsconcerningthe simulationanditsrealworldcounterpart.  o Wasthegametruetolife(ingeneralandinspecificaspects)?... o Howwouldyoumodifythesimulationtoimproveit?...why?... Ifstudentsturntoyoufor“therightanswer,”yourbestbetistoreplyin ahighlytentativemanner.Afterall,therealworldiscomplex.These questionsareclearsignsofstrongmotivationtolearnmore;listen carefullyandusethatmotivationtoleadyourstudentsintomore activities....  Commentsonindividualorteamperformancewillnotcontributetothe students’learningorsatisfaction. LedermanandRuben(1984)developedanappliedframeworkforthesystematic assessmentofcommunicationgamesandsimulationsconsistingofthreecriteria: validity,reliabilityandutility.Lederman(1984)expandedonthisframeworktoinclude discussionquestionsrelatedtoeachofthesecriteria;validityisconsideredfromtwo perspectives,“facevalidity(correspondencewithrealworldcounterparts)andconstruct validity(correspondencebetweenconceptsandtheactivitydesignedtopresentthose concepts).”Reliabilityquestions“[guide]studentsthroughareviewoftheactivity:how ithappened,whattheydid,and,finally,theimplications.”Utilityis“acomparison PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 46 betweenengagementcostsintermsoftime,money,energy,andemotionalexpenditures versustheoutcomesorbenefits.”Evaluatingutilityinvolvesacost‐benefitanalysisofthe students’perceivedexpendituresversustheirperceivedbenefits.Ratherthanlecture, instructors“engageinastructureddialogwithstudents—adialoguedesignedtoelicit everypossibleconnectionbetweentheexperienceanditscognitivecomprehension.” Inalaterwork,Lederman(1992)reviewedtheliteratureondebriefingand presentedamodelforsystematicassessmentofthedebriefingprocessconsistingoffive setsofquestionsdealingwithlearningobjectives,situationalconstraints,debriefing strategy,implementation,andtheprocessingexperience. Similarly,Thiagarajan(1992)developedaseven‐phaseproceduralmodelfor debriefing,withthephasescategorizedbythequestions:“Howdoyoufeel?...What happened?...Doyouagree?...Déjàvu?...Whatwouldyoudodifferently?...What if…?...Canyouimprovethisactivity?”Kriz(2010)discusseddebrieffacilitationusing thisphasedapproachandofferedrelatedstructuredquestionsthatmightbesuggested foruseinthestudent’slearningdiary. HertelandMillis(2002)builtontheworkofLederman(1992,1984)and Thiagarajan(1992)andsuggestthesedebriefquestionsformultiple‐rolesimulations thatareeasilyadaptabletoteam‐basedsimulations: Dealingwithemotions: 1. Howdoyoufeel? 2. Howdidyoufeelwhen_______did_______? 3. Didyoufeelingschangeduringthesimulation? Describingsimulationaction: 1. Whathappened?Howdidtheactionunfold? 2. Whatwasthescenarioabout? 3. Whatwasthemajorinterestofyourrole? 4. Whatdidyoudo? Personalizingtheaction: 1. Whydidyoutakeacertainaction? 2. Howdid(anotherrole’saction)causeyouto(re)act? 3. Whatweretheimplications(ofyouraction)? 4. Whatwereyourgreatestsuccessesandfrustrations? PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 47 5. Whatwereyourgreatestobstacles;howdidyouovercomethem? 6. Whatwouldyoudodifferentlynexttime?Why? Applythesimulationtopastandfuturelearning: 1. Whatwerethekeyissues? 2. Howdidthescenariorelatetothetypicalissuesofthecoursediscipline? 3. Whatdidyoulearn? 4. Howwastheexperienceworthwhile? 5. Whatwouldhaveoccurredifotherdecisionshadbeenmade? 6. Whatwouldyoudodifferentlynexttime?Why? Applyingthesimulationtotherealworld: 1. Howdoesthesimulationcomparewithreal‐worldbehaviors?Give examples 2. Wasitpredictable?Why? 3. Whatreal‐lifeissuesweremissinginthescenario?Whateffectdidthis have? Thatcher(1990)describesdebriefingaswhere“reflectiontakesplaceandfrom whichchangewilloccur,becauseitisthepartoftheactivitywhichfocusesonthe complexprocesseswhichtookplaceineachindividualandinthegroupasawhole.” Thatcherrecognizedthatreflectionalsooccursduringtheplayingofthesimulation gameandadaptedKolb’s(1984)learningcyclediagramtodepictthatevaluationand reflectionoccursafterresultsareobtainedfollowingeachdecisionpointduringa simulationgame.Thislearningfromreflectionwhileplayingthegamediffersfromthe learningthatoccursfromthereflectionduringthepostsimulationdebriefdiscussion. Thatcher(1990)noted: Abt(1968)identifiedthreedifferenttypesoflearningwhichareoften presentinasimulation: 1. Learningthefacts,expressedinthegamecontextanddynamics(byfacts aremeantnotonlyfacts,concepts,andgeneralizationsbutskills) 2. Learningtheprocessessimulatedbythegame 3. Learningtherelativecostsandbenefits,risksandpotentialrewardsof alternativestrategiesofdecisionmaking PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 48 Missingfromthislistislearningabouttheinterpersonalskillsaprojectmanager needstobesuccessful,e.g.“leadership,teambuilding,motivation,communication, influencing,decisionmaking,politicalandculturalawareness,negotiation,trustbuilding, conflictmanagementandcoaching”(ProjectManagementInstitute,2013a),thatoften occurswhileplayingateam‐basedsimulationgame. Thatcher(1990)describedthedebriefingprocessasaseriesofstagesthatcanbe separateorintegrated.Thesearerelatedtodifferentkindsofknowledge,different learningstyles,andtothedifferentpartsofthereflectionprocess: 1. Identifyingtheimpactoftheexperienceoneachindividual—thisoften involvesthedevelopmentofself‐knowledge.Thismaybeovertorcovert, dependingonhowthreateningtheexperiencehasbeentoeachofthe participants.Forsomeoftheparticipantstheidentificationremainsa personalprocess,whichtheymayrevealatalatertimethantheactual debriefing. 2. Identifyingandconsideringtheprocesseswhichweredevelopedinthe simulation. 3. Clarifyingthefacts,concepts,andprincipleswhichwereusedinorrelated tothesimulation. 4. Identifyingthewaysinwhichemotionwasinvolvedinorfiguredinthe simulationforeachindividualandforthegroupasawhole, 5. Identifyingthedifferentviewswhicheachoftheparticipantsformedof thenatureoftheprocessesandtheexperience.Thisidentificationisthe processbywhichalltheparticipantsbegintoexplorethecomplexityof thesysteminwhichtheywereparticipants.(Thatcher,1990) Thatcher(1990)advises“eachofthesestagesisanimportantelementinthe processofreflectionbywhichthereallearningtakesplaceforeachindividual”and“the debriefingcanbeorganizedinanumberofways,fromaninformaldiscussion,througha structureddiscussion,tosomeformofwrittenreportorcommentaryontheexperience.” Hefindsitusefultodistributea“carefullydesignedresponsequestionnaire”foreach studenttocompletepriortothedebriefingdiscussion.Thishelps“ensureeachofthe PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 49 participantshasconsideredthesignificantpointsrelatedtothesimulationindividually beforeageneraldiscussiontakesplaceandsomeimportantpersonalexperienceislost intheheatofthegeneraldiscussion.” Thecommonthemeinallthesemodelsisstudent‐definedmultidimensional reflectivelearning.Althoughthestudentssharedacommonexperience,eachwilltake awayhisorherownpersonalizedconclusionsfromtheexperience.Theroleofthe educatoristoencouragethisself‐reflectionandtoenhanceitwiththereflectionofthe otherstudentsduringthedebriefdiscussion.Thiscanbeaguideddiscussionledbythe educatorusingpreplannedquestionsimmediatelyfollowingtheactivityoritcanrequire apredebriefdiscussionrefectionandpreparationperiodforthestudent/teamtoreview whathappenedduringthesimulationgameandtoreflectontheresultsandwhatthey learnedfromtheexperiencepriortodiscussingitwiththerestoftheclass.(Hertel& Millis,2002;Heyman,1975;Lederman,1984;Lederman,1992;Thatcher,1990; Thiagarajan,1992) Instructor skill.Forsimulationgame‐basedlearningtobepedagogically legitimateratherthanjustafeelgoodexperience,educatorsmustassistthestudentsin refiningtheirknowledgebasedontheexperience.Doingsorequiresexperience guidanceskillsthatgobeyondnormalclassroomskillstoenablethemtohelpstudents “articulateacognitiveunderstandingoftheexperientialactivitiesinwhichtheyhave engaged”(Lederman,1984).Theseexperienceguidanceskillsinclude“tolerancefor ambiguity,...abilitytoobserveandinterpretbehavior,...abilitytoformquestionsand listentoanswersaboutbehaviors,...abilitytoselectappropriatedirectivenessor nondirectivenessinworkingwithstudents,...asenseoftiming,andsoundjudgmentcalls (Lederman,1984).Whilethese“experienceguidance”skillsmayhaveapplicabilityin anyclassroom,Ledermanstressesthesearearequirementinanexperience‐based learningclassroom. Instructorswhousesimulationsarenotonlyresponsibleforthelearning outcomesofthoseexperiences,theyarealso“responsibleforthehumanoutcomes—the feelingsthatstudentshaveaboutthemselvesandothersasaconsequenceof engagementintheactivityanddiscussionofthatengagement”(Lederman,1984). Accordingly,instructorsmusttakeasupportiveratherthanjudgmentalapproachto createanon‐threateninglearningenvironment. PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 50 Assessment.Assessmentoflearningismorecomplicatedthanmerelyreviewing theresultsfromperformancemeasuresthatmaybebuiltintothesimulationgame.It’s verypossiblethatthelowestscoringstudentorteamlearnsthemostbecauseofthe adversitytheydealtwith,andinthecaseofteamactivities,whattheylearnedabout workingwithothersinstressfulsituations;therefore,theemphasisshouldnotbeon whowonbutonwhattheylearnedfromtheexperience(P.H.Anderson&Lawton,2007; P.H.Anderson,Lawton,&Wellington,2008;Chin,Dukes,&Gamson,2009;Dukes& Seidner,1978;Gosenpud,1990;Taylor&Walford,1978;Teach&Murff,2007).“Much moreresearchneedstobeconductedinthe‘whatislearnedbyplayingbusinessgames’ genreandmorespecificallythelinkbetweenthecomplexityofataskandlearning” (Teach&Murff,2009). Assessmentcanincludetheobservationofbehaviorandinterpersonalskills duringtheactivityandindividualretrospectiveinterviewsthatsupplementthegroup debriefaftertheactivity(Hertel&Millis,2002;Jones,1987).“Assessmentofallaspects ofthesimulation,includingactivities,learning,student‐studentinteractions,evaluation procedures,comfortlevels,andsoon,areessential”(Hertel&Millis,2002).This assessmentcancomefrommultiplesourcesincludingdatafromquestionnaires, uninvolvedclassroomobservers,focusgroups,andinstructorself‐reflection.Salas, Rosen,HeldandWeissmuller(2009)concurandofferasetof21performance measurementsbestpracticesbasedontheoriesofhumanperformanceandpractice methodsfoundintheliteraturethatcanbeconsideredwhenplanninganassessment strategy: Performancemeasurementusuallyworksbestwhen: 1. Itcapturesmultipledimensionsofperformanceatappropriatelevels ofanalysis.... 2. Event‐basedtechniquesareusedtocapturedataatmultiplelevelsof analysis. 3. Multiplemeasuresfromvarioussourcesarecaptured. 4. ASystematicplanisinplacetointegratedatafrommultiplemeasures.. .. 5. Itcapturestheprocessesofperformance.... PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 51 6. Expertmodelsofthetaskareusedasstandardsagainstwhichto compareandevaluateperformanceprocessesandoutcomes. 7. Thecollectionandtransmissionofobjectivemeasuresisautomated. 8. Measuresaredescriptiveofperformance. 9. Performancecanbecomparedtostandardsfordesiredlevelsof performance. 10. Itisdiagnostic—whenitprovidesinsightintothecausesofperformance.... 11. Itallowsforperformancediagnosistobepartiallyorfullyautomated. 12. Itallowsforperformanceevaluationtobepartiallyorfullyautomated.... 13. Thereisflexibilitydesignedintoembeddedmeasures(different measurescanbesubstituted).... 14. Itcapturesabroadspectrumofmeasuresandthecontextof performance.... 15. Observersaretrainedtohighlevelsofreliability. 16. Observersuseprotocols.... 17. Itsupportslearning.... 18. ItallowsfortheautomatedandmanualcreationofAAR[AfterAction Review]aidsfortrainingremediationandfeedback.... 19. Itenablesautomatedscaffoldingandperformance‐basedcoaching.... 20. Whenisdrivesreal‐timecorrectivefeedback.... 21. Itisintegratedwithtrainercontrolsandfeedbackgeneration.(Salas etal.,2009) AndersonandLawton(1988)offeradifferentperspective.Theylisted11 techniquesthatcanbeusedtoevaluateperformanceonasimulationexerciseand postulatedtheirabilitytomeasurelearningobjectivesaccordingtoBloom’staxonomy. Nineofthesetechniqueswereratedstronginonetothreeofthetaxonomycategories. InorderofprogressionfromstrengthinBasicKnowledgethroughObjectiveKnowledge, theyare:  Examsonsimulationrulesandprocedures  Examsonreadingsimulationoutput  Evaluationofawrittenplan  Abilitytopredictresults PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE  Performancerelativetoimplementationoftheteam’splan  Identificationofandrecoveryfrommistakes  Relativerankingonsimulationresults  Analysispaper  Oralpresentation 52 Theremainingtwoevaluationmethodslistedare:  Examsonconceptualissues  Peerevaluations AndersonandLawton(1988)notedtheabilityofanexamonconceptualissues tomeasurelearningobjectivesinanyofthesecategorieswilldependonitsdesignandis instructor‐dependent;peerevaluationswereratedmoderateinthelowerthree categories(BasicKnowledge,Comprehension,andApplication)andweakinthehigher categories(Analysis,ObjectiveSynthesis,andObjectiveEvaluation).Theyconcluded: Nooneofthesemethodsprovidesacomprehensivemeasureofhowcompletelya studenthasmasteredthelearningobjectivesforacourse.Somelearning objectivesarejusteasiertomeasurethanothers,justassomeknowledgeis easiertodemonstratethanothers....Manyofthesemethodsevaluate[team] performanceratherthanindividualperformance.Unlesstheinstructorchooses togivethesamegradetoallteammembers...somemethod(s)ofindividual performanceevaluationmustbeutilized....Acafeteriaapproach...would providethemostcomprehensiveappraisal.(P.H.Anderson&Lawton,1988) Inresearchassessingtherelationshipbetweenfinancialperformanceona businesssimulationandothermeasuresofstudentlearning,AndersonandLawton (1990)foundveryfewsignificantrelationshipsbetweenthesimulationperformance scoreandeightoftheothermethodsusedtoassesslearning,anobservationpreviously madeinotherstudies(Greenlaw&Wyman,1973;B.Keys,1977;Wolfe,1985).Thisled themtoconcludethatperformancescoresonabusinesssimulationarenotavalid measureofstudentknowledgeorcomprehension.Theyalsofoundlittlesupportfortheir priorpostulatedhierarchyofassessmentmethods,suggestingthestrengthofa particularmethodforevaluatingalearningobjectiveaccordingtoBloom’staxonomy maybesituationalratherthanabsolute.Thisdoesnotmeanlearningisnotoccurring, PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 53 onlythatnocorrelationbetweensimulationscoresandotherassessmentmeasureswas found.ResearchbyFariaandWhiteley(1990)foundthat“certaintypesoflearningcan beimprovedbyparticipatinginasimulationgame.”. Consistentwiththisadvice,multiplemethodsaretypicallyusedtoassessstudent performance.Inasurveyofsimulationusers,AndersonandLawton(1992)foundmost respondentsusedanaverageof4.4methodstomeasurestudentperformancein simulationgames.Theyfurtherfoundthesimulationgamesinuselastedfromeightto 16classperiods,primarilyinadvancedbusinesspolicy,management,ormarketing courseswithstudentteamsofthreetofivemembers.Onaveragethesimulationgrade countedfor30%ofthestudent’scoursegrade.Teamperformancerelativetotheother teamswasusedasanevaluationmethodinalmostallcoursesaccountingfor,on average,40%ofthesimulationgrade.Thiswasfollowedbyevaluationoftheteam’s writtenplanin77%ofthecaseswithanaveragegradeimpactof16%andateam‐ writtenpaperanalyzingitsperformancein62%ofthecaseswithanaveragegrade impactof23%.Noneoftheothermeasuresaccountedformorethan17%ofthe simulationgrade.Peerevaluationwasusedtheleast(13.9%)andonaverageaccounted for13.8%ofthesimulationgrade. HertelandMillis(2002)stresstheimportanceofembeddingassessmentsinto thesimulationexperience.Thesecanbeoralpresentationsand/orwrittenpaperssuch asmanagementbriefingsandbusinessproposalsthatcanbeassessedandgradedina mannersimilartootherassignments.Theyidentifyninecriteriathatmaybeconsidered forgradingstudentperformance: 1. Demonstratedunderstandingofsubstantiveissues. 2. Understandingandproperuseofprocess. 3. Representationofroleinterests. 4. Demonstratedinitiative. 5. Qualityofwrittenwork. 6. Qualityoforalpresentations. 7. Demonstratedabilitytoworkwithothers. 8. Demonstratedleadership. 9. Effectivetimemanagement.(Hertel&Millis,2002) PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 54 Grading.Thediscussionontheimportanceofthepostgamedebriefemphasized itsroleinhelpingstudentsself‐discoverwhattheylearnedfromtheexperience.Tothe extentthattheylearnedorreinforcedconceptsortechniquesthatcanbetested,this testingisbetterdonelaterasaseparateactivityandthesimulationgameactivityitself shouldbeungradedorgradedonthebasisofthethoughtfulnessofthefeedback provided(Chinetal.,2009;Heyman,1975).Further,“simulationcoursesshouldinvolve noncompetitivegradingpractices:Studentspittedagainstoneanotherhavenoreasonto cooperatetoenhancethelearning–andhencethehigherachievement–offellow students”(Hertel&Millis,2002). Chinetal.(2009)recommendconductingashortungradedsurveyatthe conclusionofthesimulationgameandbeforethedebriefdiscussionto“capturethoughts fromtheparticipantsbeforetheyare‘contaminated’bygroupdiscussion.”Theyfurther notethatthecomparisonofdatafromapretest‐posttestapproachcanprovideamore directassessmentoftheimpactandthatthenatureofthequestions,skills‐basedor attitudinal,willdependontheobjectivesoftheactivity. HertelandMillis(2002)cautionagainstgradingstudentsindividuallyfortheir participationduringgroupactivitiesbecauseofthedifficultyinvolvedwithmonitoring allgroupactivityallthetime.Theysuggestpeerandself‐assessmentcanbeusedto determineafairgradeclaiming“peerassessmentcanbecomemoremeaningfulwhen studentshaveinputandownershipovertheprocess”andcanbeimplementedbyhaving studentscompleteafeedbackformconsideringtaskperformanceandgroupskillsafter allgroupmeetings. Whenitcomestoassigninggrades,Heyman(1975)cautions: Itisimpossibletogradesimulations,andifonetiesthescoresgainedinthegame togrades,thesimulationitselfissignificantlychanged.Inanycasegamesare excellentmotivators,sothatonepurposeofgradingisalreadyfulfilled. HertelandMillis(2002)citeresearchpreferringcriterion‐referencedgrading where“allstudentswhoreachaspecifiedlevelofcompetencyearnthedesiredand appropriategrade”and“amasteryorientationtowardlearningcanpromotepositive motivation.Studentsneedasenseofcontroloverthefinaloutcome.Theyshouldfeel PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 55 theyhaveresponsibilityfortheirowngradesandcantakepositivestepstoimprove them.” Rubricscanbedevelopedtoaidingradingsimulationgameartifactsand presentations(Hall&Ko,2008).BasedonaStateofFloridarequirementforaproject managementstudentlearningobjective(SLO)inacurriculum,onewasdevelopedfor evaluatingtheprojectmanagementskillsandpracticesemployedincompletinga capstonecoursesimulationproject(Hornyak,Peach,Bowen,Moes,&Wheeler,2006). Summary.Thissectionpresentedtheadviceforadministeringsimulationgames inaneducationalenvironmentandhighlightedtheindividualnatureofthelearningthat occurs.Inadditiontolearningbydoing,individualreflectionisimportant.The concludingdebriefdiscussionhelpsmakethislearningsalient.Itisdifficulttograde learningfromexperientialactivitiesandtheuseofmultipleassessmentmethodsis recommended.Mostsimulationgamesaremoreorientedtotheaffective,perceptual andbehavioralmodesoflearningratherthantowardthesymbolicmode;therefore, assessingtheeffectivenessofasimulationgameexperiencewilllikelybemore subjectivethanobjectiveandcanbenefitfromtheuseofgradingrubrics. Simulation Game Effectiveness Research Researchandevaluationintotheeffectivenessofsimulationgamesisanareaof controversy,bothinthesimulationliteratureandforeducationingeneral,especially whenconsideringwhetherthemethodologyandexperimentaldesignisappropriateand sufficienttojustifytheconclusions(Jones,1987). MuchoftheliteratureonassessmentfoundinSimulationandGamingwas writtenduringthe1970sand1980stoprovetheeducationaleffectivenessofsimulation games(Chinetal.,2009;Graf,2001).Recentarticleshavediscussedthelackof standardsfor,anddifficultyof,measuringtheeffectivenessofsimulationgamesas experientiallearningactivities(P.H.Anderson&Lawton,2009;Gosen&Washbush, 2004). GosenandWashbush(2004)reviewedresearchandscholarshipdealingwith assessmentofexperientiallearningapproachesandfoundattemptstoassessthevalidity ofexperientiallearningandtomeasuretheeffectivenessoftheapproach.Whilethey foundempiricalresearchsupportingboththevalidityandeffectivenessofexperiential learning,theyalsofound“along‐standingtrendofnotmeetingthehighestofresearch designandmeasurementstandards.”Althoughrigorousstudiesexist, PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 56 theredoesnotexistenoughconsistentresearchfrommethodologicallysound studiesacrossmultiplegamestoconcludesimulationsarevalid....Thisisnotto saysimulationsarenotvalid,butinsteadthereisnotenoughevidenceto concludetheyareactuallyvalid(Gosen&Washbush,2004). Whiletheyagree“learningisacomplexconstruct,hardtopindownand thereforedifficulttomeasure,”GosenandWashbush(2004)believeassessmentof effectivenessisimportantandcallformorerigor;however,Jones(1987)suggeststhis maynotbepracticalasmanyauthorshavepointedoutthedifficultiesoftesting hypothesesrelatedtoshowingthat“simulationsproducegreatergainsincritical thinking,decision‐makingandproblem‐solvingthandootherlearningmethods,orsome suchsimilargeneralization.”Forexample: DavisonandGordon(1978)pointoutthatnoevaluatoryinstrumentscanreadily encompassthemanydifferentdimensionsofbehaviourandexperienceinvolved, andTwelker(1977)emphasizestheproblemscausedbythegreatdifferences betweenindividualsimulationsandbetweentheconditionsinwhichtheyare used.Inthejargonofresearch,thereareboundtobeagreatmanyuncontrolled variables.(Jones,1987) Thecallformorerigorousresearchisnotnew.AndersonandLawton(1997) observedthatmostsimulationgameresearchisbasedontheconstructsofperceptions andattitudesfordependentvariablesbecauseweknowhowtomeasurethemand advocatedfortheuseofmoreobjectivedependentvariablesandcontrolmeasuresto accountfortheinfluenceofmoderatingvariablestoimproverigor.Theysuggesttwo modelstoestablishlearningoccurs:(1)pre‐postdesignsmeasuringchangesincognition orbehaviorforeachindividualparticipantand(2)after‐onlywithcontrolgroup experimentsexposingparticipantstoalternatepedagogiesandcomparingtheaverage difference. Theoperationalizationof“learning”posesdifficultmeasurementproblems, however,manydifferentmeasureshavebeenusedtorepresent“learning,” PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 57 including:simulationgameorexerciseperformance,objectivetestperformance, coursegrade,andself‐reports.Infact,thedefinitionandmeasurementof learningremainsanelusivegoal.Confoundingoccursinmostinstances.For example,simulationorexerciseperformancemeasuresbothknowledgeofthe conceptsandanunderstandingofthegameitself.Theuseofstandardizedtests dealingwiththegeneralcoursetopicremovesthisproblem,butintroduces others.Onesuchproblemoccurswhentheteachingmethodologyisusedwith oneobjectiveinmind,andthetestmeasuresperformancenotdirectlyrelatedto it.(J.V.Gentry&Burns,1981) GreenlawandWyman(1973)found“thedevelopmentanduseof[business] gamesexpandedenormously”intheyearsfollowing“introductionofthefirstpractical businessgamein1957.”Theyreviewedresearchstudiesthatfocusedon“gamelearning tomeetcourseobjectives”andfoundthat“althoughgameshavebeenanextremely popularandwidespreadteachingtool,verylittle‘hard’researchhasbeendoneon gaming–especiallyconcerningwhatplayerslearntomeetcourseobjectives.”They observed,“Developingafirst‐ratelearningresearchwhereboth‘soft’and‘hard’ variablesarenumerous(asingames)isanextremelydifficulttask.”Further, Asseveralauthorssuggested,gamesmaybemoreeffectiveinteachingcertain “intangible”concepts,suchasMcKenny’s(1962)planningconceptsorStrotheret al.’s(1996)“awareness,”thaninteachingexplicitfactsandrelationshipsas foundbyMoore(1967).Thisraisesacriticalpoint:stronglearningingamesmay notnecessarilybereflectedbygoodgamingperformance;conversely,absenceof learningmaynotalwaysbereflectedby“poor”performance.(Greenlaw& Wyman,1973) Keys(1977)examinedwhatheconsideredmain‐streambusinesssimulation gamesresearchbyonlyreviewing“articleswhichutilizedefinitecriteriaforthe measurementoflearningandprofessionallyacceptableresearchtechniques.”Hefound addingabusinesssimulationgamecanprovidesuperiorresultscomparedtocourses withouttheiruse,butgame‐onlyclassesweresuperiortocase‐onlyclassesonlywhen “significantinstructorguidanceisprovidedinthegame‐onlyclass.”Incontrasttomore PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 58 recentadviceintheliteratureonlettingstudentslearnfromtheexperiencewithout interferencefromtheinstructor,headvocatedtheinstructorshould“playanactiveand significantroleinguidinggamelearning.”Thisdifferingadviceislikelytheresultof Keysonlylookingatstudiesthatmeasuredlearningbyperformanceonwritten examinationsandcasestudyessaysandnottakingintoconsiderationthebehavioraland intangibleaspectsofassessment. Stainton,JohnsonandBorodzicz(2010)advocateforassessinglearningbasedon “datasourcedfromablendofvariedresearchinstruments,suchaswrittenreflective accounts,questionnaires,semistructuredinterviews,anddirectobservation.” Inarecentattempttocorrelatesimulationgameperformancescoreswith learning,andperhapsaddingfueltothecontroversyabouttheuseofsimulationscores tomeasurelearning,GosenpudandWashbush(2010)conductedanexploratorystudy correlatinggradesearnedonareflectivepaperexplainingwhythestudentscoredas theydidonthegamewiththegameperformancescoresandfoundasignificantand positivecorrelation.Thisfindingisbasedononestudyof28studentsusingatotal enterprisesimulationandmoreresearchissuggested. Withthisforewarningaboutthedifficultyandcontroversyofevaluatinglearning fromsimulationgames,thenextsectionsexamineexamplesoftypicalmethodologies usedtoevaluatetheeffectivenessofbusinesssimulationgames. Postsimulation perceptions.MostoftheresearchIfoundwasbasedonasurvey ofstudentperceptionsandattitudesatthecompletionofaneventorcourse.While studentperceptionsandattitudesarenotproofoflearning,theycanservetomeasure satisfactionwhichcanserveasaproxyfortheperceivedvalueoftheeventorcourse. “Studentattitudesareoftenpostulatedasaninterveningvariablebetweenthepedagogy andlearning;highlymotivatedstudentsarelikelytolearnmore”(J.V.Gentry&Burns, 1981).Thesesurveysareusedbysimulationgamingresearcherstocombineperception responsesintovariousconstructsmeasuringperceptionsofknowledge,confidence, teamdynamics,simulationexperiencesatisfaction,andlearningmodepreference. Resultsarereportedand,insomecases,correlatedwithdemographiccategoriesand/or simulationperformancescoresand/orpostsimulationcourseexamstoshowsupportfor relatedhypotheses.Asexemplifiedbelow,methodsrangefromasimplecalculationof theconstructmeanstorigorousstatisticalanalysisofthedata.Insomecases,qualitative PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 59 dataisalsoobtainedfromresponsestonarrativequestionswhicharethencategorized, analyzedandusedtotriangulatefurthersupportfortheresearchhypotheses. Qualitative approach example.Green(2004)analyzedtheresponsestostudent writingsonthequestions:“WhatdidIlearn?WhatdidIlearnthatIneededtolearn? WhatdoI‘take‐away’fromthisexperience?”andfoundstudentsvaluedteam experience,realizedtheimportanceofhavingastrategy,appreciatedtheintegrationof thecoursematerial,andacknowledgedmoreself‐awarenessabouttheircompetencies andrisktolerance. Quantitative approach example.Buzzetto‐MoreandMitchell(2009)useda postsimulationsurveytoassessstudentimpressionsaboutthesimulationexperience anditsimpactonlearning.Inthisstudy,studentteamscompetedagainstteamsfrom otheruniversitiesandindustrybyplayingsixroundsoftheCapstone®Business Simulation.Studentsalsocompletedonesimulationroundasindividuals,keptadiary wheretheyreflectedontheirdecisionsandprocesses,andweresubjecttoapeer assessmentandrelatedcomprehensiveexam.Apostsimulationonlinesurveywas administeredwithperceptionquestionresponsesusinga5‐pointLikert‐typescale rangingfromstronglydisagreetostronglyagree.Themeanresponsetoeachquestion wasreportedinthearticlewithitsstandarddeviation.Thereportedmeansrangedfrom 3.2to4.04andweredeemedproofofafavorablereaction(therewasnoindicationof significancetestingcomparedtoaneutralresponse).Buzzetto‐MoreandMitchellalso comparedtherankingofstudentteamsbasedonthesimulationperformancescoreand concludedtheyperformedsuccessfully.Resultsfromthecomprehensiveexamwerestill underanalysisandnotdiscussed.Whilethisstudyprovidedinformationonstudent perceptions,itdidnotassessiftherewasanydirectimpactonlearning.Thequestions usedformeasuringstudentattitudesaboutsimulationscouldbeusefulinafurtherstudy onstudentperceptionsofsimulationgames. Quantitative approach example.AdoborandDaneshfar(2006)usedanalysisof postsimulationsurveyresultstoexploretherelationshipbetweenperceivedlearning andperceptionsofsimulationrealism,simulationeaseofuse,teamtaskconflict,team emotionalconflictandsimulationperformancescore.Allsurveyquestionsusedaseven‐ pointLikert‐typeratingscaleandthecontinuoussimulationperformanceresultswere codedfromonethroughseven,e.g.ascorefrom60to64wascodedaoneandascore from96‐100wascodedaseven.Constructswereformedfromthequestionsfor PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 60 learning(21questions),taskconflict(4questions),emotionalconflict(4questions),ease ofuse(3questions)andrealism(7questions).Cronbach’salphawasabove0.91forall theconstructsexcepteaseofusewhichwas0.77. Forty‐nineteams,withatleastfourstudentsperteam,attendingasenioryear undergraduatebusinesscoursespentanaverageofsevenhoursperweekplayingThe BusinessStrategyGameoverthecourseofasemester.Eachteammanagedasimulated businessintheathleticfootwearindustry,hadtheoptiontomanufactureandsellonfour continents,hadplantstooperate,aworkforcetomanage,andhadtodealwithinventory, distribution,salesforecasting,andcurrencyexchangefluctuations.Anaverageof2.2 studentsperteamvoluntarilycompletedthequestionnaireforatotalof109studentsin thesample.Thesimulationprojectcomprised60%ofthecoursegrade;therewereno incentivesorpenaltiesassociatedwithparticipatingornotinthesurvey.(Adobor& Daneshfar,2006) Afactoranalysisofthelearningconstructrevealedthreefactorsrelatedto problem‐solvingskills,teamworkskills,andself‐as‐managerskills.Adoborand Daneshfar(2006)usedthesefactorsinaregressionanalysiswiththeotherconstructsto observe:  Realismwaspositivelyassociatedwithallthreedimensionsoflearning supportingthehypothesisthatahigherlevelofperceivedrealismis associatedwithahigherlevelofindividuallearning  Realismwasnotsignificantlyassociatedwithgroupperformance  Easeofusewaspositivelyassociatedwiththemanagerialproblem‐ solvingdimensionoflearning,butnotwiththeteamworkandself‐as‐ managerlearningdimensions  Easeofusewaspositivelyassociatedwithgroupperformance  Taskconflictwaspositivelyassociatedwiththeteamworkandself‐as‐ managerlearningdimensionsoflearning,butnotwiththemanagerial problem‐solvingdimension  Taskconflictwaspositivelyassociatedwithteamperformance  Emotionalconflictwasnegativelyassociatedwiththemanagerial problemsolvingdimensionoflearningindicatingthatahigherlevelof emotionalconflictisassociatedwithlowerlevelsofindividuallearning PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE  61 Therewasnosignificantrelationshipbetweenemotionalconflictandthe teamworkorself‐as‐managerdimensionsoflearning  Higherlevelsofemotionalconflictwereassociatedwithlowerlevelof groupperformance  Groupsize,demographics,andattitudetowardgroupworkcontrol variableswereallshowntonothaveasignificanteffectontheresults Thisstudyisanexampleofusingrigorousstatisticalanalysistechniquesto analyzeperception‐baseddata.Whiletheresultsdonotprovethattransferablelearning occurred,theydosuggestthatrealismanduser‐friendlinessinasimulationare importantfeaturestoconsiderandgroup/teamdynamicsplayanimportantroleinthe successofusingsimulationgamesinanacademicprogram. Quantitative approach example.Citingthemixedresultsofpriorresearchby others,Anderson(2005)exploredtherelationshipbetweenindividualperceptionof teamdynamicsandstudentaffectforthesimulationexerciseandtheinfluenceofteam dynamicsonsimulationperformanceresultswithanintroductorymanagementclass containing220students.Studentswererandomlyassignedtothree‐personteamsand playedtheCapsimFoundationsimulationgameforfourweeksduringthesemester. Studentsinputcross‐functionalbusinessdecisionsinroundsrepresentingoneyearof operations.Studentsreceivedextracreditontheirfinalexamiftheychosetocomplete thesurvey;172studentsprovidedusableresponsesindicatingtheirlevelofagreement with21statementsusinga5‐pointLikert‐typescalefromstronglydisagreetostrongly agree.Thedependentvariable,studentaffectforthesimulationexperiencewasbased ononeitem:“IlikedtheCapsimsimulation.”Theotherdependentvariable,simulation performance,wasa10‐pointscalebasedonaweightedcombinationofsimulation generatedbusinessmetrics.Independentvariableconstructswerestudent’sperception ofgroupcohesiveness(2items),teaminterdependence(5items),teamheterogeneity(3 items),teamopportunisticthinking(5items)andteamhypothesis‐driventhinking(5 items).Researchhypothesesweretestedrelatedtotheimpactofstudentperceptionsof teamcohesion,teaminterdependence,teamheterogeneity,teamopportunisticpractices andteamhypothesis‐driventhinkingonstudentaffectforthesimulationgameand simulationperformance.Studentpriorworkexperience(5‐pointscale)anddegree PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 62 major(businessornon‐business)wereusedtocontrolfortheireffectswhiletestingthe hypothesesusingtwo‐stephierarchicalregression.Anderson(2005)found: Studentswhoperceivetheirteamstobecohesiveandindependenthavestrong affectfortheexercise,butthisdoesnottranslateintostrongperformance. However,studentswhoperceivetheirteamstohavelowcohesiveness,low heterogeneity,highopportunisticpractices,andhighhypothesis‐driventhinking experiencehigherperformance. Thisresearchprovideseducatorswithinsightintotheeffectsofteamdynamics onstudentaffectforasimulationexerciseandonteamperformanceasmeasuredbythe simulationgame.Itdidnotassociatesimulationperformancewithlearninganddidnot discussthestudentaffectmeanscoreof3.1andstandarddeviationof1.35.Ifthe distributionwasnearnormal,manystudentswereindifferentontheiraffectforplaying thegameor,ifbimodal,therewerenearlyasmanystudentsthatdidnotlikethe experienceastherewerethatdid.Sincemostresearchersfindthatstudentslike simulationgames,furtherresearchanddiscussionofthisresultwouldbebeneficial. Quantitative approach example.BaglioneandTucci(2010)usedanalysisof postsimulationsurveyresultstoconcludethatPharmaSim,amarketingcomputer simulation,improvedtheunderstandingofmarketingconceptsfor130undergraduate studentsfromfiveclasses.Studentperceptionswithsurveyedwitha26item questionnaireusinga10‐pointExcellent(10)toPoor(1)scalefortheresponses. Individualquestionresponsesweretestedforsignificanceusingaone‐samplet‐test againstthescalemidpoint(5.5)representinganeutralresponse.Allquestions,except two,yieldedsignificantandfavorableresponses.Studentswereneutralonthetwo questionsregardingPharmaSimbeingexcitingandfun. Questionsweresummedbycategorytocreateconstructsforoverallsimulation evaluation,simulationdescription,simulationknowledge,andinstructor knowledge/integration.TheseconstructswereusedinLatentClassClusterAnalysis where48%ofthestudentswerefoundtohavehadanoverallpositiveassessmentofthe simulation,includingitseducationalvalue;founditexciting,challengingandfun;and believeditincreasedtheirknowledgeinallthecategories.Thesecondcluster representing40%ofthestudentswereneutraloverallandontheeducationalvalueof PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 63 thesimulationandfounditchallenging,butunexciting,unpredictable,hard,andnot enjoyable;theyreportinggainingknowledgeinonly2of13learningareas.Thefinal clusterrepresenting13%ofthestudentsevaluatedthesimulationpoorlyandforits educationalvalue.Furtheranalysisfoundthatthisclusterhadlessinstructorinteraction thantheothers.(Baglione&Tucci,2010) BaglioneandTucci(2010)concludedthat“studentsimprovetheirknowledge andevaluatethesimulationexperiencehigherwhentheyandfacultyspendadequate timeonit.” Thisexampledemonstratesamethodofsignificancetestingthatisnotroutinely seeninthedataanalysisofothersimulationgameresearchstudies,perhapsbecausethe scoresinthosestudiesarenotnearthescalemidpointandsignificanceisassumed.The useofclusteranalysisdemonstratedtheriskofrelyingonaveragestoassumethat everyonerespondedfavorably. Mixed method example. LainemaandLainema(2007) examinedtheuseof RealGameinacorporatelearningenvironmentwithparticipantsselectedfrommiddle management.Althoughthiswasnotanacademicprogram,theparticipantsweretypical ofwhatonewouldfindinanexecutiveMBAprogram.Thesimulationisdesignedto modelthebusinessoperationsandfunctionsofamanufacturingcompanyinan environmentwithuptoeightcompaniescompetingformarkets,suppliersandfunding. Eachcompanywasmanagedbyateamofthreeparticipants.Thepurposeofthetraining was“toproduceanauthenticdecision‐makingenvironmentandtogiveaholisticviewof howabusinessorganizationworkstoproduceprofit”(Lainema&Lainema,2007). Followingthesimulation,participantscompletedapostsimulationsurvey comprisedof13closedquestionsusingsevenpointLikert‐typescales(Poor/Excellent andDisagree/Agee)andeightunstructuredopen‐endedquestions.Mostofthe unstructuredquestionswereansweredfirst.Themeandatawasreportedfortheclosed questionstodemonstratethefavorableimpressionsoftheparticipants.These impressionsweresupportedbyquotationsfromtheopenquestionsleadingtheauthors toconclude:“Ifwewishtostrengthenthebusinessknowledgeofkeypersonnelin companies,RealGame,andsimilartypesofsimulation,seemstoofferawell‐receivedand anefficientlearningtoolforit,”adding,“Whileitisdifficultifnotimpossible,tomeasure whatpeoplehavelearned,wecanaskwhatusetheythinktheywillhavefortheirnew skillsandknowledge”(Lainema&Lainema,2007). PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 64 Mixed methods example.Inhisdoctoraldissertation,Graziano(2003)useda mixedmethodapproachofsurveysandfocusgroupinterviewstofind“remarkable consistencyintheperceptionsofstudents,thattheymakeprogressintheidentified learningoutcomesand,indoingso,developthecontentknowledgeandpersonal developmentskillsthatenhancetheirpreparationfortheworldofwork.”Hisstudy evaluatedtheuseofVirtualEnterprise,“aninterdisciplinary,technology‐driven, simulationprograminwhichstudentsdevelopandmanageglobalenterprisesina virtualbusinessenvironment.”Objectivesforthisuseofthesimulationincluded providinganopportunityforstudentstoseetherelationshipbetweencoursework learningandtheworkplace. TheVirtualEnterprisesimulationwasoriginallyintroducedintotheTourismand HospitalityDepartmentcurriculumatKingsboroughCommunityCollege(NewYork)on acceptanceofthepremiseofthevalueofexperientiallearning.Adaptationsweremade toreflectthebackgroundsofthestudents.Studentswereaskedtokeepaweeklylogand identifycompetenciesrelatedtoeachactivityperformed.Attheconclusionthey providedanarrativestatementoftheactivitiestheyperformedandthelearningthat resulted.Forthefirstseveralyears,programevaluationconsistedofanecdotalevidence andstudenttestimonialsastoitsvaluefollowedlaterbyaslightlymoreformal assessmentoftheprogram’slearningoutcomes,basedprimarilyonfacultyobservation anddiscussionswithstudents. Graziano(2003)usedaquantitativesurveyofpostsimulationperceptions (reportedasmeanscoresandpercentperceivingmoderatetoverygreatprogress) supplementedbyqualitativeinterviewdatatotriangulateandfindstudentsagreedthey madeprogressrelativetothoselearningoutcomestoanswertheresearchquestion: “Whataretheperceptionsof...studentsabouttheirlearning...”Anadditionalthree researchquestionsexploredcorrelationsbetweentheperceptionresultsand demographicfactorssuchastakingtheprogrammorethanonce,beingtaughtby differentinstructors,attendingremediationclasses,courselength,passingareading, writingand/ormathematicsexamination,andwiththecoursegradeandstudentGPA. ANOVAwasusedtoexplorethecorrelationsand,whileinteresting,theseresultswere ancillarytothequestionofoveralleffectiveness. Longitudinal methods/pretest‐posttest designs.Avarietyoflongitudinalmethods, mainlyinvolvingpretest‐simulation‐posttestapproaches,arealsofoundintheliterature PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 65 onmeasuringtheeffectivenessofbusinesssimulationgames.Mostusea preexperimentalpretest‐postdesignandsomeuseanexperimental‐controldesignto demonstratesignificantgainsaftercompletingthesimulation(Campbell&Stanley, 1963).Whiledesirable,theuseofacontrolgroupisoftennotpracticalbecauseitwould denysomestudentsthefulllearningexperienceencounteredbytheexperimentalgroup. Studentattitudesandperceptionsareoftenmeasuredinthisresearchand,dependingon thenatureoftheresearch,maybecorrelatedwithobjectivemeasurementssuchas cognitiveexamscoresandthesimulationperformancescores.Manyusemixedmethods triangulationbyincludingaqualitativeanalysisofstudentresponsestoquestionsabout theexperience.Thefollowingpretest‐posttestexamplesarecategorizedbythenatureof theprimarydependentvariableusedintheresearch. Simulation performance score.Gamlath(2009)foundbusinesssimulationgame performancewasduetoskillandnottoluckbycomparingtheresultsofalearning roundandanexamroundfortwodifferentsimulationsusedintwosequentialcourses withthesameteamsplayingeachgame.Allstudentsparticipatingreceivedapassing gradeforsubmittingacceptableplanningdocumentsanddecisionjournals,andwere awardedextrapointsbasedontheirsimulationperformancescores.Thesimulation graderepresented10%ofthecoursegrade.Gamlathobservedthatallteamsperformed similarlyinthebeginning;thehigherperformingteamstendedtoexperimentmore duringthelearningroundandaskedtheinstructorfewerquestionsthanthelower performingteams.ThisissimilartothefindingsofDavidovitchetal.(2006;2007;2008; 2009;2010)wheretheyfoundsimulationperformancescoresweresignificantly improvedbyprovidingtheabilitytoreviewandredopastdecisions.Thesimulation performancescorecanbeareliablemeasureoflearningifthegame‐playingskills developedaretransferabletorealworlddecision‐making.Gamlathalsofoundno correlationbetweenstudentperformanceonthesimulationandothertraditionalforms ofassessmentleadinghimtoconcludethatdifferentskillsarebeingassessed. Exam scores.Smalt(1999)foundusinganaccountingsimulationnearthe beginningofanacademictermwaseffectiveinimprovingperformanceonstudentfinal examinationscores.Performancewasmeasuredbysummingallthetestscores administeredduringthetermforthetwogroupsbeingcompared.Onegroup experiencedthesimulationgameearlyduringthetermandthecontrolgroupdidnot experiencethesimulationgame.AChi‐Squareanalysisofdemographicdataindicated PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 66 thegroupswerehomogeneousbasedonacademicmajor,gender,andrace.Independent t‐testsrevealednosignificantdifferencesbetweengroupsfortheotherindependent variablesexceptacomparisonofthepretestofbasicaccountingknowledgescores revealedthecontrolgrouphadabetterunderstandingofaccountingprinciplesthanthe groupabouttoexperiencethesimulationgame. Performanceofthesimulation‐experiencinggroupwasfoundtobesignificantly betteronthefinalexamination.Postcoursesurveyanalysisindicatednosignificant improvementinstudentperceptionsaboutaccountingasamajor,thequalityof instruction,theaccountingcourseingeneral,orthequalityofclassroominteractions. Theseperceptionswerenotconsideredsurprisingsinceoneinstructorconductedthe bulkofthecourseandthesimulationwasexperiencedatthebeginningofthecourseand ledbyadifferentinstructor.However,“89.9%ofexperimentalgroupparticipants indicatedthatexposuretothesimulationhadapositiveimpactontheirattitudetoward accountingingeneraland93.9%indicatedthegamehadimprovedtheirunderstanding” (Smalt,1999). Self‐assessed knowledge and competence.Seethamraju(2011)askedstudentsto self‐assesstheirknowledgeandthecompetencegainedbeforeandafterthesimulation usingaquestionnairethatwasadministeredinweekthreeofthecourse,justbeforethe simulationgame,andinweek12,justaftercompletion.Fortyquestionswithresponses usingafive‐pointLikert‐typescalefromverylowtoveryhighweregroupedintofour constructsrepresentingfourknowledgedimensions:Processconcepts&terminology, processsignificance&awareness,processmanagementandanalysis,andSAPskills.An additional18questionswereaskedonthepostsimulationquestionnaireusinga5‐point scalefromstronglydisagreetostronglyagreetoassessperceptionsandattitudesrelated totheadministration,benefits,organizationandenvironmentofthegame.Thisscale wasbiasedtowardsagreementasavalueof3correspondedtoagreeratherthana neutralneitheragreenordisagree.Constructdefinitionsandthenumberofrelated questionsweregivenforallthesedependentvariables.Examplesoftheunderlying questionswerementioned;alistofquestionsusedisavailableonrequestfromthe author.Participantsalsorespondedtonarrativequestionsaskingaboutthebestand challengingaspectsofthegameandtooffersuggestionsforimprovingthegamedesign anditsadministration. PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 67 Seethamraju(2011)usedpairedsamplest‐teststocomparethebeforeandafter ratingsandfoundasignificantgaininself‐reportedknowledgeandabilityacrossthe fourknowledgedimensions.Meandifferenceincreasesforthefourknowledge dimensionconstructsrangedfrom1.54to3.8andweresignificantforp<=.01. Postsimulationattitudescoresrangedfrom3.19to4.11forthefourconstructsand3.87 overall;allintheagreetostronglyagreerange.Thisoverallscorewasdeemed significantlybetterthantheoverallratingof3.2observedforothersimilarcourses withoutsimulation;thedetailsofthisanalysiswerenotprovided.Contentanalysisof thenarrativefeedbackwasregardedas“generallypositive.”Seethamrajunotedstudent commentspraisingtheaddedbenefitofdevelopinggroupworkskillsandobserved: Participantsappreciatedthelearningvalueofthegameandreportedlygained moresignificantskillsandknowledgethanfrompreviousteachingmethods. Participantsingeneralfeltthegameisfun,excitingandinspiredlearningwitha focusonprocessesratherthanIT,andassistedtheminappreciatingcross‐ functionalissuesandinterdependencies.(Seethamraju,2011) Keyword counts and exam scores.Klein(1980;1984)usedananalysisof keywordcountsmeasuredonmid‐termandfinalexamstoconcludetheuseofan internationalbusinesssimulationtointroducetheconceptsofbusinessriskwas moderatelymoreeffectivethanlecturealone.However,a68‐itemmultiplechoicetest administeredbeforeandafterthesimulationshowednosignificantimprovement betweentheexperimentalandcontrolgroupsandtheargumentwasmadethatthisis theresultofproblemsassociatedwithmeasuringexperientiallearning,notthatno learningoccurred: Learningissaidtohavetakenplaceifanewbehavior,anewattitude,oranew interestorvalueispresent.Theinferencehereisthatoneshoulddetectlearning bycomparingbeforeandafterbehavior. Withaprocessascomplexasasimulationgame,thetaskofisolatingthat whichislearnedfromthetotalinvolvementoftheindividualisindeedcomplex. Thedevelopmentofatestinstrumentwhichmeasuresspecificsbecomes extremelydifficultinanexperientiallearningenvironment.... PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 68 Inasimulationgame...questionsthataskforspecificresponsesmaybe inappropriate.Therespondentsmaynotbeabletoverbalizearesponse concerningaspecialconcept,yettheycanapplyittoadecision‐makingproblem. ...questionssuitedtothetextandlectureenvironmentmaynotbeappropriate totheexperientiallearningmedium.(Klein,1984) Kleinconcludedthat“simulationdoesindeedsensitizeitsuserstocritical internationalbusinessissuesthatareanintegralpartofthegame’sdecision‐making tasks(Klein,1980)andtheuseofa“simulationgameprovidesanopportunityto exercisetheskillsandknowledgeacquired.Itprovidesaviewfromadifferentbridge” (Klein,1984). Attitude measures.WilliamsandWilliams(2011)measuredpre‐andpost‐ simulationattitudesandbehaviorsofstudentgroupswhoeitherplayedaMultiple IdentificationTheorygameaboutinternationalconflicttwiceorwerepartofacontrol groupthatwatchedadocumentaryaboutconflictthenplayedthegameonce. Measurementsweretakenbefore,midway(aftercompletionofthefirstgameor documentaryviewing)andfollowingcompletion.Thecontrolandexperimentalgroups didnotdiffersignificantlyatthestartoftheexperiment.Followingtheexperiment,the experimentalgroupindicatedasignificantchange;thecontrolgroupdidnotindicatea significantchangeafterviewingthedocumentary,butdidindicateasignificantchange afterplayingthegame.Thissupportedthehypothesisthatattitudeswouldchangeasa resultofplayingthegame.Dependent(matchedpairs)t‐testswereusedtodetermine significance. Managerial and personality traits and decision‐making styles.Wellington, HutchinsonandFaria(2012)usedapretest‐posttestquasi‐experimentinvolving460 undergraduatestudentsfromaPrinciplesofMarketingcoursetoconductexploratory researchontheeffectofMerlin:AMarketingSimulationonstudents’managerialand personalitytraitsanddecisionmakingstyles.Thestudentswerefromtwodifferent iterationsofthecoursewhichranindifferentsemesters.Twopretestswere administered.Thefirsttoassessgeneralmanagerialandpersonalitytraitsbeforethe studentswereassignedasindividualstosimulationcompanies.Thesecondafter studentswereassignedtosimulationcompaniesandreceivedalectureonthe simulationtoassessdecisionmakingstyleandadditionalmanagerialandpersonality PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 69 traitsinthecontextofthesimulation.Thepostsimulationposttestrepeatedand combinedthequestionsfromthetwopretests.Atotalof133questionsusingLikert‐ typescalesweredrawnfromtheliteraturetoconstruct18managerialandpersonality traitvariables.Simulationperformancewasratedhighorlowbasedonranked performancevs.theotherparticipants.RepeatedmeasuresMANOVAwasusedto simultaneousexaminechangesovertimeandinrelationtosimulationperformance. Wellingtonetal.(2012)foundanumberofstatisticallysignificantchangesand relationships,buttheirfindingsareanexampleofwhensignificantdoesnotmeana substantialchange.Forexample,statisticallysignificantchangeswerefoundforthe variableBig5–Conscientiousnessbuttheaveragechangeforthehighperformance groupwasfromapretestscoreof3.65toaposttestscoreof3.59,adecreaseof0.06ona 5‐pointscale;thelowperformancegroupchangedfrom3.50to3.42,adecreaseof0.08. Sowhiletheresearchindicatesstudentconscientiousnessdecreasedsignificantlyasa resultofplayingMerlin,thecasualobservermightnotfindthisdecreasetobe meaningful. Summary.Researchintotheeffectivenessofusingbusinesssimulationgamesas experientiallearningactivitiesisconfoundedbythecomplexityofdefiningand measuringlearning(P.H.Anderson&Lawton,2009;Chinetal.,2009;J.V.Gentry& Burns,1981;Gosen&Washbush,2004;Greenlaw&Wyman,1973;Jones,1987;Stainton etal.,2010).Someresearcherscontinuetousethesimulationscoreasadependent variablerepresentinglearning,butmanyagreethatwinningthegameandscoringwell onothersimulator‐generatedmetricsdoesnotprovelearning.Itispossibletolearn whilescoringpoorlyandlow‐scorersmayactuallylearnmore.Lessonslearnedcan includeself‐awarenessandinterpersonalskillsinadditiontoneworrefinedtechnical knowledgeandskillrelatedtothetheoreticalunderpinningsofthesimulationmodel. Researchmethodsintotheuseofbusinesssimulationgameshavetendedtorely ofpostsimulationassessmentsofobjectivemeasuressuchassimulationandexamscores andonstudent‐reportedperceptionsandattitudes.Longitudinalresearchmeasuring presimulationandpostsimulationconstructshasbeenfoundusingbothobjectiveand perceptivemeasures.Therecenttrendappearstobetowardconductingmore longitudinalstudiestoshowlearningacrossthesimulationratherthantocompare simulationandnonsimulationmodeswhichmaybedifficulttoarrangeinanacademic degreeprogram. PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 70 Despiterepeatedcallsforgreaterrigorinmeasuringlearning,perceptionsare stillwidelyusedtoconstructvariablesinsimulationgamingresearchbecausetheyare easilymeasuredandbecausesucceedingexamscoresmaynotmeasurewhatthe simulationgameintendedtoteach.Thisseemsreasonableconsideringtheteam‐based simulationgameenvironmentismorestronglyrelatedtotheaffective,perceptualand behaviorallearningmodesthantothecognitivelearningmode(Kolb,1984).More discipline‐basedresearchiscalledfor. Thenextsectioninthischapterexplorestheavailableresearchintotheuseof projectmanagementsimulationgamesasexperientiallearningactivitiesinacademic programs.Duetothesmallnumberofstudiesfoundandtheirapplicabilitytothisstudy, eachisdiscussedinmoredetailthanwasdoneforthebusinesssimulationstudy examplesprovidedinthissection. Project Management Simulation Gaming Research Researchontheuseofbusinesssimulationgamesinacademicprograms indicatesthemostextensiveuseisingeneralmanagement,marketingand strategy/policycourses,andtoalesserextentinmanagementscience,finance,and accountingcourses;thereisnomentionofprojectmanagementsimulationgames(Faria &Wellington,2004).However,15articleswerefoundexaminingtheuseofproject managementsimulationgamesinacademicprograms.These15variedinmethodsfrom qualitativeandquantitativestudiesrelyingonpostsimulationcomments/surveysand instructorobservations(S.Al‐Jibourietal.,2005;S.H.Al‐Jibouri&Mawdesley,2001; Collofello,2000;L.S.Cook&Olson,2006;J.M.Cooper,2011;Dantasetal.,2004;Dillman &Cook,1969)tomorerigorousquantitativepretest‐posttestandexperimental‐control groupstudieswithadescribedresearchmethodology(Davidovitchetal.,2006; Davidovitchetal.,2007;Davidovitchetal.,2008;Davidovitchetal.,2009;Davidovitchet al.,2010;McCreery,2003;Pfahl,2004).Thissectionreviewsthesestudiesand recognizesaneedforadditionalresearchontheuseofprojectmanagementsimulation gamesinacademicprograms.Thelessrigorousstudiesarediscussedfirstin chronologicalorder,followedbythemorerobuststudies.Thefivearticlesby Davidovitchetal.arepresentedinorderoftheirwritingtoprovidebettercontinuity. Dillman and Cook (1969). Thearticle“SimulationinthetrainingofR&Dproject managers”istheearliestreferencefounddiscussingprojectmanagementsimulationas anELA.AtthetimeofitswritingandpresentationattheFebruary8,1969meetingof PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 71 theAmericanEducationalResearchAssociation,Dr.DillmanwasaPh.D.student associatedwiththeEducationalProgramManagementCenteratTheOhioState UniversityinColumbus.Ohio.HecurrentlyisthePrincipalofDillmanAssociatesinLos Angeles,CAandanaffiliateoftheCerritosPsychologicalCenter.Dr.Cookwasafull professorintheCollegeofEducationDepartmentofPsychology. Thisarticleispreviouslydiscussedonpage11followingtheheadingProject ManagementSimulationGames.Itincludeselevenreferences,sixoftheserelatedto simulation,threetoprojectmanagement,andoneeachtosocialsystemsanalysisand teachingproblemslaboratory. Followingtheintroductionoftheneedforamorerealisticexercise,Dillmanand Cook(1969)brieflyintroduceprojectmanagementandsimulationandthendescribethe developmentprocessforthesimulationexerciseandmaterials.Evaluationwasbasedon qualitativeanalysisofresponsestotenopen‐endedsurveyquestions.Responseswere categorizedaspositive,negative,neutral,suggestion(novalueattribution),positive‐plus suggestion,negative‐plussuggestion,ornoresponse.Mostofthequestionswererelated tothesimulationadministrationandfeedbackforimprovement,i.e.,game administrationandorganization,gamerealism,sessiontimelength,information provided,correlationwithpriorinstructionalsessions,valueofroleplaying,staff feedback,andclarityofdesireddeliverables.DillmanandCook(1969)found: Themostpositivereactionsweretowardtherealismofthesimulationexercise ...andthecorrelationofthesimulationsessionswiththepreceding instructionalsessions....Themostnegativereactionsweretothetimelengthfor eachsession...theexplicationoftherolestobeplayedandthevalueofrole playing...andthefeedbackfromstaff....Thelargestnumberofneutral responseswereobtainedontheitemsconcerningthetimelengthofeachsession. Onaseparatecritiqueform,thesimulationreceivedthehighestmentionasbeing especiallyfacilitativetolearning.(Dillman&Cook,1969) Whilefarfrombeingconsideredarigorousacademicarticle,theresultsindicate theparticipantsfoundvalueintheprojectsimulationELAduetoitsrealismand reinforcementoflearning. PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 72 Collofello (2000).CollofellodescribesacollaborativeeffortbetweenArizona StateUniversityandMotorolaUniversitytodevelopasoftwareprojectmanagement trainingcourseincorporatingaSystemDynamicssimulatorofthesoftwaredevelopment process.Dr.CollofelloisaProfessorintheDepartmentofComputerScienceand EngineeringatArizonaStateUniversityinTempe,Arizona. Hisarticlecites15references,oneontrainingeffectiveprojectmanagersandthe restrelatedtosoftwaredevelopment,modelingorprocesssimulation.Sevenofthe citationsarefromAmericanProgrammer(laterrenamedCutterITJournal),a practitionerjournal.Threeoftheotherreferencesaretosoftwareengineering textbooks,andtwoaretoproceedingsfromsoftwareprocesssimulationmodeling workshops. Followingabriefintroductionlistingmethodstoimprovesoftwareproject managementskillsandadvocatingforexplorationviaasystemdynamicssimulation model,Collofello(2000)providesanoverviewofsystemdynamicsmodelingandhowa simplemodelcandescribetheinterrelationsbetweentimespentonqualitytasks, undiscovereddefectsandabilitytomeetschedule.Hethenbrieflydescribesthe softwareprojectmanagersimulatorandfivecourseexercisesthatuseit. TheinitialdeploymentwasinaMotorolaUniversityprofessionaldevelopment coursethatraninthefallof1999andspringof2000withallparticipantsreportingthe “useofthesimulatoraddedsignificantlytothevalueofthecourse”(Collofello,2000). Collofelloclosedbystatinga“web‐basedgraduatelevelsoftwareprojectmanagement courseisalsobeingdevelopedatASU[ArizonaStateUniversity]utilizingthesimulator.” Whilenotanacademicresearcharticle,thisarticleisanexampleofthe acceptanceofthevalueofsimulationasanELA.Thismessagecouldhavebeen strengthenedwithamorerigorouspostcourseevaluationprocessataminimumand greatlyenhancedwithaproperlyconstructedresearchstudy. Martin (2000).MartindescribesasimulationcalledContractandConstructand itsapplicationinteachingprojectmanagementatthegraduateandundergraduatelevel atWarwickBusinessSchoolwhereheisalecturerinInformationSystemsand Computing. Twenty‐sixreferencesarecited:11relatedtobusinessandinformationsystem simulationsandsimulationgames,eightrelatedtoprojectmanagement(twoofthese PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 73 fromatrainingperspective),sixrelatedtogeneralmanagement(onerelatedto education),andoneonteachingpreparation. Followinganintroductionofprojectmanagementskilldevelopmentandthe benefitsofsimulation‐basedtraining,Martin(2000)describesthesimulationandits primaryuseinteachingprojectmanagementinanMBAcourse.Thisisfollowedbya discussionoftheunderlyingsimulationengine(database,schedulingengine,user interface,simulationgeneration)andadiscussionofpossibleenhancementsand implementationvariations. Thesimulatedprojectisan18‐activitychemicalplantconstructionprojectwitha simulateddurationof94weeksandabudgetof₤7million.Studentsaredividedintofive groups.Eachgroupistoldtoeitherfocusonacost,schedule,qualityormoraleobjective, ortobalanceallfouroftheseobjectives.Afterreceivingabriefingdocumentandashort presentationaboutthesimulatedproject,studentsanalyzethegiveninformation,select contractorsforthe18activities,andexecutethesimulatedproject.Duringexecution theyreceiveprojectstatusinformationandrespondtoeventsaffectingthecost, duration,quality,moraleandsafetyoftheproject.Performanceisassessedineachof thesefiveareas.Safetyisimportantnomatterwhattheprimaryobjectiveandallgroups mustobtainthesameminimumacceptablesafetyscore.Witheachgrouphavinga differentprimaryobjective,avarietyofstrategiesareimplementedprovidingdiverse resultsandarichdebriefdiscussionafterconclusionofthesimulationgame.(Martin, 2000). Performanceagainsteachofthesefiveobjectivesisscoredonacumulativebasis throughouttheprojectandanoverallscoreiscalculatedbyaveragingthefiveelements withadoubleweightgivenfortheemphasizedobjective.Afterthesimulation,students prepareashortpresentationandreflectonwhathappened,howsuccessfultheywereat implementingtheirstrategy,andwhattheylearned.Theeducatorthenleadsa discussioncomparingperformanceacrossthegroupsandnotingtheirdifferent objectives.(Martin,2000). Martin(2000)presentedatableofhypothesesandanalysisofscoresgainedby 59groupsdemonstratingthatstudents,asexpected,scorebetterintheirareaof strategicfocus.Itisnotclearhowmanystudentsactuallyparticipatedinthisstudyorif theyworkedindependentlyorinteams.StudentreactiontotheELAisreportedasbeing PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 74 generallyfavorable,butnoindicationisgivenastohowthiswasdetermined.Martin (2000)summarizedthekeylearningpointsas:  Understanding,identifyingandmanagingthenatureofthecriticalpath...  Practiceisdifferentfromtheory....  Decisionshavetobemadeunderuncertainty.  Theconsequencesofdecisionshavetobefaced.  Planningisacoreactivityinprojectmanagement. Martin(2000)reportsthesimulationwasalsousedsuccessfullyinanon‐ academicprofessionaldevelopmentsettingatanelectricitygeneratingcompanywith teamsofpeople,butnodetailswereprovided. Thisarticledescribedthesimulationanditsapplicationatahighlevel;however, itislackingthedetailsandrigorneededtobeconsideredascholarlyarticleortobe reproducedbyanotherresearcher.Itsmaincontributionisprovidinganexampleofa projectmanagementsimulationgamethatisbeingusedinbothacademicand commercialprograms. Al‐Jibouri and Mawdesley (2001).Al‐JibouriandMawdesleydescribethe developmentandinitialuseofaninternethostedsimulationgamewithgraphicaluser interface(GUI)toteachprojectplanningandcontrol.Bothauthorsareaffiliatedwith universitiesinWesternEurope.Theyprovideashorthistoryofconstruction managementgamesandthebenefitsofusinggamesfollowedbytheconsiderationsand objectivesfordevelopingagame;adescriptionofthesimulatedproject(arockandclay filldamwiththeplayertakingontheroleofthecontractor’sprojectmanager responsibleforplanning,resourceselectionanduse,controlandreportingto management);adescriptionofthestructure,mainfeaturesandGUI;andabrief descriptionofitsusein“anundergraduatecourseinplanningandcontrolandasa demonstrationtoolinanintensivecourseonplanningandcontrolforpracticing engineersandconstructionmanagers.”Thearticleisprimarilyadescriptionofthe developmentandinitialuseofthesimulation.Althoughsomequalitativeassessment dataisprovided,itisnotarigorousstudyevaluatingitseffectivenessasalearningtool. Al‐Jibouri&Mawdesley(2001)citethirteenreferences:fivearticlesdescribing constructionmanagementandschedulinggames,threearticlesprovidingan PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 75 introductiontobusinessandmanagementgames,twoconstructionmanagement textbooks,twoarticlesonmodeling,andonearticleoneffectiveteaching.Elevenof thesereferencesarecitedintheintroductionandhistoryofconstructionmanagement gamesectionsandtheremainingtwoarecitedinthesectiondiscussingreasonsfor developinggames. Studentsworkedinsmallteamsof2to4playersintheundergraduatecourse andinitiallytendedtoignoretherisksandbeoptimisticintheirplanning.When executiondidn’tgoaccordingtoplan,theyfirstignoredcostswhileattemptingtohold scheduleandlaterengagedinformalre‐planninginanattempttoalsocontrolcost. Despiteonlyfairresultsinmeetingthecostandschedulegoalsoftheproject,qualitative analysisofthenarrativequestionnaireresponses,informaldiscussionfeedbackfromthe studentsandtheobservationofthesupervisingacademicsprovidedevidencethatthe studentsmade“substantialimprovementsduringthegame”bylearninghowtoinclude projectriskmanagementintheirplanningandcontrolactivities.(S.H.Al‐Jibouri& Mawdesley,2001) Fewdetailswereprovidedabouttheuseofthesimulationintheintensivecourse forpracticingmanagersandfeedbacksuggestedthesimulationgame“shouldbeused aloneinthefutureandthatthe‘lecture’sessiononthesubjectwasnotrequired”(S.H. Al‐Jibouri&Mawdesley,2001). Al‐JibouriandMawdesley(2001)concludethat“throughcarefulconsiderationof theuserinterfaceandselectionofthecorrectprojectitispossibletobuildagamewhich addssignificantlytothetraditionalcoursestructureforsuchatopic.”Theyobservethat bothstudentsandpracticingengineersappearedtoliketheexperientiallearning providedbythegameand“particularlyvaluedtheexperienceofbeingina‘reallife’ situationwheretheycanmakedecisionsandseetheconsequencesofsuchdecisions.” Onassessingtheeffectivenessofthislearningexperience,Al‐JibouriandMawdesley opine: Itwouldbedifficulttodetermineifanymethodofeducation/trainingismore successfulthanothers.Measuresof‘successfulness’and‘effectiveness’ofa particularmethodwillbeanamalgamoffactorsandresultsthataresometimes difficulttoisolate.However,itisbelievedthattheobservationofandfeedback PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 76 fromparticipantssuggestthegameoffersbenefitsinsomeareasovertraditional methods. Assessmentoftheperformanceofthestudentsonthegamehasshown considerableimprovementsduringthegamewhichindicatesalearningprocess thatisnormallydifficulttomeasureintraditionalteachingmethodssuchas lecturesandexams. Noquantitativeevidenceofthis“considerableperformanceimprovementwhile playingthegameisprovided”andanunresolvedissueishowtoassessstudent performanceforcoursegradingpurposes.Al‐JibouriandMawdesley(2001)suggest threepossiblemethods:(1)assessingtheteampresentationsduringthereviewswith management,(2)assessingperformancewhileplayingthegame,and(3)administeringa traditionalexamattheconclusionofthecoursebasedongame’slearningobjectives. Thisarticleprovidedadescriptiveoverviewofaprojectmanagementsimulation gameusedbynon‐competitiveteamsinundergraduateandpostgraduateconstruction managementprograms.Theinternet‐hostedsimulationishighlightedbyagraphical userinterfacewithpictures,videoclips,animation,textualandgraphicreports,and eventstriggeredbyplayerdecisions.SimilartotherecommendationbyClarkandMayer (2008)thatmoreisnotnecessarilybetter,70%ofthestudentsdidnotrecommend addingmorevideotothegameandstudentopinionsweresplit50:50onwhethermore animationshouldbeadded.Thoseformorevideo“thoughtitwouldbeinteresting”(not thattheywouldlearnmore)andthoseopposedthoughtmorevideoandanimation wouldwastetimeandnotaddanythingimportanttotheexperience(S.H.Al‐Jibouri& Mawdesley,2001).Thisarticledidnotincludestatisticsdescribingtheclasssizeand composition,numberofcoursesthesimulationwasusedin,etc. Notablefindingsfromthequalitativeanalysisofthepostsimulation questionnairedataare75%ofthestudentsthoughtlectureswereneededtosupplement thesimulationtoprovidethebackgroundtheoryonprojectcontroland90%suggested theycouldlearnbetterfromthegamethanlecturesthattheoriginalplanisrarelyideal andunexpectedeventswilloccurandrequirere‐planning(S.H.Al‐Jibouri&Mawdesley, 2001).Althoughthisstudycouldbeimprovedbyapplyingamorerigorousassessment methodology,thegenericpostsimulationnarrativequestionsmightbeusefulinfuture PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 77 research.Otherthanidentificationoftheneedtodevelopperformanceassessment measures,Al‐JibouriandMawdesleymadenorecommendationsforfutureresearch. Dantas, Barros and Werner (2004).Dantasetal.believe“theinadequateuseof projectmanagementtechniquesinsoftwareprojectscanbetracedtothelackofefficient projectmanagementeducationstrategies,wherelearningbyexperienceandmotivation arekeyissues.”TheydevelopedaprojectmanagementgamebasedonaSystems Dynamicsmodelofasoftwareprojectandpresenttheresultsoftwoexperimental studiesconductedtodemonstrate“theusefulnessofgamesinprojectmanagement trainingprograms.”Dantasetal.areaffiliatedwiththeUniversidadeFederaldoRiode JaneiroinBrazil. Seventeenreferencesarecited:fiverelatingtoSystemsDynamics,fiveto softwareprojectsandmanagementstrategies,fourtotheuseofsimulationsinsoftware engineeringtraining,andoneeachtoandradogy(adultlearningstrategies),digital game‐basedlearningandsimulationeffectiveness. Followingadiscussionoftheineffectivenessofeducatingadultlearnerswith content‐centricapproachesandtheadult‐learnerpreferenceforexperientialand application‐basedlearning,Dantasetal.(2004)brieflydescribethedevelopmentand operationoftheirSystemsDynamics‐basedsimulationmodelfollowedbyasummaryof theirlearningevaluation. Thestudywasconductedwithstudentsstudyingsoftwaredevelopmentattwo universitiesinBrazil.Therewere7studentsinoneclassand8intheother.Allreceived 20minutesoftraininginprojectmanagement,notablyfunctionpointestimating,andin theuseofthesimulationpriortoplayingthesimulationgame.Theonlyresultsprovided wereinasingletablesummarizingthestudentopinionsonfivedimensionsoftheELA experience:“PMSkill”and“InterestinPM”withtheresponseoptions:“raised... indifferent...reduced;”“Game‐basedTraining”withtheresponseoptions:“good... indifferent...bad;”“PresentedLessonsLearned”withtheresponseoptions:“all...none ...lots...few;”and“Wasthetrainingfun?”withtheresponseoptions:“yes...no... much...little.”Onlyonestudentcompletedthegamewithsuccess;however,the studentfeedbacksummarizedtheexperienceas“motivating,dynamic,practicaland enjoyable.”(Dantasetal.,2004) Thisarticlediscussedasimulationgamedevelopedbytheauthorstoaidinthe developmentofsoftwareprojectmanagersanddemonstratedthatprojectmanagement PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 78 learnersreactfavorablytotheinclusionofaprojectmanagementsimulationgameinan educationalprogram.Limitationsofthisstudyincludethesmallsamplesizesand researchapproachlimitedtoanalyzingpost‐ELAattitudestowardprojectmanagement andthesimulationgame. Al‐Jibouri, Mawdesley, Scott and Gribble (2005).Inthissequeltothe2001article byAl‐JibouriandMawdesley,Al‐Jibourietal.(2005)reportinadifferentjournalon usingthesamesimulationmodelofanearthmovingprojectasamanagementgamefor teachingconstructionprojectmanagement.Asbefore,playingthegamerequired studentstoplan,makedecisionsunderuncertainty,anddealwithenvironmentaleffects andfinancialresults.Thefirstthreeauthorsarecivilengineersteachingincivil engineeringdepartmentsorschoolsatuniversitiesinTheNetherlands,theUnited KingdomandAustralia.ThefourthauthoristheTeachingFellowforstaffatthe Australianuniversityresponsibleforimprovementofteachingandlearning. Ninereferencesarecitedinthisarticle,fourrelatedtoconstructionmanagement andschedulinggames,threerelatedtoconstructionmanagement/costcontrolmodels,a genericarticleonbusinessandmanagementgames,andamanualforacomputer managementgame.The2001articlebyAl‐JibouriandMawdesleyisnotcitedeven thoughtheopeningparagraphintroducingtheimportanceofprojectplanningand controlisanear‐verbatimcopy. Inthisarticle,Al‐Jibourietal.(2005)skiptheintroductoryhistoryof constructionmanagementgamesandreasonsfordevelopinggamesfoundinthe2001 articleandprovideatwoparagraphdiscussionaboutthedifficultyinprovingthe effectivenessofthetechniquestaughtinthesecoursesandthedifficultyofassessingthe effectivenessandefficiencyofthemethodusedtoteachthem.Thelegitimacyof simulationasapedagogicaltoolisthensuggestedbythetwosucceedingparagraphs describingthenationalcivilengineeringgamerunannuallybyLoughboroughUniversity intheUK(withthewinningteamawardedaprizeandpublicityintheprofessional press)andanotherconstructionmanagementgameusedinindustrialratherthan academicenvironments.Withtheexceptionofanupdatetoacitationontheindustrial game,thesetwoparagraphsareidenticaltoparagraphsfoundintheun‐cited2001 articlebyAl‐JibouriandMawdesley. PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 79 Similartothe2001article,thisarticledescribesthesimulationgameand providesexamplescreenshotsofthesiteplan,decisionscreenandtextandgraphical reportscreens.Thissectionisre‐writtenbutcontainsessentiallythesameinformation. Differentinthisarticleisthe“experiencewiththegame”section.Ratherthana summaryoftheresultstoallthepostsurveyquestionnaireresponsesasinthe2001 article,presentationofresultsislimitedto18examplesofnarrativeresponsestoonly onepostsimulationsurveyquestionasking,“Whatifanythingdidyoulearnfromthe game,whichcouldnothavebeenlearnedmoreeasilyfromtraditionallectures?Please givereasonsforyourstatements”(S.Al‐Jibourietal.,2005).Theseweresummarizedas: Themainpointsreportedbythestudentsinclude:  Thedifferencebetweentheoryandpractice  Theimportanceofobtainingrealisticratherthanoptimisticdata  Theimportanceofcontrol  Theneedforplanningandcontrolevenwhenfacedwithanuncertain world(S.Al‐Jibourietal.,2005) Asectionnotfoundinthe2001articlebyAl‐JibouriandMawdeslydescribesthe useofasummaryplayerperformancemeasurebasedonalinearcombinationofthe differencesbetweencurrentandplannedexpenditure,income,[cash?]balance,clay heightandrockheight.Theresultsfor11teamsof3playerseachareshownasagraph oftheaverageofthismeasureforthe11teamsandasagraphshowingtheprogressof eachteamrelativetotheirownworstscore.Usingthisscore,Al‐Jibourietal.(2005) found,onaverage,teamperformanceisgoodduringtheopeningsimulationweeksofthe gamewhenthesimulatedweatherisgoodanduncertaintyeffectsareminimal.Thisis followedbyalossofcontrolandmuchworsescoreswhenuncertaintiesaffect performance.Thentheaverageperformancemeasureshowsanimprovementtrend throughthecompletionoftheproject.Thegraphofindividualgroupperformance relativetotheirownmaximumperformancemeasureshowsthatsomegroups performedmuchbetterthanotherswiththebestgroupsabletoconsistentlyimprove theirperformancescorethroughoutthesecondhalfoftheprojectandtheworstgroups onlyabletostabilizetheirperformance.Theauthorsgavenoindicationofwhetherthis measureaffectedthecoursegrade.(S.Al‐Jibourietal.,2005). PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 80 Alsonewtothisarticleisadescriptionoftheuseofthegameasarecruit‐select toolbyaconstructionservicescompany.Inthisapplicationstudentswereinvitedto attendafoursessiononeeveningperweekcourseconsistingoflecturesandthegame. Thelectureswereprovidedbyseniorpeoplefromthecompanyontopicsrelevanttoits operation.Observationofstudentsplayingthegameprovidedvaluableinsightsthat wereusedbythecompanyinmakingemploymentdecisions. Thecompanywere[sic]verypleasedwiththegameespeciallyinthewaythatit demonstratedthecomplexityofrealprojectsandtheneedtoconsidersomany apparentlyunrelatedtopicsinordertomanageit.Theycommentedonhowwell itenabledthemtoobservethebehaviouroftheparticipantsinanear‐realistic environment.(S.Al‐Jibourietal.,2005) Thisarticleprovidedanecdotalevidenceontheefficacyofusingaconstruction projectmanagementsimulationgameandanexampleofaquantitativeassessmentof learningtechniqueforasingleclassof33students.Similarresultsfromrepeateduseof thistechniquewouldstrengthenthevalidityoftheresultsandconclusions.Thisuseofa runningperformancemeasurederivedfromthesimulationresultsisatechniquefor furtherinvestigation. Ifweconsidertheadditionoftheperformancemeasurequantitativeanalysisand thediscussionconcerningtheuseofthesimulationgameasanemployeeselectiontool tobethecoreofthisarticleandthatthesubstantialrepeatofpreviouslypublished materialasnecessarytounderstandthisnewcontribution,thentheauthorsmaynotbe technicallyguiltyofself‐plagiarism;butitwouldhavebeenbettertocitetheprior2001 articlebyAl‐JilbouriandMawdesleyatthebeginningofthis2005article(American PsychologicalAssociation,2010). Cook and Olson (2006).CookandOlsondescribetheuseofateamexperiential learningactivity(ELA)simulatingaskyscraperconstructionprojectusingspaghettiand miniaturemarshmallowsasapredecessortothelectureonprojectmanagementinan operationsmanagementcourse.Dr.OlsonisDepartmentChairoftheOperationsand SupplyChainManagementDepartmentattheUniversityofSt.ThomasinMinneapolis, MNandDr.CookisanAssociateProfessorintheDepartmentofManagementatDePaul UniversityinChicago,IL.Theirarticleprovidesanoverviewoftheexercise, PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 81 administrationadviceforinstructors,andadiscussionofevidenceofeffectiveness. Theirguidanceincludesstressingtheneedtoestablishthebasisfortheexercisewiththe studentsandtotieittothelecturelearningobjectivesthroughouttheactivity. Fourteenreferencesarecited;sevenarearticlesrelatedtoteachingtechniques (twooftheseonexperientiallearningandtwooncriticalthinking),twodescribingthe useoftowerbuildingexercisesinmanagementeducation,oneonmakingfaststrategic decisions,andtextbooksonorganizationalpsychology,experientiallearningandusing MicrosoftProject,andTheGuidetoProjectManagementBodyofKnowledge(PMBOK® Guide). CookandOlson(2006)usedaquasi‐experimentalpretest‐posttestresearch designwherestudentswereaskedtoratetheiragreementusinga7‐pointtypeLikert scale(from1=stronglydisagreeto7=stronglyagree)withthestatement“Ifgiventhe examtoday,Iwouldreceivefullcreditforthequestionlisted...”followedbythe learningobjectivesforthecourse.Thearticlelistedsevenlearningobjectivesforthe projectmanagementmodel;theauthorsreportedtheresultsoft‐testsforthefour objectivesrelatedtoboththelectureandtheELAshowingasignificantincreaseinthe meanscoreforallfourofthesequestions.Thepretestwasadministeredatthe beginningofthecourseandtheposttestonthelastdayofthecourseandincludeda similarassessmentofalltheothernon‐projectmanagementmodulesinthecourse. AttributiontotheELAwasmadebyaskingstudentstoratetheoverall effectivenessofthesimulationexerciseona5‐pointLikert‐typescale.Providingno details,CookandOlson(2006)reported: Morethantwo‐thirdsofthestudentsreportedthatitwaseffectiveinhelping themunderstandtheclassmaterial.Acrossthevarioussections,the overwhelmingmajorityofstudentsfoundtheexercisetobeanextremely effectivemeanstoteachprojectmanagement. Theauthorsconcedethat“althoughthedatademonstratelearning,itisdifficult toisolatethespecificattributesoftheclassexerciseversustheensuingclasslecture”and “theattributesspecifictotheclassexerciseandclassinstructionmaybeconfounded”(L. S.Cook&Olson,2006). PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 82 Thisarticleconcludedthatbothstudentsandfacultybelieve“theuseofanELA canbeaveryeffectivemethodtopromotehigherlevelsoflearning”(L.S.Cook&Olson, 2006).However,theinternalvalidityoftheseresultsissubjecttotheeffectsof extraneousvariablessuchashistoryandmaturationduetothetestsnotbeing administeredimmediatelybeforeandaftertheELAandbecausetheposttestis administeredattheendofthecourseafterboththeELAandthesucceedinglecture withoutalecture‐onlycontrolgroup(Campbell&Stanley,1963;T.D.Cook&Campbell, 1979). Cooper (2011). CooperevaluatedthesingleplayeruseoftheHarvardBusiness PublishingProjectManagementSimulation:Scope,Resources,Scheduleinan undergraduateclassofmanagementmajorsconcentratinginprojectleadership.Dr. CooperisanAssociateProfessorintheManagementandFacilitiesDepartmentat WentworthInstituteofTechnology.Thesimulationisusedinthefirsttwooffour coursesrequiredforaBachelorofSciencedegreeinmanagementwithaconcentration inprojectleadership. Sevenreferencesarecited:threearticlesdiscussingsimulationfidelityand assessment,threeonprojectmanagement,andthesimulationfacilitator’sguide(J.M. Cooper,2011). Cooper(2011)reportsonusingthesimulationwithaclassof34undergraduate studentsintwoin‐classandthreeout‐of‐classassignments.Evaluationconsistedof analyzingthreeitems:(1)studentresponsestoaquestionnaireadministeredfollowing thesimulationexercises,(2)studentdiscussionboardpostsabouthowtokeepaproject onschedule,and(3)answerswritteninresponsetoanopen‐endedfinalexamquestion. Mostoftheitem(1)postsimulationquestionsrelatedtotherealismofthesimulation andthebelievabilityoftheresults.Studentsidentifiedmanythingstheylikedaboutthe simulationandtoaquestiondirectlyaddressingthelearningexperience,71%of respondentsagreedthatthesimulation“provide[d]avalidlearningexperienceand/or assessmentoflearning.”Fourteenpercentwereneutralandfifteenpercentdisagreedor stronglydisagreed.Cooper(2011)noted: Studentsdidnotapproachthesimulationasacomputergame.Itisimportantfor anacademicsimulationthatoutcomesarenotdeterminedbychanceorluck; insteadusersexperiencetheconsequencesoftheiractions.Aneffective PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 83 simulationexperienceshouldnotseemlikerollingdice!Participantsneedto taketheirroleseriously.Halfofthestudentsagreedthatthesimulationresults werenotdeterminedbyluckorchance[24%wereneutral;26% disagreed/stronglydisagreed]. Studentswerelesscertainaboutthesimulation’sbelievabilityandrealism. Cooper(2011)believesthisisduetothestudents’perceivedinabilitytoeasilyinfluence themoraleofthesimulatedprojectteam. Thesimulationisscoredwitha1000pointmaximumwith725pointsavailable formeetingtargetsand275pointsavailableforexceedingthem.Studentsarepermitted toplaythescenariosmultipletimesforeachassignmentandbecomefamiliarwiththe simulationlogicandcasualrelationships,especiallytheinterrelationshipbetween scheduleandbudgetandthedifficultyoffinishingbothontimeandonbudget.These insightssuggestthesimulationiseducationallyvalidasdescribedbyFeinsteinand Cannon(2002).(J.M.Cooper,2011) Thisarticleprovidedanexampleofusingpost‐ELAstudentfeedbacktoassess theefficacyofaprojectmanagementsimulationgame.Basedonthisfeedback,Cooper (2011)concludesthescoringdashboardbuiltintothesimulationandtheabilityof studentstoredoeachsimulationassignmentmultipletimessupportsskilldevelopment. McCreery (2003).McCreeryevaluatedtheintegrationofasimulationfroma three‐daycommercialprojectmanagementtrainingprogramintoaonesemesterproject managementcourse.Dr.McCreeryisanAssociateProfessorofOperationsand InnovationManagementandDirectoroftheMasterofGlobalInnovationManagement PrograminthePooleCollegeofManagementatNorthCarolinaStateUniversity.The computersimulationwasManagingbyProject,aproductofDavis&Dean,Inc.Davis& Deanisaglobalprojectmanagementeducationcompanywhoclaimstohavetrained morethan75,000projectmanagersaroundtheworld(Davis&Dean,2011a).Deanand Davisprominentlyfeaturethisarticleontheirwebsiteandclaim“[McCreery]found significantlearninginallsixteenprojectmanagementareas”(Davis&Dean,2011b).A moreaccurateclaimwouldrelatetoMcCreery’sfindingofsignificantperceptionsof learning. PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 84 Tenreferencesarecited,fiveonprojectmanagementandprojectmanager competencymodels,twoonexperientiallearning,twoonsimulationandoneisthe manualfortheSPSSstatisticalanalysissoftwarepackage. Thisquasi‐experimentalstudyusedapretest‐posttestmethodologyinvolving twoseparategraduate‐levelprojectmanagementclassestaughtinoneacademicyear andatotalof63students(29inoneclass,34intheother;datawerecombinedforthe twoclasses).Projectmanagementconceptsweretaughtduringthefirsttenweeksofthe semester,thesimulationexercisewasadministeredduringweeks11through15,and thecoursewassummarizedandwrappedupinthefinalweek16.Priortoattending classduringweek11,studentscompleted1‐2hoursofout‐of‐classpreparatoryreading regardingthesimulation.Classtimeamountingto3‐3.5hoursperweekduringweeks 11‐15wasdevotedsolelytoperformingthesimulation.(McCreery,2003) Studentsweregroupedintoteamsofthreetofivepeople.Eachstudentteam managedasimulatedprojectcontaining24activitiesandabaselinescheduleof22 weeksand$800,000budget.Teammemberscollaboratedtomakedecisionsinvolving resourceallocation,training,quality,andproblemresolutionforeachsimulationweek. Performancemetricsarecostminimization,scheduleattainmentanddeliverablequality. (McCreery,2003) Researchquestionswerenotexplicitlystatedbutdescribedasinvolvingthe assessmentofknowledgeincreaseandabilitytoapplythatknowledgeasaresultof participatingintheELAandexploringtheseinrelationtothestudents’experiencelevel andqualityofthestudents’teamprocessduringtheexercise.McCreery(2003)tested thefollowinghypotheses: H1(a)Participantswillassesstheirlevelofprojectmanagementknowledgeas greateraftercompletingtheexercisethentheywillpriortoperformingthe exercise...., H1(b)Participantswillassesstheirabilitytoapplyprojectmanagement knowledgeasgreateraftercompletingtheexercisethentheywillpriorto performingtheexercise...., H2(a)Priortoperformingtheexercise,thelevelofprojectmanagement knowledgewillbegreaterforparticipantswithhighamountsofprojectwork experiencethanforparticipantswithlowamountsofprojectworkexperience...., PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 85 H2(b)Frompreexercisetopostexercise,themagnitudeofimprovementinthe levelofprojectmanagementknowledgewillbegreaterforparticipantswithlow amountsofprojectworkexperiencethanforparticipantswithhighamountsof projectworkexperience...., H3(a)Priortoperformingtheexercise,theabilitytoapplyprojectmanagement knowledgewillbegreaterforparticipantswithhighamountsofprojectwork experiencethanforparticipantswithlowamountsofprojectworkexperience...., H3(b)Frompreexercisetopostexercise,themagnitudeofimprovementinthe abilitytoapplyprojectmanagementknowledgewillbegreaterforparticipants withlowamountsofprojectworkexperiencethanforparticipantswithhigh amountsofprojectworkexperience...., H4(a)Participantsoflowperformingteamswillshowsmallerimprovementsin levelofknowledgeandabilitytoapplythatknowledgethanwillparticipantsof highperformingteams...., H4(b)Participantsinvolvedinanegativeteamprocesswillshowsmaller improvementsinlevelofknowledgeandabilitytoapplythatknowledgethan willparticipantsinvolvedinapositiveteamprocess. Dataonknowledgeandabilitytoapplythatknowledgewerecollectedviapretest andposttestquestionnaireswithidenticalquestionsusingasevenpointLikert‐type scalerangingfromextremelylow(1)toextremelyhigh(7).Studentsrespondedtothe instructions“assessyourcurrentlevelofknowledgeineachofthefollowingareas”and “assesshowconfidentyouareinyourabilitytoapplythisknowledgeineachofthe followingareas”forthe12projectmanagementcompetenceitemslistedinTable1. Teamperformancedatawerecollectedfollowingthesimulationwithstudents respondingtotheinstruction“assessyourteamexperiencethroughoutthesimulation exercise”usingasevenpointLikert‐typescalewithchoicesrangingfromstrongly disagree(1)tostronglyagree(7)fortheteamexperienceitemslistedinTable2. Additionaldatacollectedincludedthestudents’yearsofprojectworkexperience. (McCreery,2003) PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 86 TABLE1‐PROJECTMANAGEMENTCOMPETENCEITEMS 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. 16. Project organizational design Project planning in general Assessing and managing project risks Estimating project scope Sequencing of project activities Estimating project activity times Project budgeting Allocating project resources Project change management Designing project performance measures Performing as an effective project leader Building team consensus Negotiating for resources and budgets Using earned value concepts Evaluating the performance of project personnel Managing project uncertainty TABLE2‐TEAMEXPERIENCEASSESSMENTITEMS 1. 2. 3. 4. 5. 6. 7. 8. 9. The workload was fairly balanced across all team members Team members cooperated well throughout the exercise Our team worked through the exercise in an efficient manner Team members all participated equally in the team decision making process Our team maintained a pleasant working atmosphere Our team worked out disagreements in an equitable manner Team members were highly motivated to perform well in the exercise Overall, I am satisfied with my team experience I would be willing to work with my team on an actual project in the future Improvement in knowledge and ability to apply that knowledge.McCreery(2003) foundthatstudentsstartedtheexercisewithself‐perceptionsofknowledgeandability thatwereneitherextremelyhighnorextremelylow.Mostvalueswerebetween3and5 onthe7‐pointscaleindicatingmoderateknowledgeandabilitywithroomfor improvement.Hesubtractedpostsimulationmeansforeachitemfrompresimulation meansandfoundthepostsimulationmeansincreasedby1.0unitormoreforallitems. Theseincreasesweredeterminedtobestatisticallysignificantbyperformingt‐testsfor pairedsamplesoneachitem.McCreeryconcludedhypothesesH1(a)andH1(b)were supportedandstudentssignificantlyimprovedtheirlevelofprojectmanagement knowledgeandtheirabilitytoapplythisknowledgeasaresultofparticipatinginthe PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 87 simulationgame.McCreery(2003)affirmedthisconclusionusingpostexercise discussionswithmultiplestudents: Thereisgeneralconsensusamongparticipantsthattheexerciseprovidesamuch neededcontextwithinwhichtoimproveprojectmanagementskills.Bymaking numerousdecisionsinasimulatedprojectenvironment,participantsareableto “maketheconnection”betweentheoryandpractice,therebygainingadeeper understandingofhowtomanageanactualproject. Effects of project work experience.McCreery(2003)constructedknowledgeand abilityvariableswerefromthegrandmeansofthe16knowledgeitemsandnineteam experienceitems,respectively.Studentswererankedaccordingtotheiryearsofproject managementworkexperienceandgroupedintoexperiencequartiles.Thelowranking quartilehad14of15studentswithnoprojectmanagementworkexperienceand remainingstudenthad0.2yearsofexperience;thehighrankingquartileaveraged10.7 yearswitharangefrom6to26.5years.Atabularpresentationofthesegrandmeansof knowledgeandabilityshowedthemostexperiencedquartilehadhigherpresimulation knowledgeandabilityscoresthantheleastexperiencedquartile(4.13and4.24versus 3.42and3.64,respectively). Agraphicaldisplayofthedataforeachconstructitemshowedthat“invirtually allinstances,moreexperiencedparticipantsenterintotheexercisewithhigherlevelsof knowledgeandabilitytoapplythatknowledgethanlessexperiencedparticipants” (McCreery,2003).ANOVAtestsoneachofthe16presimulationitemswithap‐valueof 0.1asacutofffoundsignificantdifferencesbetween9of16knowledgeitemsand4of16 abilityitems.Noresultsofstatisticallycomparingthegrandmeansoftheknowledge andabilityconstructswerereported.TheconclusionwashypothesisH2(a)wasstrongly supportedandhypothesisH3(a)wasmoderatelysupported:“Highlyexperienced participantsappeartocomeintotheexercisewithhigherperceivedlevelsofproject managementknowledgethandothelesserexperiencedparticipantsand,toalesser degree,greaterabilitiestoapplythatknowledge”(McCreery,2003). Thetabulardisplayofgrandmeanknowledgeandabilityalsoshowedtheleast experiencedquartileexperiencedagreaterknowledgegainfollowingthesimulation thandidthemostexperiencedquartile(1.46versus1.01),buttheincreaseinabilityto PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 88 applythatknowledgewasroughlythesameforbothofthesequartiles(1.06and1.02). Thegraphicalfiguresshowedthesedifferencesforeachconstructitem.Supportfor hypothesisH2(b)wasfoundtobemoderateatbest:ANOVAtests(p<=0.1)showedfive ofthe16knowledgeitemstohavesignificantdifferences;oneoftheseitemswasinthe oppositedirectionthanexpected.McCreey(2003)notedtheseresultsmaybeaffected bythesmallsamplesizeof15studentsineachquartilegroup.Assuggestedbythe abilityconstructdifference,HypothesisH3(b)wasnotsupported:ANOVAtests(p<= 0.1)foundthreeof16differencestobesignificantandtwoofthemintheopposite directionthanexpected.McCreery(2003)concluded: Lessexperiencedparticipantswillmakegreatergainsinlevelsofknowledge... [and]tendtoclosetheprojectmanagementknowledgegapbytakingpartinthe exercise....[all]participantsmadeequivalent,substantialgainsintheabilityto applytheirprojectmanagementknowledge.” Effects of team performance and team process.Tostudytheeffectsofteam performance,McCreery(2003)calculatedacompositeteamperformancescoreusingthe simulatedprojectcostandscheduleperformanceresultstorankthe16teamsandused ANOVAtocomparetheimprovementsmadebythe4teamsinthelowestquartileagainst theimprovementsmadebythe4teamsinthehighestscoringquartile.Hypothesis H4(a)wasnotsupported;onlyoneofthe16knowledgeitemsandnoneoftheability itemsshowedastatisticallysignificantimprovement.McCreery(2003)concluded“there isnosubstantialevidencetosupporttheviewthatobjectiveteamperformancehasan impactontheeducationalvalueoftheexercise.” Tostudytheeffectsofteamprocess,McCreery(2003)calculatedateamprocess scoreusingthenineteamexperienceassessmentitemslistedinTable2andrankedthe teamsfromlowesttohighest.Heobservedthatteammembers’assessmentsofthe qualityoftheirteamprocessvariedwidely,evenwithinteams.AgainusingANOVAtests withthehighestandlowestscoringquartiles,hefoundnocorrelationbetweenteam processscoresandimprovementsinknowledgeorintheabilitytoapplythat knowledge.WhilethisindicatednosupportforhypothesisH4(b),henotedthatmost studentsexperiencedaquitepositiveteamexperiencewithcompositeteamprocess scoresover6.00andthatthelowestteamquartilehadameanscoreof5.01.“Itleavesas PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 89 anopenquestionwhetheranextremelynegativeteamprocesswouldaffectthe educationalvalueoftheexercise”(McCreery,2003).McCreeryconcluded,“giventhe dataavailableinthisstudy,theeducationalvalueofthesimulationisnotsignificantly affectedbyeitherateam’sobjectiveperformanceorbythequalityoftheteamprocess.” Summary.McCreery’s(2003)researchdemonstratedtheeducationalvalueof usingaprojectmanagementsimulationasanELAwithamixofstudentsfrom managementandengineeringgraduateprogramswithvaryingdegreesofpriorwork experience.Theuseofp<=0.1forsignificancetestingsuggeststheresultswerenotas robustashopedforwhenconductingresearch;however,thefindingthatlearningoccurs regardlessofhowwelltheteamperformedwhileexecutingthesimulationisvaluable. Additionaldiscussionoftheinternalthreatstovalidityincludingtheself‐assessmentof knowledgeandabilitywouldbeuseful.Opportunitiesforfutureresearchinclude demonstratingtheimprovementsfoundusingthismethodologyarenotahaloeffectand theyresultintransferableimprovementsintheabilityofstudentstoperformasproject managersintherealworld. Pfahl, Laitenberger, Ruhe, Dorsch and Krivobokova (2004).Pfahletal.present“the resultsofatwicereplicatedexperimentthatevaluatesthelearningeffectivenessofusing aprocess[SystemsDynamics]simulationmodelforeducatingcomputerscience studentsinprojectmanagement.”TheauthorsareaffiliatedwiththeUniversityof KaiserslauterninGermany(wheretheexperimentwasfirstperformed),theUniversity ofOulu(Finland),andtheUniversityofCalgary(Canada)andconsiderthisresearchto beexploratoryinnature. Thirtyreferencesarelisted(citationswerefoundfor26),14relatedtosoftware engineeringandSystemsDynamicsmodeling,ninerelatedtoresearchmethods,three relatedtomanagementeducationwithsimulatorsandcasestudies,threerelatedarticles listingPfahlastheprimaryauthor,andoneondesigningandevaluatingacomputer‐ basedlearningenvironment. Theresearchapproachwasapretest–posttestcontrolgroupexperiment. Learningeffectivenesswasevaluatedbycomparingwithin‐subjectposttesttopretest scoresandbycomparingscoresbetweentheexperimentalandcontrolgroups.The experimentalgrouptrainingmoduleincludedtheuseofaSystemsDynamics(SD)model simulatingthetypicalbehaviorofasoftwaredevelopmentproject;thecontrolgroup trainingincludedtheuseofthepopularCOCOMOsoftwarecostestimationmodel.The PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 90 trainingmodulewascomposedofprojectplanningandcontrolcoursematerial.(Pfahl, 2004) Pfahletal.(2004)usedfourconstructsderivedfromthepretest/posttest questionnaire,eachrepresentedbyadependentvariable(Y.1–Y.4),tomeasure performanceandtwoadditionalconstructsfromaseparatesessionevaluation questionnaire(Z.1andZ.2)tomeasuresubjectiveperceptions: Y.1 Interestinsoftwareprojectmanagementissues(‘Interest’) Y.2 Knowledgeabouttypicalbehaviourpatternsofsoftwaredevelopment projects(‘Knowledge’) Y.3 Understandingof‘simple’projectdynamics(‘Understandsimple’) Y.4 Understandingof‘complex’projectdynamics(‘Understandcomplex’) Z.1 Availabletimebudgetversustimeneed[tocompletethetreatment](‘Time Pressure’) Z.2 Sessionevaluation Thenatureofthequestionscomprisingtheconstructsisdiscussedinthearticleand examplequestionsareprovidedintheappendixforallconstructs.Thereaderisreferred toPfahl’sPh.D.thesistoviewthecompletesetofquestions. Theexperimentalhypotheseswereintroducedas: 1. Thereisapositivelearningeffectinbothgroups(A:experimentalgroup, B:controlgroup),i.e.post‐testscoresaresignificantlyhigherthanpre‐ testscoresforeachdependentvariable. 2. ThelearningeffectingroupAishigherthaningroupB,eitherwith regardtotheperformanceimprovementbetweenpre‐testandpost‐test (relativelearningeffect),orwithregardtopost‐testperformance (absolutelearningeffect).(Pfahl,2004) Thesearerestatedasthefollowingnullhypotheses: H0,1:Thereisnodifferencebetweenpre‐testscoresandpost‐testscores withinexperimentalgroupAandcontrolgroupB. PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 91 H0,2a:Thereisnodifferenceinrelativelearningeffectivenessbetween experimentalgroupAandcontrolgroupB H0,2b:Thereisnodifferenceinabsolutelearningeffectivenessbetween experimentalgroupAandcontrolgroupB.(Pfahl,2004) Studentswererandomlyassignedtothetwogroupsandparticipatedas individuals.Thestudentsdidnotknowthepretestandposttestquestionswereidentical andtheywerenotprovidedwiththecorrectanswersuntilaftertheconclusionofthe experiment.Theinitialexperimentwasconductedontwodayswithinaweekofeach otherandstartedwith12graduatecomputersciencestudentsattheUniversityof Kaiserslauternandfinishedwithninestudents.Thesecondinstancewaswith12 graduateandpostgraduatestudentsattheUniversityofOulu,Finland,andthethird instancewaswith13seniorundergraduatestudentsattheUniversityofOulu,Canada. Thesecondandthirdinstanceswereconductedonasingleday.Studentsparticipating inthesecondandthirdrunningoftheexperimentwereenrolledinavarietyoftechnical andmathematicaldegreeprograms.ThefirstrunoftheexperimentattheUniversityof Kaiserslauternlasted130minuteswith45minutesallocatedtothetreatment. Treatmenttimewasincreasedto80minutesfortheothertworunsattheUniversityof OuluandattheUniversityofOuluafterstudentsattheUniversityofKaiserslautern indicatedtheydidn’thaveenoughtime.(Pfahl,2004) Atreatmentstartswithbothgroupsreceivinga3‐5minuteintroductiontothe maintasksofasoftwareprojectmanager,typicalplanningandcontrolproblems experienced,andtheneedtomaketrade‐offswhenmanagingthetripleconstraintofthe project.FortheexperimentalgroupusingtheSD‐modelprojectsimulator,this introductionwasfollowedbya15‐30minuteintroductiontotheprinciplesdominating softwareprojectperformanceandarole‐playinteractionwiththeSDmodel.Duringthe roleplay,thestudentactsastheprojectmanagerandispresentedwithaprojectplan thatgreatlyexceedsthedeadline.Theprojectmanagermustdecidewhichactions (basedonthepreviouslyintroducedprinciples)totakeinordertocompletetheproject ontime.Tobesuccessful,theprojectmanagermustinvokeactionsinvolvingmorethan oneoftheprinciples.Therole‐playisfollowedbyashortdiscussionofthedifferent possiblesolutionswithanexplanationoftheadvantages/disadvantagesofeach.The experimentalgroupthenreceivesa15‐30minuteintroductiontoprojectmanagement PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 92 planningmodels,includingdetailedexplanationsoftheSDmodel,followedby12‐15 minutesofapplicationexampleswithshortexercisesusingthemodel.Thecontrol groupreceivesa30‐60minuteintroductiontoprojectmanagementplanningmodelsand detailedexplanationsofusingCOCOMOintheintermediatemodefollowedby12‐15 minutesofapplicationtimewithshortexercisesusingthemodel.(Pfahl,2004) Forhypothesistesting,theresultsofone‐waypairedt‐testforH0,1andone‐sided t‐testforindependentsamplesforH0,2aandH0,2barereported.Pfajhletal.(2004)note thatatestfordatanormalityrevealedthatanormaldistributioncouldnotbeassumed andadditionalnon‐parametrictestswereperformed(Wilcoxonmatchedpairtestfor H0,1andMann‐WhitneyUtestforH0,2aandH0,2b).Resultsofthesetestsdidnotrevealany differencesandonlytheparametrictestresultsarereportedinthearticle.Further discussedistheneedtosetα=0.1forsignificancetestingduetothesmallsamplesize andthemeta‐analysistechniquesusedtointegratethefindingsfromthethreestudies. Descriptivestatisticsandresultsofthestatisticalanalysesarepresentedindetail(data tablesanddiscussion).InsummaryPfahletal.(2004)found:  EvidencefortheassumptionthatthetrainingsessioninvolvingtheSD modelinsteadofCOCOMOplusperformingarole‐playsignificantly increasesinterestinthetopicofprojectmanagement....  TraininginvolvingSDmodelandrole‐playyieldssignificantlybetter scoresforvariableY.2(knowledgeaboutempiricalpatternsinsoftware projects)thanusingCOCOMOwithoutrole‐play....  Noconsistentsignificantdifferencebetweenexperimentalandcontrol groupscouldbeobservedregardingvariableY.1,Y.3,andY.4(interestin thetopicofprojectmanagement,understandingofsimpleproject dynamics,understandingofcomplexprojectdynamics)....  Qualitativedataindicatestheinclusionofrole‐playswithSDmodelsin projectmanagementeducationisperceivedasahighlyusefulexercise.... withregardstousefulness,entertainment,difficulty,andclarity. Summary.Thiswasthemostcompletereportofaresearchstudyfoundamong thearticlesrelatedtousingprojectmanagementsimulationasanELA.Itdemonstrated theuseofapretest‐posttestcontrolledexperimentmethodologyforevaluatinglearning. PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 93 Itsdesignusingalessrobustbutstillinterestingandgenerallyrecognizedcost estimatingtoolasthereplacementfortheSDsimulatorinthecontrolgroupmayhave confoundedtheresults;however,evidencesupportingtheuseofaSystemsDynamics simulatorforteachingsoftwareprojectmanagementwasfound. Davidovitch, Parush and Shtub (2006).Davidovitchetal.researchthe performanceofparticipantsusingdifferenthistory‐keepingmodesofaproject managementsimulatordevelopedbyDr.Shtubandfind“thesimulatoras[an] educationalinnovationimprovesstudentlearningandperformance.” Atthetimeofwritingthisarticle,Mr.DavidovitchwasaPh.D.candidateinthe IndustrialEngineeringandManagementfacultyattheTechnionIsraelInstituteof TechnologyunderthesupervisionofDr.Parush,associateprofessorofpsychologyat CarletonUniversityinOttawa,CanadaandDr.ShtubatTechnion.Dr.Davidovitchis currentlyemployedasaprogrammanagerattheIsraelMinistryofDefense.Dr.Shtubis theprincipaldeveloperofProjectManagementTrainer(PMT)anditssuccessor,the ProjectTeamBuilder(PTB)simulator,andhaswrittenbooksonitsuse.Thestated purposeofthesesimulatorsis“toprovideadynamic,stochastic,simulatedenvironment forpracticingthetoolsandtechniquesofprojectmanagementandtosupportthe creationofsharedunderstandingamongprojectteams”(Shtub,2012). Davidovitchetal.(2006)usedKolb’sExperientialLearningTheory(Kolb,1984) andaconstructivistapproachastheoreticalframeworkstoexaminetheeffectofthe PTMSimulatorasanexperientiallearningtoolonadaptivetransferofknowledgeby “teachingandtrainingonetaskandthenexaminingtheperformanceofadifferenttask.” Thirty‐sevenreferencesrelatedtoprojectmanagement,learningtheory,and simulationarecitedintheintroductionsectionofthearticlesupportingdiscussionson teachingandlearningprojectmanagement,simulation‐basedlearning,theProject ManagerTrainersimulator,andtheimpactoflearninghistoryonlearningwith simulations.Fiveofthesereferencessupportthenotionofallowingthelearnerssome controloverthelearningenvironmenttogivethemamoreactiveroleinconstructing theacquiredknowledgeandtherebyenhancinglearning.Anadditionalfivearticleson curriculumdevelopmentandlearningtheoryarecitedinthemethodologyand discussionsections.(Davidovitchetal.,2006) Theresearchwasdesignedtotestthehypothesis“thatauser‐controlledor manuallearninghistorywillhaveagreaterpositiveimpactonsimulator‐basedlearning PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 94 ascomparedwiththeautomatichistoryrecording”(Davidovitchetal.,2006).ThePMT simulatorallowsuserstoreviewpriordecisionsandtore‐runthesimulationfroma priorhistoryrestorepoint.Inadditiontocomparinglearningbetweenthemanualand automatichistoryrecordingmodes,thisresearchalsotestedthehypothesis“thatif historykeepingandreviewinghasanimpactonlearning,thenparticipantswholearned withsuchamechanismshouldstillperformbetterwhenthismechanismisremoved” (Davidovitchetal.,2006). Thiswasaccomplishedbyconductingtheexperimentintwophases.InPhaseI: BasicLearning,studentswereassignedtooneoffivegroups:automatichistorywith undo,automatichistorywithoutundo,manualhistorywithundo,manualhistory withoutundo,andno‐history[controlgroup].Allweregiventhesame16activity projectscenariotorunthreetimes.InPhaseII:Transfer,studentsinallfivegroupswere givena24activitymultipleprojectscenariowithnohistorykeepingandnoundo.The projectsscenariosarepredefinedbythesimulator.Studentsaregiventaskinformation (durationdistributionandpredecessors),resourceinformation(type,availability,cost workingandcostidle)andtargetprojectcost.Performancegoalsaretofinishontime andmaximizeprofit.Abonusisaddedtoprofitforearlyfinishesandapenaltyis deductedfromprofitforlatefinishes.Thedependentvariablesusedintheresearch wereprofitattheendofthesimulationrunandelapsedtimetocompletethesimulation run.Therandomnumbergeneratorsinthesimulationengineweresettogeneratethe samenumbersforallthestudentstohelpassuredifferencesinresultswerebasedsolely onstudentdecisions.(Davidovitchetal.,2006) Thefiveexperimentalgroupswererandomlypopulatedwithstudentsfromthe poolof98fourthyearengineeringstudentswithnopriorprojectmanagement experience.Agesrangedfrom18to35.Groupsizesvariedfrom17to21students;the authorsdidnotdiscusswhythegroupsizesweren’t19and20.Studentmotivationwas stimulatedbymakingthecoursegradepartiallydependentonthesimulatedprofit achieved.(Davidovitchetal.,2006) Dataanalysisconsistedofcomparingthemeansoftheprofitandsimulationrun timeresultsgraphicallyandbyperformingt‐testsandAnalysisofVariance(ANOVA)on them.AgraphofprofitresultsforthethreesequentialPhaseIsimulationrunsshoweda clearandconsistentincrease;thecontrolgroupdidnot.Nosignificantdifferencewas PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 95 foundamongthefivemeanprofitsforthefirstPhaseIrunindicatingtheexperimental andcontrolgroupsstartedwithsimilarknowledge.Insummary,forsimulatedprofit: Havingahistorykeepingmechanismhadasignificantimpactonperformance comparedwithnohistorykeepingmechanism.Inallconditionswheretherewas ahistorymechanism,meanprofitsincreasedsignificantlybetweensimulation runs,whereastherewasnoimprovementwithouthistory.Inaddition,the highestmeanprofitattheendofthethirdsimulationwasfoundforthemanual historykeepingascomparedwiththeautomatichistorykeeping(bothwithundo capability....Havingtheabilitytoundo(restartthesimulationfromanypointin apreviousrun)hadasignificantimpactonperformance.Forbothhistory‐ keepingmodes,manualandautomatic,themeanprofitsweresignificantlyhigher thantherespectivemanualandautomaticmodeswithoutundo.(Davidovitchet al.,2006) Forsimulationrunduration,Davidovitchetal.(2006)foundmeanrundurations werelongerinthefirstrunandlongestforthegroupswithmanualhistorykeeping.The abilitytoundodidnotaffectrundurationduringthefirstrun.Secondandthirdmean simulationrundurationsdecreasedsignificantlyforeachofthefivegroups.Mean simulationrundurationsweresignificantlylongerforthemanualhistorywithundo groupduringthesecondandthirdruns. DuringPhaseIIallgroupscompletedthefourthsimulationrunusingthesame multipleprojectscenariowithouthistorykeepingandwithouttheabilitytoundo.The meanprofitoftheexperimentalgroupwasfoundtobesignificantlyhigherthanthe controlgroup.ThegroupwiththehighestPhaseIIsimulationrunprofitwasthegroup thatoperatedinPhaseIwithmanualhistorykeepingandtheabilitytoundo.From highesttolowestprofit,thePhaseIIgrouprankingwas(1)manualhistorywithundo, (2)automatichistorywithundo,(3)manualhistorywithoutundo,(4)automatichistory withoutundo,and(5)nohistory[control].Thedifferencesbetweentheautomaticand manualhistorygroupsweresignificantinbothcasesofabilitytoundoornotundo.No significantdifferenceswerefoundforthemeansimulationrundurationamongthe groups.(Davidovitchetal.,2006) PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 96 Theexperimentalgroupsachievedsignificantlyhigherprofitandshoweda betterlearningprocess...whilethecontrolgrouphadverylittlelearning....The findingsshowabetterlearningprocessfortheparticipantswithmanualhistory mechanismcomparedtotheparticipantswithautomatichistorymechanism,and withoutanysignificantdifferencebetweensimulation‐rundurations.Moreover, whileobservingtheMP[PhaseIImultipleproject]scenarioperformancewhich reflectedtransferabilities,itwasfoundthetrendpersistedandperformancewas betterforthegroupthatpreviouslyhadthemanualhistorykeepingmechanism. Thesefindingssupportthehypothesis...thatmanuallearninghistoryrecording willhaveagreaterpositiveimpactonsimulator‐basedlearningascompared withautomatichistoryrecording.Itcanbeassumedthathavingthemanual historykeepingmechanismforcesthelearnertobettermonitortheirown progress...[and]mayhavecontributedtobetterunderstandingoftheprinciples involvedwithprojectmanagementandconsequentlyprovidedthelearnerwitha bettercapabilitytotransferandhandlemorecomplexscenarios....[Further,] havingtheabilitytoundoinsimulator‐basedlearningcannotonlyimprove performance,butactuallysupportandimprovelearning.(Davidovitchetal., 2006) Davidovitchetal.(2006)discussedtheseresultsinthecontextofseveral learningtheoriesandconcluded“thehistorymechanismenabledthebuildingofstorage strengthbyprovidingthelearnerswithcontinuousaccesstothelearnedmaterial.... [and]probablyprovidedmoreexposuretopossiblesubjectmatterelements.Thisin turnnotonlystrengthenedthestorage,butmadeitpossibletobetterunderstandthe processunderlyingthelearntmaterial.” Unlikethepreviousarticlesreviewedinthisstudy,thisarticlebyDavidovitchet al.(2006)wentbeyonddescribingasimulationgameanddocumentingthatthe studentslikedit.Followinggenerallyacceptedacademicresearchprocesses, Davidovitchetal.demonstratedthevalueofprovidingstudentswithahistory‐keeping mechanismfortheirusewhensimulatingaprojectanddemonstratedthattransferable learning(formaximizingsimulatedprofit)isoptimalwhenthehistory‐keepingis implementedmanuallyandstudentsareallowedtoundoandchangedecisionsmade whilerunningasimulation.Unanswerediswhetherornotthestudentsarelearning PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 97 applicableprojectmanagementskillsthattransfertotherealworldormerelylearning howtooptimizethesimulation. Davidovitch, Parush and Shtub (2008).Buildingontheresearchdocumentedin their2006article,Davidovitch,ParushandShtub(2008)discussthelearning‐forgetting‐ relearningprocessanddescribetheirexperimentalresearchevaluatingtheeffectiveness andefficiencyofthisprocessinaprojectmanagementsimulationenvironment.In additiontotheirprior2006article,anarticlebyDr.ShtubdescribingthePMTsimulator, andthePMBOK®Guide,twenty‐ninereferencesarecitedrelatedtolearningtheories. Themainresearchquestion“washowhistoryrecordinginsimulator‐basedlearning impactsthelearning‐forgetting‐relearningprocess.”Quantitativemethodssimilarto thoseusedinthe2006studywereusedtotesthypothesesrelatedtotheforgetting phenomenon,thelengthofthebreakperiodandhistorymode(automaticormanual), andbasiclearning. Theexperimentaldesignusedthreeindependentvariables:(1)historyrecording mode(automatic,manual,none[control),(2)breakperiod(twoweeksorfourweeks), and(3)simulationrun(describedbelow).Thereweresixexperimentalgroupstofully crossthethreehistoryconditionswiththetwobreakperiodconditions.(Davidovitchet al.,2008) Thetwo‐meetingexperimentwasconductedwith66secondyearMasterof Engineeringstudentsranginginagefrom25to50.Participationwasvoluntaryand participantsreceivedagradebonusbasedontheresultsoftheirsimulationruns(with theapprovaloftheIsraelInstituteofTechnologyResearchEthicsBoard).Studentsran fouridenticalsimulation‐runsofasingleprojectscenarioduringthefirstmeetingand, followingabreakofeithertwoorfourweeks,reranthesamesingleprojectscenario followedbytwicerunningamorecomplicatedmultipleprojectscenario.Thesimulator manualhistorywithundofeaturewasavailableforthestudentsuseduringthefirst meeting(PhaseI:Basiclearning[simulationrunsonethroughfour])butnotduringthe secondmeetingafterthebreak(PhaseII:Forgettingandrelearning[simulationrunfive] andPhaseIII:Transfer[simulationrunssixandseven]).Asinthe2006study,the sequenceofrandomnumbersgeneratedforthestochasticmodelinginthesimulatorwas thesameforallstudents.Dependentvariableswereagainprofitandscenariorun durationandt‐testsandANOVAwereusedtotestforsignificantdifferencesbetweenthe datameans.(Davidovitchetal.,2008) PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 98 Asbefore,inPhaseI(basiclearning),nosignificantdifferencewasfoundamong thegroupsafterthefirstsimulationrunindicatingnoinitialimpactfromthe experimentalmanipulationsandthatallstudentsstartedwiththesameknowledgeand experience.Onceagain,thegroupswithhistoryshowedconsistentsignificant improvementsinprofitoverthenextthreesimulationrunsduringthefirstmeetingwith themanualhistorygroupscoringsignificantlybetter.Thecontrolgroupshowedno patternofconsistentgrowth.Meansimulationtimedecreasedsignificantlyforeachof thegroupsoverthefourrunsandnosignificantdifferencewasfoundbetweenthe automaticandmanualhistorygroupsandbetweenthesegroupsandthecontrolgroup. (Davidovitchetal.,2008) AfterthePhaseII(forgettingandrelearning)fifthsimulationrun,themeanprofit ofthecontrolgroupremainedsignificantlylowerthanthehistorygroups(roughlyhalf) andthemeanprofitwassignificantlylessforthegroupswiththefourweekbreakthan forthosewiththetwoweekbreak.Forthetwo‐weekbreakgroups,therewasa significantdifferencebetweentheautomaticandmanualhistorygroups;forthefour‐ weekbreakgroups,therewasnosignificantdifferencebetweenthem.Therewere significantdifferencesbetweenthemeansimulationrundurationforthetwo‐week breakgroupswiththemeantimeofthemanualhistorygroupbeinglowest,the automatichistorygroupthesecondlowest,andthecontrolgroupthemost.The simulationruntimesforthefourweekbreakgroupswereallsignificantlylongerthan thetwoweekbreakgroups;however,inthiscasetherewasnosignificantdifference betweenthemanualandautomatichistorymodegroups.Bothfourweekbreakhistory groupsweresignificantlyshorterthanthecontrolgroup.(Davidovitchetal.,2008) AfterPhaseIII(transfer)andthefinaltwosimulationruns(simulationrunssix andseven)withthemultipleprojectscenario,asubstantialandsignificantincreasewas observedforthegroupswithhistoryfromrunsixtorunseven;nosignificantincrease wasobservedforthecontrolgroups‐controlgroupprofitdecreasedslightlyfromrun sixtorunseven.Manualhistorygroupprofitwassignificantlyhigherthantheautomatic historygroupforthetwo‐weekbreakgroupsbutnotforthefour‐weekbreakgroups. Meansimulationruntimesdecreaseddramaticallyforallgroupsfromrunsixtorun seven.ComparingthefifthrunofPhaseIIwiththesixthruninPhaseIIIyieldedno significanteffectsorinteractions.(Davidovitchetal.,2008) PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 99 Insummary,thestudentsingroupswiththesimulationhistorykeepingfeature performedsignificantlybetterthanthecontrolgroupsandthemanualhistorygroups performedsignificantlybetterthantheautomatichistorygroups.Withtheexceptionof simulationrunsix,thecontrolgroupsimulationrundurationswereallgreaterthanthe rundurationsoftheexperimentalgroups.Significantprofitdifferenceswereobserved betweenthetwo‐weekbreakmanualandautomatichistorygroupsbutnotthefour‐ weekbreakgroups.Davidovitchetal.(2008)foundtheseresultsconsistentwith literaturedescribingtheforgettingphenomenonandstressedtheimportanceof designingsimulationsthatprovidestudentswiththeabilitytoreviewhistoryandideally tocontrolwhenhistoryiscaptured. Inconcludingthisarticle,Davidovitchetal.(2008)didacknowledgethe shortcomingsofmeasuringtransferoflearningwithinthesimulatorratherthantothe “realworld”andsuggestfurtherresearchinto“howwellthePMTsimulationfacilitates ‘true’projectmanagement”andhowwellit“promote[s]expertiseinreal‐lifesituations.” Thisarticlewasanothergoodexampleofprojectmanagementsimulationgaming researchthatstartedwitharesearchquestionandhypothesesandusedgenerally acceptedacademicresearchprocessestoanswerthequestion.Ifskillslearnedplayinga simulationgamearetransferabletotherealworldandimprovementinplayingthegame bybeingabletocapturehistoryandundopriordecisionstotakeadifferenceapproach therebyimprovesperformanceplayingthegame,thenskilltransfertotherealworld maybeimproved;however,moreresearchisneededtosupportthishypothesis. Davidovitch, Parush and Shtub (2007).Althoughpublishedearlierthanthe2008 articledescribedabove,thisarticleactuallysucceededitinsubmissionandbuildsonit byreportingontheresultsofanadditionalexperimentinvolvingtheperformanceof studentteamsformedfromthesamegroupofstudentsthatparticipatedinthe individualstudentsimulationexperiment. Thearticleaddsbriefdiscussionsoftheforgettingcomponentofthelearning process,team‐basedlearning,problem‐basedlearning,andtheimportanceofdebriefing. Thesediscussionsaresupportedbycitationsofeighteennewreferencesandthree referencesrepeatedfromthe2008article.(Davidovitchetal.,2007) Theresearchquestionforthemultiuserexperimentaddressedtheimpactofthe historyrecordingmodeanddebriefingonteamlearning.Twohypothesesweretested: (1)meanprofitwillbesignificantlyhigherforteamsusingthehistorymode;(2)mean PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 100 profitwillbehigherfortheteamsparticipatingindebriefingthanforteamsthatdon’t debrief.Theprocedureforselectingtheteams,thenumberofstudentsperteam,andthe natureandlengthofthedebriefsessionswerenotdescribed.(Davidovitchetal.,2007) Meanprofitisagainusedasameasureoflearningandisthedependentvariable. Independentvariablesarehistoryrecordingmode(withhistory,withouthistory), debriefing(with,without),andsimulationrun(therewerethreesimulationruns).The twohistoryrecordingconditionswerefullycrossedwiththetwodebriefingconditions tocreatefourexperimentalgroups.Allfourgroupsweregiventhesamesingle‐project managementscenarioandthesamescenariowasrunthreetimes.(Davidovitchetal., 2007) Atableofresultsshowedthesameperformancerankingaftereachsimulation run.Onaverage,thegroupofteamswithbothhistory‐keepinganddebriefsscoredthe highest.Secondhighestwasthegroupwithdebrief/nohistoryfollowedbythegroup withhistory/nodebrief.Thecontrolgroupwithoutbothscoredthelowest.Although therankingwasconsistentthroughout,nosignificantdifferenceswerefoundforeither historykeepingordebriefingafterthefirstrun.Thisindicatedthatallteamsstarted withsimilarknowledgeandexperience(thetestusedtoconfirmthiswasnotdescribed). UsingANOVAwithrepeatedmeasures,significantdifferenceswerereportedbetween thefourgroupsforboththedebriefingfactorandforthehistoryfactor.(Davidovitchet al.,2007) Theperformanceofparticipantsthatusedthehistorykeepingmechanismswas significantlybetter. Theperformanceofthesystemsengineeringstudentsthatusedthe debriefingprocesswassignificantlybetter.Theinteractionbetweenthehistory factorandthedebriefingprocessfactorwasnotfoundassignificant. ...Theuseofthedebriefingprocessforteamlearningisfoundalso powerfulinthelearningprocess.(Davidovitchetal.,2007) Theprecedingquotationcontainstheonlycommentsinthediscussionsection abouttheteam‐basedexperiment.Thediscussionsectionprimarilysummarizedthe resultsobtainedfromtheindividuallearnerexperimentpreviouslydescribedin Davidovitchetal.(2008)emphasizingthepowerofthesimulatorhistorymechanism. PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 101 Theadditionofresearchintotheuseofsimulationbyteamsisimportantbecausethisis atypicalmodeofELAwithsimulationgamesingraduatebusinessmanagement programs.Moredisclosureofthemethodologyusedfortheteam‐basedexperimentin thisstudywouldbeuseful.Additionalresearchtodemonstratethatskillslearned playingthissimulationgamearetransferabletotherealworldwouldalsobeusefulas wouldresearchshowingthosewhoimprovemorewhileplayingthisgamearemore effectiveprojectmanagersintherealworld. Davidovitch, Parush and Shtub (2009).Buildingontheirpriorresearchonthe effectivenessofusingsimulatorsinprojectmanagementacademiccoursesandthe impactofsimulatorhistoryrecordingfeaturesonlearningandforgetting(Davidovitch etal.,2006;Davidovitchetal.,2008),Davidovitch,ParushandShtub(2009)examinethe impactofprojectmanagementsimulatorfunctionalfidelityonstudentperformance. Twenty‐sevenreferencesarecitedintheintroductionsectionofthisarticle: threeareusedtodefineprojectmanagementandtocreateananalogybetweenlearning projectmanagementandlearningtoflyanairplane,threetocitepriorarticlesontheuse ofthePMT(ProjectManagementTrainer)simulator,onependingarticleonthenext generationPTB(ProjectTeamBuilder)simulator,2onjournal‐keepingenhances reflectivelearning,fiveonthevalueofgivinglearnerssomecontroloverthelearning environment,and2onsimulatorfidelityanditsinfluenceonlearningtransfer.No citationwasfoundforthetwenty‐eighthandfinalreferencelisted,thePMBOK®Guide. TwoadvancedfeaturesofthepreviouslydescribedPMTsimulatorwere employedinthisstudy:(1)theabilitytosplitactivityexecution,i.e.theabilitytostartan activity,stopworkingonitforaperiodoftime,andresumeworkonitlater;and(2)the abilitytohireanddismissresources.Studentswithaccesstotheadditionalfeatures receivedinformationontheminimum/maximumnumberofresourcesallowedandthe coststohireanddismiss.(Davidovitchetal.,2009) Theexperimentaldesignforthisstudyusedcumulativeprofitattheendofthe simulationrunasthedependentvariableandthreeindependentvariables:(1) functionalfidelity(abilitytousetheadvancedactivitysplitting/resourcemanagement featuresornot)(2)historyrecordingmode(automatic,manual,none),and(3)undo (abilitytoundoornot).Thestudyusedtengroups,fullycrossingallexperimental conditions.(Davidovitchetal.,2009) PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 102 Thepriortwo‐phaseresearchapproach(Davidovitchetal.,2006;Davidovitchet al.,2008)wasused.InPhaseI:BasicLearning,alltengroupsreceivedthesamesingle‐ projectscenariotorunthreetimesand.ThiswasfollowedbyPhaseII:Transfer,where alltengroupsweregivenanewmultiple‐projectscenarioandnogroupwasabletokeep historyorundo.Learnerconditionswereasbefore:thesimulationisstochasticin executionwithregardtoactivitydurationsandcosts(allstudentsreceivethesame sequenceofrandomnumbers);studentswererequiredtocompletethesimulated projectbyitsduedate(withpenaltiesassessed/bonusespaidforlate/earlycompletion) andtomaximizeprofit.(Davidovitchetal.,2009) Noneofthe199participatingfourthyearengineeringstudentswithagesranging from18to35hadpriorpracticalprojectmanagementexperience.Studentswere randomlydividedintothetenexperimentalandcontrolgroups.Studentsreceived verbalandwritteninformationontheuseofthesimulator,thescenarios,andtheprofit performancemeasure(which,inpart,affectedtheircoursegrade).(Davidovitchetal., 2009) Aswasreportedinthepriortwostudies(Davidovitchetal.,2006;Davidovitchet al.,2008),Davidovitchetal.(2009)found:nosignificantprofitdifferencesafterthefirst runverifyingtheassumptionthatallstudentsstartedwithsimilarknowledge;all experimentalgroups(withhistory)achievedasignificantincreaseinprofitfor subsequentPhaseI:BasicLearningsimulationrunswhilethecontrolgroups(no history/noundo)didnot;experimentalgroupswithundoperformedsignificantlybetter thanthosewithout.Theoptiontousetheenhancedfeaturestoimprovetheprofitwas foundtobe“meaningful”(Davidovitchetal.,2009): TheeffectoftheEnhancedcapabilitiesfactoronthehistory‐keepingmodedoes notexist,whiletheperformancesremainbetterforusingthemanualhistory‐ keepingmodeandusingundocapabilities. ThiscommentandtheassociatedreportingoftheANOVAresultsarenotclear.A reviewoftheprovidedgraphindicatedthatprofitwashigherwhenusingenhanced techniquesforthemanualhistory‐keepinggroupsandinconsistentfortheautomatic history‐keepinggroup(profitimprovedfortheautomaticgroupwithoutundoand worsenedfortheautomaticgroupthatcouldundo).Therewasnonoticeabledifference PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 103 forthecontrolgroupwithnohistory/undoafterthefirstandsecondruns,butthemean profitwasnoticeablyhigherforthecontrolgroupwithaccesstotheadvancedfeatures afterthethirdrun. FollowingPhaseII:Transfer,whereallgroupsaredeniedaccesstohistoryand cannotundo,studentswhohadaccesstohistoryinPhaseIperformedsignificantly betterthanthosewhodidn’thaveaccesstohistoryinPhaseI;studentswithprioraccess tomanualhistory‐keepingperformedbetterthanthosewithpriorautomatichistory‐ keeping;andstudentswithpriorabilitytoundodidsignificantlybetterthantheir counterpartswithoutthispriorability(Davidovitchetal.,2009).Alltheseresultsare consistentwiththepriorfindingsofDavidovitchetal.(2006;2008).Whencomparing theeffectofaddingtheabilitytousetheadvancedfeatures,areviewofthetableof resultsindicatedprofitwassignificantlyhigherforthegroupswiththeabilitytousethe enhancedfeaturesiftheydidnothavetheabilitytoundotheirpriorwork;thegroups withundocapabilityperformedslightlybetterthantheircounterpartsiftheydidnot haveaccesstotheenhancedfeatures.Davidovitchetal.(2009)describethisasfollows: Theenhancedprojectmanagementcapabilitiesfactorkeepstheperformances andtheresultsofthesignificantdifferencesbasedonthehistory‐keepingmode. Moreover,theimprovedperformancesintheinitiallearningphasearekeptfor thetransfertoadifferentscenariophasewhenconsideringtheenhanced capabilitiesfactor....Theresultsindicatethatusingadvancedproject managementcapabilitiesenhancedlearningforallusergroups....foradvanced capabilities,thecomparisonbetweenhavinghistorymechanismwithundoand withoutundo,duringbasiclearningphase,showedbetterperformancesfor thosehavingundocapability.Thefindingssuggesthavingtheabilitytoundoin simulator‐basedlearningcanactuallysupportandimprovelearningandnotonly improveperformance....Theuseofadvancedresourcemanagementcapabilities enhancesthelearningofprojectmanagement. Summary.Thisarticleisanotherexampleofprojectmanagementsimulation gamingresearchthatstartswitharesearchquestionandhypothesesandusesgenerally acceptedacademicresearchprocessestoanswerthequestion.Thisarticlestrengthened theconclusionsreachedintheauthors’priorstudiesabouttheeffectsofsimulation PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 104 history‐keepingandundoonlearning.Theconclusionsaboutthebenefitofaddingthe advancedfeaturestothesimulatorappeartobebasedonmixedresults.More discussionwouldbeusefulofthePhaseIIobservationthatthegroupswithhistoryand theabilitytousetheseadvancedfeaturesandundounderperformedthecounterpart groupswithoutaccesstotheadvancedfeatures. Additionalresearchdemonstratingthatskillslearnedplayingthissimulation gamearetransferabletotherealworldwouldbeusefulaswellasresearch demonstratingthatthosewhoimprovemorewhileplayingthisgamearemoreeffective projectmanagersintherealworld. Davidovitch, Parush and Shtub (2010).Inthepriorarticles,Davidovichetal. (2006;2007;2008;2009)discussedresearchonusingthePMTsimulatorasanELAtool forindividuallearners.Thisarticledescribestheuseofasimilarresearchparadigmwith amultiuserversionofthesimulatorcalledProjectTeamBuilder(PTB)toexplore trainingprojectmanagerstocollaborateonresourceusageinasimulatedmatrix organizationrequiringthesharingofresources. Fifty‐threereferencesarecitedintheirintroductionanddiscussionofthe importance,challengesandmechanismsofwhatDavidovitchetal.(2010)term “collectiveprojectlearning.”Thisdiscussionincludesthebenefitofgroupintegrationof knowledgewhere“gapsinindividualknowledgecanbefilledinbytheknowledgeof otherteammembers,”theneedinamatrixorganizationforcollaborationinanoften competitiveenvironment,benefitsandtypicaldeficienciesofpostprojectreviews,key successfactorsforandbarriersagainstimprovingprojectlearning,Kolb’s(1984) experientiallearningmodel(emphasizingthe“criticalroleofreflectiveobservation”) anditsextensiontotheKolbTeamLearningExperience(asdescribedbyKayes,Kayes, andKolb(2005a;2005b)).Anadditionalsevenreferencesarecitedinthediscussionof results. Theexperimentaldesignforthisstudyagainusedcumulativeprofitattheendof thesimulationrunasthedependentvariableandmeasureofeffectivenessformanaging theproject.Independentvariableswere:(1)historyrecording(using,notusing),(2) teamdebriefingprocedure(structured,informal),and(3)experience(graduatestudent withprojectmanagementexperience,undergraduatewithoutprojectmanagement experience).Thestudyusedeightgroupsofteams,fullycrossingallexperimental conditions.Eachteamwascomposedofthreeteammembers[assumedtoberandomly PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 105 selectedbasedontheauthors’pastarticles(Davidovitchetal.,2006;Davidovitchetal., 2007;Davidovitchetal.,2008;Davidovitchetal.,2009),butnotspecifiedinthisarticle]. InPhaseI:BasicTeamScenario,eachteammembermanagedasingleseven activityprojectwithresourcesassignedfromaresourcepoolcommontothethreeteam members(21totalactivities).Eachscenariowasrunfourtimes.InPhaseII:Transfer,a moredifficultmultipleprojectscenariowaspresentedwithone16activitysub‐project andtwoeightactivitysub‐projects(32totalactivities).Bothscenarioshadduedates andbonus/penaltyawardsforearly/latecompletion.Theauthorsdidnotstatewhether thethreeteammembersagainmanagedonesubprojecteachinPhaseIIorwhetherthey collectivelymanagedallthreeasasingleprojectmanagerteam.(Davidovitchetal., 2010) Atotalof132studentsparticipatedassubjectsinthestudy:60graduate engineeringstudentsintheexperiencedgroupand72undergraduateengineering studentsintheinexperiencedgroup.Agesrangedfrom18to45andstudentswereboth maleandfemale(nofurtherdescriptivestatisticswereprovided).Thisimpliesthere were20teamsintheexperiencedgroupsand24intheinexperiencedgroups.History keepingisdescribedinthepriorarticles(Davidovitchetal.,2006;Davidovitchetal., 2007;Davidovitchetal.,2008;Davidovitchetal.,2009).Thegroupswithdebriefing wereabletochatviaatoolembeddedinthesimulationwhilecompletingarunandused awrittenoutlinetoconductastructureddebriefingaftercompletingtherun.An appendixtothearticledescribedthedebriefingprocedure.Groupswithoutdebriefing werepermittedtodebriefinformally,it’snotclearwhetherornottheyhadaccesstothe chatfeatureduringarun.(Davidovitchetal.,2010) Agraphofmeanprofitresultsfortheeightgroupsshowedatrendof improvementoverthefivesimulationrunswiththefourgroupswithdebriefingscoring thehighestafterrunstwothroughfive;thehighestscoringgroupafterrun5wasthe experiencedgroupwithhistoryanddebriefing.Forthefirstrun,nosignificant differencesbetweenthefactorswerefoundexceptformeanprofitwith/withouthistory. Thisverifiedthatallstudentsstartedwithsimilarknowledge,andunliketheprior studies,thehistoryfeaturewasfoundtoenhancelearningduringthefirstrun.ANOVA withrepeatedmeasurestestsindicatedthatforthePhaseI:BasicLearningsequenceof foursimulationruns: PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE  106 Theexperienceoftheexperiencedgroupenhancesteamlearningforall fourruns....  Thehistorymechanismenhancesteamlearningforthefirstrunaswell asfortheadditionalthreeruns....  Thedebriefingmechanismenhancesteamlearning....  Learningisbetterforexperiencedprojectmanagersgroupshaving historymechanismthanexperiencedprojectmanagerswithouthistory mechanismcapabilitiesduringthefourruns....  Learningisbetterforexperiencedprojectmanagersgroupshaving debriefingprocedurethansuchexperiencedprojectmanagerswithout debriefingprocedureduringthefourruns....[and]  Learningisbetterforusershavinghistorymechanismthatuses debriefingprocedurethanforusershavinghistorymechanismwithout debriefingprocedure,duringthefourruns.(Davidovitchetal.,2010) Davidovitchetal.(2010)reportedthatfollowingPhaseII:Transfer,themean profitoftheexperiencedgroupswashigherthanthemeanprofitoftheinexperienced groups;however,areviewoftheplotofmeanprofitresultsvs.runontheprovided graphshowstheinexperiencedgroupwithdiscussionandnohistoryoutperformingthe experiencedgroupwithdiscussionandnohistory.Thisobservationisnotdiscussedin thearticle. AnalysisofPhaseII:TransferANOVAtestresultsledtothefollowingsummaryof results:  Theexperiencedprojectmanagersgroupsenhanceteamlearningduring thetransfertoadifferentscenariophase....  Thehistoryfactorenhancesteamlearningduringthetransfertoa differentscenariophase[note:thehistorymechanismwasnotavailable tostudentsduringthisphaseintheearlierresearchperformedbythe authors....(Davidovitchetal.,2006;Davidovitchetal.,2007;Davidovitch etal.,2008;Davidovitchetal.,2009).Thisarticledidnotstipulate whetherornotthehistorymechanismwasavailableforuseduringthe simulationrunofPhaseIIduringthisstudy] PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE  107 Thedebriefingfactorenhancesteamlearningduringthetransfertoa differentscenariophase....  Nosignificantinteractionwasfoundbetweenexperiencedproject managers’factorandhavinghistorymechanism....  Nosignificantinteractionwasfoundbetweenexperiencedproject managers’factorandusingdebriefingprocedure....[and]  Nosignificantinteractionwasfoundbetweenhavinghistorymechanism andusingdebriefingprocedure.Davidovitchetal.(2010) Theselastthreefindingsappeartocontradictastatementinthediscussionof resultsthat“theinteractionoftheexperiencedgroupwiththehistorymechanismwas significant.”WhilethiswasfoundtobethecaseduringPhaseI:BasicLearning,itdoes notappeartobethecaseduringPhaseII:Transfer.Furtherdiscussionofthese observationswouldclarifythis. Summary.ThiscontrolledexperimentbyDavidovitchetal.(2010)demonstrated thatthepriorfindingofenhancedperformanceofindividualstudentshavingaccesstoa historymechanismasindividualsusingthePMTsimulatorisalsothecaseforteamsof studentscollaboratingonresourcesharingusingthePTBsimulator.Thisstudyfurther demonstratedtheadvantageofteamsusingastructuredprocesstodebrieffollowing completionofaprojectrun. Project management research summary.Mostarticlesdiscussingtheacademic useofsimulationgamesdescribefindingsrelatedtotheuseofmarketing,strategy managementortotalenterprisemanagement.Anunderresearchedareaistheuseof projectmanagementsimulations.Onlyfifteenarticlesresearchingprojectmanagement simulationwerefoundinavarietyofjournalsandproceedingsandnoneinSimulation andGamingorDevelopmentsinExperientialLearning,thejournalandproceedingsof ABSEL.All15ofthesearticlesmadeaknowledgecontribution,butonlysevencouldbe consideredacademicallyrigorous,andfiveofthissevendocumentedaseriesofstudies bythesameresearchers.ExceptforthisseriesbyDavidovitchetal.(2006;2007;2008; 2009;2010)whichequatedlearningeffectivenesstotheamountofsimulatedprofit,a questionablepractice,alltheprojectmanagementsimulationstudiesmeasuredstudent perceptionsandattitudes.Ofthese,onlyMcCreery(2003)andPfahl(2004)reported doingthisinanacademicallyrigorousmanner.Bothusedacombinationofpretest‐ PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 108 posttestmethodswithsignificantfindingsandaqualitativeanalysisofstudent commentstoconcludetheuseofthesimulationwaseffectiveandaddedvaluetothe educationalexperience.Acommonthemeamongstallthearticlesisstudentslikedthe overallexperience,thecontextitprovidedforimprovingskillsandtheopportunityit providedtopracticetheirdecision‐makingskills. Summary Simulationgamingasanexperientiallearningactivityisunderpinnedbythe seminalworkofKolb(Kolb,1984)buildingonthe“intellectualoriginsofexperiential learningfromtheworksofJohnDewey,KurtLewinandJohnPiaget.”Theuseof businesssimulationgamesinacademicprogramshasgrowndramaticallysincethe AmericanManagementAssociationdevelopedasimulation‐basedtrainingcoursein 1956(Fariaetal.,2009).Despitethisgrowthandtheassociatedresearchattestingtoits effectiveness,controversyremainsaboutthebestwaytoevaluatethiseffectivenessand therearerepeatedcallsformorerigorousresearchonthemethodsusedtoadminister simulationgamesandonassessmentofthelearningthatresults(P.H.Anderson& Lawton,2009;Crookall,2010;Fariaetal.,2009).Thisisespeciallytrueforproject managementsimulationgameswhereveryfewarticlesarefoundintheliteraturethat canbeconsideredacademicallyrigorous. Oneareawherethereappearstobeagreementisonthedifficultyofassessing learningbecausesimulationgameloserscanlearnmorethanthewinnersandvaluable lessonscanbelearnedthathavenothingtodowiththeeducator’sintention(Gosenpud, 1990;Gosenpud&Washbush,1994;Greenlaw&Wyman,1973).Accordingly,the statementofbroadlearningobjectivesforsimulationgameactivitieshasbeendeemed acceptable,ifnotdesirable(Hertel&Millis,2002;Jones,1987)withstudentsdefining whattheyfindmostvaluable(Gosenpud,1990).Theimportanceofdebriefingtohelp makethislearningsalienthasbeenemphasized(Lederman,1984;Lederman,1992)and reemphasized(Crookall,2010).DespiteAndersonandLawton’s(2009)callforless assessmentonperceptionsoflearninginsimulations,thissuggeststhemeasurementof perceptionsoflearningmaybeasimportantasfindingvalidmeasuresofcognitive learningandmoreresearchintowhatislearnedisappropriate(Teach&Murff,2007; Teach&Murff,2009).RespondingtoAndersonandLawton’scall;Cronan,Leger,Robert, BabinandCharland(2012)comparedtheresultsobtainedfromapostsimulationself‐ assessmentsurveywithresultsfroma60questionobjectiveexaminationandfoundself‐ PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 109 assessedmeasureresultscorrelatedwithobjectivemeasures.Thissuggestsself‐ assessmentsurveysarevalidproxiesforlearning. AlthoughtheprojectmanagementsimulationresearchbyDavidovitchetal. (2006;2007;2008;2009;2010)isquiterigorous,itsfocusonasinglemetric,simulated projectprofit,doesnotnecessarilydemonstratewhatthestudentshavelearnedfrom thisexperienceotherthantheabilitytodobetteratplayingthegame.Thereisno discussionofwhatprojectmanagementskillsarebeingimprovedandwhetherthese improvedskillsaretransferabletothepracticeofmanagingrealprojects,apoint concededintheir2008article.Theirresearchisimportantbecauseitdemonstrates rigorousresearchpracticesandtheuseofaprojectmanagementsimulationgame performancemetricasadependentvariableinacademicresearch.However,theriskof solelybasingconclusionsoflearningeffectivenessonsuchametricisthatitdoesnot takeintoaccountthatlearningcanoccurwhilelosingthegame.Asnotedearlier, GreenlawandWyman(1973)foundthat“strong‘learning’ingamesmaynotnecessarily bereflectedbygoodgamingperformance”and“converselyabsenceoflearningmaynot alwaysbereflectedby‘poor’performance.”GosenandWashbush(2004)concluded simulation“performanceisnotaproxyforlearning,anditisinappropriatetoassess simulationsusingperformanceasameasureoflearning.” Amoreappropriatestartingpointforresearchintothevalueandeffectivenessof usingaprojectmanagementsimulationasacapstoneexperientiallearningisthe researchbyMcCreery(2003).Hisresearchmeasuredstudentperceptionsofproject managementknowledgeandtheabilitytoapplythatknowledgebeforeandafter experiencingaprojectmanagementsimulationgameandfoundsignificantincreasesin both.Areplicationofthisaspectofhisstudyusingadifferentsimulationinadifferent venuewouldaddtothebodyofknowledgeand,ifsimilarresultsarefound,suggest generalizability.ToexpandonMcCreery’swork,questionsfromtheresearchconducted byBuzzetto‐MoreandMitchell(2009)andAhn(2008)canbeadaptedtothe longitudinalmethodologyusedbyMcCreerytoadditionallyresearchchangesinstudent perceptionsofsimulationgamesandMcCreery’smethodcanbesimilarlyadaptedto studyperceptionchangesrelatedtoteam‐basedlearningactivities.Tobegintoaddress AndersonandLawton’s(2009)advocacyformoreobjectivemeasures,assessmentsof cognitivelearningrelatedtoprojectmanagementschedulingandprogressreportingcan PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 110 alsobeadded.TorespondtoTeachandMurff’s(2009)callforresearchonwhatis learned,qualitativequestionscanbeusedtoprobeforthis. Thischapterreviewedtheliteraturerelatedtousingsimulationgamesas experientiallearningactivitiesinacademicbusinessandprojectmanagementdegree programs.Followingadiscussionofexperientiallearningandrecommendationsfor conductingsimulationgames,examplesoftypicalresearchwereexploredforboth businessmanagementandprojectmanagementgames.Additionalresearchisneeded andastartingpointforadditionalprojectmanagementresearchisidentified.The methodologyforconductingthisresearchisdescribedinthenextchapter. PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 111 Methodology Thisresearchisastudyofoneuniversity’sapproachtousingaparticularproject managementsimulationgameasacapstoneexperientiallearningactivityinitsgraduate MS/MBAprogramforexperiencedprofessionals.Itspurposeistodetermineifthis applicationofaprojectmanagementsimulationgameisregardedasavaluableand effectiveuseofclasstime.ThemethodologybuildsontheapproachtakenbyMcCreery (2003)usingadifferentprojectmanagementsimulationandadifferentdelivery approachatadifferentuniversity.ThisstudyreplicatesMcCreery’sapproachto measurechangesinperceptionsofprojectmanagementknowledgeandtheabilityto abilitythisknowledgebycomparingpresimulationandpostsimulationsurvey responses.Inadditiontothisreplication,changesinattitudesandperceptionsof simulationgamesandteamdynamicsareexploredbyaddingadaptionsofMcCreery’s teamexperiencequestionsandBuzzetto‐MoreandMitchell’s(2009)andAhn’s(2008) simulationsatisfactionquestions.Analysisofpostsimulationqualitativedataisusedto supportthefindings.Similarfindingswillsuggestthegeneralizabilityofresultstoother deliveryapproachesusingprojectmanagementsimulationgamesinanacademic environment. Thischapterdiscussestheresearchparadigm,researchapproach,research design,surveyinstruments,pilotstudy,researchparticipants,administrationprocedure, dataprocessingandanalysisprocedure,andethicalconsiderations. Research Paradigm Thisresearchusesmixedmethodstoanswertheresearchquestionofwhetheror nottheuseofaprojectmanagementsimulationasacapstoneactivityinaparticular executivelevelMBAprogramisavaluableandeffectiveuseofclasstimeandtuition dollars.Mixedmethodsresearchisaninquiryapproach“thatcombinesorassociates bothqualitativeandquantitativeformsofresearch”(Creswell,2009).Theuseof concurrent(parallel)mixedmethodsproceduresprovidesforamorecomprehensive analysisoftheresearchproblem(Creswell,2009;Mertens,2010).Sidebysideanalysis ofthequantitativeandqualitativeresultsallowseachtoreinforceeachother(Creswell, 2009),e.g.,inthecaseofthisstudy,findingsofsignificantgainsinknowledgeand confidenceintheabilitytoapplythatknowledgearefurtherunderstoodandmaybe reinforcedbythelearningthemesthatemergefromthequalitativeanalysis. PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 112 Conventionalwisdomholdsthattheresearcher’sviewoftheworldandhis chosenparadigmunderpinsthechosenresearchapproach;however,theuseofmixed researchmethodscanbeproblematicfromaphilosophicalperspective.Mostmixed methodresearchersstatepragmatismfortheirparadigmbutnotallresearchers“agree withparadigmpluralismasastartingpoint”(Tashakkori&Teddlie,2010a).Thisis furtherexasperatedbydisagreementregardingtheneedtoevenacknowledgean underlyingparadigmandtheroleparadigmsserveintheresearchprocess(Mertens, 2010).“Thea‐paradigmaticstancestatesthat,formanystudiesconductedwithinreal worldsettingsespeciallyinappliedfields,paradigmsorconceptualstancesare unimportanttopractice(e.g.,TashakkoriandTeddlie,2003)”(Tashakkori&Teddlie, 2010b). Thedebateregardingwhichparadigmservesasthefoundationformixed methodsresearchhasresultedintheidentificationofseveralconceptualstancesin additiontothesingleparadigmstancewhichincludespragmatism,criticalrealism,and thetransformativeparadigm(Tashakkori&Teddlie,2010b).TashakkoriandTeddlie describefourothers:thesubstantivetheorystanceclaimsthattheoriessuchascritical racetheoryorattributiontheoryaremoreimportantthanphilosophicalparadigms;the complementarystrengthsstancestatesthatthedifferentmethodsshouldbekeptas separateaspossibletocapitalizeonthestrengthofeachparadigmaticpositionsuchas constructivismorpostpositivism;themultipleparadigmsstancearguesthatasingle paradigmdoesnotapplytoeachinstanceofmixedmethodsresearch,thebestparadigm isresearchsituationspecific;andthedialecticstanceassumesallparadigmsareuseful andcanbecombinedinasinglestudytoprovideamorethoroughanswertothe researchquestion. Whilepragmatismisfrequentlycitedalongwithrealismasparadigmsformixed methodsresearch,theunderlyingphilosophyofthisstudyisbestdescribedbythe paradigmofpostpositivismowingtoitsprimaryuseofquasi‐experimentalquantitative methodsandcomparisonofresultswithanotherstudysuggestingthepossibilityof generalizabilitytoonereality.Althoughaqualitativemethodisusedinthisstudyto reinforceorrefutetheconclusionsfromthequantitativestudy,thismethodcollected andanalyzednarrativeresponsedatainadispassionate,objectivemannersuggestingan epistemologymoreassociatedwithapostpositivismparadigmthanaconstructivistor pragmaticparadigm(Mertens,2010).Thissimultaneouscollectionofquantitativeand PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 113 qualitativedatawiththeoreticaldriveorprioritygiventothequantitativedataanalysis supplementedbyqualitativeanalysiscanbereferredtobythenotation“QUAN+qual” wherethecapitalletters“indicatethetheoreticaldriveorpriority(coremethods)given inastudy”(Creswell,2010). Research Approach TheQUAN+qualapproachtakeninthisresearchisdepictedinFigure3.The quantitativeapproachtothisstudyadministersthesamesurveyquestionsbeforeand afterthestudentsexperiencetheprojectmanagementsimulationgame.Thequalitative approachsupportsorcontradictsthesefindingswithananalysisofnarrativeresponses toadditionalpostsimulationsurveyquestions. FIGURE3‐RESEARCHAPPROACH Quantitative method.Thequantitativeapproachtothisstudyusesapretest‐ posttestpreexperimentaldesignasillustratedinFigure4.Datawascollectedfromfour groupsofstudents.Sequencedintime,GroupAwasapilotstudyconductedwithanon‐ academiccorporateclasstopilotthemethodologyandGroupsBandDwereayearapart inthegraduateprogramofinterest.GroupCwascomposedofstudentsfromasystems engineeringandmanagementgraduateprogramwithasingleprojectmanagement course.GroupAresultsarediscussedinthischapterandGroupBandDresultsare discussedintheResultschapter.AnalysisofGroupCresultsisoutsidethescopeofthis studyandanopportunityforfurtherresearch.GroupsA,B,CandDarelaterreferredto asCohortPD,OC2010,SEM2011andOC2011,respectively. PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 114 FIGURE4‐PRETEST‐POSTTESTDESIGN Group A: O1A ‐‐‐‐‐‐‐‐‐‐ X ‐‐‐‐‐‐‐‐‐‐ O2A Group B O1B ‐‐‐‐‐‐‐‐‐‐ X ‐‐‐‐‐‐‐‐‐‐ O2B Group C O1C ‐‐‐‐‐‐‐‐‐‐ X ‐‐‐‐‐‐‐‐‐‐ O2C Group D O1D ‐‐‐‐‐‐‐‐‐‐ X ‐‐‐‐‐‐‐‐‐‐ O2D O1 = Presimulation survey O2 = Postsimulation survey X = Simulation game Studentperceptionsaresurveyedimmediatelypriortobeginningthesimulation gameandagainimmediatelyaftertheconcludingdebriefdiscussion.Analysisofthedata isperformedtodeterminethesignificanceofthedifferences. Qualitative methods. Open‐endedquestionsinthepostsimulationsurveyand presentationsmadeduringthesimulationdebriefprovidequalitativedataforfurther assessmentofparticipantattitudestowardsthevalueofthesimulationgameexperience. Additionalqualitativedataiscollectedfollowingcompletionofthe21SCHProject ManagementCorePhaseoftheMS/MBAgraduateprogramtoassesssatisfactionwith theoverallprojectmanagementlearningexperience.Thislatterdataisscannedfor commentsregardingthesimulationexperienceandananalysisofthisdataisincluded withthequalitativeassessmentresults. Research Design Variables.Perceptionsmeasuredareprojectmanagementknowledge(K1 [presimulation]andK2[postsimulation]),confidenceintheabilitytoapplythat knowledge(C1andC2),usefulnessofsimulationgamesingeneral(SG1andSG2), usefulnessofthesimulationgameexperienced(SP1andSP2),attitudetowards teamworkingeneral(TG1andTG2),andcurrentteamsatisfaction(T1,TP1,T2and TP2).Alsoassessediseachstudent’sabilitytosolvefundamentalproblemsinvolving thecriticalpathmethodandearnedvaluemanagement(E1,N1,P1andE2,N2andP2, whereP1=E1+N1andP2=E2+N2).Theconstructsforthesevariablesandthe demographicdatacollectedisdiscussedlaterinthischapterinthesectiondiscussingthe surveyinstruments. PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 115 Method.FollowingMcCreery’s(2003)approach,t‐testsformatchedpairs (dependentt‐tests)determinethestatisticalsignificanceofdifferencesbetweenthe presimulationandpostsimulationmeasurementsforeachofthecohortgroups. RelationshipstotheresearchhypothesesofthisstudyaredepictedinFigure5.The resultsofstatisticaltestsareconsideredsignificantforp<.05. Qualitative Analysis.Thestatisticalresultsarecomparedwithacategorizationof thequalitativefindingscollectedduringthepostsimulationdebriefsessionsand reportedonthepostsimulationandprogramevaluationsurveys. FIGURE5‐RESEARCHMODEL Survey Design SurveyquestionsarebasedonthepriorworkofMcCreery(2003),Buzzetto‐ MoreandMitchell,(2009),andAhn(2008).Thepresimulationsurvey,foundin AppendixC,iscomposedof74questions.Fiveofthesequestionsaredemographicand 58combinetoformtheattitudeandperceptionvariables.Table3liststhevariablesand theirassociationwiththe58attitudequestions. PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 116 TABLE3‐ATTITUDEVARIABLEQUESTIONS Variable (1=presimulation; 2=postsimulation) Knowledge Self‐Assessment (K1, K2) Knowledge Application Confidence (C1, C2) Teamwork Attitude (McCreery) (T1, T2) Current Team Satisfaction (TP1, TP2) Teamwork Attitude (Generic) (TG1, TG2) Generic Simulation Attitude (SG1, SG2) This Simulation Attitude (SP1, SP2) Questions 16 questions: 1A‐1F 16 questions: 2A‐2F 9 questions: 3A‐3I 10 questions: 3A‐3I, 4N 2 questions: 4C, 4K 6 questions: 4A, 4B, 4E, 4G, 4I, 4L 8 questions: 4D, 4F, 4H, 4J, 4M, 4O, 4P, 4Q Theremaining11questionscomprisetheknowledgeapplicationtestquestions relatedtonetworkdiagramsandearnedvaluemanagement.Table4showsthe associatedvariablesandtheirrelationshiptothequestions.Thenetworkanalysis questionsareadaptedfromanexercisefoundinthe“DevelopingaProjectPlan”chapter ofoneofstudents’projectmanagementtextbooks(Gray&Larson,2008).Theearned valuequestionsaskthestudenttoanalyzetheprogresspresentedinafigurefoundin the“ProgressandPerformanceMeasurementandEvaluation”chapterofthesametext (Gray&Larson,2008). TABLE4‐PROBLEM‐SOLVINGQUESTIONS Variable (1=presimulation; 2=postsimulation) Network Analysis Ability (N1, N2) Earned Value Analysis Ability (E1, E2) Problem‐solving Ability (P1, P2) Questions 4 questions: 5A‐5D 7 questions: 6A‐6G P1 = N1 + E1, P2 = N2 + E2 Table5liststhedemographicdatacollectedinthepresimulationsimulation. TABLE5‐PRESIMULATIONDEMOGRAPHICVARIABLES Years Project Experience Years Work Experience Gender Prior Formal Project Management Training (type and duration) Background (technical/non‐technical) Question sources.The16knowledgeself‐assessmentquestions(1A‐1F),16 knowledgeapplicationconfidencequestions(2A‐2F)and9of10currentteam satisfactionquestions(3A‐3I)areidenticaltothoseusedbyMcCreery(2003). PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 117 Simulationattitudequestions4Athrough4Mareslightre‐phrasingsofBuzzetto‐ MoreandMitchell’s(2009)questionstoadaptthemforuseinpretestandposttest surveys.Questions4Othrough4QarefromasurveyusedbyAhn(2008).Inbothof thesecases,theresponsescalewasexpandedfromfivepointstosevenpointstobe consistentwiththescalesusedbyMcCreery(2003):  Questions1and2:1=ExtremelyLowto7=ExtremelyHigh  Questions3and4:1=StronglyDisagreeto7=StronglyAgree. Thepostsimulationsurveyquestionsareidenticaltothepresimulationsurvey questionsexceptthe“thissimulation”attitudequestionsarere‐phrasedtoaskabout impressionsofthesimulationjustcompletedversuspreviouslyaskingabout expectationsfortheupcomingsimulation.Forexample,thepresimulationquestion,“I believethissimulationwillbefunandexciting”changedto“thissimulationwasfunand exciting”inthepostsimulationsurvey.Thefivedemographicquestionsofthe presimulationsurveywerereplacedbythesixdemographicquestionslistedinTable6 andthefiveopen‐endedquestionslistedinTable7.Thepostsimulationsurveyisfound inAppendixD. TABLE6‐POSTSIMULATIONDEMOGRAPHICVARIABLES Industry Job Title Education Level Age Global Origin Annual Income TABLE7‐POSTSIMULATIONOPEN‐ENDEDQUESTIONS What did you like about the just‐completed simulation experience? What did you learn from this experience? Excluding changes to the computer simulation itself, what changes do you think could be made to the overall simulation experience to better prepare you to manage projects? What changes do you recommend be made to the computer simulation itself? Elaborate on any other thoughts you have regarding learning project management with a computer‐based simulation game. Other qualitative data sources.Inadditiontothepresimulationand postsimulationsurveys,qualitativedataisderivedfromthesimulationdebriefteam PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 118 presentationsthatprecedetheadministrationofthepostsimulationsurveyandfromthe ProjectManagementCorePhaseexitsurvey.Duringthedebrief,eachteampresentsits finalstatusreporttomanagementdiscussingitsmetricsandvariances,aprojectaudit reportdescribingtheteamstrategyandlessonslearned,andrespondstothequestion, “WhatdidyoulearnaboutprojectmanagementfromparticipatingintheSimProject simulation?”Thehandoutdescribingthedeliverablesassociatedwiththisteamdebrief presentationinfoundinAppendixE.Table8listsrelevantopen‐endedquestionsfrom theProjectManagementCorePhaseexitsurvey. TABLE8‐RELEVANTPROJECTMANAGEMENTCOREPHASEEXITSURVEYQUESTIONS If you were given an option to change only 3 things in the Project Management Core Program, what would those 3 things be, and why would you change them? If you were given an option to retain only 3 things in the Project Management Core Program, what would those 3 things be, and why would you keep them? Survey Structure.Toeasethementalburdenontherespondentandtoprovidea clearframeofreference,questionsweregroupedintosectionsbasedontheiroverall topicandscaletypewitheachsectionpresentedononepageinamatrixformat.While someauthorsofe‐mailandweb‐basedresearchbooksdiscouragetheuseofmatrix questionsbecauseofextraworkwithinascreen(Schonlau,Elliott,&Fricker,2002), othersemphasizethegeneralneedtogivetherespondentthesenseofasmooth progression,thusfacilitatingarapidprogressionthroughthequestionnairebygrouping questionswiththesamescaletogether(Czaja&Blair,2005).AlreckandSettle(1995) advisethatgroupingitemsbybothtopicandscaletypeis“theidealsituationbecause therearebothalogicalsequenceandahighdegreeoftimeandspaceeconomy.”Cooper andSchindler(2006)emphasizetheimportanceofprovidingastrongandobviousframe ofreferenceforthequestionswithclearlydefinedtransitionsbetweensectionsto delimitchangesinframeofreference. Groupingquestionsbytopic,whichinseveralcaseshereisalsogroupingthe questionsconstructingavariable,cancauseconcernoverorderingeffectswith respondentsselectingthesameratingforallanswers.Areviewofthedatacollected showedthisrarelyoccurred,andwhenitdid,itwasneverforallsectionsofthesurvey. Sincethetotalquestioncountwasabove70,followingtheaforementionedadvicethatit isbettertoimprovethecooperationoftherespondentsandreducetheircognitive PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 119 burdenbyorderingthequestionsbytopicinsteadofrandomizingthemappearstobe valid. ConsistentwithguidelinesofferedbyCzajaandBlair(2005)basedonthework ofDillman(2000),thepapersurveysweresevenpagesinlengthwithsufficientlyspaced precodedresponsecategoriesarrangedvertically.Topicalquestionsusingthesame scaleweregroupedtogetheronthesamepage.Toavoidconfusion,noreversingof questionswasdone. Thenetworkanalysisknowledgeapplicationquestionspresentedthe respondentwithatableofsevenactivitieswithpredecessoranddurationdataandasked twoquestionsaboutthecriticalpathandtwoquestionsaboutactivityslacktime.Each questionwasscoredaoneforacorrectanswerandazeroforanincorrectanswer.A respondentansweringallfourquestionscorrectlyearnsascoreoffour. Theearnedvalueknowledgeapplicationquestionspresentedtherespondent withacumulativeearnedvaluegraphshowingPV,EV,AC,BAC,andEACforaprojectin progressandaskedsevenquestionsrequiringanalysisandinterpretationofthe graphicaldata.Correctanswersscoredaoneandincorrectanswersscoredazero.A respondentansweringallsevenquestionscorrectlyearnsascoreofseven. Pilot Study Thesurveywaspilotedusingthe19participantsinGroupA,acorporate professionaldevelopmentseminarserieswithsimilarcontenttotheproject managementmaster’sdegreeprogramofinterest.UnlikeMcCreery’s(2003)results, whichwerebasedon63participants,thematched‐pairt‐testsyieldedsignificant increasesforsome,butnotall,individualquestionmeans;similartohisresults,these testsdidyieldsignificantdifferencesfortheknowledgeandconfidencetoapplythat knowledgegrandmeanscomprisingtheKandCvariablesunderstudy. Thereliabilityofusingthesegrandmeans,exceptgenericteamworkattitude (TG)whichisthemeanresponseofonlytwoquestions,wasconfirmedasthescales representedbytheK,C,TP,SG,andSPvariableshadCronbach’sα>.8.Thesevaluesare listedinTable9. PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 120 TABLE9‐PILOTTESTSCALERELIABILITY Variable K C TG TP SG SP Description Knowledge Confidence to apply knowledge Teamwork attitude Current team satisfaction Generic simulation attitude This simulation attitude Cronbach's α 1 2 (Pre‐sim.) (Post‐sim.) .934 .938 .948 .963 .771 .606 .95 .96 .941 .883 .895 .887 Onaverage,participantsreported:  Significantincreaseinprojectmanagementknowledge(M=5.29,SE= 0.15)fromtheirpresimulationperceptions(M=4.80,SE=0.16),t(18)=‐ 7.72,p<.05,r=.88.  Significantincreaseinconfidenceinabilitytoapplyprojectmanagement knowledge(M=5.16,SE=0.18)fromtheirpresimulationperceptions(M =4.80,SE=0.16),t(18)=‐3.07,p<.05,r=.59. Favorableattitudes,butnosignificantincreaseswereseenfortheteamand genericsimulationperceptions.Onaverageparticipantsreported:  Nosignificantincreaseingeneralteamworkattitude(M=5.53,SE= 0.196)fromtheirpresimulationperceptions(M=5.58,SE=0.173),t(17) =0.399,p>.05,r=.10.  Nosignificantincreaseincurrentteamsatisfaction(M=5.526,SE= 0.285)fromtheirpresimulationperceptions(M=5.500,SE=0.232), t(18)=‐.122,p>.05,r=.03.  Nosignificantincreaseingeneralattitudetowardstheuseofsimulations asalearningactivity(M=5.15,SE=0.201)fromtheirpresimulation perceptions(M=5.13,SE=0.19),t(17)=‐0.138,p>.05,r=.03. However,onaverage,participantsdidreportasmallbut  Significantincreaseinfavorableattitudetothisparticularsimulation experience(M=5.35,SE=0.211)fromtheirpresimulationperceptions (M=5.1,SE=0.190),t(17)=‐1.968,p<.05,r=.43. PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 121 Ontheschedulenetworkandearnedvalueanalysisproblems,participants demonstrated  Nosignificantincreaseintheirabilitytosolveschedulenetworkanalysis problems(M=2.68,SE=0.316)fromtheirpresimulationability(M= 2.58,SE=0.336),t(18)=‐1.000,p>.05,r=.22.  Significantincreaseinintheirabilitytosolveearnedvalueanalysis problems(M=3.6316,SE=0.406)fromtheirpresimulationability(M= 2.68,SE=0.297),t(18)=‐2.282,p<.05,r=.47.  Significantincreaseintheiroverallabilitytosolveanalysisproblems(M =6.3158,SE=0.490)fromtheirpresimulationability(M=5.26,SE= 0.512),t(18)=‐2.727,p<.05,r=.54. Itisnoteworthythatthisprofessionaldevelopmentcohortsimulationexperience includedaninstructor‐led20minutediscussionoftheearnedvaluemanagement methodologywiththeentireclassduringoneoftheinterimprogressreportingmeetings andonewouldexpecttheretobeasignificantincreaseinabilitytosolveearnedvalue analysisproblems.InkeepingwithHeyman’s(1975)advicethat“youarenotthereto teach...thesimulationdoestheteaching,”instructorledtutorialdiscussionsofthistype wereavoidedduringthesimulationactivityfortheMS/MBAcohortsthatarethesubject ofthisstudy. Participants Acombinedtotalof60peopleparticipatedinthefourgroupsexperiencingthe simulationgame.Table10liststhesimulationdates,totalclasstimeallottedtothe simulationgame,participantcountsandcontacthoursofinstructionpriortostartingthe simulationgameforeach.GroupA,thecorporatemanagers,participatedinthe previouslydescribedpilotstudy.GroupsBandDarethesubjectsofthisstudy;GroupC datawascollectedforafuturestudy.Individualclasssizesaresmallduetothe specializednatureoftheseprograms.Intheacademicgroups,studentswereassignedto teamstomaximizediversityatthefirstclassmeeting;theteamsinthecorporate programwerecomprisedofmanagersfromdifferentfunctionalareaswithincommon businesscomponents.Inallcases,studentsworkedtogethertocompleteteam assignmentsbeforebeginningtheprojectsimulation. PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 122 TABLE10‐SIMULATIONPARTICIPANTS Class Corporate managers (single company) Group A (PD) Project Management Graduate Program Group B (OC2010) Systems Engineering and Management Graduate Program Group C (SEM2011 Project Management Graduate Program Group D (OC 2011) Total number of participants Simulation dates / Total class time allotted March 23‐25, 2010 20 hours June 10‐12, 2010 16 hours April 15, 22, 29, 2011 12 hours June 10‐11, 2011 July 15, 2011 16 hours Number of Participants Contact Hours of Instruction Prior to Starting Simulation 19 176 14 220 13 28 14 204 60 Pilot study participants.Thepilotstudywasconductedattheconclusionofa professionaldevelopmentprogramheldfor19cross‐functionalmanagerschargedwith developingastrategytoimprovetheprojectmanagementpracticesandperformanceat theirenergyservicescompany.Thisprogramwassimilartotheacademicgraduate programunderstudyindurationandcontent,butwithouttheassessmentrigorrequired forgraduatecoursework.Classeswereheldfrom8AMto5PMinahotelmeetingroom ontwoconsecutivedayspermonthfor11monthswithlunchandbreaksnacks provided.Thetwelfthmonth’ssessionswereheldatthecorporateheadquartersto provideeachteamwithitsownbreakoutroomfortheconductofthesimulation.All19 managersparticipatedinthispilot.Onaverage,theyreported12.3yearsofwork experienceand7.9yearsofexperiencewithprojects;therewere14menand5women. Project management graduate program participants.Thegraduateprogram understudyischaracterizedbysmallclasssizesduetoitsspecializednatureand premiumtuition.Twenty‐eightstudentsfromtheprojectmanagementgraduate program(14inGroupBand14inGroupD)completedfiveprecedingproject managementcoursesandanintegratedcourseinorganizationalbehaviortotaling17 SCHofacademicstudy.Topicalcontentwassimilartothatreceivedbythecorporate managers(GroupA)butwithsubstantialacademicrigor.Thethirdclass(GroupC)was attheconclusionofasingleprojectmanagementoverviewcoursefromasystems engineeringandmanagementgraduateprogramwithsimilaradultlearners.Although notfromtheprogramunderstudy,thisthirdinstanceisofinterestbecauseofits PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 123 similaritytotheprogramstudiedbyMcCreery(2003)andisasubjectforpotential futureresearch. Classesinthegraduateprojectmanagementprogram(GroupsBandD)areheld ononeconsecutiveThursday‐Friday‐SaturdaypermonthintheExecutiveEducation AreaoftheSchoolofManagementbuildingfrom8AMto5PM;breakfast,lunchand breaksnacksareprovided.Studentscomplete9monthsofcourseworkpriorto participatinginthesimulationgame. Duetothesmallclasssizes,randomsamplingisnotperformed;allstudentsare invitedtoparticipateinthestudypriortothestartofthesimulation.Noincentivesto participateorpenaltiesfornotparticipatingareassessed.Theinstructorleavesthe roomandisnotawareofwhodidordidnotparticipateinthesurvey;however,mostdo participate. Procedure Inthissimulationgame(brieflydescribedinAppendixF),studentteamstakeon theroleoftheprojectmanagementcoreteamresponsibleforselectingandmanaging theresourcesassignedtoasimulatedproductdevelopmentproject.Thissection providesachronologicaldescriptionoftherelatedpresimulationinstructionand preparatoryassignments,presimulationsurveyadministration,simulation administration,andpostsimulationsurveyadministration. Presimulation instruction.Studentsparticipatinginthegraduateprogram leadingtoeitheraM.S.orMBAwithanemphasisinprojectmanagementcompleteover 200contacthoursofprojectmanagementandintegratedorganizationbehavior courseworkpriortobeginningexecutionofthesimulatedproject.Theyareintroduced tothesimulationgamethreemonthspriortoitsexecutionduringtheirfourthsequential projectmanagementcoursecoveringprojectplanningandexecutionintheformofan assignment.Theprimarypurposeoftheassignmentisforstudentstoapplythe conceptsofprojecttime,resourceandcostmanagementbyfirstdevelopingabaseline projectscheduleandbudgetfromgivenstandards,thenbydevelopingastaffing managementplanandarevisedbaselinescheduleandbudgetbasedonanalysisof availableresourcesandtheircharacteristics.Thesecondarypurposeoftheassignment isforthestudentstobecomefamiliarwiththesimulationscenarioandtoprepareaplan inadvanceofexecutionday.ThisassignmentisavailableforreviewinAppendixB. Aftersubmittingthebaselineplanfromgiveninformation,studentsreceivethecorrect PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 124 solutiontouseasthestartingpointforcompletingparttwooftheassignment;theteams presentthisrevisedplaninasimulatedmeetingwiththesponsoramonthlaterinclass. Presimulation survey administration.Instructionsforcompletingthesurveysare givenverballybytheinstructoralongwithassurancesofanonymityandvoluntary participation.Tohelpassureanonymity,theinstructorleavestheroomduring completionandcollectionofthesurveysisadministeredbyastaffmemberorstudent. Thepresimulationsurveyisadministeredatthebeginningoftheclassperiodpriortoa reviewofpresimulationquizandtheteamsbreakingouttobeginthesimulationgame. Allstudentscompletedthepresimulationsurvey. Simulation administration.Thesimulationisadministeredfollowingthe philosophythatstudentsshouldlearnfromtheexperienceandnotfromthefacilitator. Instructorinteractionwiththeclassoccursprimarilyduringthesimulationkick‐off briefing,duringinterimteamstatusreviewswithmanagementandduringthecombined finalpresentationtomanagementandsimulationexperiencedebrief.Studentsalso interactbrieflywiththeinstructorduringtheprocessinganddisplayofresultsattheend ofeachsimulatedworkperiod.Exceptforthepreviouslymentionedreviewofearned valuemanagementduringthepilot,theinstructordoesnotteachorconsultduringthe administrationofthesimulation. Simulation time allocation.Sixteenhoursofclasstimeareallocatedtoplaying anddebriefingthesimulationgame.Thistimeisdividedintofourfour‐hourclass modules.Morningmodulesrunfrom8AMtonoonandafternoonmodulesrunfrom1to 5PM.Playbeginsinanafternoonmodulestartingat1PMandendingat5PM. Module one – the first four hours.Duringthefirstmodule,studentsfirst participateinapresimulationkick‐offbriefingheldfortheentireclass,thenmeet privatelyasteamstoreviewtheirstrategyandentertheirresourcebidsforthefirst roundofpreplay.Onceallbidsarein,studentsreturntotheclassroomwherethe instructorinitiatesthesimulatorpreplayroundandstudentsreceivetheresultsoftheir preplaybids.Additionalpreplayroundsareheldasneededforteamsthatlostbidsand stillneedtohireresourcesforthefirstworkround.Preplayendswhenteamsare satisfiedtheyareadequatelystaffedandtheinstructorinitiatesthefirstworkround. Oneortwoworkroundsarecompletedduringthefirstfour‐hourmodule. PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 125 Thekick‐offmeetingisusedbytheinstructorto:  Administerthepresimulationsurvey  Describehowthesimulationwillbeadministered  Reviewtheresultsofthepresimulationquizasarefresheroftherulesof thegame  Answeranyquestionsregardingtherulesoradministration  Instructtheteamstobeginroundoneofpreplaybyenteringtheir resourcebids Thevariationinthenumberofworkperiodscompletedduringthissessionisthe resultofsometeamsneedingmorethanonepreplayroundtoobtaintheirstarting resources.Forthetwocohortgroupsunderstudy,GroupBrequired2preplayrounds andGroupDrequired3preplayrounds.Asaresult,GroupBcompletedonesimulated workperiodduringthisopeningsessionandGroupDcompletedtwo.Thisfirstfour‐ hourmoduleendswithallteamsmeetingtogetherwiththeinstructortoviewtheresults ofthejust‐endedsimulatedworkperiodandtoreceivetheassignmenttopreparea statuspresentationformanagementthatwillbedeliveredatthebeginningofthenext modulesessionfollowinga30minuteteamworkperiod.Inkeepingwiththecapstone natureofthisassignment,teamsmustdecideforthemselvesbasedonpriorlearningand experienceswhattheyaregoingtopresentandhowtheywillformatthepresentation. Nospecificationsortemplatesaregivenforthispresentationtomanagement. Module two – the next morning.Thesecondmodulebeginswithashortteam workperiodtocompletetheirpresentationpreparationandtoenterdecisionsforthe nextworkperiodround.Thisisfollowedbya“meetingwithmanagement”withthe instructoragainplayingtheroleastheprojectsponsor.Teamspresenttheirstatus, receiveencouragementbutnoguidance,andviewthecompletionresultsforthenext simulatedworkperiod.Threetofouradditionalworkperiodroundsarecompleted duringthisfour‐hoursession,eachwithtypically30minutestoreviewresults,discuss neededchangestotheplan,andenterdecisionsforthenextsimulatedworkperiod. Afterreceivingtheresultsofthefinalsimulatedworkperiodofthemodule,studentsare againadvisedtoprepareanotherinterimstatuspresentationformanagementwith contentandformatoftheirchoosingthatistobedeliveredatthebeginningofthethird four‐hourmodule. PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 126 Module three – schedule variation.Dependingonthecourseschedule,thethird four‐hoursimulationmodulebeginsafterlunchonthesameday(GroupB)oramonth laterwhenstudentsreturntocampus(GroupD).Thisdifferenceisduetoachangein coursescheduletoprovidemoretimeforcompletingassignmentsrelatedtothe nonsimulationportionofthecourse.Theextendedformatisplannedtobethenormfor futurecohorts.Duringthethirdmodule,studentscompleteanadditionalfivesimulated workperiods.Followingthereceiptofresultsforthesimulatedworkperiodconcluding thissession,studentsreceivethespecificationsforthefinalpresentation.Thefinal presentationisacombinedfinalprojectstatusreport,projectauditreportandstatement oflessonslearnedwithadefinedoutline.Thehandoutwiththerequirementsforthis presentationisshowninAppendixE. Module four – who won?Thefourthandfinalfour‐hourmodulebeginswitha shortworkperiodtocompletedecisionsforthenextroundor,inthecasewherethe simulationwascompletedattheendofthepriorsession,alongerworkperiodto completethefinalpresentation.Intheformercase,theremainingroundsarecompleted withinthefirsttwohoursofthemodule,anhourisallowedtopreparethefinal presentation,andthefinalpresentationsanddebriefareconductedduringthefourth hour.Inthelattercase,studentsareinformedofthetimeofthefinalpresentationand arepermittedtoscheduletheirclassarrivalandteampreparationtimeastheyseefit. Duringthesefinalstatuspresentations,theinstructorallowsthestudentstoself‐ facilitateandonlyasksprobingquestionsofthepresentingteamiftheotherteamsdo not.Followingthelastpresentation,theinstructorshareshisobservationsonwhat occurredandfacilitatesadebriefdiscussion. Postsimulation survey administration.Thepostsimulationsurveyisadministered immediatelyfollowingthepostsimulationteampresentationsanddebriefdiscussion. Theinstructoragainleavestheroomtohelpassureanonymity.Onestudentdidnot completethepostsimulationsurveyandanothertwosurveyswereunusablebecausethe studentdidnotincludetheanonymousidentificationinformationrequiredtoallowa longitudinalstudyontheirindividualresponses. Data Processing and Analysis Thisstudyusesquantitativemethodstoacceptorrejecttheresearchhypotheses. Theseconclusionsaresupplementedbythethemesfoundinthequalitativedataand, togetherwiththeresultsofthequantitativeanalysis,triangulatetoanswerthe PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 127 fundamentalquestion:Istheuseofaprojectmanagementsimulationasacapstone activityaneffectivetechniqueforaddingvaluetoanexecutiveMBAprogram specializinginprojectmanagement?Thissectiondescribestheresearchmethodsused. Avoiding sample selection bias.Sinceweareconcernedwiththeeffectivenessof asimulationgameinaparticularprogramusingaspecificdeliverymodel,andthe populationofinterestissmallandidentifiable,acensusofallstudentsparticipatingin thesimulationgameistaken.Thiseliminatesthepossibilityofsampleselectionbiasand isnecessarybecausesamplingmighteliminateimportantcasesfromthestudyand credibilityrequiresconsiderationofallmembersofthetargetpopulation(D.R.Cooper& Schindler,2006).Inthecaseofstudentsattendingacourseinaclassroom,dataiseasy tocollectandmoststudentsarewillingtoparticipateiftheiranonymityisassured. Inspectionofthedatashowswhethertheelementsarequitedifferentfromeachother andsuggestwhethersamplingmightbepossibleforlargerpopulationsofinterest (Burns&Burns,2008). Initial data review. Dataarereviewedforextremevaluesandmissingdata elements.Inthecaseofamissingdataelement,theaverageansweroftheother respondentsisenteredforthemissingdata(Burns&Burns,2008;Fowler,2009). Data reliability.Thisstudyusesparametrictestsbasedonthenormal distributionandaccordingly,datamustbecheckedtoverifytheymeettheassumptions ofparametrictests:normallydistributeddata,homogeneityofvariance,intervaldata andindependence(Field,2009).Dataarecheckedfornormaldistributionusingthe Kolmogoroc‐SmirnovandShapiro‐Wilktests.Homogeneityofvariancebetweenthe cohortgroupsistestedusingLevene’stest.Theattitudinalsurveydataresponsesare treatedasintervalandvalidforparametrictesting;thesurveyscalesusedimplyan equaldifferencebetweeneachrating(Field,2009). Independenceisassumedsince,inthecaseofthepresimulation,participantsdo notseethequestionnairebeforetheyareaskedtocompleteit.Afterwardstheyare immediatelydistractedfromdiscussingitbytheneedtogetstartedwithreviewingtheir teamstrategyforexecutingthesimulationprojectandplacinginitialbidsforresources. This,combinedwithnotknowingthesimilarityofthepostsimulationquestionnaire, suggeststheirpostsimulationresponseswillnotbeinfluencedbyotherparticipants. Thereliabilityofusingthegrandmeanoftheattitudequestionstocreatethe variablespreviouslylistedinTable3isassessedbycalculatingCronbach’sAlpha. PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 128 Cronbach’sAlphavaluescloseto1indicateahighdegreeofinternalconsistencyand reliabilityamongthesetofquestionresponses(Sekaran,2003).Acceptablelevelsfor Cronbach’sAlphaare0.7andaboveandcoefficientsof0.9andaboveareconsideredto indicateexcellentstrengthofassociation(Burns&Burns,2008). Hypothesis testing.Dependentt‐testsareusedtotestforsignificantdifferences betweenpresimulationandpostsimulationmeasures.Theresultsareconsidered significantforp<.05. Qualitative analysis.Narrativedataiscollectedfromthesimulationgame participantsduringtheirfinalpresentationanddebriefdiscussions,fromtheir postsimulationsurveyqualitativequestionresponses,andfromcourseandprogram evaluationsurveys.Thesedataarecategorizedtoreportemergentthemesand,inthe caseoftheexitsurveys,favorableandunfavorablecommentsarecountedandreported. Postsimulation team presentations.Followingcompletionoftheproject simulationgame,eachteampreparesandpresentsashort(20minute)PowerPoint presentationcontainingthefinalprojectstatusreporttothesponsor,aprojectaudit reportdiscussinglessonslearnedonthesimulatedproject,andasummaryofwhatthey learnedaboutprojectmanagementfromparticipatinginthesimulation.Thisisfollowed byaninstructor‐leddebriefdiscussionoftheoverallexperience.Presentationand debriefdiscussioncontentiscategorizedandemergentthemesarereported. Postsimulation survey open‐ended question responses.Thepostsimulationsurvey askedparticipantstorespondtofouropen‐endedquestions: 1. Whatdidyoulikeaboutthejust‐completedsimulationexperience? 2. Excludingchangestothecomputersimulationitself,whatchangesdoyou thinkcouldbemadetotheoverallsimulationexperiencetobetter prepareyoutomanageprojects? 3. Whatchangesdoyourecommendbemadetothecomputersimulation itself? 4. Elaborateonanyotherthoughtsyouhaveregardinglearningproject managementwithacomputer‐basedsimulationgame. Answerstoeachquestionwerecategorizedandanalyzedtoidentifyemergent themes. PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 129 Course and program evaluation surveys.Course,ProjectManagementCorePhase, andgraduateexitsurveysareexaminedforcommentsrelatedtothesimulation.These commentsarenotedandcountedasappropriate. Ethical Considerations Studentswereadvisedthatparticipationinthisstudywasvoluntaryand anonymous.Theconsentprocesswascomprisedofaverbalstatementpriorto beginningthesimulationsimilarto: Iamconductingresearchintotheeffectivenessofprojectmanagement simulationgamesandwouldappreciateitifyouwouldparticipateinthis researchbycompletingashortsurveybeforeandafterthesimulation. Participationisentirelyvoluntaryanddoesnotaffectyourcoursegradeinany way.Tohelpprotectyouranonymity,Ionlyneedyoutoindicatewhichteamyou areamemberofandtoassignanindividualcode,knownonlytoyou,thatwill allowcomparingpresimulationandpostsimulationsurveyresponsesby anonymousindividual.Ifyouarenotcomfortableansweringanyofthe demographicquestions,youmayleavethemblankastheyaretheretodescribe thecompositionofthesampleandarenotvitalforthisresearch.Completingthe surveywillindicateyourconsent.Ifyouchoosenottoparticipateyoumaytakea breakuntilwereconvenein30minutestobeginthesimulation. IcompletedtheNationalInstitutesofHealth(NIH)web‐basedtrainingcourse “ProtectingHumanResearchParticipants”andtheresearchplanwasreviewedand approvedbytheInstitutionalReviewBoardofTheUniversityofTexasatDallaswhereI amemployedandwheretheresearchwasconducted. Summary Thisresearchusesamixedmethodsapproachtostudyoneuniversity’sapproach tousingaprojectmanagementsimulationgameasacapstoneexperientiallearning activityinitsgraduateMS/MBAprogramforexperiencedprofessionals.The quantitativeapproachbuildsonthepriorworkofMcCreery(2003)andallowsfor comparisonstobemadetopositgeneralizabilityaboutchangesinperceptionsof knowledgeandabilitytoapplythatknowledge.Additionaladaptationsprovidefor PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 130 determinationofchangesinattitudesrelatedtoteamworkandsimulationgamesas experientiallearningactivities.Additionalquestionsprovideformeasuringgainsinthe abilitytoanalyzeprojectnetworkandearnedvaluedata.Thequalitativeapproach examinespostsimulationandpostcoursesurveydatatoidentifyemergentthemesof whatstudentslearnedfromtheexperiencetohelpexplainandreinforcethequantitative findings.Apilotstudyvalidatedthemethodologyandtheprocedureforconductingthe study PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 131 Results Thischapterdescribesandanalyzesthequantitativedataobtainedfromthe presimulationandpostsimulationsurveysandthequalitativedataobtainedfromthe postsimulationdebriefsession,postsimulationsurvey,andtheProjectManagementCore Phaseexitsurvey.Thefirstsectiondescribestheresultsofacheckofthedatafor completenessandoutliersandthetreatmentusedformissingdata.Thesecondsection checkstheassumptionsofparametricdata,importantbecauseofthesmallsamplesize. Thethirdsectiondescribesthediversityofthestudentparticipants.Thefourthsection analyzesthedatabasedonthehypothesesunderstudy.Thefifthsectionsummarizes thequalitativeresults.Thesixthsectionreportsadditionalquantitativedatafroma courseevaluationsurvey. Presimulationandpostsimulationsurveyquantitativedataareanalyzedusing SPSSforWindowsPASWStatistics18.0.2.Theresultsofstatisticaltestsareconsidered significantforp<.05. Initial data analysis Theinitialanalysisofthesurveydataconsistedofcalculatingtheresponserate andcheckingforandtreatingextremevaluesandmissingdataelements.Noextreme datavalues(outliers)werefound. Response rate.AsdiscussedintheSamplePopulationsectionofthe MethodologyChapterandsummarizedinTable10,60peoplein4cohortgroups participatedinthesimulationgamewithpresimulationandpostsimulationsurveys. GroupAwasthepilotgroupcomprisedofparticipantsinacorporatetrainingprogram withcontentandcontacthoursthataresimilartothetwocohortgroupsofinterestin thegraduatedegreeprogramwithemphasisinprojectmanagement.Thepilotgroupis categorizedasPD(ProfessionalDevelopment)inthedata.GroupsBandDarefromthe graduatedegreeprogramunderstudyandarecategorizedinthedataasOC2010and OC2011.Datawasalsocollectedforparticipantsattendingasingleexecutivegraduate‐ levelacademiccourseinprojectmanagement(GroupC)whocompletedsimilar academicassignmentstoGroupsBandD,butexperiencedfewercontacthoursof presimulationprojectmanagementstudy.GroupCcompletedthesimulationactivityin thetimeperiodbetweenthesimulationactivityforGroupsBandDandiscategorizedas SEM2011inthedata.AlthoughGroupCisnotthesubjectofthisresearch,thedatawas PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 132 availableandsomeobservationsandcommentsareincludedtosuggestopportunities forfutureresearch.Simulationactivityforallgroupswasfacilitatedbythesame instructor. Ahighresponseratewasexpectedsincethiswasanin‐classcensusofsimulation participants(Fowler,2009);however,theuseofapapersurveyallowedrespondentsto skipquestionsornotprovidethepropercodetoallowmatched‐pairlongitudinal analysisofthedata.Onehundredpercentofsimulationparticipantscompletedthe presimulationsurvey.AllparticipantsinGroupsAandB(PDandOC2010)completed thepostsimulationsurvey.OneGroupC(SEM2011)participantwasabsentonthelast dayofsimulationanddidnotcompletethesurvey.OneGroupD(OC2011)participant didnotcompleteasurvey.TwoGroupDparticipantsfailedtoentertheirunique identifieronthepostsimulationsurveythuspreventinglongitudinalpairingand renderingthepostsimulationsurveysuselessforthisstudy.Surveyresponserates variedfrom79%to100%andarelistedinTable11. Aninitialcheckofthedatatoverifyitsusabilityrevealed10instancesofasingle missingdataelementwithinaparticipant’sattitudinalresponse,5inthepresimulation surveyand5inthepostsimulationsurvey.FourofthesemissingelementsareinGroups BandD,thetwogroupsunderstudy.With58attitudinalquestionspersurveyand60 and56respondentsforthepre‐andpost‐simulationsurvey,respectively;thisisan overallmissingdatarateof.0014%forthepresimulationsurveyand.0015%forthe postsimulationsurvey.Thereasonforincompletedatacanbeaccidentalordeliberate foravarietyofreasons,buttheotherdatafromthatparticipantdoesn’thavetobe ignored(Field,2009).Toavoidexcludingrespondentsfromtheanalysisbecauseof these10casesofamissingdataelement,theaverageansweroftheotherrespondentsis enteredforthemissingdata(Burns&Burns,2008;Fowler,2009). Presimulation survey data review.Allparticipantscompletedthepresimulation surveyinallfourcohortgroups.Inthetwocohortgroupsfromthegraduateprogram understudy(GroupsBandD,OC2010andOC2011),oneGroupDrespondentdidnot answeronequestion,anelementoftheteamperformance(TP1)variable. Inthesingle‐courseacademiccohort(GroupC–SEM2011),oneindividualdid notreplytothreeoftheattitudinalquestions,onequestioneachintheelementsofthe knowledge(K1),confidence(C1)andteamperformance(TP1)variables.Asecond PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 133 individualinGroupD(SEM2011)didnotrespondtooneattitudinalquestion,anelement ofthegenericsimulationperception(SG1)variable. Inallfivecasesofamissingdataelementinthepresimulationsurvey,themean oftheotherresponsesforthatquestionwassubstitutedforthemissingelementtoavoid reducingthesamplesizeduetorandommissingdata(Burns&Burns,2008;Fowler, 2009). Postsimulation survey data review.OneGroupA(PD)persondidnotcomplete anyofthe17simulationattitudequestionsandthisindividualwasexcludedfromthe matched‐pairt‐testpilotanalysisinvolvingthetwosimulationperceptionvariables. InGroupB(OC2011),onemissing“confidenceintheabilitytoapplyproject managementknowledge”questionresponsewasreplacedbythemeanoftheother responsesforthatquestionandonepersonskippedallthenetworkandearnedvalue analysisquestions.Sinceitisreasonabletoassumethatthislatterpersondidnotforget howtosolvetheseproblemsbutchosetonotrepeatdoingso,theassociated presimulationnetworkandearnedvalueanalysisscoreswerecopiedtotheassociated postsimulationvariables.Thisassumesthepersonlearnednothingtoimprovethis scoreduringthesimulation,apossiblereasonforchoosingtonotcompletethese sectionsofthesurvey. Randommissingdataelementswerereplacedwiththemeanforone respondent’sknowledgeself‐assessmentquestioninGroupC(SEM2011)andGroupD (OC2011),andonegenericsimulationattitudequestioninGroupD.OneGroupCand twoGroupDrespondentsdidnotcompletethescheduleandearnedvalueanalysis problems.Onceagain,theassumptionistheycanstillanswerthesetotheextentthey didbeforethesimulationandtheirpresimulationresponsesarecopiedintothe associatedpostsimulationdata. Table11summarizestheresultsofthedatareviewandtheactionstaken.Table 12andTable13summarizethemissingdatavaluesanalysisbyvariable. PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 134 TABLE11‐DATAREVIEWSUMMARY Presimulation Survey Group Responses (rate) A (PD) 19 (100%) B (OC2010) 14 (100%) C (SEM2011) D (OC2011) 13 (100%) 14 (100%) Missing Data Treatment None None 1 – K1 1 – C1 1 – TP1 1 – SG1 1 – TP1 Postsimulation Survey Usable Responses (rate) n/a 19 (100%) n/a 14 (100%) Replaced missing data element with mean of other responses 12 (92%) 11 (79%) Missing Data 1 person skipped all 17 simulation questions 1 person skipped network and earned value questions 1 – C2 1 – K2 1 – TP2 1 person skipped network and earned value questions 1 – K2 1 – SG2 2 people skipped network and earned value questions Treatment Omitted from matched‐ pair t‐tests for simulation perceptions (SG and SP) Include in analysis copying presimulation score of 6 Replace missing data element with mean of other responses Include in analysis copying presimulation score of 5 Replace with mean of other responses Include in analysis copying presimulation scores of 1 and 2 TABLE12‐PRESIMULATIONQUESTIONRESPONSERATEBYVARIABLE Variable Knowledge Self‐Assessment (K1) Knowledge Application Confidence (C1) Current Team Satisfaction – McCreery (T1) Current Team Satisfaction (TP1) Teamwork Attitude (TG1) Generic Simulation Attitude (SG1) This Simulation Attitude (SP1) Number of Possible Responses Number of Missing Values Percent Missing Values 1A‐1F (16) 960 1 .001 2A‐2F (16) 960 1 .001 3A‐3I (9) 540 2 .004 3A‐3I, 4N (10) 600 2 .003 4C, 4K (2) 120 0 0 4A, 4B, 4E, 4G, 4I, 4L (6) 360 1 .003 4D, 4F, 4H, 4J, 4M, 4O, 4P, 4Q (8) 480 0 0 Questions PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 135 TABLE13‐POSTSIMULATIONQUESTIONRESPONSERATEBYVARIABLE Variable Knowledge Self‐Assessment (K2) Knowledge Application Confidence (C2) Current Team Satisfaction – McCreery (T2) Current Team Satisfaction (TP2) Teamwork Attitude (TG2) Generic Simulation Attitude (SG2) This Simulation Attitude (SP2) Number of Possible Responses Number of Missing Values Percent Missing Values 1A‐1F (16) 896 2 .002 2A‐2F (16) 896 1 .001 3A‐3I 504 1 .002 3A‐3I, 4N (10) 559 1 .002 4C, 4K (2) 110 0 0 4A, 4B, 4E, 4G, 4I, 4L (6) 330 1 .003 4D, 4F, 4H, 4J, 4M, 4O, 4P, 4Q (8) 440 0 0 Questions Verification of assumption of parametric data Normal distribution.ResultsoftheKolmogorov‐SmirnovandShapiro‐Wilktest fordistributionofdataareshownforthecombineddatasetofGroupsBandD(OC2010 andOC2011),thetwogroupsinthegraduateprogramunderstudy,inTable14andfor thesplitdatasetinTable15.Bothtestsindicatethedataareassumedtobenormalfor theK1‐K2andC1‐C2matchedpairt‐testsforasignificantincreaseinperceived knowledgeandabilitytoapplythisknowledge.Forexample,forthepresimulationself‐ perceptionofknowledge(K1),D(25)=.069,ns(p=.200). Inthecombineddataset,atleastoneofeachoftheotherpre‐postsimulation test‐pairingvariablesissignificantlynon‐normal,p<.05,andisindicatedbyhighlight shading. Thesplitdatasetresultsindicatethatifthecohortgroupsareconsidered independently,accordingtotheKolmogorov‐Smirnovtest,theadditionalpairingsofT1‐ T2,TP1‐TP2,SG1‐SG2,SP1‐SP2andN1‐N2areassumedtousenormaldata,p<.05. WiththeexceptionofSG2(p=.038),theShapiro‐Wilktestalsoindicatesthedatafor thesevariablesmaybeassumedtobenormal,p<.05. PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 136 TABLE14‐COMBINEDGROUPSBANDDDATASETTESTSOFNORMALITY Tests of Normality OC K1 C1 T1 TP1 TG1 SG1 SP1 N1 E1 P1 K2 C2 T2 TP2 TG2 SG2 SP2 N2 E2 P2 Kolmogorov‐Smirnov Statistic df Sig. .069 25 .200 .116 25 .200 .141 25 .200 .173 25 .052 .160 25 .097 .153 25 .132 .012 .199 25 .003 .218 25 .205 25 .008 .211 25 .005 .135 25 .200 .165 25 .076 .135 25 .200 .140 25 .200 .177 25 .042 .163 25 .086 .139 25 .200 .170 25 .059 .165 25 .076 .170 25 .060 Statistic .981 .967 .926 .925 .930 .925 .930 .815 .889 .894 .955 .939 .903 .901 .916 .884 .954 .888 .922 .944 Shapiro‐Wilk df 25 25 25 25 25 25 25 25 25 25 25 25 25 25 25 25 25 25 25 25 Sig. .911 .581 .069 .068 .086 .066 .089 .000 .011 .013 .320 .141 .022 .019 .042 .009 .306 .010 .056 .178 PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 137 TABLE15‐TESTSOFNORMALITYSPLITBYCOHORTGROUP Tests of Normality K1 C1 T1 TP1 TG1 SG1 SP1 N1 E1 P1 K2 C2 T2 TP2 TG2 SG2 SP2 N2 E2 P2 Kolmogorov‐Smirnova Cohort OC2010 OC2011 OC2010 OC2011 OC2010 OC2011 OC2010 OC2011 OC2010 OC2011 OC2010 OC2011 OC2010 OC2011 OC2010 OC2011 OC2010 OC2011 OC2010 OC2011 OC2010 OC2011 OC2010 OC2011 OC2010 OC2011 OC2010 OC2011 OC2010 OC2011 OC2010 OC2011 OC2010 OC2011 OC2010 OC2011 OC2010 OC2011 OC2010 OC2011 Statistic .153 .147 .108 .232 .183 .164 .180 .188 .249 .177 .176 .248 .203 .132 .214 .230 .252 .245 .269 .190 .149 .127 .191 .194 .182 .175 .192 .186 .157 .249 .152 .215 .115 .206 .152 .226 .232 .148 .160 .209 df 14 11 14 11 14 11 14 11 14 11 14 11 14 11 14 11 14 11 14 11 14 11 14 11 14 11 14 11 14 11 14 11 14 11 14 11 14 11 14 11 Sig. .200 .200 .200 .102 .200 .200 .200 .200 .019 .200 .200 .057 .122 .200 .081 .106 .016 .065 .007 .200 .200 .200 .180 .200 .200 .200 .171 .200 .200 .056 .200 .166 .200 .200 .200 .123 .040 .200 .200 .195 Shapiro‐Wilk Statistic .933 .938 .983 .904 .893 .949 .902 .930 .887 .930 .890 .886 .923 .942 .830 .822 .871 .811 .843 .921 .914 .951 .943 .903 .910 .892 .896 .895 .914 .872 .926 .846 .943 .941 .916 .864 .891 .946 .923 .913 df 14 11 14 11 14 11 14 11 14 11 14 11 14 11 14 11 14 11 14 11 14 11 14 11 14 11 14 11 14 11 14 11 14 11 14 11 14 11 14 11 Sig. .334 .492 .989 .206 .089 .633 .120 .411 .073 .413 .081 .125 .242 .547 .012 .018 .044 .013 .018 .325 .178 .653 .464 .203 .159 .146 .099 .161 .177 .081 .267 .038 .456 .538 .190 .064 .084 .590 .247 .268 PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 138 Homogeneity of the cohort groups.InspectionofTable16showsthattheself‐ perceptionscoresare,onaverage,slightlyhigherforGroupB(OC2010)thanforGroupD (OC2011).Table17reportstheresultsofLevene’stestforequalityofvariancesandt‐ testsfortheequalityofmeans. Forallvariablesinthiscombineddataset,thevariancesarefoundtobeequal. Forexample,theperceptionofknowledgevariableK1,F(1,26)=1.66,ns(p=.209). Forallvariables,exceptpresimulationSG1andSP1,therewerenosignificant differencesbetweenthemeans,p>.05.ForSG1,onaverage,studentsinGroupBhada morefavorableviewofgenericsimulationsasalearningtool(M=5.94,SE=0.228)than thoseinGroupD(M=4.71,SE=0.339).Thisdifferencewassignificantt(26)=3.003,p< .05.ForSP1,onaverage,studentsinGroupBalsohadamorefavorableviewofthe simulationusedinthecourse(M=5.78,SE=0.194)thanstudentsinGroupD(M=4.96, SE=0.291).Thisdifferencewassignificantt(26)=2.346,p<.05. Correspondingsignificantdifferencesarenotfoundfortherelated postsimulationvariables,SG2andSP2.Followingthesimulation,onaverage,studentsin GroupBstillhadamorefavorableviewofgenericsimulationsasalearningtool(M= 5.48,SE=0.222)thanthoseinGroupD(M=5.01,SE=0.422).Thisdifferencewasnot significantt(23)=1.050,p>.05.ForSP2,onaverage,studentsinGroupBalsohada morefavorableviewofthesimulationusedinthecourse(M=5.47,SE=0.203)than studentsinGroupD(M=5.02,SE=0.388).Thisdifferencewasnotsignificantt(23)= 1.094,p>.05. PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 139 TABLE16–COHORTGROUPMEANSCORES Group Statistics K1 C1 T1 TP1 TG1 SG1 SP1 N1 E1 P1 Years of experience with projects Years working as a professional K2 C2 T2 TP2 TG2 SG2 SP2 N2 E2 P2 Cohort OC2010 OC2011 OC2010 OC2011 OC2010 OC2011 OC2010 OC2011 OC2010 OC2011 OC2010 OC2011 OC2010 OC2011 OC2010 OC2011 OC2010 OC2011 OC2010 OC2011 OC2010 OC2011 OC2010 OC2011 OC2010 OC2011 OC2010 OC2011 OC2010 OC2011 OC2010 OC2011 OC2010 OC2011 OC2010 OC2011 OC2010 OC2011 OC2010 OC2011 OC2010 OC2011 OC2010 OC2011 N 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 11 14 11 14 11 14 11 14 11 14 11 14 11 14 11 14 11 14 11 Mean 5.29 4.78 4.96 4.68 5.52 5.46 5.52 5.44 5.93 5.39 5.94 4.71 5.78 4.96 2.21 2.14 2.00 1.36 4.21 3.50 5.25 8.86 10.75 14.57 5.62 5.35 5.46 5.33 5.87 5.51 5.83 5.48 6.07 5.95 5.48 5.01 5.47 5.02 2.21 2.36 2.07 2.55 4.29 4.91 Std. Deviation .577 .756 .710 .902 1.292 1.101 1.293 1.115 .829 1.003 .854 1.267 .728 1.089 1.672 1.512 1.468 1.737 2.424 2.710 6.296 8.347 7.552 7.939 .661 .661 .839 .746 .997 1.379 1.074 1.423 .703 .850 .832 1.399 .758 1.287 1.369 1.502 1.639 1.968 2.016 2.773 Std. Error Mean .154 .202 .190 .241 .345 .294 .346 .298 .221 .268 .228 .339 .194 .291 .447 .404 .392 .464 .648 .724 1.683 2.231 2.018 2.122 .177 .199 .224 .225 .266 .416 .287 .429 .188 .256 .222 .422 .203 .388 .366 .453 .438 .593 .539 .836 PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 140 TABLE17‐RESULTSOFINDEPENDENTSAMPLEST‐TESTBETWEENCOHORTGROUPS Independent Samples Test Levene's Test for Equality of Variances t‐test for Equality of Means K1 C1 T1 TP1 TG1 SG1 SP1 N1 E1 P1 Yrs of exper. with projects Equal variances assumed Equal variances not assumed Equal variances assumed Equal variances not assumed Equal variances assumed Equal variances not assumed Equal variances assumed Equal variances not assumed Equal variances assumed Equal variances not assumed Equal variances assumed Equal variances not assumed Equal variances assumed Equal variances not assumed Equal variances assumed Equal variances not assumed Equal variances assumed Equal variances not assumed Equal variances assumed Equal variances not assumed Equal variances assumed Equal variances not assumed F 1.660 1.873 .183 .893 .353 .541 .469 1.168 2.109 3.372 .055 .513 .254 ‐.011 1.038 26 .375 .277 .307 ‐.354 .907 .902 24.634 .376 .277 .307 ‐.356 .909 .143 26 .888 .065 .454 ‐.868 .997 .143 25.362 .888 .065 .454 ‐.869 .998 .175 26 .863 .080 .456 ‐.859 1.018 .175 25.451 .863 .080 .456 ‐.860 1.019 26 .136 .536 .348 ‐.179 1.251 1.540 25.104 .136 .536 .348 ‐.180 1.252 26 .006 1.226 .408 .387 2.065 3.003 22.793 .006 1.226 .408 .381 2.071 26 .027 .821 .350 .102 1.541 2.346 22.673 .028 .821 .350 .097 1.546 26 .907 .071 .603 ‐1.167 1.310 .119 25.740 .907 .071 .603 ‐1.168 1.311 26 .300 .643 .608 ‐.606 1.892 1.058 25.296 .300 .643 .608 ‐.608 1.894 26 .469 .714 .972 ‐1.283 2.712 .735 25.681 .469 .714 .972 ‐1.284 2.713 ‐ 26 1.291 ‐ 24.176 1.291 .208 ‐3.607 2.794 ‐9.351 2.137 .209 ‐3.607 2.794 ‐9.372 2.158 .902 .158 3.003 .078 2.346 .640 .431 .119 .641 1.058 .059 .810 1.679 .206 Sig. (2‐ df tailed) 26 .054 2.019 24.310 .290 1.540 .222 Mean Differ‐ ence .513 Sig. t .209 2.019 95% Confidence Interval of the Difference Lower Upper ‐.009 1.036 Std. Error Differ‐ ence .254 .735 PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 141 Independent Samples Test Levene's Test for Equality of Variances t‐test for Equality of Means Years working as a prfssnl K2 C2 T2 TP2 TG2 SG2 SP2 N2 E2 P2 Equal variances assumed Equal variances not assumed Equal variances assumed Equal variances not assumed Equal variances assumed Equal variances not assumed Equal variances assumed Equal variances not assumed Equal variances assumed Equal variances not assumed Equal variances assumed Equal variances not assumed Equal variances assumed Equal variances not assumed Equal variances assumed Equal variances not assumed Equal variances assumed Equal variances not assumed Equal variances assumed Equal variances not assumed Equal variances assumed Equal variances not assumed F .120 Sig. (2‐ df tailed) 26 .203 ‐ 1.305 ‐ 25.935 1.305 .978 1.023 23 Mean Differ‐ ence ‐3.821 .203 ‐3.821 2.928 ‐9.842 2.199 .317 .272 .266 ‐.278 .823 1.023 21.621 .318 .272 .266 ‐.280 .825 23 .685 .132 .322 ‐.535 .799 .416 22.589 .681 .132 .318 ‐.526 .790 .775 23 .446 .368 .475 ‐.614 1.350 .745 17.608 .466 .368 .494 ‐.671 1.407 .695 23 .494 .347 .499 ‐.685 1.379 .672 18.160 .510 .347 .516 ‐.737 1.431 .376 23 .710 .117 .310 ‐.525 .759 .368 19.342 .717 .117 .318 ‐.548 .781 23 .305 .472 .449 ‐.457 1.400 .989 15.419 .338 .472 .477 ‐.542 1.485 23 .285 .450 .412 ‐.401 1.302 1.029 15.324 .319 .450 .438 ‐.481 1.382 ‐.260 23 .798 ‐.149 .575 ‐1.340 1.041 ‐.257 20.575 .800 ‐.149 .582 ‐1.361 1.063 ‐.657 23 .517 ‐.474 .721 ‐1.966 1.018 ‐.643 19.432 .528 ‐.474 .738 ‐2.015 1.067 ‐.651 23 .521 ‐.623 .957 ‐2.603 1.356 ‐.627 17.686 .539 ‐.623 .995 ‐2.716 1.469 Sig. .732 .001 .187 .670 .639 .432 .544 .468 .026 .872 3.113 t .410 .091 1.050 3.711 .066 1.094 .528 .475 .683 .417 3.845 .062 95% Confidence Interval of the Difference Lower Upper ‐9.841 2.198 Std. Error Differ‐ ence 2.928 PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 142 Scale Reliability Scalereliabilityisassessedusingtheinternalconsistencymethodby determiningCronbach’sαforthequestionscomposingeachvariable.Underlying statisticsreviewedforeachscaleincludetheCorrectedItem‐TotalCorrelationand Cronbach’sAlphaifItemDeletedreportedintheItem‐TotalStatisticsTable.Corrected Item‐TotalCorrelationvalueslessthan.3indicateaweakcorrelationwiththeoverall scaleandsuggestthatitembedroppedfromthescale.ItemswithCronbach’sAlphaif ItemDeletedvaluessubstantiallygreaterthantheoverallαsuggestthatitembedeleted toimprovetheoverallscalereliability(Field,2009). Table18andTable19showtheSPSSoutputfromthesecalculationsfortheK1 variable.AllCorrectedItem‐TotalCorrelationvaluesaresubstantiallygreaterthan.3and allCronbach’sAlphaifItemDeletedvaluesarelessthantheoverallCronbach’sAlphaof .924,indicatingallitemsarerelevantintheconstructbeingmeasured.Thesame inspectionprocesswasfollowedfortheothervariables.Scalereliabilityforthevariables usedinthisstudyissummarizedinTable20. TABLE18‐SCALEK1‐RELIABILITYSTATISTICS Reliability Statistics Cronbach's Alpha Based on Cronbach's Standardized Alpha Items N of Items .924 .928 16 PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 143 TABLE19‐SCALEK1‐PRESIMULATIONKNOWLEDGEITEM‐TOTALSTATISTICS Item-Total Statistics Scale Mean Scale Corrected if Item Variance if Item-Total Deleted Item Deleted Correlation Squared Multiple Correlation Cronbach's Alpha if Item Deleted K1_ORGDESGN 75.86 113.534 .727 .727 .917 K1_PROJPLAN 75.25 111.750 .830 .855 .914 K1_PROJRISK 75.64 113.720 .545 .722 .922 K1_ESTSCOPE 75.50 112.111 .650 .760 .919 K1_SEQACT 74.93 115.106 .668 .668 .919 K1_ESTDUR 75.43 116.180 .680 .736 .919 K1_BUDGETNG 75.82 108.967 .758 .839 .915 K1_RESALLOC 75.25 109.083 .831 .898 .913 K1_CHGMGT 75.64 113.794 .622 .650 .920 K1_DESPERFMEAS 75.96 112.036 .579 .852 .922 K1_EFFLDRSHP 75.21 120.101 .581 .797 .921 K1_TEAMCONS 75.04 116.925 .566 .832 .921 K1_NEGOT 75.39 116.766 .542 .746 .922 K1_EVM 76.18 109.411 .608 .759 .921 K1_EVALPERS 75.18 115.560 .565 .830 .921 K1_MNGUNCERT 75.75 119.231 .503 .665 .923 TABLE20‐SCALERELIABILITY‐GROUPSBANDD(OC2010ANDOC2011) Variable K C T TP TG SG SP Description Knowledge Confidence to apply knowledge Teamwork attitude (McCreery) Current team satisfaction Teamwork attitude (generic) Generic simulation attitude This simulation attitude Cronbach's α 1 2 (Pre‐sim.) (Post‐sim.) .924 .924 .932 .949 .927 .946 .933 .954 .562 .672 .928 .939 .840 .869 ReviewoftheCronbach'sαscalereliabilityscoresindicatepotentialissueswith thevariablesTGandSP. PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 144 TGisagenericteamworkattitudevariablecomprisedofthemeanoftwo questionsbasedonBuzzetto‐MoreandMitchell’s(2009)postsimulationsurvey:  GroupProjectshelppreparestudentstobeabletoworkinprofessional groupsinthefuture  Ienjoyworkingingroups Thesearetheonlyquestionsinthesurveyaddressingteamsandteamworkthat arenotspecifictotheteamofparticipantsperformingthesimulation.Otherthan evaluatingeachitemindividually,thisisnottreatableandthegenericteamworkattitude scaleremainswithrelativelylowreliability. Cronbach’sαforSP1canbeimprovedfrom.840to.943andSP2canbe improvedfrom.869to.925byeliminatingthequestions“Ibelievethissimulationwillbe difficult”and“Thissimulationwasdifficult”fromthepresimulationandpostsimulation surveys,respectively.TheCorrectedItem‐TotalCorrelationfortheseitemswere ‐.253and‐.026,respectively.Accordingly,anadditionalanalysisisperformedwiththis itemremovedfromtheSPscaleandconsideredindependently. McCreery’s(2003)teamworkassessmentvariables(T1andT2)are strengthenedbytheadditionofthecurrentteamquestionfromBuzzetto‐Moreand Mitchell’s(2009)survey,“Iwouldbewillingtoworkwithmyteamonanactualproject inthefuture.”Consequently,theconstructTPwillbeusedratherthanTforassessing teamworkattitudesrelatedtotheparticipant’ssimulationteam. ThescalereliabilitysummarywiththisrevisedconstructstructureforSPis showninTable21. PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 145 TABLE21‐REVISEDSCALERELIABILITY‐GROUPSBANDD(OC2010ANDOC2011) Variable K C T TP TG SG SP‐D Description Knowledge Confidence to apply knowledge Teamwork attitude (McCreery) Current team satisfaction Teamwork attitude (generic) Generic simulation attitude This simulation attitude minus difficulty question Cronbach's α 1 2 (Pre‐sim.) (Post‐sim.) .924 .924 .932 .949 .927 .946 .933 .954 .562 .672 .928 .939 .943 .925 Withtheexceptionofthepreviouslydiscussedgenericteamworkattitude construct(TG)basedononlytwoquestions,theresultsofCronbach’sαanalysissupport theconclusionthatthedefinedsurveymeasuresallhavehighreliabilities,Cronbach’sα >.9(Burns&Burns,2008). Participant demographics StudentsareassignedtoteamswhentheybegintheProjectManagementCore Phaseofthegraduateprogram.Theseassignmentsaremadewiththeintentto maximizediversityontheteamswithregardstogender,workexperience,ethnicity,and industry.Inadditiontodemographicdatarelatedtothesecharacteristics,demographic datawascollectedinthepre‐andpost‐simulationsurveysregardingprofessional background(technicalornon‐technical),program‐startingeducationallevel,prior projectmanagementtraining/education,andannualincometofurtherdeterminethe diversityoftheparticipants.Theintentofthisresearchisnottoexplorecorrelations withthesedemographiccategories;however,thedataprovidesforthispossiblefuture research. Gender and technical background.Thetwocohortgroupsunderstudycontained identicalgender(64%male/36%female)andtechnical/non‐technicalbackground (57%technical/43%non‐technical)distributions.Allteamshadatleastonewoman (seeFigure6)andamixoftechnicalandnon‐technicalbackgrounds(seeFigure7). PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 146 FIGURE6‐TEAMGENDERMIX FIGURE7‐TEAMTECHNICAL/NON‐TECHNICALBACKGROUNDMIX Educational background.Priortobeginningtheirgraduatestudies,60%ofthe studentsineachcohorthadexperiencednoformalprojectmanagementtrainingor education.Thosewithpriortrainingoreducationhadexperiencedmorethan26contact PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 147 hours,butonlyonepersonreportedmorethan125contacthours.Figure8depictsthis forthetwocohortgroupsunderstudy. Priorprojectmanagementtrainingandeducationwasnotoneofthefactors consideredwhenassigningteamsatthestartoftheprogram;however,Figure9shows theteamsarealsodiverseinthiscategory.Exploringrelationshipsbetweenteam compositionandteamperformanceisanopportunityforfutureresearch. FIGURE8‐CONTACTHOURSOFPRIORPROJECT MANAGEMENTTRAININGOREDUCATION PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 148 FIGURE9‐PRIORTRAINING/EDUCATIONBYTEAM Years of experience with projects and years of professional experience.Overall, participantsreportedanaverageof7yearsofexperienceworkingwithprojectsand12.7 yearsworkingasaprofessional.TheexperiencedistributionsareshowninFigure10 andFigure11. Onaverage,GroupB(OC2010)participantshad3.6feweryearsofexperience withprojects(M=5.25,SE=1.68)thanGroupD(OC2011)participants(M=8.86,SE= 2.23).Thisdifferencewasnotsignificantt(26)=‐1.291,p>.05;however,itdid representamediumsizeeffect,r=0.24. Onaverage,GroupB(OC2010)participantshad3.8feweryearsofprofessional workexperience(M=10.75,SE=2.02)thanGroupD(OC2011)participants(M=14.57, SE=2.12).Thisdifferencewasalsonotsignificantt(26)=‐1.305,p>.05;r=0.25. PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE FIGURE10‐PROJECTEXPERIENCE FIGURE11‐PROFESSIONALEXPERIENCE 149 PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 150 Region of origin.Studentsoriginatedfrommultipleregionsoftheworld,but primarilyfromNorthAmerica.Thisdemographicdatawascollectedonthe postsimulationsurveyanddataismissingforfourGroupD(OC2011)students.The mixtureofworldregionoforiginbycohortgroupandbyteamisshowninFigure12and Figure13. FIGURE12‐WORLDREGIONOFSTUDENTORIGIN PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 151 FIGURE13‐WORLDREGIONOFORIGINBYTEAM Prior education level.Fourstudents,twoineachofthecohortgroupsreported previouslyearningamaster’sdegree.Allotherstudentsenteredtheprogramwith bachelor’sdegrees. Industry and job title.Studentdiversityofworkindustrybycohortgroupis showninFigure14andbyteaminFigure15;diversitybyjobtitleisshowninFigure16 andFigure17,respectively.Industrieswritteninforthe“other”categorywerefinance, semiconductors,retail,technologyanddesign/architecture.Jobtitleswritteninforthe “other”categorywereanalyst,accountmanager,teamleadandprojectdesigner. Annual income.Studentdiversityinannualincomebycohortgroupisshownin Figure18andbyteaminFigure19. PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE FIGURE14‐INDUSTRYBYCOHORTGROUP FIGURE15‐INDUSTRYBYTEAM 152 PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE FIGURE16‐JOBTITLEBYCOHORTGROUP FIGURE17‐JOBTITLEBYTEAM 153 PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE FIGURE18‐ANNUALINCOMEBYCOHORTGROUP FIGURE19‐ANNUALINCOMEBYTEAM 154 PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 155 Descriptive statistics AppendixGcontainstablesshowingthedescriptivestatisticsforeachvariable anditscomponentquestions.Thisdatawascollectedusing7‐pointLikert‐typescaleson thesurveyinstrumentandtreatedasintervaldata.Higherscoresindicatehigherself‐ assessmentsofprojectmanagementknowledge,abilitytoapplythatknowledge, strongerfavorableattitudesrelatedtoteamsandsimulations,ormorecorrectanswers ontheabilitytoanalyzeschedulenetworksandearnedvalueperformance. Hypothesis testing Thissectionreportstheresultsofthequantitativeanalysisofthedatarelatedto testingthehypothesesposedbytheresearchquestions.Testingofthesehypotheses usesdependentt‐teststocomparetheresultsobtainedfromthepresimulationand postsimulationsurveys.Theresultsofthesedependentt‐testsarepresentedinTable22 (descriptivestatistics),Table23(correlations)andTable24(testresults).Sinceweare lookingforincreases,itisappropriatetouseone‐tailedsignificance,obtainedby dividingthereportedtwo‐tailedsignificancebytwo.Effectsizesassociatedwiththese testsarecalculatedusingtheformula,r=SQRT(t2/(t2+df).(Field,2009). Do participant knowledge and confidence to apply knowledge self‐assessments increase following the simulation experience? H1: Participantswillassesstheirprojectmanagementknowledgelevelhigher aftercompletingthesimulationgame. Onaverage,participantsassessedsignificantlygreaterprojectmanagement knowledgeaftercompletingthesimulation(M=5.5,SE=0.132)thanbeforestartingthe simulation(M=5.01,SE=0.148),t(24)=4.942,p<.01,r=.71. H2: Participantswillassesstheirabilitytoapplyprojectmanagement knowledgehigheraftercompletingthesimulationgame. Onaverage,participantsassessedsignificantlygreaterconfidenceintheirability toapplyprojectmanagementknowledgeaftercompletingthesimulation(M=5.4,SE= PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 156 0.157)thanbeforestartingthesimulation(M=4.75,SE=0.162),t(24)=5.639,p<.01,r =.75. TABLE22‐PAIREDSAMPLESSTATISTICS Paired Samples Statistics Knowledge Confidence 25 .739 .148 C2 5.40 25 .786 .157 C1 4.75 25 .810 .162 5.71 25 1.168 .234 T1 5.39 25 1.188 .238 Team Experience (enhanced) TP2 5.68 25 1.225 .245 TP1 5.38 25 1.199 .240 Teamwork (generic) TG2 6.02 25 .757 .151 TG1 5.56 25 .939 .188 Generic Simulation Attitude SG2 5.27 25 1.117 .223 SG1 5.35 25 1.187 .237 Project Simulation Attitude SP2 5.28 25 1.026 .205 SP1 5.39 25 .955 .191 Project Simulation Attitude ‐ difficulty SP2_D 5.51 25 1.156 .231 SP1_D 5.47 25 1.113 .223 S2_HARD 3.64 25 1.604 .321 S1_HARD 4.76 25 1.985 .397 Earned Value Analysis Problem Overall Problem‐ Solving Score 5.01 Std. Error Mean .132 T2 Network Analysis Problem K1 25 Std. Deviation .661 N Team Experience (McCreery) Project Simulation Difficulty Question K2 Mean 5.50 N2 2.28 25 1.400 .280 N1 2.24 25 1.640 .328 E2 2.28 25 1.768 .354 E1 1.80 25 1.658 .332 P2 4.56 25 2.347 .469 P1 4.04 25 2.638 .528 PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 157 TABLE23‐PAIREDSAMPLESCORRELATIONS Paired Samples Correlations Knowledge K2 & K1 N 25 Correlation .752 Sig. .000 Confidence C2 & C1 25 .737 .000 Team Experience (McCreery) T2 & T1 25 .743 .000 Team Experience (enhanced) TP2 & TP1 25 .775 .000 Teamwork (generic) TG2 & TG1 25 .482 .015 Generic Simulation Attitude SG2 & SG1 25 .304 .140 Project Simulation Attitude SP2 & SP1 25 .426 .034 Project Simulation Attitude ‐ difficulty Project Simulation Difficulty Question 25 .442 .027 25 .417 .038 Network Analysis Problem SP2_D & SP1_D S2_HARD & S1_HARD N2 & N1 25 .714 .000 Earned Value Analysis Problem E2 & E1 25 .546 .005 Overall Problem‐Solving Score P2 & P1 25 .643 .001 PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 158 TABLE24–PAIREDSAMPLETESTRESULTS Paired Samples Test Paired Differences 95% Confidence Interval of the Std. Difference Std. Error Dev. Mean Lower Upper .499 .100 .287 .699 t 4.942 df 24 Sig. (2‐ tailed) .000 Knowledge K2 ‐ K1 Mean .493 Confidence C2 ‐ C1 .654 .580 .116 .414 .893 5.639 24 .000 Team Experience (McCreery) Team Experience (enhanced) Teamwork (generic) T2 ‐ T1 .316 .845 .169 ‐.032 .665 1.872 24 .073 TP2 ‐ TP1 .297 .814 .163 ‐.039 .632 1.823 24 .081 TG2 ‐ TG1 .460 .877 .175 .098 .822 2.623 24 .015 SG2 ‐ SG1 ‐.079 1.360 .272 ‐.640 .483 ‐.289 24 .775 SP2 ‐ SP1 ‐.110 1.063 .213 ‐.549 .329 ‐.517 24 .610 SP2_D ‐ SP1_D S2_HARD ‐ S1_HARD N2 ‐ N1 .034 1.199 .240 ‐.461 .529 .143 24 .888 ‐1.120 1.965 .393 ‐1.931 ‐.309 ‐2.850 24 .009 .040 1.172 .234 ‐.444 .524 .171 24 .866 E2 ‐ E1 .480 1.636 .327 ‐.195 1.155 1.467 24 .155 P2 ‐ P1 .520 2.124 .425 ‐.357 1.397 1.224 24 .233 Generic Simulation Attitude Project Simulation Attitude Project Simulation Attitude ‐ difficulty Project Simulation Difficulty Question Network Analysis Problem Earned Value Analysis Problem Overall Problem‐ Solving Score How do participant opinions regarding their team experience in the program and group work in general change as a result of this simulation experience? H3: Participantswillreportahigheropinionoftheirteamexperienceafter completingthesimulationgame. SimilarresultswereobservedusingboththeconstructsbasedonMcCreey’s (2003)teamattitudequestions(T1andT2)andtheenhancedconstructsaddingthe teamworkquestion(TP1andTP2)fromBuzzetto‐MoreandMitchell’s(2009)survey. UsingtheconstructbasedonMcCreey’s(2003)teamattitudequestions,on average,participantsreportsignificantlygreatersatisfactionwiththeirteamexperience PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 159 intheprogramaftercompletingthesimulation(M=5.71SE=0.234)thanbefore startingthesimulation(M=5.39,SE=0.238),t(24)=1.872,p<.05,single‐tailed,r=.36. UsingtheenhancedteamexperienceconstructbasedonMcCreey’s(2003)team attitudequestionsandBuzzetto‐MoreandMitchell’s(2009)teamexperiencequestion, onaverage,participantsreportedsignificantlygreatersatisfactionwiththeirteam experienceintheprogramaftercompletingthesimulation(M=5.68SE=0.245)than beforestartingthesimulation(M=5.38,SE=0.240),t(24)=1.823,p<.05,single‐tailed, r=.35. H4: Participantswillreportahigheropinionofgroupprocessesingeneralafter completingthesimulationgame. Onaverage,participantsassessedsignificantlygreaterpositiveopinionongroup workingeneralaftercompletingthesimulation(M=6.02,SE=0.151)thanbefore startingthesimulation(M=5.56,SE=0.188),t(24)=2.623,p<.01,singletailed,r=.47. How do participant opinions regarding the use of simulations as a learning tool change as a result of this simulation experience? H5: Participantswillreportahigheropinionontheuseofsimulationsasa learningtoolaftercompletingthesimulationgame. Onaverage,therewasnosignificantchangeofopinionontheuseofsimulations asalearningtoolaftercompletingthesimulation(M=5.27,SE=0.223)thanbefore startingthesimulation(M=5.35,SE=0.237),t(24)=‐.289,p>.05,r=.06. H6: ParticipantswillreportahigheropinionofthesimProjectsimulationafter completingthesimulationgame. Onaverage,therewasnosignificantchangeofopinionontheprojectsimulation gameusedinthecourseaftercompletingthesimulation(M=5.51,SE=0.231)than beforestartingthesimulation(M=5.47,SE=0.223),t(24)=‐.143,p>.05,r=.03. PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 160 Thesemeansscoresareslightlyhigherthanthoseobtainedwiththeoriginal constructthatincludedtheopinionquestiononsimulationdifficulty.Analysisusingthe initialconstructalsoindicatedtherewasnosignificantchangeofopinionontheproject simulationgameusedinthecourseaftercompletingthesimulation(M=5.28,SE= 0.205)thanbeforestartingthesimulation(M=5.39,SE=0.191),t(24)=‐.517,p>.11,r =.06. Consideringthesinglequestionregardingperceiveddifficultyofthesimulation, therewasasignificantdecreaseintheperceptionofhowdifficultthesimulationwas aftercompletingthesimulation(M=3.64,SE=0.321)thanhowdifficultthesimulation wasgoingtobebeforestartingthesimulation(M=4.76,SE=0.397),t(24)=‐2.850,p< .01,r=.50. Are students better able to develop and interpret schedule network diagrams and earned value data as a result of the simulation experience? H7: Participantswillbebetterabletosolveprojectscheduleandearnedvalue analysisproblemsaftercompletingthesimulationgame. Onaverage,therewasnosignificantchangeinabilitytoanswertheschedule networkanalysisquestionsaftercompletingthesimulation(M=2.28,SE=0.280)than beforestartingthesimulation(M=2.24,SE=0.328),t(24)=.171,p>.05,r=.03. Onaverage,therewasnosignificantchangeinabilitytoanswertheearnedvalue analysisquestionsaftercompletingthesimulation(M=2.28,SE=0.354)thanbefore startingthesimulation(M=1.80,SE=0.332),t(24)=1.467,p>.05,r=.29 Therewasnosignificantchangeintheoverallproblem‐solvingscoreafter completingthesimulation(M=4.56,SE=0.469)thanbeforestartingthesimulation(M= 4.04,SE=0.528),t(24)=1.224,p>.05,r=.24 Qualitative Results Thissectionsummarizesthequalitativedatacollectedfromthesimulationgame participantsduringtheirdebriefpresentationanddiscussions,fromtheirpostsimulation surveyqualitativequestionresponses,andfromcourseandprogramevaluationsurveys. Postsimulation team presentations.Teamstendedtoincludegenericlessons learnedaspartoftheirpresentationoftheproject‐specificauditandalsoincluded PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 161 simulation‐specificobservationsduringtheiranswertowhattheylearnedaboutproject managementingeneral.Accordingly,bothsectionsoftheteampresentationswere analyzedforsalientthemesrelatingtothepracticeofprojectmanagementinthereal world.Projectresourcemanagementwascommentedonthemost,followedby teamworkandmonitoring&control. Theme 1: Resource management is critical.Itisimportanttostartwith appropriateandcapableresourcesandtohavethemhiredwhenneeded.It’salso importanttobesureresourcesareassignedtoalltasks;otherwise,lesscapable resourcesmaybeassignedtocompletethetasktokeeptheprojectmovingandthiscan betime‐consumingandexpensive.Donotsecuremoreresourcesthanareneededas thistoocanbeexpensivewhentheyarepaidforidletimeorassignedtotasksthatare incompatiblewiththeirbackground.Multipleresourcecross‐functionalassignmentson ataskcanbeeffectiveiftherightskilltypesandpersonalitiesarecombined.The converseisalsotrue;assigninganadditionalresourcewithoutappropriateskillstoa taskwillnotproducethedesiredschedulecompressionandwillalsonegativelyaffect cost. Theme 2: Teamwork is important. Presenterscommentedontheneedforthe studentteamstoapplytheirsocioculturalskillstoworktogetheronplanningforthe simulationandplayingthegame,especiallywhenworkingthroughthestressofthe unknown.Theneedforteammemberinvolvement,planningaheadandcommunication wasstressed.Theuseofvisualaidsduringteamworkingsessionswasfoundtobe useful.Thesimulationgamehighlightedtheimportanceofconsensusbuildinginteam decision‐making;italsodemonstratedthefrustrationthatoccurswhenconsensusis slowly,ornot,reached.Inthesecasesitishelpfultohaveadesignatedteamleaderwho considersalltheopinions,makesadecisionandmovestheteamon;however,thiscan leaveteammembersfeelingdisenfranchisedandunmotivatedforfutureperformance andtheteamleadermustbeawareofthisandactaccordinglytominimizethiseffect. Theme 3: Monitor results and adapt as needed. Thesimulationreinforcedthe valueofplanningaheadandadaptingtoanewstrategyifthecurrentonefails.Havinga planinthebeginningprovidesabaselinetomeasureperformanceagainst.Change controlsareneeded.It’simportanttomonitortrendsrelatedtothebaselineandto differentiatebetweenessentialandinessentialdata.Ittakesjustonewrongdecision duringaphasetohugelyimpactthesimulation;therefore,itisimportanttoanalyze PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 162 decisionsineveryphaseandmakeadjustmentswhenevernecessary.Periodresults needtobereviewedthoroughlyandusedasthebasisforfuturedecisions.Itiseasierto shortentheschedulethanreducecost. Postsimulation survey open‐ended question responses. Thepostsimulation surveyaskedparticipantstorespondtofouropen‐endedquestions: 1. Whatdidyoulikeaboutthejust‐completedsimulationexperience? 2. Excludingchangestothecomputersimulationitself,whatchangesdoyou thinkcouldbemadetotheoverallsimulationexperiencetobetter prepareyoutomanageprojects? 3. Whatchangesdoyourecommendbemadetothecomputersimulation itself? 4. Elaborateonanyotherthoughtsyouhaveregardinglearningproject managementwithacomputer‐basedsimulationgame. Answerstoeachquestionwerecategorizedandanalyzedtoidentifyemergent themes. What they liked about the just completed simulation experience. Resource managementreceivedthemostcommentsfollowedbynear‐equalnumbersofcomments relatedtocompetition,planning,teamworkandanalysis/tools.Lessfrequently mentionedweretheaction‐orientedpracticeofprojectmanagementtechniques,the abilitytoexperiencethewholeprocess,experiencingthepressureofbeingtheproject manager,thedebriefdiscussions,decision‐makingpractice,andchangemanagement. Resource Management. Elevenresourcemanagementcommentsrelatedtothe needtoanalyzeeachresource’scharacteristics,assignthemappropriatelytotaskswhile takingintoaccounttheirefficiencyanditsrelationshiptothebaselinescheduleand budget,considerdiversityeffects,andtoobserveandrespondtotheirperformanceon tasks. Competition. Eightcompetitioncommentsincludedthefunofcompetingagainst theotherteamsforresources,comparingperformanceresultsaftereachworkperiod, andlearningfromeachotherduringtheprojectstatusmeetingswiththesponsorandat theconclusionofthesimulation. Planning. Eightparticipantsreportedlikingthepresimulationplanningexercise topreparethebaselineschedulebasedonnominalresourcesandastrategyincluding thestaffingmanagementplanandaccompanyingrevisionstothenominalbaseline.The PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 163 simulationgameexperienceemphasizedtheimportanceofplanningandhavingan executionstrategy;studentsenjoyedseeinghowtheirstrategiesplayedoutduringthe simulatedprojectexecution. Teamwork. Eightparticipantscommentedfavorablyonworkingasateamtoplan andmanagetheexecutionofthesimulatedproject.Inadditionto“greatteam experience”andsimilarcomments,moreverbosecommentsonteamworkincluded“[it] showedtheimportanceofteamworkinordertosucceedinaproject”and“Ialsoenjoyed splittingthetaskswithmyteam.” Analysis/Tools. Sevenparticipantsmentionedenjoyingusingthetoolsanddatato analyzetheprojectresultsandusetheresultsofthisanalysistoguidedecisionmaking. Onecommented,“ItmademerealizethatIneedtolearntointerpretthedataIam presentedwith.” Recommended changes to the experience excluding the simulation itself. Recommendationsforchangestotheadministrationoftheexperiencegroupedintofour categories:none(9),teamworktime(6),practice(6),andotherunrelatedsuggestions (11). None. Ratherthanwrite“none”orleavetheanswerblank,oneofthenine respondentsinthiscategorywrote,“IfeelsohappyIgottoparticipateinthissimulation asitallowedmetohavealittlebitofwhatrolesaprojectmanagerplaysandhowhis decisionsaffectsthesuccessoftheproject.” Teamwork time.Threeoftherespondentssuggestedmoretimeshouldbe allowedfortheworkperiodsbetweenmilestones,onesuggestedlesstime,one preferredfixedtimesthatdidn’tflextoincreasepressureontheteams,andone preferredfewermilestoneswithoutsuggestingwhethermoretimeshouldbeallowedfor theremainingworkperiods. Practice.Fiverespondentssuggestedafewpracticeroundsbeplayedbefore startingthegameandonesuggestedtheentiregamebereplayedto“seeifyoucan improve.” Other.Othersuggestionsweretohaveliveinstructionontopicsthatarecovered intheuserguide(twomentions),assignanexperiencedchampiontoeachteam,conduct moreprojectreviewswiththesponsor,requirespecificformatsforthein‐progress projectreviewswiththesponsor,includemoreissues/events,compareperformance againstthemodifiedteambaseline,andchangeteamassignmentsduringthesimulation. PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 164 Recommended changes to the simulation. Themostfrequentcommentregarding changingthecomputersimulationitselfwas“none”ornoresponsebyninerespondents followedbysixcommentseachinthecategoriesof“userguide”and“reporting.”Four respondentscommentedontherulesregardingreleaseofresources,twoontheuseof events,twoonresourceassignmentflexibility,andtherewerefiveotherunrelated comments. User guide.Userguidesuggestionsincludedmorescreenshotsofthesimulation “sowearenotscramblingtolearnallthesoftwareinadditiontoinputtingproject managementdecisions,”theadditionofjobdescriptionsofthesimulatedresource categories,clarificationonthepartialassignmentofresources,morediscussiononthe functionalitymetric,andmoreinformationingeneral. Reporting.Reportingsuggestionswereforaneasierwaytoproducechartsand graphs,aCostPerformanceIndex(CPI)report,trendreportsshowingpriorworkperiod results,EarnedValueManagement(EVM)reports,separationoftrainingcostsfromtotal projectcosts,andmorewrittenfeedback. Rules.Fourrespondentssuggestedmodifyingtheresourcehiring/firingprotocol toeliminatetheunrealisticpracticeofbiddingonmoreresourcesthanareneededand allowingtheimmediateterminationofthebackupsiftheteamwonallthebids. Events.Onerespondentwouldliketohaveseentheinclusionofmoreeventsand onerespondentwouldhavepreferredteam‐specific“eventsbasedonindividualteam dynamics.” Other.Othercommentsrequestedawarningmessagetopreventdecision submittalifnoresourcesareassignedtoatask,moreneedforresourcetraining,fewer workperiods,adecisionsummaryscreenpriortosubmittal,andconsiderationofvirtual teamdiversityincalculations(thislastsuggestionisbuiltintothesimulationand discussedintheuserguide). Other thoughts about learning project management with a computer‐based simulation game.Responsesconcerningotherthoughtsgroupedintofourcategories: favorable(11comments),noneornoresponse(6),unrealistic(2),andother(6). Favorable.Favorablecommentsmadebyelevenoftherespondentsincluded:  “IfeltlikeIlearnedalot.Irealizedwhatdatawasmoreimportantthan others.ThoughImayhavethoughtsomethingwasimportantearly,Iwas forcedtoreevaluatewhatinfowasuseful.” PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE  165 “Itprovidedanexcellentbaselineexperiencetocomponentsofproject management.”  “Greatexperience;canrelatetoreallifePMexperience”  “Greatalternativeto‘reallife’projects.Someitemswereunrealisticbut mostwererealistic.”  “Ihavelearntalotespeciallyinthehumanresourceareaofproject management”  “Greatidea.Givesstudentsasclosetoa‘realworld’experienceasyou can.” Threeoftherespondentswithsimilarfavorablecommentsalsosuggested playingmoresimulationgameswouldbebeneficial. Unrealistic.Tworespondentscommentedthatthesimulationwasnotrealistic, referringtothemannerinwhichresourcesarehiredandreleasedandtheabilityto “game”thesystembybiddingonmoreresourcesthanneedingandimmediately releasingonesthataren’tneeded:  “Itisnotreallyarealworldexperiencesinceinarealworldexperiencea projectmanagercannothireandfirepeopleinthesameperiodiftheyare notneeded.”  “Simulationwillneverbeanaccuratereflectionoftherealworld.There willalwaysbeawayto‘game’thesystem.Iseenowayaroundthis limitation.” Other.Oftheothercomments,onerespondentwrote,“Iwouldratherhave learnedonarealprojectwithrealpeople.”Tworespondentswouldhavepreferredto workaloneonthesimulation,onefoundtheexperience“somewhatexhausting,”another wroteonly“planahead,”andonecommentedthatseniorprojectmanagersmaynotget asmuchbenefitfromthesimulationgameexperienceasjuniorprojectmanagers. Course evaluation survey responses. Duetoaproceduralerror,nocourse evaluationsurveywasissuedforthefirstcohortgroup.Thesecondcohortgroup completedacourseevaluationsurveythatexplicitlyasked,“Whatcommentswouldyou liketomakeaboutthesimulation?”Fiveofsixresponseswereveryfavorable,echoing previouscommentsfromthepostsimulationsurveyresponsesandoneresponsealso repeated“simulationisgoodforjr.projectmanager,butnotforsr.projectmanager”and added,“shouldaddmorediscussionshowtolinksimulationtoreallifeproject.” PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 166 Program evaluation survey responses. Programevaluationsurveysissuedasthe completionofthe21SCHProjectManagementCorePhaseoftheprogramincludethe questions: 1. Ifyouaregiventheoptiontochangeonly3thingsintheProject ManagementCoreProgram,whatwouldthose3thingsbe,andwhy wouldyouchangethem? 2. Ifyouaregiventheoptiontoretainonly3thingsintheProject ManagementCoreProgram,whatwouldthose3thingsbe,andwhy wouldyoukeepthem? Bothcohortgroupswereaskedtocompletethissurvey.Norespondentsto question1mentionedchangingoreliminatingthesimulation.Fourof21respondents mentionedretainingthesimulationinresponsetoquestion2. Graduate program exit survey responses.StudentsgraduatingwithaMasterof ScienceorMasterofBusinessAdministrationdegreeareaskedtocompleteasurvey shortlyaftertheirgraduationtoprovidefeedbackontheiroverallgraduateeducation experience.ResponsestothissurveytendtoconcentrateonthemorerecentBusiness andMBACorePhasesoftheprogramratherthantheProjectManagementCorePhase. Onthequestionofwhataspectsoftheprogramwillyourememberthemost, respondentsalmostunanimouslycitedthetwoweekinternationalstudytripwithafew favorablecommentsabouttheclassenvironmentandprofessors. Respondingtothequestionofsuggestionsforimprovingtheprogram,one respondentreactedfavorablytotheuseofsimulationsintheprogram,“Wehavedone2 simulationprojects,1inProjectManagementand1instrategicManagement.Iwould liketoseemoresimulationprojectthroughouttheprogram.”Therewerenonegative commentsaboutusingsimulations. Course Evaluation Survey Quantitative Data Beginningwiththesecondcohortgroup,thecourseendingsurveyforthecourse withthecapstonesimulationgameaskedrespondentstoindicatetheiragreementwith fivestatementsrelatedtothesimulation.Thequestionsandtheresponsesshownin Table25indicateafavorableviewontheuseofthesimulationgameinthecourse. PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 167 TABLE25‐COURSEEVALUATIONRATINGS OC2011 The simulation increased my knowledge of fundamental project management principles The simulation experience improved my ability to work with and lead project teams My team worked together effectively on the simulation I enjoyed the simulation experience I recommend this simulation be kept in the program SA A N D SD Rating (5) (4) (3) (2) (1) (mean) 3 2 1 1 4 5 0 1 1 4.3 4 5 5 3 1 2 1 4.6 4.4 4.6 Summary of Results Table26summarizestheresultsofthequantitativehypothesistesting.H1 throughH4aresupportedindicatingstudentsreportedsignificantincreasesinproject managementknowledge,abilitytoapplythatknowledge,teamexperienceattitude,and attitudetowardsfuturegroupwork.H5andH6arenotsupportedindicatednochange inattitudeaboutusingsimulationgamesasexperientiallearningactivities,eitherin generalorforthissimulationexperienceinparticular.H7wasnotsupportedindicating noincreaseinabilitytosolveprojectscheduleornetworkanalysisproblems. Analysisofteampresentationanddebriefcommentsandnarrativeresponsesto surveyquestionssupporttheconclusionthatthesimulationgamewaseffectiveand addedvalue.Thisconclusionisreinforcedbyresponsestospecificcourseevaluation ratingsrelatedtothesimulationgameexperience. PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 168 TABLE26‐SUMMARYOFRESULTS Hypothesis H1: Participants will assess their project management knowledge level higher after completing the simulation game. H2: Participants will assess their ability to apply project management knowledge higher after completing the simulation game. H3: Participants will report a higher opinion of their team experience after completing the simulation game. H4: Participants will report a higher opinion of group processes in general after completing the simulation game. H5: Participants will report a higher opinion on the use of simulations as a learning tool after completing the simulation game. H6: Participants will report a higher opinion of the simProject simulation after completing the simulation game. H7: Participants will be better able to solve project schedule and earned value analysis problems after completing the simulation game. Result Significance (one‐tailed) Supported P < .01 Supported P < .01 Supported P < .05 Supported P < .01 Not supported ns, p = .388 Not supported ns, p = .444 Not supported ns, p = .117 PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 169 Discussion Thischapterdiscussestheresultsofthisstudyintothevalueandeffectivenessof usingaprojectmanagementsimulationgameasacapstoneexperientiallearningactivity inagraduateexecuteMS/MBAdegreeprogramwithinemphasisinproject management.ThechapterbeginswithadiscussionofKolb’s(1984)perspectiveson experientiallearninginthecontextofusingtheSimProjectprojectmanagement simulationgameasanexperientiallearningactivity.Thisisfollowedbyadiscussionof theresultsandacontrastwithpriorsimulationgamingresearch.Thechapterconcludes withadiscussionofthelimitationsofthisstudyandasummaryofthefindings.The implicationsofthisresearchandrecommendationsforfutureresearcharediscussedin thenextchapter. Experiential Learning and the Project Management Simulation FavorablereactiontoplayingSimProjectisconsistentwithKolb’s(1984)finding thatadultlearnerspreferexperientiallearningmethodsthatallowthemtotestthe relevanceandapplicationofideasinacontextthatfacilitatescomparisonwiththeir “ownaccumulatedexperienceandwisdom....Fortheseadults,learningmethodsthat combineworkandstudy,theoryandpracticeprovideamorefamiliarandtherefore moreproductivearenaforlearning.”Kolbidentifiedfourlearningmodesinhis experientiallearningmodel:affective,perceptual,symbolicandbehavioral(thismodelis describedintheLiteratureReviewchapter). Theaffectivelycomplexlearningenvironmentrequiresactivitiessimilartothose performedasaprofessionalorreflectionontheexperiencetogenerateinsightsandself‐ discovery(Kolb,1984).BothoftheseaspectsarecharacteristicofplayingSimProject: thesimulationoftheprofessionalprojectmanagerexperienceduringthecourseofthe gameandthereflectionthatoccursaspartoftheperiodicprogramreviewsandduring thedebriefsessionafterthegame. Perceptivelycomplexlearningenvironmentsrequireconsideringatopicfrom differentperspectivesandindifferentways(Kolb,1984).Thismodeoflearningisalsoa componentofusingSimProjectasanexperientiallearningactivity.Theuseofteams, selectedtomaximizetheirdiversity,asthemakersofgamedecisionshelpsassurethe considerationofdifferentperspectiveswhileplanningforandplayingtheproject simulationgame.Theinstructor‐facilitatedperiodicprogressreviewsandformal PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 170 “reportstomanagement”helpstimulateinquiryandreflectionasaguidetofutureaction andcircumventanyparticipant’sdesiretomerelyevaluatebasedonthecorrectnessof thedecisions. Symbolicallycomplexlearningenvironmentsinvolveabstract“problem[s]for whichthereisusuallyarightanswerorabestsolution”(Kolb,1984)wherethelearner isguidedandconstrainedbytherulesandtheteacheristheacceptedexpert.Thismode sharessome,butnotall,characteristicswiththeSimProjectexperience.The performanceofprojectworkandthehumanresourcesthatperformitarebothabstract sincetheyaresimulatedonthecomputer.Also,therearerulesofthegameanda performancegoaltoachieve.However,herethebestsolutionisnotknownandthe instructorplaysafacilitativeratherthanexpertrole. Behaviorallycomplexlearningenvironmentsemphasizeapplyingknowledgeor skillstoapracticalproblemthatneednothaveacorrectorbestanswer;“butitdoes havetobesomethingthelearnercanrelateto,value,andfeelsomeintrinsicsatisfaction fromhavingsolved”(Kolb,1984).Thefocusofbehaviorallycomplexlearningison completingthetaskandthelearnerbeingresponsiblefordecidingonacourseofaction andmanaginghisorhertimewithintheconstraintsofpossiblecheckpointsanda deadline.Thisisanaccuratedescriptionoftheenvironmentencounteredforthis applicationofSimProjectasanexperientiallearningactivity. Inthecontextofthisstudy,theSimProjectexperiencehasastrongbehavioral learningmodeandexhibitsstrongcharacteristicsoftheaffectiveandperceptualmodes, andtoalesserdegree,thesymbolicmodewherelearnerpreferencestendtobefor thinkingaloneandavoidinggroupexercisesandsimulations.TeamsplayingSimProject cansomewhatmitigatethepreferenceforsymboliclearningbydividingthetasksofeach workperiodamongtheteammemberssuchthattheneedforindividualthinkingcanbe providedfor.Whilemostteamswereobservednotpreferringtosubdividethework, dividingdecisionperiodtaskswasobservedinseveralinstanceswhereteamshadoneor morememberswithstrongpreferencesforsymboliclearning. Discussion of Findings Althoughseveralvariablesdidnotmeetalltheassumptioncriteriafor parametrictests,mostdidandallwereassumedtobeusablesince“mostparametric testsarerelativelyinsensitivetoslightviolationsofassumptions...anddotendtobe usedifpossible.”(Burns&Burns,2008). PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 171 Research Question 1 – Support for Hypotheses 1 and 2.Researchquestion1asked whetherparticipantperceptionsofprojectmanagementknowledgeandconfidencein theirabilitytoapplythisknowledgeincreasedasaresultofthesimulationgame experience.Hypothesis1statedperceptionsofprojectmanagementknowledgewould increaseandHypothesis2statedperceptionsinabilitiestoapplythisknowledgewould increase. Finding 1.Onaverage,participantsassessedsignificantlygreaterproject managementknowledgeaftercompletingthesimulation(M=5.5,SE=0.132)than beforestartingthesimulation(M=5.01,SE=0.148),t(24)=4.942,p<.01,r=.71.This 0.49increaseinthemeanissmallerthanthe1.26significantincreaseobservedby McCreery(2003)inhisstudyusingadifferentprojectmanagementsimulationgameas anelementinagraduateprojectmanagementcourseatNorthCarolinaStateUniversity’s CollegeofManagement.However,asshowninFigure20,thestudentsinMcCreery’s studystartedwithaloweraverageself‐assessmentandhadmoreroomtogrow,perhaps becausethiswastheirfirstcourseinprojectmanagementandtheywerenotinaproject managementspecialtyprogram. FIGURE20‐COMPARISONOFAVERAGEKNOWLEDGERATINGS This Study Post McCreery (2003) Pre 1.00 2.00 3.00 4.00 5.00 6.00 7.00 Self‐Assessment of Knowledge 1 = Extremely Low 7= Extremely High McCreery(2003)reportedthattheknowledgeincreasewasgreaterfortheleast experiencedquartileofstudents(1.46forthosewithnooralmostnoprojectwork experience)andlessforthemostexperiencedquartile(1.01forthoseaveraging10.7 yearsofexperience).Inbothofthesecases,theincreasesweregreaterthanfoundinthis PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 172 studybutthepostsimulationknowledgescoreswereless(4.88[lowexperience]and 5.14[highexperience]versus5.5[thisstudy]).Thisisnotunexpectedasthestudentsin thisstudywerecompletingtheirsixthcourseinprojectmanagementinaproject managementspecialtyprogramwithareportedaverageof12.7yearsofwork experienceandsevenyearsofexperienceworkingwithprojects.Withthisexperience andadditionalcoursework,itisnotsurprisingthatthesestudentswouldreportahigher thanmid‐rangeself‐assessmentofknowledgepriortotheELA. Thepostsimulationaverageratingof5.5indicatesafairlyhighself‐assessmentof knowledge.Sincethereisalwaysmoretolearn,onewouldnotexpectverymany studentstobeboldenoughtoratetheirknowledgeasextremelyhighandscores between5and6arereasonable. Finding 2.Onaverage,participantsassessedsignificantlygreaterconfidencein theirabilitytoapplyprojectmanagementknowledgeaftercompletingthesimulation(M =5.4,SE=0.157)thanbeforestartingthesimulation(M=4.75,SE=0.162),t(24)= 5.639,p<.01,r=.75.Again,asshowninFigure21,theparticipantsinthisstudy reportedasmallerincreasebutahigheraverageratingofconfidenceintheirabilityto applyprojectmanagementknowledgethanMcCreery(2003)observedinhisstudy (unliketheknowledgerating,McCreeryfoundnosignificantdifferenceintheaverage increasebetweenlowexperienceandhighexperiencequartiles[1.02and1.06, respectively]). FIGURE21‐COMPARISONOFAVERAGECONFIDENCERATINGS This Study Post McCreery (2003) Pre 1.00 2.00 3.00 4.00 5.00 6.00 7.00 Self‐assessment of confidence in ability to apply knowledge 1 = Extremely Low 7 = Extremely HIgh PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 173 Whilethe0.65increaseinperceptionsoftheabilitytoapplyprojectmanagement knowledgeisnotasdramaticasfoundbyMcCreery(2003),theratingof5.4indicatesa fairlyhighself‐assessmentofconfidence. Conclusion.Consideringthelargeeffectsizesoftheseresultsandthethemes expressedinthequalitativedata,itappearsstudentsbelievetheirprojectmanagement knowledgeandconfidenceintheirabilitytoapplythisknowledgeincreasedasaresult oftheSimProjectexperientiallearningactivity.Inusingasimulationgameasacapstone activity,itisnotsurprisingtoseemodestgainsinknowledgeandconfidence.Large gainswouldimplystudentsstartedwithlittleornoknowledgeandconfidence.This mightbethecaseifthesimulationwasusedatthebeginningofthecoursewithaless experiencedgroupofstudents;butinthisstudy,anysignificantgainwithexecutive educationstudentsismeaningful. Research Question 2 – Support for Hypotheses 3 and 4.Researchquestion2 exploredhowparticipantopinionsregardingtheirteamexperienceintheprogramand groupworkingeneralchangedasaresultofplayingthesimulationgame.Hypothesis3 statedparticipantswouldreportahigheropinionoftheirteamexperienceafter completingthesimulationgame.Hypothesis4statedparticipantswouldreportahigher opinionofgroupprocessesingeneralaftercompletingthesimulationgame. Finding 3.Satisfactionwiththestudents’currentclassteamwasassessedusing thenine“teamexperienceassessment”itemsfromMcCreey’s(2003)study.Unlike McCreery’sstudywhichonlyassessedthequalityofthestudentteamexperience followingthecompletionofthesimulationgame,thisstudypolledsatisfactionboth beforeandafterthesimulationgame.AlsounlikethestudentsinMcCreery’sstudy, thesestudentshadalreadyparticipatedwiththeirclassteamonmultipleassignments overaspanofninemonthsbeforeworkingtogetherasamanagementteamtoexecute thesimulatedproject.Aspartofthepresimulationsurvey,studentsassessedtheirprior teamexperienceonthepriorexercisesandactivitiesintheprogram.Followingthe simulation,theywereaskedtoassesstheirteamexperiencethroughoutthesimulation exerciseusingthesamequestionsandratingscale(1=stronglydisagree;7=strongly agree):  Theworkloadwasbalancedacrossallteammembers  Teammemberscooperatedwellthroughouttheprogram  Ourteamworkedinanefficientmanner PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE  174 Teammembersallparticipatedequallyintheteamdecisionmaking process  Ourteammaintainedapleasantworkingatmosphere  Ourteamworkedoutdisagreementsinanequitablemanner  Teammemberswerehighlymotivatedtoperformwellintheteam exercisesandactivities  Overall,Iamsatisfiedwithmyteamexperience  Iwouldbewillingtoworkwithmyteamonanactualprojectinthefuture [AppendicesCandD] Averagescoreswerefavorablebothbeforeandafterthesimulationand,on average,participantsreportedsignificantlygreatersatisfactionwiththeirteam experienceintheprogramaftercompletingthesimulation(M=5.71SE=0.234)than beforestartingthesimulation(M=5.39,SE=0.238),t(24)=1.872,p<.05,single‐tailed, r=.36.Theaverageincreaseinagreementwiththesurveystatementsof0.32suggestsa slightlyhigherlevelofsatisfactionwiththeirteamexperienceplayingthesimulation gamethanontheotherpriorteamassignmentsintheprogram.Thisscoreislessthan the6.29compositeprocessmeanscorereportedbyMcCreery(2003).Onepossible explanationforthishigheraverageisMcCreery’steamsstartedwithoutaneighttonine monthlegacyofworkingtogetherandhadfeweropportunitiestobecomedissatisfied witheachother. Finding 4.TwoquestionsfromBuzzetto‐MoreandMitchell’s(2009)study comprisedtheconstructusedtoassessgenericgroupworkattitudes:  Groupprojectshelppreparestudentstobeabletoworkinprofessional groupsinthefuture  Ienjoyworkingingroups[AppendicesCandD] Onaverage,participantsassessedsignificantlygreaterpositiveopinionongroup workingeneralaftercompletingthesimulation(M=6.02,SE=0.151)thanbefore startingthesimulation(M=5.56,SE=0.188),t(24)=2.623,p<.01,singletailed,r=.47. This0.46increaseto6.02ona7‐pointscale(1=stronglydisagree;7=stronglyagree) indicatesahighlevelofagreementwiththesetwogenericteamworkattitudequestions andamorefavorableattitudeaftercompletingthesimulationgame. PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 175 Conclusion.Thegraduatedegreeprogramunderstudyemphasizesleadership andgroupworkduringtheprojectmanagementcorephaseanditisreassuringandnot surprisingthatopinionsarehighlyfavorableaboutgroupworkandworkingwiththeir assignedprojectteams.A0.46pointincreaseingenericteamworkattitudeand0.32 pointincreaseinattitudetowardworkingwiththeirspecificteamaftercompletingthe simulatedprojectsuggesttheprojectsimulationgameexperiencereinforcedandslightly strengthenedtheirattitudesaboutlearningonteamsinanacademicenvironmentand ontheplayingofsimulationgamesinparticular.Consideringthisincreaseandthe teamworkthemesexpressedinthequalitativeresults,itappearsstudentopinionoftheir teamexperienceandgroupworkingeneralwasfavorableandimprovedasaresultof participatinginthesimulationgame.Thisisespeciallymeaningfulconsideringtheonly statedlearningobjectiveinthecoursesyllabusisto“demonstrateyourabilitytoworkas ateamtoplanandexecuteasimulateproject.” Research Question 3 – Lack of Support for Hypotheses 5 and 6.Researchquestion 3asked,“Howdoparticipantopinionsregardingtheuseofsimulationsasalearningtool changeasaresultofthissimulationexperience?”Hypothesis5stated,“Participantswill reportahigheropinionontheuseofsimulationsasalearningtoolaftercompletingthe simulationgame”andhypothesis6stated,“Participantswillreportahigheropinionon theSimProjectsimulationaftercompletingthesimulationgame.”Neitherhypothesisis supportedbythisresearch. Finding 5.Thepresimulationandpostsimulationsurveyquestionsforassessing genericsimulationattitudesareadaptedfromapostsimulationsurveyconductedby Buzzetto‐MoreandMitchell(2009)andusedthe7‐pointscale(1=stronglydisagree;7= stronglyagree):  Simulationsallowstudentstoseehowcourseconceptsareappliedinreal worldpractice  Simulationsprovidevaluablereal‐worldexperience  Computersimulationshelponebetterunderstandthedecisionmaking processthatoccursinprofessionalpractice  Simulationscanmakeclassmorefun  Simulationshelpstudentsbuildprofessionalskills  Iwouldliketoseemoresimulationsinfutureprograms[AppendicesC andD] PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 176 AsshowninFigure22,studentsindicatedamoderatelystrongagreementwith thestatementsindicatingfavorableattitudestowardstheuseofsimulationinthe classroombothbeforeandafterthesimulationgameexperience.However,onaverage, therewasnosignificantchangeofopinionontheuseofsimulationsasalearningtool aftercompletingthesimulation(M=5.27,SE=0.223)thanbeforestartingthe simulation(M=5.35,SE=0.237),t(24)=‐.289,p>.05,r=.06.Althoughdisappointing fromtheperspectivethatthesimulationexperiencedidnotresultinanincreasein favorableattitudetowardtheuseofsimulationgamesasanexperientiallearningmode, thisfindingsuggeststhattherewasnotahalo‐effectaccountingforthesignificant increasespreviouslydiscussedforhypotheses1–4.Studentsstartedthesimulation gamewithafairlyhighopinionontheuseofsimulationgamesasateachingtooland retainedthisopinionattheendofthesimulation. FIGURE22‐COMPARISONOFPRE‐ANDPOST‐SIMULATIONATTITUDESONSIMULATION Generic simProject Post Pre simProject difficulty 1 2 3 4 5 6 7 Rating on use of simulation as learning tool 1 = strongly disagree 7 = strongly agree Finding 6.ThequestionsforassessingattitudesaboutSimProjectarebasedon postsimulationsurveysconductedbyBuzzetto‐MoreandMitchell(2009)andAhn (2008).AsshowninTable27,thepresimulationsurveysusedslightlyre‐phased questionstocorrectthecontext. PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 177 TABLE27–SIMPROJECTATTITUDEQUESTIONS Presimulation Survey This simulation will be a valuable learning experience This simulation will help me understand key course concepts I think I will be pleased with my performance on the simulation I think I will enjoy the computer simulation I believe this simulation will help me feel more confident about my abilities to manage projects I believe this simulation will be educational I believe this simulation will be fun and exciting I believe this simulation will be difficult Postsimulation Survey This simulation was a valuable learning experience This simulation helped me understand key course concepts I am pleased with my performance on the simulation I enjoyed the computer simulation As a result of the simulation, I now feel more confident about my abilities to manage projects This simulation was educational This simulation was fun and exciting This simulation was difficult Onaverage,therewasnosignificantchangeofopinionregardingSimProject aftercompletingthesimulation.Thiswasthecaseforboththeconstructexcludingthe“I believethissimulationwillbedifficult”question[(M=5.51,SE=0.231)versus presimulation(M=5.47,SE=0.223),t(24)=‐.143,p>.05,r=.03]andfortheconstruct includingit[(M=5.28,SE=0.205)versuspresimulation(M=5.39,SE=0.191),t(24)= ‐.517,p>.11,r=.06].AsshowninFigure22,studentsbeganthesimulationgamewith fairlyfavorableexpectationsandtheseexpectationswereconfirmedasmetbythe postsimulationsurveyresults. Consideringthequestionregardingperceiveddifficultyofthesimulationthat wasremovedfromtheSP(thissimulation)variableconstruct,studentsperceiveda significantdecreaseinhowdifficultthesimulationwasaftercompletingthesimulation (M=3.64,SE=0.321)fromhowdifficultthesimulationwasgoingtobebeforestarting thesimulation(M=4.76,SE=0.397),t(24)=‐2.850,p<.01,r=.50.Onepossible explanationistheystartedthesimulationwithasmallamountofapprehensionthatwas relievedbyafavorableexperiencebuttherearenoqualitativefindingsspecifically supportingthisconclusion. Conclusion.Althoughthesetwofindingsindicatenosupportforthehypotheses relatedtomorefavorableattitudesaboutsimulationgamesingeneralandtoSimProject inparticular,thefactthatattitudesstartedoffmoderatelypositive,andremainedso, reinforcestheiruseinanacademicprogramandthelackofasignificantincreaseshould notnecessarilybeaconcern.Hadtherebeenanunfavorableattitudeorasignificant decreaseinratingsfollowingthesimulationgameexperience,thenthiswouldsuggesta problemrequiringfurtherinvestigationandmitigation. PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 178 Basedontheobservedintensityofactivityduringtheteamworkingsessionsand noqualitativethemessuggestingstudentsthoughtmanagingthesimulatedprojectwas easy,thedecreaseindifficultyratingdoesnotappeartoindicatethesimulationwasnot achallengingandrewardingexperience. Research Question 4 – Lack of Support for Hypotheses 7.Researchquestion4 askedwhetherstudentswerebetterabletodevelopandinterpretschedulenetwork diagramsandanalyzeearnedvaluedataasaresultofcompletingthesimulation experience.Duringtheirpriorcoursework,studentscompletedindividualassignments requiringthemtoproduceandanalyzeaschedulenetworkdiagramandtosolveseveral earnedvalueanalysisproblems.Sinceallstudentssuccessfullycompletedthese assignments,thepresimulationassessmentprovidesanindicationoftheirretentionof theseabilitiesandthepostsimulationassessmentindicatesiftheyimprovedonthem duringthecourseofthesimulation. Fortheschedulenetworkanalysisproblem,studentsaregivenatablelisting sevenactivitieswiththeirpredecessorsanddurationsandaskedtoidentifytheactivities onthecriticalpath,theprojectduration,andtheslacktimefortwoactivities.Each correctanswerreceivesonepointandaperfectscoreis4. Fortheearnedvalueproblem,studentsaregivenagraphicaldisplayof cumulativeearnedvaluedataforaprojectinprogressandaskedtoassesshowwellthe projectisperformingrelativetoscheduleandbudgetandsixadditionalquestionsare askedrequiringknowledgeofearnedvaluevariancesandindices.Eachcorrectanswer receivesonepointandaperfectscoreis7. Theseschedulenetworkandearnedvaluescoresarecombinedfortheoverall problem‐solvingabilityscorewhichcanrangefromzeroto11. Resultshereweresurprisinganddisappointingfromboththeretentionof knowledgeandabilityperspectiveandfromthelearningfromthesimulation perspective. Finding 7. Onaverage,therewasnosignificantchangeintheoverallproblem‐ solvingscoreaftercompletingthesimulation(M=4.56,SE=0.469)thanbeforestarting thesimulation(M=4.04,SE=0.528),t(24)=1.224,p>.05,r=.24.Thiswasalsothe caseforthecomponentquestions:  Onaverage,therewasnosignificantchangeinabilitytoanswerthe schedulenetworkanalysisquestionsaftercompletingthesimulation(M PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 179 =2.28,SE=0.280)thanbeforestartingthesimulation(M=2.24,SE= 0.328),t(24)=.171,p>.05,r=.03.  Onaverage,therewasnosignificantchangeinabilitytoanswerthe earnedvalueanalysisquestionsaftercompletingthesimulation(M= 2.28,SE=0.354)thanbeforestartingthesimulation(M=1.80,SE= 0.332),t(24)=1.467,p>.05,r=.29 Reviewofthequestionnaireresponsesrevealedatendencyforstudentstonot doacompleteforwardandbackwardpassanalysisofthenetworkdiagramresultingin missingoneoftheactivityslacktimequestionsandtoleaveoffthenegativesignwhen answeringtheearnedvaluevariancequestions;otherscomplainedaboutnothaving accesstotheformulas. Conclusion.Thepoorscoresonthepresimulationquestionssuggestabitof carelessnessandperhapsalackofknowledgeretentionwhenansweringthenetwork analysisquestions,andalackofretentionoftheearnedvalueconceptsfromprior coursework.Thetimemanagementmodulewasdeliveredthreemonthspriortothe presimulationquestionnairebeingissuedandtheearnedvaluemodulewastwomonths prior.Itappearsongoingpracticeoron‐the‐jobapplicationisneededfortheseconcepts tostayfreshinmind.Italsoappearsstudentsdonotrefreshthemselvesonthis knowledgewhileplayingthegameeventhoughtheteamsaremanagingschedulesand preparingperformancereportstomanagement.Sincethepilotgroupdidexperiencea significantearnedvaluescoreincreasewhenashort“chalktalk”washeldduringoneof thestatusreviews,thisshouldbeconsideredforfutureiterationsifrefreshmentof earnedvalueconceptsistobeoneofthelearningobjectives.Anotheroptionwouldbeto requireapplicationoftheconceptsinthereportstomanagementduringthesimulation game. Limitations of this research Thesizeandnatureoftheprogramunderstudyresultedinsurveyingand observingasmallnumberofstudents(28).Ideallymorestudentswouldhavebeen availabletoincreasethestatisticalpoweroftheresults.Thesamplesizeusedisnearly sufficienttodetectlargeeffects(r=.5)withastatisticalpowerof.8,butlargersample sizesareneededtodetectsmallereffectsizeswiththesamestatisticalpower(Field, 2009). PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 180 Ethicalconsiderationsrequiredtheuseofapreexperimentalpretest‐posttest methodologyratherthanatrueexperimentwithacontrolgroup.Atruecontrolgroup wouldrequiredoingnothingwithsomestudentswhileothersexperiencedthe simulationgame.Substitutinganalternateactivitywouldnotbeatruecontrolsituation and,asPfahl(2004)experienced,wouldconfoundtheresults. Summary ThereisastrongrelationshipbetweenusingSimProjectasanexperiential learningactivityandKolb’s(1984)affective,perceptualandbehavioralmodesof experientiallearningandalessstrongrelationshipwiththesymbolicmodeoflearning. Theuseofthissimulationrespondstolearnerswithapreferenceforanyofthese learningmodes;however,learnerswithastrongpreferenceforsymboliclearningmay belesssatisfiedthantheothers. Aftercompletingtheprojectmanagementsimulationgame,studentsperceiveda significantincreaseinprojectmanagementknowledge,confidenceintheirabilityto applythisknowledge,favorableattitudetowardsworkingwiththeirteamandfavorable attitudetowardsgroupwork,ingeneral.Theincreasesinknowledgeandconfidence perceptionsweresmallerthanseenbyMcCreery(2003)butthesedifferencesare explainedbythedifferencesinclasscomposition.Onaverage,McCreery’sstudentswere lessexperiencedandwereattendingtheirfirstprojectmanagementcoursewhilethe studentsinthisstudyweremoreexperiencedandattendingtheirsixthproject managementcourse.Consequently,McCreery’sstudentsreportedlowerknowledgeand confidencescorespriortoparticipatinginthesimulationandcomparablescoresafter completingthesimulation.Thisdifferenceinstartingpointmayalsobeduetoproject managementbeingacasualareaofinterestforMcCreery’sstudentswhereasitisa primarypointofinterestforthestudentsintheprogrambeingstudied.After completing15SCHofprojectmanagementandanintegrated3SCHorganization behaviorcoursefocusedonprojectleadership,itisnotsurprisingthatstudentsinthis studywouldreporthigherlevelsofknowledgeandconfidencepriortobeginningthe simulation. Nosignificantattitudeimprovementswerefoundforthegenericuseof simulationgamesasexperientiallearningactivitiesorforthespecificuseofSimProject asaprojectmanagementELA,butattitudesweremoderatelyfavorableandremainedso acrosstheexperience.Thislackofchangemaybeviewedasacontrolandstrengthens PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 181 thefindingofsignificancefortheincreasesinperceivedknowledgeandconfidenceand improvedattitudetowardsteamworkandtheirteam. Nosignificantimprovementintheabilitytodevelopandanalyzeaschedule networkdiagramortointerpretagraphicalpresentationofcumulativeearnedvalue datawasfound.Althoughthisisdisappointingtotheinstructor,itisnotsurprising consideringthisimprovementisnotastatedlearningobjectiveforplayingthegameand nothingwasexplicitlybuiltintotheexperiencetopromoteitsoccurrence. Thenextchapterfurthersummarizesthisstudy,discussesitsimplicationsfor practiceandrecommendstheconductofadditionalresearch. PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 182 Conclusion Theuseofsimulationsinacademicbusinesscourseshasgrownconsiderably sinceonewasdevelopedfortrainingbytheAmericanManagementAssociationin1956; theyarenowoftenthecentralmodeofinstructioninsomecourses(Fariaetal.,2009). Theaimofthisstudyistodetermineifonesuchuse,theapplicationoftheSimProject projectmanagementsimulationgameasacapstoneexperientiallearningactivityina particularprojectmanagementgraduatedegreeprogram,providesvaluetothestudent andisaneffectiveuseofclasstime. Areviewoftheliteraturefoundanactiveassociationencouragingandreporting researchontheuseofsimulationgamesinacademicbusinessmanagementprograms. Thisreviewexploredthreerelevantbodiesofknowledge:learningtheory,simulation gameapplicationandsimulationgameeffectiveness.Mostliteratureonbusiness simulationgameeffectivenessexplorestheuseofmarketing,strategyortotalenterprise simulationgames;relativelyfewarticlesreportresearchontheuseofproject managementsimulationgames.Thescarcityofresearchontheuseofproject managementsimulationgamesasapedagogicaltoolidentifiesagapinman’sknowledge. Thereissubstantialcontroversyintheacademiccommunityonhowto determinesimulationeffectivenessduetothedifficultyindefiningwhatpeopleactually learnfromsimulationgamesandhowtomeasureit;asaresult,therearerepeatedcalls formorerigorousresearchonthemethodsusedtoadministersimulationgamesandon theassessmentoflearningthatresults(P.H.Anderson&Lawton,2009;Crookall,2010; Fariaetal.,2009).Oneareaofagreementisthatthesimulationgameresultsor performancescores(e.g.,simulatedmarketpenetration,profit,scheduleperformance,or stakeholdersatisfaction)arenotvalidindicatorsoflearningandvaluablelessonscanbe learnedthathavenothingtodowiththeeducator’sintention(Gosenpud,1990; Gosenpud&Washbush,1994;Greenlaw&Wyman,1973).Consequently,when researchingtheeffectivenessofbusinesssimulationgames,studentperceptionsof learningarefrequentlyusedasaproxyformoredirectmeasuresoflearningand longitudinalpreexperimentalandpostsimulationexperiment‐controlmethodsare recommendedforresearchingeffectiveness(P.H.Anderson&Lawton,1997).The literaturesearchrevealedonly15articlesexaminingtheuseofprojectmanagement PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 183 simulationgamesinacademicprograms;one,byMcCreery(2003),wasappropriateto buildonforthisstudy. Contribution to Knowledge Thisstudycontributestotheknowledgeonusingprojectmanagement simulationgamesinacademicprogramsbybuildingonMcCreery’s(2003)longitudinal approachofmeasuringincreasesinperceptionsofknowledgeandabilitytoapplythat knowledgeintwoways:(1)Byaddingasimilarlongitudinalstudyofattitudestoward teamworkandsimulationgamesbyadaptingpostsimulationsurveyitemsusedby McCreery,Buzzetto‐MoreandMitchell(2009)andAhn(2008),and(2)Byexploring whetherstudentsimprovetheirabilitytodevelopandinterpretschedulenetwork diagramsandinterpretearnedvalueperformancegraphsasaresultofparticipatingin theprojectsimulationgame.Analysisofpostsimulationnarrativequestionresponses helpsusunderstandwhatstudentsbelievetheylearnedfromtheexperience. ThisstudyconfirmedMcCreery’s(2003)findingthatplayingaproject managementsimulationgameimprovesstudentself‐perceptionsofprojectmanagement knowledgeandconfidenceintheabilitytoapplythatknowledgeandfurtherfoundthat studentattitudestowardsteamworkingeneralandtheirclassroomteaminparticular improvedsignificantlyasaresultoftheexperience.Categorizationofnarrativeresponse datasupportthesefindingsand,takentogether,bothsupporttheconclusionthatit appearsstudentsfoundtheprojectmanagementsimulationgametobeavaluable experienceandaneffectiveuseofclasstime. Althoughopinionsontheuseofsimulationgamesasalearningpedagogywere notstrengthenedasaresultofplayingtheprojectmanagementsimulationgame,they werefavorablebeforetheactivityandremainedso.Thislackofchangeisviewedasa controlandstrengthensthefindingofsignificancefortheincreasesinperceived knowledgeandconfidenceandimprovedattitudetowardsteamworkandtheirteam. Asurprisingfindingwasnochangeinstudentabilitytoconstructandanalyze schedulenetworkdiagramsandtointerpretgraphicalearnedvaluedataasaresultof participatinginthesimulationexperience.Whilethesewerenotexplicitlystated learninggoalsforthesimulationgameintheprogrambeingstudied,onewouldexpect thesetechniquestobeappliedduringthecourseofthesimulation,therebyrequiring recallandreinforcementofpriorlearning. PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 184 Implications ConsistentwiththefindingsofKeys(1977),theimplicationsofthislastfinding suggestthatratherthanfollowconventionalwisdomandrelyonthesimulation experiencetodotheteaching,instructorinterventionand/orenhancedguidelinesfor studentsmaybenecessaryifsimulationgamesareintendedtoreinforcespecific learningoutcomes,evenwhenthesimulationgameisbeingplayedasacapstoneactivity andpriorlearningisassumed. Fortheprogramunderstudy,criticalpathnetworkanalysisandearnedvalue managementtechniquescouldunobtrusivelybestrengthenedbyrequiringtheteamto answerspecificquestionsabouttheirscheduleandearnedvalueperformanceduringthe interimreviewswithmanagement,ratherthanrelyingonthemtodecidewhatis relevantandreportingonit.Areviewofcriticalpathmethodsandearnedvalue techniquescouldthenbeincludedduringaninterimreviewsessionifneeded. Thisstudyalsodemonstratesthatifacapstonesimulationgameexperienceisto beanythingotherthanafunactivitytowrapupacourseorprogramphase,itis importanttoclearlyidentifylearningobjectivesforitsuseandtodevelopandapply measurementsoftheintendedlearning.Inthecaseoftheprogramunderstudy,the statedlearningobjectivewastotietogethereverythinglearnedinthepriorcoursework andtodemonstratetheabilitytoworkasateamtoplanandexecuteasimulatedproject. Sincetheemphasisisontheteamexperience,basedontheresultsofthisstudy,this objectiveisbeingmet.Thereisanopportunitytoreinforcepriorlearningbyproviding moreguidanceonwhatshouldbereportedduringtheinterimreviewswith management;however,suchguidanceactscountertothenormthatteamsfigureoutfor themselveswhatshouldbereportedbasedontheirpriorexperiencesandcoursework. Itisuptotheeducatortodecideonanappropriateapproach. Anotherrecommendationistoconsidertheincorporationofamoreformalself‐ assessmentoflearningthatrequiresmorethoughtthanisnormallygivento postsimulationsurveyquestionsthatareusuallycompletedinahurryattheendofclass. Thiscouldbeintheformofarequiredjournalthatiskeptduringtheconductofthe simulationand/oraformalreportonwhatwaslearnedfromtheexperienceafterits conclusion. PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 185 Opportunities for further study Furtherstudyintowhatstudentsthinktheylearnfromprojectmanagement simulationgameswouldservetoidentifyopportunitiestoenhancethislearningandto identifyneedsthatarenotbeingmet. Littleisknownabouttheextenttowhichprojectmanagementsimulationsare usedinacademicprograms,howtheyaredeployedandhoweffectivenessismeasured. Astudyoftheuseofprojectmanagementsimulationgamesinotherprogramswould createadialoguewithintheprojectmanagementacademiccommunityandfoster improvementintheconductofprojectsimulationgamesbysharingtechniquesand lessonslearned.Suchastudycouldinclude:  Nameandsupplierofthesimulationgame  Howdeployed(e.g.,distributedthroughoutcourse,capstoneatendofa singlecourse,capstoneatendaseriesofcourses)  Projectmanagementprinciplesappliedduringthegame(e.g.,simProject requiredconsiderationofresourcecharacteristicsandteamdynamicsto estimateactivitydurations,biddingtoacquireresourcesandestablish theircost,afixedworkbreakdownstructureandactivitysequencewith noopportunityforfasttrackingorscopemodification,12fixed‐scope workperiodswithvariabledurationandcostbasedontheresources acquiredandtheassignmentstrategy,andreportstomanagementbased onmonitoringandcontrolproceduresestablishedbythestudentteams)  Natureandtimespentonanypregamesetupassignments  Playerteamsize  Totaltimespentplayingthesimulationgame  Numberofgamingsessions  Sessionlength  Venue:in‐classorvirtualbetweenclasssessions  Learningobjectives  Assessmentmethod(s)forstudentlearningandsatisfaction  Gradeweight(e.g.,25%ofcoursegrade)  Summaryofassessmentfindings  Instructorroleduringthegame PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE  186 Timingandnatureofgroupdiscussions(e.g.interimstatusreview,final statusreview,endingdebrief)  Lessonslearnedontheconductofprojectmanagementsimulation games Alternatively,thisstudycouldbeexpandedtoresearchtheuseofallformsof experientiallearninginprojectmanagementcoursework,e.g.studentinitiatedcharity projects,industryconsultingprojects,etc. Inconclusion,thisstudyprovidesevidencethatstudentsinanEMBAprogram emphasizingprojectmanagementvaluetheinclusionoftheSimProjectproject managementsimulationgameinthecurriculumandfinditaneffectivepedagogicaltool toincreasetheirprojectmanagementknowledgeandconfidenceintheirabilitytoapply thatknowledge,andtoimprovetheiralreadyfavorableattitudetowardsteamwork. SimilarityofresultstothepriorstudybyMcCreery(2003)suggestthatthesefindings maybegeneralizabletootherprojectmanagementsimulationgamesandothergraduate businessschoolprograms.Inadditiontoenjoyinghands‐onpracticewiththetoolsof projectmanagementinacompetitiveenvironment,studentslearnimportantlessons relatedtoprojectplanning,resourcemanagement,teamwork,performancemonitoring andcontrol. Educatorsconsideringtheadditionofaprojectmanagementsimulationgameto theircurriculumshouldbemindfuloftheimportanceofdebriefsessionsandconsider requiringformalself‐assessmentsoflearningthataremoresubstantialthanshort answersurveyquestions. PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE Appendix A Literature Search Journal Sources  AcademyofManagementJournal  AcademyofManagementLearning&Education  AcademyofManagementReview  AdministrativeScienceQuarterly  Agriculture&AppliedEconomicsAssociation  AmericanPsychologist  BritishJournalofEducationalPsychology  CommunicationEducation  Computers&Education  ComputerSimulationandLearningTheory  DevelopmentsinBusinessSimulationandExperientialExercises  DevelopmentsinBusinessSimulationandExperientialLearning  Economist  Education+Training  EducauseReview  Engineering,ConstructionandArchitecturalManagement  EuropeanManagementJournal  FinancialPracticeandEducation  HumanResourceDevelopmentQuarterly  IEEETransactionsonEducation  IEEETransactionsonEngineeringManagement  InformationandSoftwareTechnology  InnovationsinEducationandTeachingInternational  InterdisciplinaryJournalofE‐LearningandLearningObjects  InternationalJournalofEngineeringEducation  InternationalJournalofProjectManagement  JournalofBusiness  JournalofBusinessResearch  JournalofBusinessEducation  JournalofCollegeTeaching&Learning 187 PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE  JournalofContingenciesandCrisisManagement  JournalofEconomicEducation  JournalofEducationforBusiness  JournalofEngineeringEducation  JournalofEuropeanIndustrialTraining  JournalofExperientialLearningandSimulation  JournalofInformationSystemsEducation  JournalofInformationTechnology  JournalofInternationalBusinessStudies  JournalofManagement  JournalofManagementDevelopment  JournalofManagementEducation  JournalofMarketingEducation  JournalofTeachinginInternationalBusiness  JournaloftheConstructionDivision  JournalofResearchonTechnologyinEducation  ManagementLearning  MMAFallEducators’Conference–2007  OntheHorizon  PersonnelReview  PerspectivesonAcademicGamingandSimulation  PhysicsEducationResearchConference–2006  PMIGlobalCongressProceedings  PMIToday  ReviewofBusinessResearch  ReviewofEducationalResearch  SchoolLeadershipandManagement  SimulationandGaming  SimulationModelingPracticeandTheory  SoftwareEngineering&KnowledgeEngineering,ProceedingsoftheAnnual InternationalConferenceon 188  SystemsEngineeringandModeling,InternationalConferenceon  TeachingSociology PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE  TheoryintoPractice  TrainingandDevelopmentJournal  TrainingMediaReview 189 PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 190 Appendix B Presimulation Team Assignment UTDPMPlastics–ProjectPlanningAssignment BackgroundInformation UTDPM Plastics is a 10‐year old plastics manufacturing company located in North Central Texas. They specialize in developing parts for industrial use using injection molding and extrusion technologies. Their specific specialty lies in the area of developing made‐to‐order parts for the automotive after‐market, although their product catalog includes products used within many industries, both marketed directly to consumers and those sold to manufacturers and retailers. UTDPM Plastics is a privately owned company with next fiscal year projected revenues of $30 million. In recent years, the company has begun to broaden its capabilities by developing in‐ house design and engineering expertise. This approach has allowed the company to expand its business opportunities by developing products for other firms that lack specific knowledge of plastics engineering and/or manufacturing. The impetus to expand in‐house engineering capabilities at UTDPM has been identified by upper management as a necessary means for continuing to enhance business opportunities and generating revenue in this highly competitive marketplace. The goal of UTDPM is to continue to develop in‐house engineering and new product development to a level that will provide a sustainable competitive advantage for the firm over competitors in both the local and national markets. This includes the objective to re‐ engineer existing products and develop new products in a cost‐effective manner while meeting customer needs. UTDPM also seeks to use exclusivity agreements and patents to protect its revenue generation for these products. And, recognizing the value of project management procedures, UTDPM committed to improving their new product development execution through superior project management three years ago. You have been hired to replace the experienced project leadership team that led the move to superior project management. Unfortunately for UTDPM, they decided to cash in on their experience and explore opportunities at a competing firm. Your new management is concerned that this competing firm may capture a large share of UTDPM’s target market if PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 191 the product launch they were planning before their departure is delayed or over‐budget. Here’s what you know about the project they were planning: ProjectObjectiveStatement Demonstrate UTDPM’s new product development prowess by capitalizing on a new commercial market opportunity with product launch 260 work‐days or less after initiation at a cost not to exceed $380,000. Milestones The project is divided into 12 work periods, each ending with an associated milestone. Some milestones describe multiple rather than unique events, but management has instructed you to stay with these milestones as they represent phase boundaries requiring an approval to proceed. This means the start of non‐critical path sequential tasks may be delayed or become critical because of a phase boundary. WBSandWBSDictionary The departed team identified 58 project tasks and grouped them into 9 functional work packages including a project management work package containing 12 tasks, one for each work period: 1. Market Assessment 2. Procurement 3. Supplier Quality 4. Design 5. Engineering 6. Engineering Quality 7. Manufacturing 8. Commercialization 9. Project Management This structure is similar to past projects and management has approved this structure as part of the scope baseline for this project. You will not be able to add, delete or edit the description of these tasks. Your preference is to refer to these “tasks” as “activities” to be PMBOK® Guide compliant, but you realize that “task” is part of the corporate culture and management has signaled that you shouldn’t diddle with culture until you’ve proven yourselves. Since some milestones are related to the completion of several tasks, you decide to group all the milestones together in the WBS under a tenth summary task called PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 192 “Milestones” (you hesitate to call this a work package because no work is performed here). You note that each milestone will have at least two predecessors: a project management task and a task that is probably on the critical path. OtherTaskInformation The departed project management team also estimated durations for each task based on historical information and expert input. You have been instructed to go with these estimates as you don’t have a better source of information nor time to go looking. You note that the duration estimating assumption was effort‐driven based on one resource per task working full time. For a worker of normal competence, eight hours of planned work effort will be completed in one day. The prior team also mapped each task into one of the twelve periods (you’d rather call them “phases” but keep this thought to yourself after the feedback on “tasks” vs. “activities”). Since the milestones are being treated as phase boundaries you also make a note to make sure you there is a predecessor link to the prior milestone if needed to keep the task starts in the proper period. This seems to be as far as the prior project management team progressed as you’ve been unable to find any other work product related to developing a project schedule. Fortunately you bumped into the former leader of the departed project management planning team at a local PMI® chapter meeting and, not wanting to burn any bridges, she sends you her notes to help you complete your planning effort. You look these over and decide that, combined with the WBS information you received from your new boss, you have enough information to prepare a baseline schedule and budget. Notesfromtheformerprojectmanager  With the following exceptions, tasks within each work package are performed in sequence o “Identify vendors” and “Develop and Issue RFQ” can start in parallel o “Train sales team” and “Advertising campaign” and “Show functional model at trade show” can start in parallel (if there are no other dependencies) o “Assess RFQ responses and select vendors” starts after ”Qualify supplier” completes and Milestone 10 PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE  193 The project begins with “Market Assessment.” Its completion is designated as Milestone 1 and is followed by the start of “Design” and “Commercialization” in Period 2  “Engineering” follows “Design”  The other work package starts are a bit more complicated: o “Procurement” and “Engineering Quality” can start after the completion of “Release pre‐production specifications” and Milestone 5 o “Supplier Quality” can begin after the completion of “Issue sample (production equivalent)” o “Manufacturing” can start in Period 8 after the completion of “Perform supplier process capability” and “Build functional model”  “Issue sample (production equivalent)” requires the completion of both “Identify vendors” and “Develop and Issue RFQ.” Its work needs to be performed during Period 7.   “Perform supplier process capability” is a predecessor of o “Approve sample parts” o “Design validation activities” o “Test prototype” o “Process engineering plan” o “Show functional model at trade show” o “Milestone 7” Starting “Identify testing requirements” also needs the completion of “Develop marketing program”   “Release pre‐production specifications” is a predecessor of o “Identify vendors” o “Develop and issue RFQ” o “Issue sample” o “Build functional model” o “Evaluate design specifications” o “Develop testing protocol for prototype” o “Milestone 5” “Build functional model” is a predecessor of PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE  o “Design validation activities” o “Test prototype” o “Process engineering plan” o “Show functional model at trade show” o Milestone 6 194 “Design Transfer activities” also needs the completion of “Evaluate results of tests and identify weaknesses”  “Product release meetings” also needs the completion of “Design transfer activities”  “Develop production plan” also needs the completion of “Validation design review” and “Evaluate results of tests and identify weaknesses”  “Develop production control plan” also needs the completion of “Qualify supplier”  “Contracting for deliveries” also needs the completion of “Assess RFQ responses and select vendors”  “Production pilot test” also needs the completion of “Product release meetings”  The following depend on the completion of “Develop marketing program”  o “Identify testing requirements” o “Train sales team” o “Advertising campaign” o “Show functional model at trade show” o “Milestone 2” “Product launch” requires the completion of o “Production release” o “Train sales team” o “Advertising campaign” o “Show functional model at trade show” With this information, the table of milestones, and the estimated durations, you are confident you can quickly validate the desired schedule. You recall the need to make sure all tasks are scheduled in the proper time period. This may require adding a milestone as a predecessor to some tasks. PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 195 ResourcePool Management provided a list of available resources and advised these are procured though a bidding process with other project teams just before initiation of the execution phase. You have some concern about this but realize you need to get over it as this is life in the fast lane. Since each person you add to the team is charged against your project budget, your goal should be to fully utilize any resource you hire. There is no overtime. Each resource’s personal characteristics and the nature of the task determine whether they can complete the task with the estimated amount of effort. Training and managerial actions are available to influence their characteristics. The cultural background of resources for project teams impacts on cohesion and team performance. A diverse group is more effective than one with minimal diversity; however, a group that is too diverse may be dysfunctional. No more than two resources may be assigned to any task. Hired resources become available for assignment in the succeeding time period. For example, if you successfully bid on a resource prior to executing Period 1, they will be available for assignment to tasks in Period 2. There will be a Period 0 with multiple rounds for hiring the initial team. A released resource is immediately removed from the team and not available for assignment in that period and not available for rehire until two periods later (if not hired by another project team in the interim). Training Management also provided information on available training in case your planned resources need development. This is a good thing as some of the resources look like they may need some development and, due to the bidding process, there is no guarantee you can hire the best resources available. ManagerialActions A list of managerial actions available during execution is also provided. You remember from your MBA courses that doing some of these things may be a good idea when leading people. These may be applied in an attempt to motivate, punish, or develop the team. PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 196 YourAssignment Using the given information: 1. Prepare a project schedule using the critical path method and determine the size of the project completion buffer in work‐days. 2. Prepare a baseline budget using this CPM schedule and the estimates provided by the departed project management team. Calculate the size of the management reserve/projected overrun. 3. Prepare a project staffing plan based on your analysis of the resource requirements and the available resources. Since you are competing for resources, a backup plan may be in order! 4. Prepare a schedule and cash flow forecast assuming you obtain all your desired resources at the planned bid rates. Be sure to adjust work/durations as appropriate. 5. Analyze variances versus the baseline schedule and budget. Identify any needed gap‐closing actions. Put them in your plan and redo 4 and 5. Finishing early and under‐budget is a good thing, late and/or over‐budget is not. 6. Prepare a summary presentation suitable for reviewing the baseline and your forecast with management. Be prepared to explain all variances as the prior project management team had an outstanding reputation with management and your team is relatively unproven. Deliverables Post three files on Blackboard: 1. MS Project file a. Set Baseline with CPM schedule and baseline budget from (1) and (2) above b. Active plan reflects resource assignments and associated changes from (3) – (5) 2. Staffing Plan (MS Office or PDF document) showing hiring, training, release, etc. plan by period. Initial staffing for Time Period 1 occurs during Time Period 0. List plans for Time Periods 0 – 11. 3. MS PowerPoint presentation from (6) above. Do not show the people you plan to hire or discuss your bidding strategy as the other teams will be bidding against you during Time Period 0. PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 197 Period‐endingMilestones(PhaseBoundaries) All are also preceded by the project management activity for the prior period. All but Milestone 12 are followed by the project management activity for the next period. No work on succeeding tasks may commence until all work preceding the milestone is completed and approved. You may assume approval is automatic and consumes no time or budget. Preceding Tasks Business Evaluation Design specs. Develop marketing program Milestone 1 Succeeding Tasks Design and Development Plan Develop Preliminary Marketing Plan Identify testing requirements 2 Train sales team Advertising campaign Risk Analysis Train sales team 3 Design labeling 4 Design verification activities Advertising campaign Initial engineering specs. Identify vendors Release pre‐production specifications 5 Develop and issue RFQ Build functional model Evaluate design specifications Identify vendors Develop and issue RFQ Build functional model Issue sample (production equivalent) 6 prototype Evaluate design specifications Approve sample parts Perform supplier process capability Develop testing protocol for Design validation activities 7 Show functional model at trade show Approve sample parts Test prototype Test prototype Process engineering plan prototype Design validation activities Develop testing protocol for Validation design review 8 Evaluate results of tests and identify weaknesses PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 198 Process engineering plan Show functional model at trade show Approve model design Evaluate results of tests and Qualify supplier 9 identify weaknesses Develop production plan Qualify supplier Product release meetings Design transfer activities Assess RFQ responses and select 10 Develop production plan vendors Develop production control plan Contracting for deliveries 11 Submit production purchase order Product launch 12 Celebrate!!! PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 199 WBSDictionary Work Package Task Name 1 Market Assessment 1 Evaluate market 1 1 1 2 2 2 2 2 3 3 3 3 Task Description Conduct full market research study to identify market segments, pricing, and final confirmation of product features Develop Business Identify key customers and gaining preliminary commitments or opportunity contracts in order to secure a baseline contract to justify continuation of the project Customer preference Interviewing and conducting focus groups and surveys to identify most study desirable product characteristics Business evaluation Project screening used to identify costs, including revenue streams and (NPV, etc.) net cash flows, for the viability assessment Procurement Identify vendors Create a viable vendor pool for all material and service requirements, including performance criteria such as delivery, material or service quality, and pricing Develop and issue RFQ Identify all purchased materials and service requirements and develop requests for quotation for each requirement. Issue RFQ Issue sample Issue purchase order for sample quantities to be used in first run (production equivalent) production plan Assess RFQ responses Evaluate all supplier responses to RFQ and notify those selected and select vendors Supplier Quality Perform supplier Assessment by quality control and procurement of suppliers’ capability process capability with respect to product characteristics, delivery, timeliness, and pricing Approve sample parts Quality control and manufacturing test and approve production equivalent sample orders for raw materials and parts Qualify Supplier Using results from sample parts assessment, formally notify suppliers, plant representatives and procurement of all suppliers qualified to bid Phase Est. (Work Work Period) (hours) Est. Labor Rate 1 96 $50 1 112 $90 1 168 $50 1 32 $125 6 56 $50 6 48 $50 7 40 $75 11 80 $50 7 112 $50 8 64 $75 10 80 $50 PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE Work Package 4 4 4 4 4 4 4 5 5 5 5 5 5 5 5 5 Task Name Task Description for contracts for materials and services Design Design and High level structural design of the product, including plans and schedules development plan for product completion Design specs. Detailed technical drawings and schematics for the product, including all equipment needs to create the final product Identify testing Detail critical product specifications, acceptable tolerances and product requirements liability limits Risk analysis Identify significant product usage risk and adherence to product standards. Include an assessment of acceptable levels of product tolerance. Design labeling Developing design labeling and packaging for the finished product Approve design Final assessment of product design characteristics matched to preliminary customer specifications Engineering Initial engr. specs. Converting product design specifications into engineering templates Design verification Validate the consistency of product functionality, product design and activities engineering plans Verification design Formal review with engineering, design, and marketing to finalize review product design Release pre‐production Formal approval and sign‐off on preliminary product specifications for specifications review and comment Build functional model Develop product prototype Design validation Develop protocol for verification of product design activities Validation design Perform desk check (structured walk through) of product design review Approve model design Evaluate results from design review and secure final approval from engineering, design and manufacturing 200 Phase Est. (Work Work Period) (hours) Est. Labor Rate 2 48 $50 2 176 $50 3 80 $50 3 4 80 40 $125 $50 4 4 32 40 $50 $50 5 56 $75 5 32 $50 5 6 80 144 $50 $75 8 40 $50 9 32 $125 9 32 $75 PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE Work Package Task Name 5 Design transfer activities 6 Engineering Quality 6 Evaluate design specifications 6 6 6 6 7 7 7 7 7 7 7 7 Develop testing protocol for prototype Test prototype Evaluate results of tests and identify weaknesses Product release meetings Manufacturing Process engineering plan Develop production plan Develop production control plan Approve production parts Contracting for deliveries Submit production purchase order Production pilot test Task Description Develop the process to support the transfer the product to manufacturing Conduct quality assessment, including quality engineering, on product designs. Create upper and lower control limits for product component manufacturing Identify specific testing protocol for each product specification, document for repeatability and benchmarking Perform tests to valid all product characteristics and identify significant deviations from upper and lower control limit boundaries Evaluate results of prototype tests from a product quality perspective, identify characteristics outside of control limits, and implement plan for correction Gain required sign‐off approval from representatives from engineering, manufacturing, design, and quality control Convert engineering and design specifications to an operations plan for plant work flow and design for manufacturing Identify the specific machine and manpower resources needed to produce the requirements for the product Develop the schedule for raw materials, shipping, and packaging against the sales forecast and requirements plan Assess and approve first‐run production of product components 201 Phase Est. (Work Work Period) (hours) 10 56 Est. Labor Rate $75 6 80 $50 7 64 $50 8 80 $50 9 48 $50 10 24 $125 8 120 $50 10 48 $50 11 68 $50 11 40 $50 Specify exact terms for schedules and quantities of manufacturing supplies, including quantity release schedule Issue detailed production requirements for production pilot test 11 64 $50 12 16 $50 Test production run within normal plant operations, staffing, and 12 40 $50 PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE Work Package 7 7 8 8 Debugging production system Production release 8 Commercialization Develop preliminary marketing plan Develop marketing program Train sales team 8 Advertising campaign 8 8 9 9 Show functional model at trade show Product launch Project Management Project Management 9 Project Management 9 Project Management 9 Project Management 9 Project Management 8 Task Name Task Description resource requirements for operational stability Identify and correct any significant deviations from process operations and product outcomes Issue formal sign‐off from manufacturing to accept product into the production system Develop timetable, responsibilities and costs for creating and implementing the marketing program Creating a plan to identify customers by segment, promotional programs, pricing structures, and distribution channels Specific product training for sales personnel with the purpose of having them knowledgeable regarding the product during conversations with potential customers Develop detailed advertising plan, including media schemes, scripts, and public relations activities (trade shows and trade journal promotion) Design display and coordinate delivery and presentation of prototype at selected trade shows Plan for and implement the formal announcement of the new product Overseeing all project tasks for the period, developing status reports and project updates, and maintaining project team staffing and commitment Overseeing all project tasks for the period, developing status reports and project updates, and maintaining project team staffing and commitment Overseeing all project tasks for the period, developing status reports and project updates, and maintaining project team staffing and commitment Overseeing all project tasks for the period, developing status reports and project updates, and maintaining project team staffing and commitment Overseeing all project tasks for the period, developing status reports and project updates, and maintaining project team staffing and commitment 202 Phase Est. (Work Work Period) (hours) Est. Labor Rate 12 32 $50 12 24 $50 2 40 $90 2 120 $90 3 176 $50 3 224 $50 8 24 $90 12 1 24 200 $125 $125 2 112 $125 3 112 $125 4 104 $125 5 120 $125 PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE Work Package Task Name 9 Project Management 9 Project Management 9 Project Management 9 Project Management 9 Project Management 9 Project Management 9 Project Management Phase Est. (Work Work Task Description Period) (hours) Overseeing all project tasks for the period, developing status reports and 6 72 project updates, and maintaining project team staffing and commitment Overseeing all project tasks for the period, developing status reports and 7 88 project updates, and maintaining project team staffing and commitment Overseeing all project tasks for the period, developing status reports and 8 24 project updates, and maintaining project team staffing and commitment Overseeing all project tasks for the period, developing status reports and 9 32 project updates, and maintaining project team staffing and commitment Overseeing all project tasks for the period, developing status reports and 10 40 project updates, and maintaining project team staffing and commitment Overseeing all project tasks for the period, developing status reports and 11 104 project updates, and maintaining project team staffing and commitment Overseeing all project tasks for the period, developing status reports and 12 80 project updates, and maintaining project team staffing and commitment Training and managerial action budget of $25,000 is allocated at $3,125 per period for periods 1‐8 203 Est. Labor Rate $125 $125 $125 $125 $125 $125 $125 PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 204 ResourceInformationandMetrics Standard Rate is the rate paid on their last project. Resources may reject your bid if it is below what they think they should be paid. Training is the amount of previous instruction in related skills. You may send resources for additional training if you think it would enhance their ability to perform work effectively. Skill is degree of expertise. Skill levels can be enhanced through additional training. Experience relates to the length of time the individual has worked in their field. More experienced people tend to be more efficient and (at least initially) adapt faster to working on project teams. Education relates to the level and relevance of education completed. Work Ethic is the set of principles that individuals have about performing the job. A stronger work ethic means that the project team member is disposed to work more diligently. Reputation is the general belief about an individual’s character. It may also be described as the state of being well thought of. The better the individual’s reputation, the easier it is to hire other team members, retain top management support, and keep stakeholders happy. Public Relations skills include employee communications, media relations, advertising, and community relations. They are the ability of a person to present an appropriate “face” to external stakeholders. Flexibility is a measure of the adaptability of a person to a change in circumstance and the ability to handle changes. Interpersonal Skills are goal‐directed behaviors conducted in a face‐to‐face environment. They enable a person to relate to and interact with others. PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 205 Education Reputation Work Ethic Public Relations Flexibility Interpersonal Skills Cultural Background American American American American Spanish Chinese German Spanish Spanish American American Japanese American American Japanese German German American Chinese American Chinese German American American American Spanish Spanish German Experience Gender Male Male Male Female Male Female Male Male Male Male Male Male Male Female Female Male Male Male Male Female Female Female Male Female Male Male Female Male Skill Level Engineer Engineer Engineer Engineer Engineer Junior Marketing Specialist Junior Marketing Specialist Junior Marketing Specialist Junior Marketing Specialist Junior Marketing Specialist Junior Marketing Specialist Junior Marketing Specialist Junior Marketing Specialist Junior Marketing Specialist Junior Marketing Specialist Junior Product Designer Junior Product Designer Junior Product Designer Junior Product Designer Junior Product Designer Junior Product Designer Junior Product Designer Junior Product Designer Junior Product Designer Junior Product Designer Marketing Manager Marketing Manager Marketing Manager Age 24 26 27 28 28 21 26 24 29 24 25 27 28 25 26 22 22 22 23 22 22 23 28 21 27 36 34 24 Training Category $57 $86 $46 $69 $48 $72 $50 $75 $51 $76 $53 $80 $58 $87 $47 $70 $48 $72 $45 $68 $56 $84 $58 $87 $47 $70 $47 $70 $56 $84 $55 $82 $55 $82 $41 $62 $51 $76 $56 $84 $49 $74 $51 $76 $59 $88 $54 $81 $58 $87 $81 $122 $76 $114 $102 $153 78 59 75 83 79 43 50 53 51 58 48 41 46 40 52 46 40 44 34 38 39 41 45 36 33 75 81 70 96 82 80 89 85 78 80 64 71 62 77 56 64 65 82 69 69 49 60 67 58 61 50 64 48 75 70 100 59 59 56 50 60 58 56 46 46 52 54 49 52 54 55 42 36 42 39 33 44 45 35 47 39 81 96 67 62 61 51 66 54 50 49 45 45 65 47 63 53 46 61 52 53 55 58 56 56 58 44 54 43 78 73 74 71 57 68 71 71 39 43 33 41 37 41 37 39 39 42 43 51 43 51 51 57 59 41 49 47 90 97 89 63 86 71 62 75 53 51 62 72 54 49 59 55 56 50 55 56 55 66 49 65 67 61 71 59 85 79 82 70 58 67 72 70 68 54 66 54 49 52 57 67 51 64 41 47 33 46 40 35 36 45 38 38 83 61 57 40 32 33 36 35 66 52 72 60 63 50 63 56 66 54 33 34 43 45 39 39 41 43 38 42 81 67 94 58 65 55 61 59 66 58 61 52 68 55 64 70 70 53 49 43 52 41 40 47 49 48 58 59 64 70 88 Standard Rate Resource Name Brandon Plude Christian Cepeda Darren Mohammad Ja Hauser Jose Fernandez An Wu Bing Heilbronner Ceandro Apellániz Felipe Garcia Fernando Bleakley Guy Geer Hekirou Ohitsuji Kurt Darcangelo Natasha Cremin Tani Ashida Armand Oswald Bud Zimmermann Gregory Treglia Kyung‐Yul Zheng Madelene Emel My‐Trang Ding Nadine Bathurst Sean Stanger Virgen Couch Wayne Brousseau Alejandro Córdoba Ana Garcia Baron Heilbronner Overtime Rate 85 72 86 67 63 64 79 88 89 93 79 94 51 45 52 44 52 51 57 61 67 65 Reputation 71 70 61 50 70 59 57 100 99 90 83 100 17 17 16 21 23 69 55 65 58 60 Education 75 63 60 64 60 74 55 86 79 82 68 83 34 29 34 29 24 59 46 48 57 51 Experience $114 90 76 96 82 77 91 $118 84 73 67 76 100 82 $84 72 70 74 41 66 76 $74 75 76 56 48 65 85 $69 71 72 81 45 62 81 $81 81 91 82 59 73 73 $81 64 86 75 49 56 86 $214 84 100 99 60 66 80 $207 76 98 73 50 70 98 $166 79 79 77 62 98 81 $222 69 75 89 62 90 83 $218 80 100 78 54 75 70 $130 90 74 86 63 93 95 $122 80 95 68 58 100 90 $108 92 85 89 51 76 90 $90 78 100 76 65 98 76 $98 100 82 80 58 81 93 $129 79 91 65 71 69 80 $110 79 91 71 61 55 65 $122 85 85 77 49 54 82 $104 70 78 76 70 64 73 $111 67 78 55 61 58 68 Skill Level Interpersonal Skills $76 $79 $56 $49 $46 $54 $54 $143 $138 $111 $148 $145 $87 $81 $72 $60 $65 $86 $73 $81 $69 $74 Training Cultural Background American American German Japanese Spanish German Chinese American American American American American German Japanese American American American German American American American American Flexibility Gender Male Female Female Male Female Male Female Female Male Male Male Male Female Male Male Female Male Male Female Male Male Male Public Relations Age 31 35 33 31 30 32 31 37 40 47 39 42 26 33 33 28 36 29 23 27 28 25 Work Ethic Category Marketing Manager Marketing Manager Operations Specialist Operations Specialist Operations Specialist Operations Specialist Operations Specialist Project Manager Project Manager Project Manager Project Manager Project Manager Quality Engineer Quality Engineer Quality Engineer Quality Engineer Quality Engineer Senior Product Designer Senior Product Designer Senior Product Designer Senior Product Designer Senior Product Designer Overtime Rate Resource Name Darren Babbitt Melodee Pledger Carrie Frankfurter Chigemi Washi Coronie Alonso Emery Zimmermann Genji Nguyen Jane Barto Kurt Nordine Leonard Rockholt Tyrone Leigh Tyrone Rutten Adelle Hitzig Chigemi Yagi Jessie Taffe Monique Korbar Stephen Asuncion Alder Heilbronner Danielle Stupp Guy Slain Mike Deblanc Stephen Adams 206 Standard Rate PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 207 Training Cost per Training Name Benchmarking Financial Skills Refresher Interpersonal Skills Introduction to Planning Market Research Tools Negotiation Techniques Principles of Quality Process Engineering Project Evaluation Project Management 101 Description Students will identify project management processes and develop strategies for obtaining information on best practices and implementing these strategies in their organizations. Use of NPV and IRR for project selection and portfolio management Successful completion of this course will enable students to be conversant in project selection criteria and to prepare cost‐benefit analysis. Students will learn effective leadership techniques, group behavior and decision making. They will also practice persuasive communications, conflict resolution, and influence tactics. Elements in effective scope management, scheduling, resource management, and risk assessment. An introduction into focus group administration, business‐to‐business survey methods, and data collection and analysis. Basic skills and practice of effective negotiation. Introduction of Deming's Principles of Total Quality Management, statistical process control, and Taguchi quality cycles. Intermediate level course to learn best practices of organizational process design and improvement. Kaizen principles will be discussed. Intermediate level course in project tracking and control techniques. Basic project management, covering a survey of the major relevant skills and knowledge to manage or perform effectively on projects. resource Maximum Duration Seats $800 4 days 3 $600 3 days 5 $600 3 days 7 $1,000 5 days 4 $400 2 days 2 $600 3 days 5 $600 3 days 5 $600 3 days 2 $1,000 5 days 5 $1,000 5 days 5 PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 208 ManagerialActions Managerial Action Name Company Sponsored Family Event Disciplinary Action Management Recognition Award Milestone Celebration Monetary Bonus One‐on‐one Chat Pizza Party Verbal Warning Description Company hosts a picnic for team members and their families. Project Manager applies formal disciplinary sanctions for employee's behavior. Notation is placed in personnel file. Top management sends letters to all team members on project accomplishments. Team celebrates completion of current milestone. Project Manager announces nominal monetary award for excellent performance. Project Manager calls in team member for informal discussion and corrective suggestions. Project Manager throws a pizza party for the entire team. Project Manager applies informal disciplinary action without notation in file but warning if repeat performance occurs. Cost per resource $75 $0 $50 $1,000 $2,500 $0 $10 $0 PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 209 Appendix C Presimulation Survey Thank you for participating in this survey! To help assure your responses are anonymous; please enter a unique identifier that is not traceable to you as an individual: ________________________ Which Team are you on? ________ Team A ________ Team B ________ Team C ________ Team D Enter this information in response to questions 12 and 13 and retain this sheet as you will need to enter this same information on the post‐simulation survey. PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 210 1. Assess your current level of knowledge in each of the following areas: 1 = Extremely Low ‐‐‐ 7 = Extremely High A. Project organizational design 1 2 3 4 5 6 7 B. Project planning in general 1 2 3 4 5 6 7 C. Assessing and managing project risks 1 2 3 4 5 6 7 D. Estimating project scope 1 2 3 4 5 6 7 E. Sequencing of project activities 1 2 3 4 5 6 7 F. Estimating project activity times 1 2 3 4 5 6 7 G. Project budgeting 1 2 3 4 5 6 7 H. Allocating project resources 1 2 3 4 5 6 7 I. Project change management 1 2 3 4 5 6 7 J. Designing project performance measures 1 2 3 4 5 6 7 K. Performing as an effective project leader 1 2 3 4 5 6 7 L. Building team consensus 1 2 3 4 5 6 7 M. Negotiating for resources and budgets 1 2 3 4 5 6 7 N. Using Earned Value concepts 1 2 3 4 5 6 7 O. Evaluating the performance of project personnel 1 2 3 4 5 6 7 P. Managing project uncertainty 1 2 3 4 5 6 7 PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 211 2. Assess how confident you are in your ability to effectively apply your knowledge in each of the following areas: 1 = Extremely Low Level of Confidence ‐‐‐ 7 = Extremely High Level of Confidence A. Project organizational design 1 2 3 4 5 6 7 B. Project planning in general 1 2 3 4 5 6 7 C. Assessing and managing project risks 1 2 3 4 5 6 7 D. Estimating project scope 1 2 3 4 5 6 7 E. Sequencing of project activities 1 2 3 4 5 6 7 F. Estimating project activity times 1 2 3 4 5 6 7 G. Project budgeting 1 2 3 4 5 6 7 H. Allocating project resources 1 2 3 4 5 6 7 I. Project change management 1 2 3 4 5 6 7 J. Designing project performance measures 1 2 3 4 5 6 7 K. Performing as an effective project leader 1 2 3 4 5 6 7 L. Building team consensus 1 2 3 4 5 6 7 M. Negotiating for resources and budgets 1 2 3 4 5 6 7 N. Using Earned Value concepts 1 2 3 4 5 6 7 O. Evaluating the performance of project personnel 1 2 3 4 5 6 7 P. Managing project uncertainty 1 2 3 4 5 6 7 PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 212 3. Assess your team experience on exercises and activities in this program prior to today 1 = Strongly Disagree ‐‐‐ 7 = Strongly Agree A. The workload was balanced across all team members 1 2 3 4 5 6 7 B. Team members cooperated well throughout the program 1 2 3 4 5 6 7 C. Our team worked in an efficient manner 1 2 3 4 5 6 7 D. Team members all participated equally in the team 1 2 3 4 5 6 7 decision making process E. Our team maintained a pleasant working atmosphere 1 2 3 4 5 6 7 F. Our team worked out disagreements in an equitable manner 1 2 3 4 5 6 7 G. Team members were highly motivated to perform well 1 2 3 4 5 6 7 in the team exercises and activities H. Overall, I am satisfied with my team experience 1 2 3 4 5 6 7 I. I would be willing to work with my team on an actual project in the future 1 2 3 4 5 6 7 PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 213 4. Assess your view of using simulations as a learning tool 1 = Strongly Disagree ‐‐‐ 7 = Strongly Agree A. Simulations allow students to see how course concepts are applied in real world practice B. Simulations provide valuable real‐world experience C. Group projects help prepare students to be able to work in professional groups in the future D. This simulation will be a valuable learning experience E. Computer simulations help one better understand the decision making process that occurs in professional practice F. This simulation will help me understand key course concepts 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 G. Simulations can make class more fun 1 2 3 4 5 6 7 H. I think I will be pleased with my performance on the simulation I. Simulations help students build professional skills 1 2 3 4 5 6 7 1 2 3 4 5 6 7 J. 1 2 3 4 5 6 7 I think I will enjoy the computer simulation K. I enjoy working in groups 1 2 3 4 5 6 7 L. I would like to see more simulations in future programs 1 2 3 4 5 6 7 M. I believe this simulation will help me feel more confident about my abilities to manage projects N. I would be willing to work with my team on an actual project in the future O. I believe this simulation will be educational 1 2 3 4 5 6 7 P. I believe this simulation will be fun and exciting Q. I believe this simulation will be difficult 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 214 5. For the following network, develop an activity‐on‐node project network. Complete the forward and backward pass, compute the activity slack, and identify the critical path. Activity Predecessor Time (weeks) A None 4 B A 5 C A 4 D B 3 E C,D 6 F D 2 G E,F 5 A. What activities are in the critical path? B. How many weeks are in the critical path? C. What is the slack time for activity C? D. What is the slack time for activity F? PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 215 6. Respond to the questions below based on the following graphical project information. A. How is this project performing (select the best answer)? 1. Ahead of schedule and under budget 2. Ahead of schedule and over budget 3. Behind schedule and under budget 4. Behind schedule and over budget 5. I have no idea As of “Today” in the chart” B. What is the value in dollars of the schedule variance? __________ C. What is the value in dollars of the cost variance? __________ D. What is the value of the schedule performance index? __________ E. What is the value of the cost performance index? __________ F. What is the value of the to‐complete performance index? __________ G. What will the value of the SPI be at the end of the project? __________ PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 216 7. How many years of on‐the job experience have you had as a project manager/director, project coordinator, project team member, and other project‐related participant? ______ years 8. How many years have you worked as a professional (this is typically the time since you graduated from college with a bachelor’s degree)? ______ years 9. What is your gender? Male Female 10. What formal project management training have you had prior to this course? Include an estimate of the number of contact or classroom hours. 11. How would you describe your background? Technical Non‐technical 12. What team are you a member of? 13. Print a unique identifier that will facilitate correlation with a post‐simulation questionnaire. PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 217 Appendix D Postsimulation Survey 1. Assess your current level of knowledge in each of the following areas: 1 = Extremely Low ‐‐‐ 7 = Extremely High A. Project organizational design 1 2 3 4 5 6 7 B. Project planning in general 1 2 3 4 5 6 7 C. Assessing and managing project risks 1 2 3 4 5 6 7 D. Estimating project scope 1 2 3 4 5 6 7 E. Sequencing of project activities 1 2 3 4 5 6 7 F. Estimating project activity times 1 2 3 4 5 6 7 G. Project budgeting 1 2 3 4 5 6 7 H. Allocating project resources 1 2 3 4 5 6 7 I. Project change management 1 2 3 4 5 6 7 J. Designing project performance measures 1 2 3 4 5 6 7 K. Performing as an effective project leader 1 2 3 4 5 6 7 L. Building team consensus 1 2 3 4 5 6 7 M. Negotiating for resources and budgets 1 2 3 4 5 6 7 N. Using Earned Value concepts 1 2 3 4 5 6 7 O. Evaluating the performance of project personnel 1 2 3 4 5 6 7 P. Managing project uncertainty 1 2 3 4 5 6 7 PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 218 2. Assess how confident you are in your ability to effectively apply your knowledge in each of the following areas: 1 = Extremely Low Level of Confidence ‐‐‐ 7 = Extremely High Level of Confidence A. Project organizational design 1 2 3 4 5 6 7 B. Project planning in general 1 2 3 4 5 6 7 C. Assessing and managing project risks 1 2 3 4 5 6 7 D. Estimating project scope 1 2 3 4 5 6 7 E. Sequencing of project activities 1 2 3 4 5 6 7 F. Estimating project activity times 1 2 3 4 5 6 7 G. Project budgeting 1 2 3 4 5 6 7 H. Allocating project resources 1 2 3 4 5 6 7 I. Project change management 1 2 3 4 5 6 7 J. Designing project performance measures 1 2 3 4 5 6 7 K. Performing as an effective project leader 1 2 3 4 5 6 7 L. Building team consensus 1 2 3 4 5 6 7 M. Negotiating for resources and budgets 1 2 3 4 5 6 7 N. Using Earned Value concepts 1 2 3 4 5 6 7 O. Evaluating the performance of project personnel 1 2 3 4 5 6 7 P. Managing project uncertainty 1 2 3 4 5 6 7 PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 219 3. Assess your team experience throughout the simulation exercise 1 = Strongly Disagree ‐‐‐ 7 = Strongly Agree A. The workload was fairly balanced across all team members 1 2 3 4 5 6 7 B. Team members cooperated well throughout the exercise 1 2 3 4 5 6 7 C. Our team worked in an efficient manner 1 2 3 4 5 6 7 D. Team members all participated equally in the team 1 2 3 4 5 6 7 decision making process E. Our team maintained a pleasant working atmosphere 1 2 3 4 5 6 7 F. Our team worked out disagreements in an equitable manner 1 2 3 4 5 6 7 G. Team members were highly motivated to perform well in the exercise H. Overall, I am satisfied with my team experience 1 2 3 4 5 6 7 1 2 3 4 5 6 7 I. 1 2 3 4 5 6 7 I would be willing to work with my team on an actual project in the future PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 220 4. Assess your view of using simulations as a learning tool 1 = Strongly Disagree ‐‐‐ 7 = Strongly Agree A. Simulations allow students to see how course concepts are applied in real world practice B. Simulations provide valuable real‐world experience C. Group projects help prepare students to be able to work in professional groups in the future D. This simulation was a valuable learning experience E. Computer simulations help one better understand the decision making process that occurs in professional practice F. This simulation helped me understand key course concepts 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 G. Simulations can make class more fun 1 2 3 4 5 6 7 H. I am pleased with my performance on the simulation 1 2 3 4 5 6 7 I. Simulations help students build professional skills 1 2 3 4 5 6 7 J. I enjoyed the computer simulation 1 2 3 4 5 6 7 K. I enjoy working in groups 1 2 3 4 5 6 7 L. I would like to see more simulations in future programs 1 2 3 4 5 6 7 M. As a result of the simulation, I now feel more confident about my abilities to manage projects N. I would be willing to work with my team on an actual project in the future O. This simulation was educational 1 2 3 4 5 6 7 P. This simulation was fun and exciting Q. This simulation was difficult 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 221 5. For the following network, develop an activity‐on‐node project network. Complete the forward and backward pass, compute the activity slack, and identify the critical path. Activity Predecessor Time (weeks) A None 4 B A 5 C A 4 D B 3 E C,D 6 F D 2 G E,F 5 A. What activities are in the critical path? B. How many weeks are in the critical path? C. What is the slack time for activity C? D. What is the slack time for activity F? PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 222 6. Respond to the questions below based on the following graphical project information. A. How is this project performing (select the best answer)? 1. Ahead of schedule and under budget 2. Ahead of schedule and over budget 3. Behind schedule and under budget 4. Behind schedule and over budget 5. I have no idea As of “Today” in the chart” B. What is the value in dollars of the schedule variance? __________ C. What is the value in dollars of the cost variance? __________ D. What is the value of the schedule performance index? __________ E. What is the value of the cost performance index? __________ F. What is the value of the to‐complete performance index? __________ G. What will the value of the SPI be at the end of the project? __________ PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 223 7. What did you like about the just‐completed simulation experience? 8. Excluding changes to the computer simulation itself, what changes do you think could be made to the overall simulation experience to better prepare you to manage projects? PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 224 9. What changes do you recommend be made to the computer simulation itself? 10. Elaborate on any other thoughts you have regarding learning project management with a computer‐based simulation game. PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 11 What best describes the industry you work in? General Construction Oil, Gas, Petrochemical or Natural Resources Telecommunications IT/IS Pharmaceuticals Management Services Banking Consulting Other: ___________________________ 12. What best describes your JOB TITLE? Project Manager Functional Manager Cost Estimator/Scheduler Finance/Accounting Sales and Marketing Training, Mentoring or Consulting Human Resources Engineering/Technical Support Procurement/Purchasing/Expediting Other: ____________________________ 13. What is your highest level of formal education? High School Junior College Bachelors Degree Masters Degree PhD 225 PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 226 14. What was your PMP practice exam score? Less than 34.5 34.5 – 38.4 38.5 – 41.9 42.0 – 45.9 Greater than 46 15. What best describes the part of the world you originate from? North America Central/South America European Union Russia/Eastern Europe Middle East Africa Asia Pacific 16. What is your total gross income (in U.S. dollars) over the past twelve months from you primary source of employment (e.g., salary plus cash bonuses before taxes and other withholding)? Less than $20,000 $20,001 ‐ $40,000 $40,001 ‐ $60,000 $60,001 ‐ $80,000 $80,001 ‐ $100,000 $100,001 ‐ $120,000 $120,001 ‐ $140,000 More than $140,000 17. What Team were you on? 18. What is the unique identifier you assigned to the pre‐simulation survey? PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE Appendix E Project Simulation Final Presentation ProjectSimulation–FinalPresentation FinalStatusReport     Final Metrics: Time/Cost/Functionality/Stakeholder/Overall Baseline Completion Date vs. Actual Completion Date Baseline Budget vs. Actual Cost Variance Explanations ProjectAuditReport        Original strategy – what was your original approach? How did you divide responsibilities on your team? What worked? What didn’t work? What would you do differently the next time? Some things to consider o Were the right people and talents hired? o Did you establish appropriate planning and control systems? o Did the project conform to plan? Why or why not? What lessons did you learn that you would pass on to future project teams? TheSimulation  What did you learn about project management from participating in the SimProject simulation? 227 PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 228 Appendix F SimProject – About the Simulation SimProject, an Engaging Experience Inmakingthecaseforeducationalgames,Quinn(2005)suggeststhefocus shouldbeondesigninglearnerenvironmentsandexperiencesratherthanjustcontent, emphasizingyouhaveto: [Place]thelearnersintoacontextwheretheyhavetomakedecisions, understandwhythosedecisionsareimportant,wanttomakethosedecisions, andknowthatthereareconsequencesofthosedecisions.Ifyoudon’t,you’re likelytobedoomingyouraudiencetoineffective,andreally,user‐abusive learning....[Learnersshouldbeput]intothepositionofmakingdecisionsthat makethecontentmeaningfuland…alignedwithhowtheyneedtoapplythe knowledge. SimProject,thesimulationusedinthisstudy,doesthisbyrequiringtheanalysis ofavailableresourcesandthecreationofastaffingmanagementplanwhichisthenused asthebasisforresourceselectionduringthesimulation.Followingeachworkperiod, performancefeedbackisanalyzedandcomparedwiththeoriginplananddecisionsare madeforthenextworkperiod.Thissimulationwasdevelopedtoprovide“virtual‘first‐ hand’experienceinmanagingprojects”byitsdevelopers,Dr.JeffreyPintoandDr.Diane ParenteofThePennsylvaniaStateUniversity(SimProfessionals,2009).Theyprovide thisoverviewofSimProject: Withtherapidriseofcomputersandadvancedtechnologyintheclassroomhave comenewpedagogicalapproachestoteaching.Oneofthemostpowerfulofthese approachesistheuseofcomputersimulationstodemonstrate“realworld” businesspractices,theinter‐relatednessofvariousbusinessfunctions,andthe roleofcompetitivedecision‐makinginbusiness.Computersimulations encourageteamdevelopment,collaboration,globalthinking,andapredilection toconsidertheramificationsofdecisionsandtheireffectonthebottomline—in otherwords,manyoftheskillsthatareusefultoprojectmanagersandteam PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 229 membersinbusiness.Thepurposeofthissimulationistotietogethermanyof thesalientchallengesofprojectmanagementinordertogivestudentsthe deepestpossibleunderstandingofthecomplexitiesinvolvedinundertakinga project.Thegoalofthesimulationwillbetohavestudentsmanageaprojectfrom initiationtocompletion.Withinthisframeworkthestudentwillneedtoemploy anddevelopskillspertinenttopersonnelselectionandtraining,motivation, conflictmanagement,andstakeholdermanagement.Studentswillberequiredto useplanningandschedulingtechniques,suchasworkbreakdownstructures, PERT/CPM,scopedevelopment,andriskanalysis.Thetopicalcoveragewillhave approximatecoincidencewiththeProjectManagementInstitute’sBodyof Knowledge(PMBOK),insuringthatthestudentsgainexposuretothosetopics recognizedbythekeyprofessionalorganizationforprojectmanager.(Pinto& Parente,2003) Playing the game. Thissimulationgameistypicallyplayedbystudentteamswhodevelopaplan involvingfourtypesofdecisions:resourcehiringandrelease,resourcetrainingto improveexpectedperformance,managerialactionstoinfluenceresourceperformance, andassignmentofresourcestoactivities(Pinto&Parente,2003).Thestudentteams competeagainsteachotherforacquisitionofresourcesfromacommonresourcepool basedontheiranalysisofgiveninformation:  Fixedworkbreakdownstructurewithfixedsequencingofactivities  Eachactivityassignedtooneoftwelveprojectphasesorworkperiods  Estimateddurationsandcostbudgetsforeachactivity  Resourcepoolwithsufficientresourcesofeachcategoryforallteams,but notwithequalperformancecharacteristics  Trainingcatalogofavailablecourses,durations,andcosts  Managerialactioncatalogwithcosts(Pinto&Parente,2003) Thestudentteamsdecidewhichresourcestobidonbasedonanalysisofthe giveninformationwhichincludesthefollowingdemographicinformationandmetrics foreachavailableresource(Pinto&Parente,2003): PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 230  Demographicinformation o Resourcename o Category(Engineer,JuniorMarketingSpecialist,JuniorProduct Designer,MarketingManager,OperationsSpecialist,Project Manager,QualityEngineer,SeniorProductDesigner) o Standardandovertimepayrates(overtimeisnotallowed) o Age o Gender o Culturalbackground(American,Chinese,German,Japanese, Spanish)  Metrics o Training o Skilllevel o Experience o Education o Reputation o WorkEthic o PublicRelations o Flexibility o InterpersonalSkill Resourcemetricinformationisgivenasapercentagebetweenzeroand100. Resourceefficiencyontasksisbasedonanundisclosedformulainvolvingthevaluesof thesemetrics,theresourcedemographicsandthecompositionofthesimulatedproject team(Pinto&Parente,2003).Typicallytaskstakelongerthanthegivenduration estimateifonlyoneresourceisassignedtothetask. Priortothestartofthesimulationthestudentteamsengagein“preplay”tobid onandobtaintheirstartingresources.Intheeventoneormoreteamsdoesnotwin theirbidsandobtaintheneededresources,thepreplayroundisrepeateduntileach teamissatisfiedtheyhaveacquiredtheresourcestheyneedtogetstarted.Oncethe virtualprojectteamsarestaffed,thestudentteamsentertheirdecisionsandthe instructoractivatesthesimulationengineforthefirstworkperiod. PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 231 Atthecompletionofeachsimulatedworkperiod,thestudentteamsreceive projectsummaryinformationandstatisticsandtaskactualsinformation.Theproject summaryinformationincludes:workperiodfinishdate,projectbeginningbudget,work periodcosts,anycostadjustmentsresultingfromtheoccurrenceofunforeseenevents, remainingbudgetandpercentagescoresforteamefficiency,cohesion,compositionand longevity.Projectstatisticsreporttheteam’sperformancerelativetotheotherstudent teamsaspercentilesinfourcategories:cost,time,functionalityandstakeholder satisfaction.Regardlessofhowwelleachteamisdoing,theteamwiththebest performanceineachofthesefourcategoriesisshownwithascoreof100%andtheteam withtheworstperformancereceivesascoreof0%.Teamsinthemiddlereceiveascore indicativeoftheirstandingrelativetotheotherteams.Forexample,allteamscould havefinishedtheworkperiodaheadofthebaselineschedule.Theteamfinishingthe earliestwouldreceiveatimescoreof100%andtheteamfinishinglatestwouldreceivea scoreof0%eventhoughtheybothfinishedearlyandregardlessofthedifferenceintheir finishdates(Pinto&Parente,2003). Taskactualsperformancedataisprovidedforeachresourcebyactivitylisting percentagesforeffectiveness,allocation,andefficiency;hoursworked;andcost. Changesinresourcemetricvaluesindicatetheresultsoftraining,managerialactions, eventsandteammorale(Pinto&Parente,2003). Followingeachworkperiod,studentteamsreviewtheirresults,adjusttheirplan asrequired,andsubmittheirfourtypesofdecisionsforthenextworkperiod(resource bidsandreleases,training,managerialactionsandtaskassignments). Thereportedwinnerofthesimulationgameistheteamreceivingtheoverall percentilescoreof100%. PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 232 Appendix G Descriptive Statistics for Variable Components TABLE28‐DESCRIPTIVESTATISTICSFORPRESIMULATIONKNOWLEDGEVARIABLE(K1) Descriptive Statistics K1_ORGDESGN Minimum 3 Maximum 7 Mean 4.68 Std. Deviation .945 Variance .893 K1_PROJPLAN 28 4 7 5.29 .937 .878 K1_PROJRISK 28 3 7 4.89 1.197 1.433 K1_ESTSCOPE 28 2 7 5.04 1.138 1.295 K1_SEQACT 28 3 7 5.61 .916 .840 K1_ESTDUR 28 4 7 5.11 .832 .692 K1_BUDGETNG 28 2 7 4.71 1.182 1.397 K1_RESALLOC 28 3 7 5.29 1.084 1.175 K1_CHGMGT 28 3 7 4.89 1.066 1.136 K1_DESPERFMEAS 28 3 7 4.57 1.260 1.587 K1_EFFLDRSHP 28 4 7 5.32 .670 .448 K1_TEAMCONS 28 4 7 5.50 .923 .852 K1_NEGOT 28 3 7 5.14 .970 .942 K1_EVM 28 2 7 4.36 1.393 1.942 K1_EVALPERS 28 3 7 5.36 1.026 1.053 K1_MNGUNCERT 28 3 6 4.79 .833 .693 K1 28 3.50 6.25 5.033 .710 .504 N 28 PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 233 TABLE29‐DESCRIPTIVESTATISTICSFORPRESIMULATIONCONFIDENCEVARIABLE(C1) Descriptive Statistics C1_ORGDESGN N 28 Minimum 2 Maximum 7 Mean 4.46 Std. Deviation 1.036 Variance 1.073 C1_PROJPLAN 28 3 7 5.25 1.110 1.231 C1_PROJRISK 28 2 6 4.50 1.291 1.667 C1_ESTSCOPE 28 2 7 4.86 1.297 1.683 C1_SEQACT 28 3 7 5.11 1.031 1.062 C1_ESTDUR 28 2 7 4.82 1.156 1.337 C1_BUDGETNG 28 2 6 4.50 1.171 1.370 C1_RESALLOC 28 3 7 5.11 1.197 1.433 C1_CHGMGT 28 2 7 4.75 1.236 1.528 C1_DESPERFMEAS 28 2 7 4.43 1.200 1.439 C1_EFFLDRSHP 28 4 7 5.25 .799 .639 C1_TEAMCONS 28 4 7 5.29 1.049 1.101 C1_NEGOT 28 3 7 4.93 1.016 1.032 C1_EVM 28 1 7 3.93 1.538 2.365 C1_EVALPERS 28 3 7 5.14 1.079 1.164 C1_MNGUNCERT 28 3 7 4.75 1.005 1.009 C1 28 3.25 6.19 4.817 .809 .654 TABLE30–DESCRIPTIVESTATISTICSFORPRESIMULATIONTEAMEXPERIENCE(T1) Descriptive Statistics T1_WRKLD N 28 Minimum 1 Maximum 7 Mean 5.07 Std. Deviation 1.844 Variance 3.402 T1_COOP 28 4 7 5.82 1.218 1.484 T1_EFFIC 28 4 7 5.68 1.056 1.115 T1_EQPART 28 1 7 4.50 1.915 3.667 T1_PLEAS 28 1 7 5.79 1.475 2.175 T1_DISAGR 28 4 7 5.86 .970 .942 T1_MOTIV 28 1 7 5.39 1.524 2.321 T1_SATIS 28 2 7 6.00 1.122 1.259 T1_WILLNG 28 2 7 5.32 1.847 3.411 T1 28 3.1 7.0 5.5 1.2 1.388 PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 234 TABLE31‐DESCRIPTIVESTATISTICSFORPRESIMULATIONTEAMEXPERIENCE(TP1) Descriptive Statistics T1_WRKLD N 28 Minimum 1 Maximum 7 Mean 5.07 Std. Deviation 1.844 Variance 3.402 T1_COOP 28 4 7 5.82 1.218 1.484 T1_EFFIC 28 4 7 5.68 1.056 1.115 T1_EQPART 28 1 7 4.50 1.915 3.667 T1_PLEAS 28 1 7 5.79 1.475 2.175 T1_DISAGR 28 4 7 5.86 .970 .942 T1_MOTIV 28 1 7 5.39 1.524 2.321 T1_SATIS 28 2 7 6.00 1.122 1.259 T1_WILLNG 28 2 7 5.32 1.847 3.411 S1_TEAM 28 2 7 5.39 1.663 2.766 TC1 28 3.1 7.0 5.482 1.1858 1.406 TABLE32‐DESCRIPTIVESTATISTICSFORPRESIMULATIONGENERICTEAMATTITUDE(TG1) Descriptive Statistics S1_GRP N 28 Minimum 3 Maximum 7 Mean 5.50 Std. Deviation 1.202 Variance 1.444 S1_EGRPS 28 3 7 5.82 1.056 1.115 TG1 28 4 7 5.66 .943 .890 TABLE33‐DESCRIPTIVESTATISTICSFORPRESIMULATIONGENERICSIMULATIONATTITUDE (SG1) Descriptive Statistics S1_APPL N 28 Minimum 2 Maximum 7 Mean 5.32 Std. Deviation 1.565 Variance 2.448 S1_VAL 28 2 7 5.18 1.517 2.300 S1_DEC 28 2 7 5.21 1.618 2.619 S1_FUN 28 3 7 5.57 1.103 1.217 S1_HELP 28 2 7 5.11 1.449 2.099 S1_MORE 28 3 7 5.57 1.289 1.661 SG1 28 2.833 7.000 5.327 1.230 1.513 PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 235 TABLE34‐DESCRIPTIVESTATISTICSFORPRESIMULATIONPROJECTSIMULATIONATTITUDE (SP1) Descriptive Statistics S1_VAL N 28 Minimum 2 Maximum 7 Mean 5.68 Std. Deviation 1.467 Variance 2.152 S1_UND 28 3 7 5.61 1.286 1.655 S1_PERF 28 2 7 5.11 1.286 1.655 S1_ENJOY 28 2 7 5.39 1.524 2.321 S1_CONFID 28 2 7 5.18 1.565 2.448 S1_EDUC 28 3 7 5.79 1.197 1.434 S1_FUN 28 3 7 5.43 1.200 1.439 S1_HARD 28 1 7 4.75 1.974 3.898 SP1 28 3.125 7.000 5.36607 1.000537 1.001 TABLE35‐DESCRIPTIVESTATISTICSFORPRESIMULATIONTECHNICALKNOWLEDGE APPLICATION(N1,E1,P1) Descriptive Statistics N1 N 28 Minimum 0 Maximum 4 Mean 2.18 Std. Deviation 1.565 Variance 2.448 E1 28 0 6 1.68 1.611 2.597 P1 = N1 + E1 28 0 9 3.86 2.549 6.497 PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 236 TABLE36‐DESCRIPTIVESTATISTICSFORPOSTSIMULATIONKNOWLEDGEPERCEPTION(K2) Descriptive Statistics K2_ORGDESGN N 25 Minimum 3 Maximum 7 Mean 5.57 Std. Deviation .955 Variance .913 K2_PROJPLAN 25 3 7 5.72 .891 .793 K2_PROJRISK 25 3 7 5.32 1.108 1.227 K2_ESTSCOPE 25 3 7 5.52 .963 .927 K2_SEQACT 25 5 7 6.08 .640 .410 K2_ESTDUR 25 3 7 5.36 1.114 1.240 K2_BUDGETNG 25 3 7 5.20 .957 .917 K2_RESALLOC 25 4 7 5.88 .927 .860 K2_CHGMGT 25 3 7 5.40 .957 .917 K2_DESPERFMEAS 25 3 7 5.00 1.041 1.083 K2_EFFLDRSHP 25 4 7 5.76 .879 .773 K2_TEAMCONS 25 4 7 6.00 .816 .667 K2_NEGOT 25 3 7 5.56 .917 .840 K2_EVM 25 2 7 4.84 1.405 1.973 K2_EVALPERS 25 3 7 5.48 .963 .927 K2_MNGUNCERT 25 4 7 5.32 .690 .477 K2 25 4 6 5.50 .661 .437 TABLE37‐DESCRIPTIVESTATISTICSFORPOSTSIMULATIONCONFIDENCEPERCEPTION(C2) Descriptive Statistics C2_ORGDESGN N 25 Minimum 3 Maximum 7 Mean 5.52 Std. Deviation 1.005 Variance 1.010 C2_PROJPLAN 25 3 7 5.68 .988 .977 C2_PROJRISK 25 3 7 5.12 1.054 1.110 C2_ESTSCOPE 25 3 7 5.56 1.083 1.173 C2_SEQACT 25 4 7 5.84 .850 .723 C2_ESTDUR 25 4 7 5.40 .957 .917 C2_BUDGETNG 25 3 7 5.04 1.020 1.040 C2_RESALLOC 25 3 7 5.80 1.000 1.000 C2_CHGMGT 25 3 7 5.28 1.173 1.377 C2_DESPERFMEAS 25 2 7 4.96 1.207 1.457 C2_EFFLDRSHP 25 4 7 5.76 .831 .690 C2_TEAMCONS 25 3 7 5.68 .945 .893 C2_NEGOT 25 4 7 5.54 .865 .749 C2_EVM 25 2 7 4.48 1.558 2.427 C2_EVALPERS 25 3 7 5.48 1.005 1.010 C2_MNGUNCERT 25 3 7 5.32 .988 .977 C2 25 4 7 5.40 .786 .618 PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 237 TABLE38‐DESCRIPTIVESTATISTICSFORPOSTSIMULATIONTEAMEXPERIENCE(T2) Descriptive Statistics T2_WRKLD N 25 Minimum 1 Maximum 7 Mean 5.36 Std. Deviation 1.440 Variance 2.073 T2_COOP 25 2 7 5.92 1.187 1.410 T2_EFFIC 25 3 7 5.80 1.190 1.417 T2_EQPART 25 1 7 4.92 1.681 2.827 T2_PLEAS 25 3 7 6.08 1.256 1.577 T2_DISAGR 25 2 7 6.00 1.323 1.750 T2_MOTIV 25 2 7 5.92 1.288 1.660 T2_SATIS 25 3 7 6.00 1.291 1.667 T2_WILLNG 25 2 7 5.40 1.780 3.167 T2 25 2.67 7.00 5.7111 1.16843 1.365 TABLE39‐DESCRIPTIVESTATISTICSFORPOSTSIMULATIONTEAMEXPERIENCE(TP2) Descriptive Statistics T2_WRKLD N 25 Minimum 1 Maximum 7 Mean 5.36 Std. Deviation 1.440 Variance 2.073 T2_COOP 25 2 7 5.92 1.187 1.410 T2_EFFIC 25 3 7 5.80 1.190 1.417 T2_EQPART 25 1 7 4.92 1.681 2.827 T2_PLEAS 25 3 7 6.08 1.256 1.577 T2_DISAGR 25 2 7 6.00 1.323 1.750 T2_MOTIV 25 2 7 5.92 1.288 1.660 T2_SATIS 25 3 7 6.00 1.291 1.667 T2_WILLNG 25 2 7 5.40 1.780 3.167 S2_TEAM 25 1 7 5.36 1.912 3.657 TP2 25 2.5 7.0 5.676 1.2248 1.500 TABLE40‐DESCRIPTIVESTATISTICSFORPOSTSIMULATIONGENERICTEAMWORKATTITUDE (SG2) Descriptive Statistics S2_GRP N 25 Minimum 3 Maximum 7 Mean 5.92 Std. Deviation .997 Variance .993 S2_EGRPS 25 5 7 6.12 .726 .527 TG2 25 4 7 6.02 .757 .573 PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 238 TABLE41‐DESCRIPTIVESTATISTICSFORPOSTSIMULATIONGENERICSIMULATIONATTITUDE (SG2) Descriptive Statistics S2_APPL N 25 Minimum 2 Maximum 7 Mean 5.04 Std. Deviation 1.338 Variance 1.790 S2_VAL 25 2 6 4.44 1.356 1.840 S2_DEC 25 2 7 5.03 1.209 1.461 S2_FUN 25 3 7 6.08 1.115 1.243 S2_HELP 25 2 7 5.36 1.221 1.490 S2_MORE 25 2 7 5.70 1.399 1.958 SG2 25 3 7 5.27 1.117 1.247 TABLE42‐DESCRIPTIVESTATISTICSFORPOSTSIMULATIONPROJECTSIMULATIONATTITUDE (SP2) Descriptive Statistics S2_VAL N 25 Minimum 2 Maximum 7 Mean 5.56 Std. Deviation 1.356 Variance 1.840 S2_UND 25 2 7 5.24 1.300 1.690 S2_PERF 25 2 7 5.60 1.500 2.250 S2_ENJOY 25 2 7 5.92 1.352 1.827 S2_CONFID 25 1 7 4.76 1.690 2.857 S2_EDUC 25 2 7 5.60 1.384 1.917 S2_FUN 25 3 7 5.88 1.092 1.193 S2_HARD 25 1 7 3.64 1.604 2.573 SP2 25 3 7 5.28 1.026 1.053 TABLE43‐DESCRIPTIVESTATISTICSFORPOSTSIMULATIONTECHNICALKNOWLEDGE APPLICATION(N2,E2,P2) Descriptive Statistics N2 N 25 Minimum 0 Maximum 4 Mean 2.28 Std. Deviation 1.400 Variance 1.960 E2 25 0 6 2.28 1.768 3.127 P2 25 1 9 4.56 2.347 5.507 PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 239 References References AACSBAccreditationCoordinatingCommittee,&AACSBAccreditationQuality Committee.(2007).AACSBassuranceoflearningstandards:Aninterpretation. Retrieved5/27,2011,from http://www.aacsb.edu/publications/whitepapers/AACSB_Assurance_of_Learning.p df ABSEL.(2011).AboutABSEL.Retrieved7/21,2011,from http://absel2011.wordpress.com/about Adobor,H.,&Daneshfar,A.(2006).Managementsimulations:Determiningtheir effectiveness.TheJournalofManagementDevelopment,25(2),151‐168. Ahn,J.(2008).Applicationofexperientiallearningcycleinlearningwithabusiness simulationgame.(UnpublishedDoctorofEducation).ColumbiaUniversity, Al‐Jibouri,S.,Mawdesley,M.,Scott,D.,&Gribble,S.(2005).Theuseofasimulationmodel asagameforteachingmanagementofprojectsinconstruction.TheInternational JournalofEngineeringEducation,21(6),1195. Al‐Jibouri,S.H.,&Mawdesley,M.J.(2001).Designandexperiencewithacomputergame forteachingconstructionprojectplanningandcontrol.Engineering,Construction andArchitecturalManagement,8(5/6),418. Alreck,P.L.,&Settle,R.B.(1995).Thesurveyresearchhandbook(2nded.).Chicago: Irwin. PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 240 AmericanPsychologicalAssociation.(2010).Publicationmanualoftheamerican psychologicalassociation(6th[correct2printinged.).Washington,DC:American PsychologicalAssociation. Anbari,F.T.(2010).Distanceeducationinprojectmanagement.Paperpresentedatthe PMIGlobalCongress2010NorthAmerica,Washington,D.C. Anderson,P.H.,&Lawton,L.(1988).Assessingstudentperformanceonabusiness simulationexericse.DevelopmentsinBusinessSimulation&ExperientialExercises,15 Anderson,P.H.,&Lawton,L.(1990).Therelationshipbetweenfinancialperformance andothermeasuresoflearningonasimulationexercise.DevelopmentsinBusiness Simulation&ExperientialExercises,17 Anderson,P.H.,&Lawton,L.(1992).Asurveyofmethodsusedforevaluatingstudent performanceonbusinesssimulations.Simulation&Gaming,23(4),490‐498.doi: 10.1177/1046878192234010 Anderson,P.H.,&Lawton,L.(1997).Demonstratingthelearningeffectivenessof simulations:Whereweareandwhereweneedtogo.DevelopmentsinBusiness Simulation&ExperientialLearning,24 Anderson,P.H.,&Lawton,L.(2003).Theopimaltimingforintroducingbusiness simulations.DevelopmentsinBusinessSimulation&ExperientialLearning,30 Anderson,P.H.,&Lawton,L.(2007).Simulationperformanceanditseffectivenessasa PBLproblem:Afollow‐upstudy.DevelopmentsinBusinessSimulation&Experiential Learning,34 PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 241 Anderson,P.H.,&Lawton,L.(2009).Businesssimulationsandcognitivelearning: Developments,desires,andfuturedirections.Simulation&Gaming,40(2),193‐216. doi:10.1177/1046878108321866 Anderson,P.H.,Lawton,L.,&Wellington,W.J.(2008).Goalorientationandsimulation performance.DevelopmentsinBusinessSimulation&ExperientialLearning,35,329‐ 335. Anderson,J.R.(2005).Therelationshipbetweenstudentperceptionsofteamdynamics andsimulationgameoutcomes:Anindividual‐levelanalysis.JournalofEducationfor Business,81(2),85‐90. Ashleigh,M.,Ojiako,U.,Chipulu,M.,&Wang,J.K.(2012).Criticallearningthemesin projectmanagementeducation:Implicationsforblendedlearning.International JournalofProjectManagement,30(2),153‐161.doi: 10.1016/j.ijproman.2011.05.002 Baglione,S.L.,&Tucci,L.A.(2010).Usingamarketingcomputersimulationtoimprove learninginaprinciplesofmarketingcourse.ReviewofBusinessResearch,10(4),47‐ 57. Biggs,W.D.(1990).Introductiontocomputerizedbusinessmanagementsimulations.In J.W.Gentry(Ed.),Guidetobusinessgamingandexperientiallearning(pp.23‐35). EastBrunswick,NJ/London:Nichols/PB. Burns,R.B.,&Burns,R.A.(2008).BusinessresearchmethodsandstatisticsusingSPSS. ThousandOaks,CA;London:SAGE. PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 242 Buzzetto‐More,N.,&Mitchell,B.C.(2009).Studentperformanceandperceptionsina web‐basedcompetitivecomputersimulation.InterdisciplinaryJournalofE‐Learning andLearningObjects,5,73‐90. Campbell,D.T.,&Stanley,J.C.(1963).Experimentalandquasi‐experimentaldesignsfor research.Chicago:R.McNally. Chin,J.,Dukes,R.,&Gamson,W.(2009).Assessmentinsimulationandgaming:Areview ofthelast40years.Simulation&Gaming,40(4),553‐568.doi: 10.1177/1046878108327585 Clark,R.C.,&Mayer,R.E.(2008).E‐learningandthescienceofinstruction:Proven guidelinesforconsumersanddesignersofmultimedialearning(2nded.).San Francisco,CA:Pfeiffer. Collofello,J.S.(2000).University/industrycollaborationindevelopingasimulation‐ basedsoftwareprojectmanagementtrainingcourse.Education,IEEETransactions on,43(4),389‐393. Cook,D.L.,&Granger,J.C.(1976).Currentstatusofprojectmanagementinstructionin americancollegesanduniversities.TheAcademyofManagementJournal,19(2), 323‐328. Cook,L.S.,&Olson,J.R.(2006).Thesky'sthelimit:Anactivityforteachingproject management.JournalofManagementEducation,30(3),404‐420. Cook,T.D.,&Campbell,D.T.(1979).Quasi‐experimentation:Design&analysisissuesfor fieldsettings.Boston:HoughtonMifflin. PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 243 Cooper,D.R.,&Schindler,P.S.(2006).Businessresearchmethods(9thed.).Boston: McGrawHill. Cooper,J.M.(2011).Crisismanagementandprojectleadershipskills:Assessingthe educationalvalidityofaprojectmanagementsimulation.Unpublishedmanuscript. Creswell,J.W.(2009).Researchdesign:Qualitative,quantitative,andmixedmethod approaches[null](3rded.).ThousandOaks,CA:Sage. Creswell,J.W.(2010).Mappingthedevelopinglandscapeofmixedmethodsresearch.In A.Tashakkori,&C.Teddlie(Eds.),Handbookofmixedmethodsinsocial&behavioral research(2nded.,pp.45‐68).ThousandOaks,CA:ThousandOaks,CA:SAGE Publications. Cronan,T.P.,Léger,P.,Robert,J.,Babin,G.,&Charland,P.(2012).Comparingobjective measuresandperceptionsofcognitivelearninginanERPsimulationgame. Simulation&Gaming,43(4),461‐480.doi:10.1177/1046878111433783 Crookall,D.(2010).Seriousgames,debriefing,andSimulation/Gamingasadiscipline. Simulation&Gaming,41(6),898‐920. Czaja,R.,&Blair,J.(2005).Designingsurveys:Aguidetodecisionsandprocedures. ThousandOaks,CA:SagePublications,Inc. Dantas,A.R.,Barros,M.O.,&Werner,C.M.L.(2004).Asimulation‐basedgamefor projectmanagementexperientiallearning.Proceedingsofthe16thAnnual InternationalConferenceonSoftwareEngineering&KnowledgeEngineering(SEKE), Banff,Canadá, PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 244 Davidovitch,L.,Parush,A.,&Shtub,A.(2006).Simulation‐basedlearninginengineering education:Performanceandtransferinlearningprojectmanagement.Journalof EngineeringEducation,95(4),289‐299. Davidovitch,L.,Parush,A.,&Shtub,A.(2008).Simulation‐basedlearning:Thelearning– forgetting–relearningprocessandimpactoflearninghistory.Computers& Education,50(3),866‐880.doi:DOI:10.1016/j.compedu.2006.09.003 Davidovitch,L.,Parush,A.,&Shtub,A.(2009).Theimpactoffunctionalfidelityin simulator‐basedlearningofprojectmanagement.InternationalJournalof EngineeringEducation,25(2),333‐340. Davidovitch,L.,Parush,A.,&Shtub,A.(2010).Simulator‐basedteamtrainingtoshare resourcesinamatrixstructureorganization.EngineeringManagement,IEEE Transactionson,57(2),288‐300. Davidovitch,L.,Shtub,A.,&Parush,A.(2007).Projectmanagementsimulation‐based learningforsystemsengineeringstudents.SystemsEngineeringandModeling,2007. ICSEM'07.InternationalConferenceon,17‐23. Davis&Dean.(2011a).Davis&dean:Amilestonemanagementcompany.Retrieved 01/04,2013,fromhttp://www.davisdean.com Davis&Dean.(2011b).Whattheresearchsays.Retrieved01/05,2013,from http://www.davisdean.com/what‐the‐research‐says.html Dennis,E.E.,&Smith,S.P.(2006).Findingthebestbusinessschoolforyou:Lookingpast therankings.Westport,CT:Praeger. PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 245 Dillman,D.H.,&Cook,D.L.(1969).SimulationinthetrainingofR&Dprojectmanagers. [Washington,D.C.]:DistributedbyERICClearinghouse,. Dukes,R.L.,&Seidner,C.J.(Eds.).(1978).Learningwithsimulationsandgames.Beverly Hills,Calif.:SagePublications. Faria,A.J.(1998).Businesssimulationgames:Currentusagelevels‐‐anupdate. Simulation&Gaming,29(3),295‐308.doi:10.1177/1046878102238606 Faria,A.J.,Hutchinson,D.,Wellington,W.J.,&Gold,S.(2009).Developmentsinbusiness gaming:Areviewofthepast40years.Simulation&Gaming,40(4),464‐487.doi: 10.1177/1046878108327585 Faria,A.J.,&Wellington,W.J.(2004).Asurveyofsimulationgameusers,former‐users, andnever‐users.Simulation&Gaming,35(2),178‐207.doi: 10.1177/1046878108322225 Faria,A.J.,&Whiteley,T.R.(1990).ANEMPIRICALEVALUATIONOFTHEPEDAGOGICAL VALUEOFPLAYINGASIMULATIONGAMETNAPRINCIPLESOFMARKETING COURSE.DevelopmentsinBusinessSimulation&ExperientialExercises,,1753‐57. Feinstein,A.H.,&Cannon,H.M.(2002).Constructsofsimulationevaluation.Simulation &Gaming,33(4),425‐440.doi:10.1177/1046878102238606 Field,A.P.(2009).DiscoveringstatisticsusingSPSS(3rded.).London:SAGE. Fowler,F.J.(2009).Surveyresearchmethods(4thed.).ThousandOaks,CA:Sage Publications. PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 246 Fritzche,D.J.,&Cotter,R.V.(1990).Guidelinesforadministeringbusinessgames.InJ.W. Gentry(Ed.),Guidetobusinessgamingandexperientiallearning(pp.74‐89).East Brunswick,NJ/London:Nichols/PB. Gamlath,S.L.(2009).Fieldtestingtwosimulationgames:Dowinnerswinconsistently? OntheHorizon,17(4),388‐396. Gentry,J.V.,&Burns,A.C.(1981).Operationalizingatestofamodeloftheuseof simulationgamesandexperientialexercises.DevelopmentsinBusinessSimulation& ExperientialExercises,8,48‐52. Gentry,J.W.(Ed.).(1990a).Guidetobusinessgamingandexperientiallearning.East Brunswick,NJ/London:Nichols/GP. Gentry,J.W.(1990b).Whatisexperientiallearning?InJ.W.Gentry(Ed.),Guideto businessgamingandexperientiallearning(pp.9‐20).EastBrunswick,NJ/London: Nichols/PB. Gentry,J.W.,&Burns,A.C.(1997).Thoughtsaboutthemeasurementoflearning:The caseforguidedlearningandassociatedmeasurementissues.Developmentsin BusinessSimulation&ExperientialLearning,24,241‐246. Gosen,J.,&Washbush,J.(2004).Areviewofscholarshiponassessingexperiential learningeffectiveness.Simulation&Gaming,35(2),270‐293.doi: 10.1177/1046878108322225 PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 247 Gosenpud,J.(1990).Evaluationofexperientiallearning.InJ.W.Gentry(Ed.),Guideto businessgamingandexperientiallearning(pp.301‐329).EastBrunswick, NJ/London:Nichols/PB. Gosenpud,J.,&Washbush,J.(1994).Whatsimulationusersthinkplayersshouldbe learningfromsimulations.DevelopmentsinBusinessSimulation&Experiential Learning,21,96‐99. Gosenpud,J.,&Washbush,J.(2010).Therelatonshipbetweenlearningandperformance inatotalenterprisesimulation:Revisitedandnewdata.DevelopmentsinBusiness Simulation&ExperientialLearning,37,95‐98. Graf,L.A.(2001).ABSEL'scontributionstoexperientialLearning/Experientialexercises: Thedecadeofthe1970s.Simulation&Gaming,32(1),40‐65.doi: 10.1177/1046878102238606 Gray,C.F.,&Larson,E.W.(2003).Projectmanagement:Themanagerialprocess(2nd ed.).NewYork:McGraw‐Hill/Irwin. Gray,C.F.,&Larson,E.W.(2008).Projectmanagement:Themanagerialprocess(4thed.). NewYork:McGraw‐Hill/Irwin. Graziano,R.(2003).Thevirtualenterprisesimulation:Students'perceptionsofan experiential,activelearningstrategyforbusinessandcareereducation.Hofstra University)..(UMINumber:3088556). PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 248 Green,J.C.(2004).Studentreactionstotheuseofacomputer‐basedsimulationasan integratingmechanismforaMBAcurriculum.DevelopmentsinBusinessSimulation andExperientialLearning,,31286‐289. Greenlaw,P.S.,&Wyman,F.P.(1973).Theteachingeffectivenessofgamesincollegiate businesscourses.Simulation&Games,4(3),259‐294. Hall,O.P.,&Ko,K.(2008).MeasuringMBAlearningoutcomesusingbusinesssimulations Workingpaperseries,PepperdineUniversity. Hays,R.T.(2005).Theeffectivenessofinstructionalgames:Aliteraturereviewand discussion.(No.TechnicalReport2005‐004).Orlando,FL:NavalAirWarfareCenter TrainingSystemsDivision. Hertel,J.P.,&Millis,B.J.(2002).Usingsimulationstopromotelearninginhigher education.Sterling,VA:StylusPublishing,LLC. Heyman,M.(1975).Simulationgamesfortheclassroom.Bloomington,Ind.:PhiDelta KappaEducationalFoundation. Hornyak,M.J.,Peach,E.B.,Bowen,A.,Moes,W.,&Wheeler,R.(2006).Examiningproject managementinbusinesssimulations:Studentandfacultyviews.Developmentsin BusinessSImulationandExperientialLearning,,33107‐117. Hutcheson,J.(1984).Educatingprojectmanagersfortheconstructionindustryin australia.InternationalJournalofProjectManagement,2(4),220‐224.doi: 10.1016/0263‐7863(84)90039‐5 PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 249 Jones,K.(1987).Simulations:Ahandbookforteachersandtrainers(2nded.).London: KoganPage. Kayes,A.B.,Kayes,D.C.,&Kolb,D.A.(2005a).Developingteamsusingthekolbteam learningexperience.Simulation&Gaming,36(3),355‐363.doi: 10.1177/1046878108322225 Kayes,A.B.,Kayes,D.C.,&Kolb,D.A.(2005b).Experientiallearninginteams.Simulation &Gaming,36(3),330‐354.doi:10.1177/1046878108322225 Keys,B.(1977).Reviewoflearningresearchinbusinessgaming.ComputerSimulation andLearningTheory,3,173‐184. Keys,B.,&Wolfe,J.(1990).Theroleofmanagementgamesandsimulationsineducation andresearch.JournalofManagement,16(2),307‐336. Keys,J.B.,&Biggs,W.D.(1990).Areviewofbusinessgames.InJ.W.Gentry(Ed.),Guide tobusinessgamingandexperientiallearning(pp.48‐73).EastBrunswick, NJ/London:Nichols/PB. Klein,R.D.(1980).Anempiricalevaluationoftheeffectivenessofasimulationgamein teachinginternationalbusiness.(Doctoraldissertation,GeorgiaStateUniversity). (ProquestDissertationandThesesdatabase).(UMINumber8120885). Klein,R.D.(1984).Addinginternationalbusinesstothecoreprogramviathesimulation game.JournalofInternationalBusinessStudies,15(1),151. Kolb,D.A.(1984).Experientiallearning:Experienceasthesourceoflearningand development.EnglewoodCliffs,NJ:Prentice‐Hall. PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 250 Krathwohl,D.R.(2002).Arevisionofbloom'staxonomy:Anoverview.Theoryinto Practice,41(4),212‐218. Kriz,W.C.(2010).Asystemic‐constructivistapproachtothefacilitationanddebriefing ofsimulationsandgames.Simulation&Gaming,41(5),663‐680. Lainema,T.,&Lainema,K.(2007).Advancingacquisitionofbusinessknow‐how:Critical learningelements.JournalofResearchonTechnologyinEducation,40(2),183‐198. Larréché,J.(1987).Onsimulationsinbusinesseducationandresearch.Journalof BusinessResearch,15(6),559‐571.doi:10.1016/0148‐2963(87)90039‐7 Larson,E.W.,&Gray,C.F.(2011).Projectmanagement:Themanagerialprocess(5thed.). NewYork:McGraw‐Hill/Irwin. Lederman,L.C.(1984).Debriefing:Acriticalreexaminationofthepostexperience analyticprocesswithimplicationsforitseffectiveuse.Simulation&Gaming,15(4), 415‐431.doi:10.1177/0037550084154002 Lederman,L.C.(1992).Debriefing:Towardasystematicassessmentoftheoryand practice.Simulation&Gaming,23(2),145‐160.doi:10.1177/1046878192232003 Lederman,L.C.,&Ruben,B.D.(1984).Systematicassessmentofcommunicationgames andsimulations:Anappliedframework.CommunicationEducation,33(2),152. Martin,A.(2000).Asimulationengineforcustomprojectmanagementeducation. InternationalJournalofProjectManagement,18(3),201‐213.doi:10.1016/S0263‐ 7863(99)00014‐9 PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 251 Mayer,B.W.,Dale,K.M.,Fraccastoro,K.A.,&Moss,G.(2011).Improvingtransferof learning:Relationshiptomethodsofusingbusinesssimulation.Simulation& Gaming,42(1),64‐84.doi:10.1177/1046878110376795 Mayer,R.E.(2002).Roteversusmeaningfullearning.TheoryintoPractice,41(4,Revising Bloom'sTaxonomy),226‐232. Maylor,,Harvey.(2001).Beyondtheganttchart:Projectmanagementmovingon. EuropeanManagementJournal,19(1),92. McCreery,,JohnK.(2003).Assessingthevalueofaprojectmanagementsimulation trainingexercise.InternationalJournalofProjectManagement,21(4),233. Megarry,J.(1978).Retrospectandprospect.InR.McAleese(Ed.),Perspectiveson academicgaming&simulation3().London:K.Page. Meier,R.C.,Newell,W.T.,&Pazer,H.L.(1969).Simulationinbusinessandeconomics. EnglewoodCliffs,N.J.,:Prentice‐Hall. Mertens,D.M.(2010).Researchandevaluationineducationandpsychology:Integrating diversitywithquantitative,qualitative,andmixedmethods(3rd.ed.).LosAngeles: Sage. Mintzberg,H.(2004).ManagersnotMBAs:Ahardlookatthesoftpracticeofmanaging anddevelopingpeople.SanFrancisco:Berrett‐Koehler. Pate,G.S.,&Parker,H.A.(1973).Designingclassroomsimulations.Belmont,Calif.:Lear Siegler/Fearon. PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 252 Peters,V.A.M.,&Vissers,G.A.N.(2004).Asimpleclassificationmodelfordebriefing simulationgames.Simulation&Gaming,35(1),70‐84.doi: 10.1177/1046878108322225 Pfahl,,Dietmar.(2004).Evaluatingthelearningeffectivenessofusingsimulationsin softwareprojectmanagementeducation:Resultsfromatwicereplicated experiment.InformationandSoftwareTechnology,46(2),127. Pfeffer,J.,&Fong,C.T.(2002).Theendofbusinessschools?lesssuccessthanmeetsthe eye.AcademyofManagementLearning&Education,1(1),78‐95. Pinto,J.K.,&Parente,D.H.(2003).SimProjectinstructorsmanualV1.2.NewYork: McGrawHill/Irwin. ProjectManagementInstitute.(2009).Handbookofaccreditationofdegreeprogramsin projectmanagement(3rded.).NewtownSquare,PA:ProjectManagementInstitute. ProjectManagementInstitute.(2010a).AboutPMI.RetrievedJuly14,2010,from http://www.pmi.org/AboutUs/pages/About‐PMI.aspx ProjectManagementInstitute.(2010b).Thepowerofprojectmanagement.Retrieved 8/18,2010,fromhttp://www.pmiteach.org/your‐free‐ downloads/Power_of_Project_Management_WhitePaper.pdf ProjectManagementInstitute.(2010c).Theprojectmanagementinstituteglobal accreditationcenterforprojectmanagementeducationprograms(GAC).Retrieved 7/14,2010,fromhttp://www.pmi.org/CareerDevelopment/Pages/Global‐ Accreditation‐Center.aspx PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 253 ProjectManagementInstitute.(2010d).Projectmanagementinstitute:Whoweareand whatwedo.Retrieved08/30,2010,from http://www.pmi.org/AboutUs/Pages/FactSheet.aspx ProjectManagementInstitute.(2011).Projectmanagementinstituteprofessional(PMP) handbook.Retrieved3/18,2011,from http://www.pmi.org/en/Certification/~/media/PDF/Certifications/pdc_pmphand book.ashx ProjectManagementInstitute.(2012).DirectoryofGACaccreditedprograms.Retrieved October31,2012,fromhttp://www.gacpm.org/Directory‐of‐GAC‐Programs.html ProjectManagementInstitute.(2013a).Aguidetotheprojectmanagementbodyof knowledge(PMBOKguide)(5thed.).NewtownSquare,PA:ProjectManagement Institute,Inc. ProjectManagementInstitute.(2013b,PMIfactfile.PMIToday:ASupplementtoPM Network,27(1) Quinn,C.N.(2005).Engaginglearning:Designinge‐learningsimulationgames.San Francisco,CA:Pfeiffer. SAGSET.(2011).Publications.Retrieved7/21,2011,fromhttp://www.sagset.org SAGSET.(2013).Publications.Retrieved1/20,2013,fromhttp://www.sagset.org Salas,E.,Rosen,M.A.,Held,J.D.,&Weissmuller,J.J.(2009).Performancemeasurement insimulation‐basedtraining:Areviewandbestpractices.Simulation&Gaming, 40(3),328‐376.doi:10.1177/1046878108322225 PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 254 Salas,E.,Wildman,J.L.,&Piccolo,R.F.(2009).Usingsimulation‐basedtrainingto enhancemanagementeducation.AcademyofManagementLearning&Education, 8(4),559‐573. Schlesinger,P.F.(1996).Teachingandevaluationinanintegratedcurriculum.Journalof ManagementEducation,20(4),479‐499. Schonlau,M.,Elliott,M.N.,&Fricker,R.D.(2002).Conductingresearchsurveysviae‐mail andtheweb.SantaMonica,CA:RAND. Seethamraju,R.(2011).Enhancingstudentlearningofenterpriseintegrationand businessprocessorientationthroughanERPbusinesssimulationgame.Journalof InformationSystemsEducation,22(1),19‐29. Sekaran,U.(2003).Researchmethodsforbusiness:Askillbuildingapproach(4thed.). Hoboken,NJ:JohnWiley&Sons. Shtub,A.(2012).ProjectmanagementsimulationwithPTBprojectteambuilder.New York:Springer. SimProfessionals.(2009).Aboutthesimulation.RetrievedMarch18,2012,from http://www.simprojectonline.com/AboutUs.aspx Smalt,S.W.(1999).Integrationofagameintoacollegeaccountingprinciplescourse: Studentperformanceandstudentperceptions.(DoctoralDissertation,TheUnion Institute).(ProQuestDissertationsandThesesdatabase).(UMINo.9958873). PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 255 Snow,S.C.,Gehlen,F.L.,&Green,J.C.(2002).Differentwaystointroduceabusiness simulation:Theeffectonstudentperformance.Simulation&Gaming,33(4),526‐ 532.doi:10.1177/1046878102238606 SouthernAssociationofCollegesandSchoolsCommissiononColleges.(2009).The principlesofaccreditation:Foundationsforqualityenhancement(4thed.).Decatur, Georgia:SouthernAssociationofCollegesandSchoolsCommissiononColleges. Stainton,A.J.,Johnson,J.E.,&Borodzicz,E.P.(2010).Educationalvalidityofbusiness gamingsimulation:Aresearchmethodologyframework.Simulation&Gaming, 41(5),705‐723.doi:10.1177/1046878109353467 Starkey,K.,&Tiratsoo,N.(2007).Thebusinessschoolandthebottomline.Cambridge,UK: CambridgeUniversityPress. Tashakkori,A.,&Teddlie,C.(2010a).Currentdevelopmentsandemergingtrendsin integratedresearchmethodology.InA.Tashakkori,&C.Teddlie(Eds.),Handbookof mixedmethodsinsocial&behavioralresearch(2nded.,pp.803‐826).Thousand Oaks,CA:ThousandOaks,CA:SAGEPublications. Tashakkori,A.,&Teddlie,C.(2010b).Overviewofcontemporaryissuesinmixed methodsresearch.InA.Tashakkori,&C.Teddlie(Eds.),Handbookofmixedmethods insocial&behavioralresearch(2nded.,pp.1‐41).ThousandOaks,CA:Thousand Oaks,CA:SAGEPublications. Taylor,J.L.,&Walford,R.(1978).Learningandthesimulationgame.BeverlyHills,Calif.: SagePublications. PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 256 Teach,R.D.,&Murff,E.R.T.(2007).Assessingparticipantlearninginabusiness simulation.DevelopmentsinBusinessSimulationandExperientialLearning,,3476‐ 84. Teach,R.D.,&Murff,E.R.T.(2009).Learninginhibitorsinbusinesssimulationsand games.DevelopmentsinBusinessSimulationandExperientialLearning,,36191‐197. Thatcher,D.C.(1990).Promotinglearningthroughgamesandsimulations.Simulation& Gaming,21(3),262‐273.doi:10.1177/1046878190213005 Thiagarajan,S.(1992).Usinggamesfordebriefing.Simulation&Gaming,23(2),161‐173. Thornton,G.C.,&Cleveland,J.N.(1990).Developingmanagerialtalentthrough simulation.AmericanPsychologist,45(2),190‐199.doi:10.1037/0003‐ 066X.45.2.190 Waggener,H.A.(1979).Simulationvscasesvstext:Ananalysisofstudentopinion. JournalofExperientialLearningandSimulation,1,113‐118. Wellington,W.J.,Hutchinson,D.,&Faria,A.J.(2012).Anexploratorystudyoftheimpact ofasimulationexerciseonthemanagerialandpersonalitytraitsandthedecision makingstylesofmarketingstudents.DevelopmentsinBusinessSimulationand ExperientialLearning,,39132‐140. Williams,A.J.,&Williams,R.H.(2011).Multipleidentificationtheory:Attitudeand behaviorchangeinasimulatedinternationalconflict.Simulation&Gaming,42(6), 733‐747.doi:10.1177/1046878111429228 PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 257 Wolfe,J.(1985).Theteachingeffectivenessofgamesincollegiatebusinesscourses:A 1973‐1983update.SimulationGaming,16(3),251‐288. Wolfe,J.(1990).Theevaluationofcomputer‐basedbusinessgames:Methodology, findings,andfutureneeds.InJ.W.Gentry(Ed.),Guidetobusinessgamingand experientiallearning(pp.279‐300).EastBrunswick,NJ/London:Nichols/PB. Wolfe,J.(1993).Ahistoryofbusinessteachinggamesinenglish‐speakingandpost‐ socialistcountries:Theoriginationanddiffusionofamanagementeducationand developmenttechnology.Simulation&Gaming,24(4),446‐463.doi: 10.1177/1046878193244003 Wolfe,J.(1997).Theeffectivenessofbusinessgamesinstrategicmanagementcourse work.Simulation&Gaming,28(4),360‐276. Yeaple,R.N.(2006).DoesitpaytogetanMBA?.Orlando,FL:TheCenterforManagement Education. Zantow,K.,Knowlton,D.S.,&Sharp,D.C.(2005).Morethanfunandgames: Reconsideringthevirtuesofstrategicmanagementsimulations.Academyof ManagementLearning&Education,4(4),451‐458.

the value and effectiveness of project management simulation as a

Related documents

Products

Support

the value and effectiveness of project management simulation as a

Related documents

Add this document to collection(s)

Add this document to saved

Suggest us how to improve StudyLib