Susan P. Maude Iowa State University smaude@iastate.edu 2013 Early On Faculty Colloquium Early On Center for Higher Education June 3, 2013 Introduction Definition of Professional Development (PD) What we know about PD (+ and -) What we don’t know; What we don’t want to lose Where are we going? To increase your awareness on current research on Professional Development (PD). To evaluate your practices with the growing evidence-based literature (own PD and PD provided to others). To begin examining and possibly exploring other ways to modify future Early On Faculty Colloquia. National Professional Development Center on Inclusion (NPDCI, 2008) “Professional development is facilitated teaching and learning experiences that are transactional and designed to support the acquisition of professional knowledge, skills, and dispositions as well as the application of this knowledge in practice … The key components of professional development include: a) characteristics and contexts of the learners (i.e., the “who” ); b) content (i.e., the “what” of professional development); and c) organization and facilitation of learning experiences (i.e., the “how”).” NPDCI, 2008 http://npdci.fpg.unc.edu/sites/npdci.fpg.unc.edu /files/resources/NPDCI_ProfessionalDevelopmentI nEC_03-04-08_0.pdf Is here today…… The “what” (latest research, policy, funding) The “how” (online, instructional strategies, etc.) Rarely do we in EI receive PD on the What, How, and For Whom that fits our particular work. Exceptions have included…. ◦ USDOE/OSEP Faculty Institutes (early 1990’s)**** ◦ USDOE/OSEP – Funded PD Projects: SCRIPT/Natural Allies Initiatives FPG/UNC-Chapel Hill ◦ State Departments Early Intervention Consortia (early 1990’s but many have been discontinued) – Rock on Early On Recall an exemplary PD experience YOU participated in as a “receiver”. Think of the three components of the PD definition….Who, What, How Activity: With your neighbor, share what made that PD experience exemplary for you? What “feelings” did that PD event have on you? Why did that PD event become an exemplary one for you? Any key who, what, how to share? Discipline Specific Knowledge, Skills, Dispositions Cross Disciplinary KSD Interdisciplinary KSD Thorp, E.K. & McCollum, J.A. (1988, 1994). Defining the infancy specialization in early childhood special education. In J.Jordan, J. Gallagher, P. Huttinger, & M. Karnes (Eds.), Early Childhood Special Education: Birth to Three, pp. 147-162. Reston, VA: Council for Exceptional Children. CIPPP FPG Snapshot and in ECTE The Center to Inform Personnel Preparation, Policy, and Practice (CIPPPP) in Early Intervention and Preschool Education funded by OSEP from 2003-2008 to collect, synthesize, and analyze information related to: 1. EI/ECSE certification and licensure requirements, 2. Quality of training programs, and 3. Supply and demand of professionals in all EI/ECSE disciplines. Nationally, decline of the family voice in higher education; Some critical content areas (FCP, IFSPs, Teaming, NLE, SC) for EI still have limited focus in personnel preparation across 5 disciplines: OTs, PTs, ECSEs, SLPs, and Multidisciplinary programs – FCP was stronger; SC was limited across all disciplines; Limited use of national standards (4 out of 18 sets of standards met 80% or better) for EC/EI/ECSE (and blended programs); and Potential for inconsistent application of national standards in IHE programs (disconnect between SEAs and national standards). Bruder & Dunst (2005). Personnel preparation in recommended early intervention practices: Degree of emphasis across disciplines. TECSE, 25, 25-33. Stayton et al., (2012). Comparison of state certification and professional association standards in early childhood education. TECSE, 32, 24-37. http://www.uconnucedd.org/projects/per_prep/resources.html CRITICAL EARLY INTERVENTION CONTENT AREAS CLASS/ APPLIED ASSIGNMENT INTRO: Family Story Interview a family child w/disability home language CLAD ASSESSMENT Play-based Assessment w/family CURRICULUM PIWI Play Group FAMILIES Resource Family Design, Facilitate, and Evaluate an IFSP Familycentered practices (FCP) IFSP X Teaming NLE ? Service Coordination X X X X ? X X X X X X X X X/? X X X Addressing the needs of young children with disabilities and their families; Understanding and working effectively with infants and toddlers; and Building young children’s competence and interest in mathematics. http://www.fpg.unc.edu/sites/default/files/res ources/snapshots/FPG_Snapshot68_2013.pdf Accreditation of ECTE programs is voluntary, limited the benefits of state and national attempts at quality assurance (there are issues with NCATE/CAEP on what professional standards to use in reviewing blended EC/ECSE programs); Dearth of research about barriers and facilitators for faculty in implementing high-quality college curricula that aligns with EBP. Preservice students in ECTE programs receive limited coursework working with children with disabilities, infants and toddlers; Promising IHE programs and approaches for preparing students – yet no nationwide system of EBP for preparing workforce to enter the field (“a cottage industry of path breaking initiatives”) Limited follow-up of preservice students once they enter the field on a) competence, b) fidelity of implementation of effective practices, and faculty use of emerging, EB curricula. Horm, Dr., Hyson, M., & Winton, P. (2013). Research on ECTE: Evidence from three domains and recommendations for moving forward. JECTE, 34(1), 95-112. National survey (Ray and colleagues, 2006) indicated no difference in coverage of diversity content in NCATE- and non-NCATE- accredited ECTE programs. Little known about HOW the PD is delivered in accreditation programs. Research indicating that graduate students in ECTE preparation programs, the group most likely to be faculty, do not have exposure to adult learning principles (Maxwell, Lim, & Early, 2006) suggests that future faculty are not being adequately prepared to implement PD practices. Winton, P. (2010). Professional development and quality initiatives: Two essential components of an early childhood system. In P.W. Wesley & V. Buysse (Ed.), The Quest for Quality: Promising Innovations for Early Childhood Programs. Baltimore: Brookes. http://www.edutopia.org/blo g/project-happinessempathy-randy-taran Empathy helps one understand people whose values, views, and behaviors are different from one’s own (Calloway-Thomas, 2010). One definition for empathy from the perspective of the teacher in the context of ECE is as follows: Empathy, in the context of early care and education, is the ability to: feel what the child or family member is feeling, understand what the child or family member is feeling, communicate that understanding to them, and then respond in a way to help meet their needs. Nancy F. Peck, 2012 Supporting the Development of Reflective Supervision Honoring Diversity in College Students and the Families they Support Use of Dilemmas as Instructional Strategy Closing session – Performance http://www.edutopia.org/blog/film-festival-kindness-empathyconnection?utm_source=SilverpopMailing&utm_medium=email&utm _campaign=enews%20051513%20remainder&utm_content=&spMaili ngID=6150212&spUserID=MjcyOTI0NzE4MjMS1&spJobID=7361404 8&spReportId=NzM2MTQwNDgS1 http://youtu.be/HfHV4-N2LxQ Recent research syntheses on adult learning strategies and teacher development provide some empirical basis for designing effective professional development (Trivette, 2005; Trivette, Dunst, Hamby, & O’Herin, 2009; Darling-Hammond, Wei, Andree, Richardson, & Orphanos, 2009) Thanks to Camille Catlett, CIES Conference, 2013 One thing we can say with certainty about professional development is that workshops alone are not effective if building skills or dispositions is the desired outcome (Fixsen, Naoom, Blase, Friedman, & Wallace, 2005) Thanks to Camille Catlett, CIES Conference, 2013 One-time events and training workshops are consistently the PD method of preference in early childhood, despite the fact that short-term, one-time trainings have little or no impact on quality improvements. (Zollitsch & Dean, 2010, The Organisation for Economic Co-operation and Development, 2011) Thanks to Camille Catlett, CIES Conference, 2013 Staff Development Inservice Training Preservice Training In situ/Consultation/Coaching Induction/Mentoring Web Training Materials Only Shared Inquiry Other Snyder et al., 2012. Infants and Young Children Coaching/Performance Feedback Consultation Mentoring Peer Support Group Communities of Practice/Shared Inquiry Assignments Job Aids Back-Home plans Handouts Refresher Sessions Follow-up visit Follow-up phone call or email Follow-up letter/packet Individualized learning Contract Discussion Board/Chat room No follow-up Snyder et al., 2012. Infants and Young Children Desired impact (learning outcomes from low to high) INSTRUCTIONAL STRATEGIES USED Attitudes, values role playing field application case studies Skill role playing field application Knowledge reading lecture demonstration observation interviewing problem solving brainstorming discussion Awareness reading lecture Low guided reflection self-analysis clinical supervision guided reflection follow-up plans coaching High Complexity of synthesis and application required A model for matching training approach to desired training outcomes and complexity of application. (Adapted from Harris [1980].) Research synthesis of 79 studies of accelerated learning, coaching, guided design, and just-intime-training 58 randomized control design studies and 21 comparison group studies 3,152 experimental group participants and 2,988 control or comparison group participants Combination of studies in college and noncollege settings Learner outcomes included learner knowledge, skills, attitudes, and self-efficacy beliefs Trivette, C.M. et al. (2009). Characteristics and consequences of adult learning methods and strategies. Winterberry Research Syntheses, Vol. 2, Number 1. a Planning Introduce Engage the learner in a preview of the material, knowledge or practice that is the focus of instruction or training Illustrate Demonstrate or illustrate the use or applicability of the material, knowledge or practice for the learner Application Practice Engage the learner in the use of the material, knowledge or practice Evaluate Engage the learner in a process of evaluating the consequence or outcome of the application of the material, knowledge or practice Deep Understanding Reflection Engage the learner in self-assessment of his or her acquisition of knowledge and skills as a basis for identifying “next steps” in the learning process Mastery Engage the learner in a process of assessing his or her experience in the context of some conceptual or practical model or framework, or some external set of standards or criteria a Donovan, M. et al. (Eds.) (1999). How people learn. Washington, DC: National Academy Press. 3 6 1.2 Planning Introduce Application Practice RefUnderstanding lection MEAN EFFECT SIZE (d) 1.0 0.8 0.6 0.4 0.2 0.0 Introduce Illustrate Practice Evaluate Reflection ADULT LEARNING METHOD CHARACTERISTICS Mastery Effect Sizes for Introducing Information to Learners Number Studies Effect Sizes Mean Effect Size (d) 95% Confidence Interval Pre-class exercises 9 9 1.02 .63-1.41 Out of class activities/self-instruction 12 20 .76 .44-1.09 Classroom/workshop lectures 26 108 .68 .47-.89 Dramatic readings 18 40 .35 .13-.57 Imagery 7 18 .34 .08-.59 Dramatic readings/imagery 4 11 .15 -.33-.62 Practices Effect Sizes for Illustrating/Demonstrating Learning Topic Number Studies Effect Sizes Mean Effect Size (d) 95% Confidence Interval Using learner input 6 6 .89 .28-1.51 Role playing/simulations 20 64 .87 .58-1.17 Real life example/real life + roleplaying 6 10 .67 .27-1.07 Instructional video 5 49 .33 .09-.59 Practices Effect Sizes for Learner Application Number Studies Effect Sizes Mean Effect Size (d) Real life application + role playing 5 20 1.10 .48-1.72 Problem solving tasks 16 29 .67 .39-.95 Real life application 17 83 .58 .35-.81 Learning games/writing exercises 9 11 .55 .11-.99 Role playing (skits, plays) 11 35 .41 .21-.62 Characteristics 95% Confidence Interval Effect Sizes for Learner Evaluation Number Studies Effect Sizes Mean Effect Size (d) 95% Confidence Interval Assess strengths/weaknesses 14 48 .96 .67-1.26 Review experience/make changes 19 35 .60 .36-.83 Practices Effect Sizes for Learner Reflection Number Studies Effect Sizes Mean Effect Size (d) 95% Confidence Interval Performance improvement 9 34 1.07 .69-1.45 Journaling/behavior suggestion 8 17 .75 .49-1.00 Group discussion about feedback 16 29 .67 .39-.95 Practices Effect Sizes for Self-Assessment of Learner Mastery Number Studies Effect Sizes Mean Effect Size (d) 95% Confidence Interval Standards-based assessment 13 44 .76 .42-1.10 Self-assessment 16 29 .67 .39-.95 Practices Study 2: Promoting Adoption of Family-Systems Intervention Practices • 473 Part C early intervention practitioners • 5 types of training (presentations, day and multiday workshops, field-based and enhanced fieldbased) • Participants randomly assigned to complete the study outcome measure at one or six months after training • Outcome measure included items on the usefulness of the training and the extent to which the training improved their abilities to work with families Four Major Components of the Family-Systems Intervention Model that Constituted the Focus of Inservice Training CAPACITY-BUILDING HELPGIVING PRACTICES FAMILY CONCERNS AND PRIORITIES FAMILY MEMBER STRENGTHS SUPPORTS AND RESOURCES Characteristics of the Different Types of In-service Training Type of In-Service Training Conference Presentations Half Day/ Full Day Workshops Multi-Day Workshops Field-Based Training Enhanced Field-Based Training Trainer introduction of the practice + + ++ ++ ++ Trainer illustration of use of the practice ± + + ++ ++ Trainee application/use of the practice - ± + + ++ Trainee evaluation of his/her use of the practice - - ± + ++ Trainee reflection on his/her learning - - ± + ++ Trainee assessment of learner mastery - - - ++ ++ Multiple learning sessions - - + ++ ++ Practice Characteristics NOTE. - = No activity or opportunity, ± = limited opportunity, + = multiple opportunities, and ++ = multiple and varied opportunities. PLAN Introduce and Illustrate RECYCLE APPLICATION Identify Next Steps in the Learning Process Practice and Evaluate INFORMED UNDERSTANDING Reflection and Mastery Active Learner Involvement Participants’ Judgments of the Benefits of the Five Types of Training 32 MEAN PARTICIPANT RATING Usefulness Abilities 30 28 26 24 22 20 Presentations Day Workshops Multi-Day Workshops Field-Based Enhanced FieldBased TYPE OF TRAINING Interaction Between Hours and Type of Training for Improved Family Systems Intervention Abilities 32 Multiday Workshops PARTICIPANT RATINGS 30 Field-Based Enhanced Field-Based 28 26 24 22 20 10 20 HOURS OF TRAINING Conclusions • PALS includes practices that have been found effective in promoting practitioner adoption of different kinds of early childhood practices • Participatory adult learning is an effective implementation method • Additional analyses of the use of PALS are expected to shed light on which characteristics of the practices matter most in terms of affecting learner outcomes • Another research synthesis study has looked at participants “values and beliefs” in adopting a practice (Toolbox of strategies might not be the best way?)? Real life application Helping Hand 4:06 Example of project based learning http://www.youtube.com/watch?v=T7PK8sWwixI&feature=s hare&list=PLvzOwE5lWqhQWsPsW5PQQ5gj5OBewwgUw http://youtu.be/T7PK8sWwixI Key Learning Application ◦ ◦ ◦ ◦ Includes infants/toddlers with diverse abilities Families Sustained (over a semester) Criteria/Performance Checklist Is intensive and ongoing, with multiple, sequenced, active learning experiences; smaller numbers are more successful <20 Is grounded in specific practice-focused content Builds on the learner’s current level of understanding Includes large doses of learner selfassessment of his/her learning against a set of standards, criteria, or expert feedback Is aligned with instructional goals, learning standards, and curriculum materials Thanks to Camille Catlett, CIES Conference, 2013 Relationship-based training efforts (coaching, consultation, technical assistance, mentoring, communities of practice, peer study groups) are promising but largely unproven methods. To date studies are “far from conclusive and offer little in the way of showing advantages of one over the other.” (Zollitsch & Dean, 2010) Research grants from Early On? Thanks to Camille Catlett, CIES Conference, 2013 Recommended Practices New Type of PDI – Future Idea for Michigan to host? Division for Early Childhood of the Council for Exceptional Children (DEC/CEC). Commission – ◦ ◦ ◦ ◦ ◦ 12-15 individuals Barbara Smith, UC-Denver is Chair Synthesizing current research Will be ready by January, 2014 Update at the DEC International Conference in San Francisco, CA October, 2013 www.dec-sped.org/conference DEC – national professional organization Reduction of practitioners at our DEC ◦ CSEFEL NTI – longer sessions, “ready to go into classroom resources” Have had several PD subgroups addressing “other types of PD” to host ◦ IHE Consortium (pre/post national conference) ◦ Practitioner focused institute ◦ Family voice interwoven (strong in early 90’s, weaker, coming back) ◦ *Lindbloom (1959) New Possibilities Using Evidence-Based Practices That Support Inclusion Day 1 out of 2 Days Time Participant Options 8:00 - 9:00 Registration/Check in/Continental Breakfast Orientation to New Possibilities – Types of learning opportunities, who’s at New Possibilities to learn from/with, 9:00 – 9:30 what came before/what comes after 9:30 – 9:45 Transition to Participant Options Strand 1, Part 1 Strand 2, Part 1 Strand 3, Part 1 Strand 4, Part 1 Grab Bag Embedded Learning Literacy and SocialEnvironments that Technology Toolboxes, 9:45 – 12:45 Emotional Development Support Each Child resources Snyder/Wolfe Trivette/Fox West/XX Edelman Catlett 12:45 – 2:00 2:00 – 5:00 How it will be structured… 5:30 – 7:00 7:00 - ?? Lunch (provided) Strand 1, Part 2 Embedded Learning Strand 2, Part 2 Strand 3, Part 2 Literacy and SocialEnvironments that Emotional Development Support Each Child I Want More Opportunity to talk with instructor and/or other participants about session content and applications; ask questions Strand 4, Part 2 Technology Participant Options I Want to Try It I Want Something New Opportunity to apply content with other participants Networking reception with presenters Dinner on your own Toolbox A Grab Bag Toolboxes, resources I Want a Break Independent processing time Planning Introduce PRIOR to the Event/At the Event Illustrate Application Practice At the Event/After the Event Evaluate Deep Understanding Reflection AFTER the Event Mastery a Donovan, M. et al. (Eds.) (1999). How people learn. Washington, DC: National Academy Press. 6 5 What does that mean for us who are Providers of Professional Development ◦ How do we link our own PD efforts to EBP? ◦ How do we evaluate the impact of different models ◦ How do we not lose “professional or family wisdom” in the process ◦ New Early Childhood Personnel Center – via Mary Beth Bruder (SEAs and IHEs) ◦ Center for Goodness Science – video Office of Special Education Programs US Department of Education Address the need for highly qualified personnel to work with infants, toddlers, and preschool with disabilities The Early Childhood Personnel Center (University of Connecticut) A national resource on personnel standards, competencies, and recommended practices for professional development for personnel providing services to infants, toddlers, and preschool children with disabilities and their families. The Center will: ◦ Bring together national professional organizations to facilitate the development of a set of recommended personnel standards for personnel serving infants, toddlers, and preschool children with disabilities and their families. Update recommended practices related to professional development for personnel providing services to infants, toddlers, and preschool children with disabilities and their families Assist States in aligning their personnel standards to the recommended personnel standards for personnel providing services to infants, toddlers, and preschool children with disabilities and their families and linking those standards to State competencies and certification or licensure requirements ◦ Assist State agencies and IHEs in developing partnerships with each other to support the alignment between preservice and inservice training for personnel providing services to infants, toddlers, and preschool children with disabilities and their families Assist States in developing and implementing integrated early childhood professional development systems so that all early childhood personnel, including those working in Child Care, Early Head Start, Head Start and State-funded Pre-K, have the competencies to effectively serve infants, toddlers, and preschool children with disabilities ◦ Assist States in developing and implementing integrated early childhood professional development systems to ensure that IDEA Part C and Part B preschool programs and personnel in each State are included within the State’s professional development initiatives Mary Beth Bruder, PhD and colleagues The Early Childhood Personnel Center (University of Connecticut) Co-teach and Teach Graduate Supervise Students Level Courses Designing, Delivering and Evaluating Interventions Supervise Students Conducting Education/ Research Use Technology in Supervision and Teaching Participate on Research Team Conduct Research Synthesis Implement Research Studies Research and Inquiry Personnel Preparation Leadership Training for ECI Higher Education Faculty Review Grants, Manuscripts and other Publications Present at State, Regional, National Conferences Provide PD Activities Scholarship Community Engagement and Outreach for Practice and Policy Scale Up Evidence-Based Practices into Service Delivery Models Using Implementation Science Collaborate with Others to Address Family/Child Need Implement Evidencebased Practice in High Need Programs Engage in Self Assist State Inquiry Early Childhood Develop and and Organizations and Agencies Submit a Grant Submit ManuscriptsParticipate in National Reflection to Develop Organizations’ Application Practices and Policy for Publication Policy Initiatives ADMINISTRATION AND MANAGEMENT Data Collection and Analyses of Personnel Policies; Literature Reviews, Syntheses and Papers Review of DEC Personnel Practices, Standards; Refine Standards across Disciplines General TA through Meetings, Webinars, Product Development and Dissemination Targeted TA and Dissemination to Specific Populations on Personnel Procedures, Policies and Practices Intensive TA to 8 States to Develop, Implement and Sustain a Model for Preparation of Quality Personnel Infants and Young - Implement and Sustain Leadership through Collaborations; Training to Scale up Systems of Personnel Development EVALUATION and CONTINUOUS IMPROVEMENT Organizational Chart Director Mary Beth Bruder (UConn) Executive Management Team Co-Director George Sugai (UConn) Carl Dunst (Puckett Institute) Larry Edelman (U of Colorado) Lynn Kagan (Columbia Teachers College) Coordinator Tierney Giannotti (UConn) External Evaluator Kathleen Hebbeler (SRI) Evaluation Team Mary Louise Hemmeter (Vanderbilt) Jeannette McCollum (U of Illinois) Vicki Stayton (Western Kentucky University) Project Consultants/Contractors Roxanne Kaufman (Georgetown) Pip Campbell (Jefferson University) Toby Long (Georgetown) Division of Early Childhood(S. Mulligan) Dale Mann (Interactive Inc.) Claudia Dozier (KU- ABS/BCBA) Carol Trivette (Puckett Institute) Maureen Greer (Emerald Consulting) University of CT Regional Associate Director Mary Beth Bruder Post Doc (TBH) Gabriela Freyre Calish FL State University Regional Associate Directors Juliann Woods Mary Frances Hanline Post Doc (TBH) Cindy Vail University of KS Regional Associate Directors Eva Horn David Lindeman P. Doc Stephanie Parks University of OR Regional Associate Director Jane Squires P. Doc Lois Pribble Mary Jo Noonan Regional Advisory Board Regional Advisory Board Regional Advisory Board Regional Advisory Board Partner Organizations and Project Advisory Board AUCD CEC HECSE ITCA NAECS-SDE NAEYC NASDSE NASDTEC CCSSO Part B/619 Consortia NHSA NRCP Break out sessions will address “innovative approaches” yet we will need to continue building the evidence (beyond professional or family wisdom) for some of these practices. What changes to your Early On Colloquium would you embrace? Prior to, During, After? Last Clicker Activity…… Develop systematic approaches to data collection on critical content-area requirements in ECTE. Move beyond counting courses and credits to examine patterns in course content, methods, and field experiences. Develop, field test, and rigorously evaluate innovative, evidence-based approaches to ECTE. Develop and evaluate a variety of tools to assess preservice students’ competencies, beliefs, and attitudes. Develop, field test, and rigorously evaluate practical approaches to faculty development. Winton, P. (2010). Professional development and quality initiatives: Two essential components of an early childhood system. In P.W. Wesley & V. Buysse (Ed.), The Quest for Quality: Promising Innovations for Early Childhood Programs. Baltimore: Brookes. Center for Goodness Kindness Scientist 3:13 http://www.youtube.com/watch?v=sUcxoNFiomY&feature =share&list=PLvzOwE5lWqhQWsPsW5PQQ5gj5OBewwgUw http://www.youtube.com/watch?v=sUc xoNFiomY&feature=share&list=PLvzO wE5lWqhQWsPsW5PQQ5gj5OBewwgUw Think about this as a key underpinning of our work in EI. Web site where you can search for course resources in EI/EC/ECSE on aspects of diversity http://projects.fpg.unc.edu/~crosswalks/toolbox/index.c fm Make sure to click all the links – some will lead you to a site where you can search by types of instructional strategies case studies, assignments, videos, etc. Thanks to Camille Catlett, CIES Conference, 2013 Rubric Refined External Evaluator Scored Pre and Time 1 Syllabi Faculty/Staff Scored Pre Syllabi No reliability training across evaluators SIX MAJOR ELEMENTS; 22 INDICATORS COURSE DESCRIPTION (4) COURSE OBJECTIVES (5) TEXTS, READINGS, & RESOURCES (3) ASSIGNMENTS (5) GUEST SPEAKERS (2) IN-CLASS INSTRUCTIONAL EXPERIENCES (3) RATING GUIDELINES: EXTENT THE SYLLABUS EMPHASIZES CORE VALUES ◦ ◦ ◦ ◦ LITTLE OR NONE SOME SIGNIFICANTLY EXTENSIVELY 1 2 3 4 TOTAL SCORE = 4 X 22 = 88 Performance Measure 2b Course Syllabi Results 80 Scoring by KCC Faculty Raw Score (max = 88) 70 60 50 40 30 20 10 0 103 133 158 159 170 221 243 262 Time 1 (9/11) 46 41 38 41 51 47 40 40 Time 2 (10/12) 72 65 71 65 61 81 73 73 Kirkwood EC Course