National Mentoring Summit January 2013 Agenda • • • • • • • Mentor Video Program Description Research Overview Key Findings Conclusions and Continual Improvement Lessons Learned: Mentoring for Academic Gains Action Planning Video: Be a Mentor • http://www.youtube.com/watch?v=YyK_03DccSY Program Model • Afterschool, Summer, High School Placement • 650 hours/year, + 1000 hours in school • Weekly academic mentoring • Accelerated lessons, college trips, career exposure • Positive culture and youth development approach 2% Only of nonprofits have this “gold-standard” randomized research RCT Study Design $4 million over 7 years Over 3 years (2006, 2007, 2008), 951 students & families apply and interview for 3 cohorts 50% accepted through lottery 50% assigned to control group Researchers test & survey: 4 summers and 3 school years + one 9th grade follow up Two-year results and Summer Snapshot published in 2011 by P/PV Four-year results published in 2013 By MDRC Randomized Control Trial (RCT) Study: funded by Wallace, WT Grant, Smith Richardson, Spencer, Atlantic Philanthropies, Bank of America Randomization – in Brief 1. Hear about HA at school, neighborhood, etc. 2. Interview 99% - enter lottery. 3. Lottery 50% enter program: “treatment” “Treatment” Access to full program – may never attend. “Control” Never able to access program. But given list of other after-school & summer options Annual testing ($120 payment), surveys, Follow-up from researchers Researchers • Dr. Jean Grossman – Princeton University, P/PV, MDRC • Dr. Leigh Linden – Columbia University, University of Texas- Austin • Dr. Carla Herrera – P/PV Outcome Measures Collected • Hypothesis: - With high dosage and structure, Higher Achievement would have a measurable impact on academic outcomes and high school placement, by first impacting attitudes and behaviors. • 2 key outcomes: – Standardized Test Scores – Application, Acceptance to, and Matriculation at Competitive High Schools Other outcomes of interest: – – – – – Behavior Academic Attitudes Perceptions of Peer and Adult Support Participation in HA and Other OST Programs Engagement in Academic Activities and High School-Related Activities KEY FINDINGS Two-year follow-up study “Higher Achievement’s intensive year-round program had a significant impact on youth’s standardized reading and math test scores.” “The longstanding Higher Achievement model is making a difference in the academic lives of motivated, at-risk students who could easily fall through the cracks.” Summer 2010 Snapshot High levels of attendance and retention, during tricky middle school years: 97% of summer participants also in after-school No summer learning loss. Also - no impact on test scores, compared to control group, over the course of one summer: 2010 Four-year follow up study Academic Impacts First- Second- FourthYear FU Year FU Year FU Math problem-solving 0.03 0.10* 0.11* Reading comprehension * = Statistical significance. 0.02 0.08† 0.04 Academic and Enrichment Activities Activity Community service First-Year FU Treatmt T-C Diff (%) (%) 53 0 Second-Year FU Fourth-Year FU Treatmt T-C Diff Treatmt T-C Diff (%) (%) (%) (%) 60 4 74 4 Presented ideas to a group out of school Visited a college 63 6† 64 9* 69 11* 72 28** 73 28** 78 25** Read books out of school Writing out of school 74 73 3 7* 79 74 4 3 85 82 8† 14** Visited a business 56 4 61 8* 70 14** Events with OST 77 10** 80 6† 84 9* Academic contests at OST 68 13** 68 11** 71 16** Conclusions • Year-round, multi-year: high dosage yields results • No effect after 1 year, only after 2 years: – Academic results take time • Reading gains level with control group in 4-year follow-up: – Control and treatment made gains. Reading can be more self-directed. Math requires more instruction. Continual Improvement • Common Core Standards Alignment: – Curricula and Technology • Explicitly teach writing skills • Staff training to improve inference skills for reading comprehension • “How Children Succeed” & Social/Emotional Skills – Piloted three assessments: PEAR HSA, SAYO, Gallup Student Poll + Grit Scale Annual Operating Plan and Individual Work Plans, tied to Strategic Plan. Annual Retreat: Outcomes and RCT Results Continuous Improvement Cycle Mid-Year Retreats: Org & City course corrections Update individual work plans Monthly Dashboard and Site Observation Discussions: National & Local Staff School Partners Quarterly Dashboard, linked to Financial Model Maximizing Mentors • • • • • Orientation Ongoing mini-trainings Scripted lessons Nightly session feedback Monthly observations One Mentor’s Perspective • Ms. Liesa – Her involvement, academic gains of scholars – Mentor training and ongoing support – Best practices in academic mentoring Table Activity– Think, Pair, Share! • What are the goals for your mentoring program? • What challenges do you have to meeting your goals? • What resources do you have to meet your goals? What resources do you have to overcome your challenges? • What support and/or trainings do you have to put in place to maximize your resources and meet the goals of your mentoring program? ACTION PLANNING "Whatever affects one directly, affects all indirectly. I can never be what I ought to be until you are what you ought to be.” - Dr. Martin Luther King