April 23, 2013 Mahnomen, MN Ximena Suarez-Sousa, Ph.D. Boyd Bradbury, Ph.D. Basics of the Study 2,800 students 140 parents 400 teachers 7 schools Grades 3-12 Mixed-Methods Early childhood, social services, health services Quantitative Findings 1. Appraisal of Academic Influences Inventory 2. Outcome Variables (MCA II and MAP) 3. Predictor Variables (e.g. absenteeism, tardy, free and reduced lunch) 4. Predictive Models 5. Differences and similarities between the American Indian and White populations 1. Appraisal of Academic Influences Inventory (AAII) (Bradbury, 2006) Constructs (9) Structure Computer-base 3-5 grades: Dichotomous items (Yes/No) (quantitative data) 6 – 12 grades: Scale items (1-5) (quantitative data) All grades: Open ended questions (qualitative data) Technical integrity Validity Reliability CSV data from schools to MSUM (SPSS). 2. Outcome Variables MCA Math MCA Reading MAP Math MAP Reading Cumulative GPA 3-5 and 6-12 2010-2011 MCA – Math (3-5) MAP – Math (3 – 5) MCA – Reading (3-5) MAP – Reading (3-5) MCA – Math (6-12) MAP – Math (6-12) MCA – Reading (6-12) MAP – Reading (6-12) Cumulative GPA (6-12) 3. Predictor Variables Demographic data Lunch code Absenteeism Household configuration Gender among others Appraisal Data (AAII) – Self (e.g., “I feel that I do my best in school”) – Parents and Home Environment (e.g., “I feel safe at home” – Own Behavior (e.g., “I get in trouble in school”) – Teacher (e.g., “My teacher makes sure that I know how to do something before moving onto something else”) – Relationships (e.g., “My classmates influence my behavior and performance in school more than any other thing”) – School Environment (e.g., “I feel safe at school”) MCA – Math (3-5) MAP – Math (3 – 5) Tardiness (.19*) MCA Math Native American MCA Reading (.84**) MAP Math Native American MAP Reading (.85**) MCA – Reading (3-5) MAP – Reading (3-5) Percept. of Teacher Discipline (.53**) (-.22*) MCA Reading Native MAP Reading American Native American MCA Math (.84**) Tardiness (.28**) MAP Math (.85**) MCA – Math (6-12) MAP – Math (6-12) MAP Reading MCA Reading (.85**) (.73**) MAP Math MCA Math NA NA Percept. of Behavior (.37**) Percept Parents (.36**) MCA Reading (.26**) MCA – Reading (6-12) MAP – Reading (6-12) MCA Math MAP Math (.78**) Percept. of Behavior (.40*) Percept. of School (.32**) MAP Math (.85**) (.26**) MAP Reading MCA Reading (.34**) NA Percep. of Parents MCA Reading MAP Reading Percept. of Parents (.34**) NA .23** GPA (.46**) (.25**) Absence Discipline Percept. of Behavior (-.28**) (-.23*) (.47**) Cumulative GPA (6-12) MCA Reading (.46**) Percept. of Relats. (.19*) Percept. of School (.20**) Discipline (-.40**) Cum. Absence GPA (-.52*) NA Percept. of Teacher Tardiness -.42** (.20*) Percept. of Self (.27**) 4. Predictive Models Multiple regression analysis Cause effect relationships Effect size MCA – Math (3-5) MCA Reading score (an increase increases MCA Math scores) Perception of Own Behavior (an increase increases MCA Math scores) Model fit R2 = .43 The MCA Reading score and the student’s perception of own behavior can predict 43% of the changes on students’ MCA Math scores. MCA – Reading (3-5) MCA Math score (an increase increases MCA Reading scores) Model fit R2 = .48 The MCA Math score can predict 48% of the changes on students’ MCA Reading scores. MAP – Math (3-5) MAP – Reading (3-5) MAP Reading score (an increase increases MAP Math score) MAP Math score (an increase increases MAP Reading score) Model fit R2 = .80 Model fit R2 = .80 The MAP Reading score can predict 80% of the changes on students’ MAP Math Reading scores. The MAP Math score can predict 80% of the changes on students’ MAP Reading scores. MCA – Math (6-12) MCA Reading score (an increase increases MCA Math score) Model fit R2 = .53 (53%) The MCA Reading score can predict 53% of the changes on students’ MCA Math scores. MCA – Reading (6-12) MCA Math score (an increase increases MCA Reading score) Model fit R2 = .34 (34%) The MCA Math score can predict 34% of the changes on students’ MCA Reading scores. MAP – Math (6-12) MAP Reading score (an increase increases MAP Math score) Model fit R2 = .72 (72%) The MAP Reading score can predict 72% of the changes on students’ MAP Math scores. MAP – Reading (6-12) MAP Math score (an increase increases MAP Reading score) Model fit R2 = .72 (72%) The MAP Math score can predict 72% of the changes on students’ MAP Reading scores. Cumulative GPA (6-12) Number of absences (an increase reduces GPA) Number of tardy days (an increase reduces GPA) Perception of student’s own behavior (an increase increases GPA) Model fit R2 = .48 (48%) These three variables have the power to predict 48% of the change on students’ cumulative GPA. 5. Differences and Similarities between Native American and White Students Grades 3-5 Grades 6-12 3-5 Native American students: White students: Total: 278 (58% boys). Total: 608 (53% boys). SES: 65% received free SES: 22% received free lunch lunch Family Structure: Family Structure: Approximately 25% lived Approximately 66% lived with both biological with both biological parents. parents. 3-5 Native American students: White students: While approximately half of this group had never been involved in disciplinary problems (46%), 13% had received a minimum of 6 and up to 36 disciplinary sanctions. Absenteeism or tardiness were not prevalent in this group, about 25% of students were absent from school between 10 and 41 times and 18% of them arrived late at school between 10 and 89 times. Disciplinary data were significantly unavailable for this group and we cannot determine whether it was an omission in the data collection process or whether there were no data to report, in which case each student could have received a “zero” score on this variable; however, that was not the case. Approximately 18% of students were absent from school between 10 and 26 times. 3-5 Ethnicity Native American M White MCA Math 53.26 64.73 t(315) = -3.307, p < .05 MCA Reading 55.84 64.97 t(625) = -3.844, p < .001 MAP Math 223.99 212.04 t(635) = -3.307, p < .05 MAP Reading 211.26 202.52 t(625) = 4.562, p < .001 Discipline 3.75 1.33 t(89) = 2.046, p < .05 Absenteeism 7.03 5.73 t(709) = 2.523, p < .05 Tardiness 7.54 1.15 t(615) = 5.168, p < .001 Perception of Self 29.93 29.91 Groups are equivalent Perception of Teacher 18.87 19.76 t(569) = -4.605, p < .001 Perception of Parents 11.89 12.73 t(616) = -4.121, p < .001 Perception of Own Behavior 7.14 7.64 t(526) = 22.877, p < .05 Perception of School Environment 7.58 7.67 Groups are equivalent Perception of Relationships 12.76 13.16 t(510) = -2.190, p < .05 Variable t value and Level of Statistical Significance M 6-12 Native American students: White students: Total: 485 (44% boys). Total: 1358 (51% boys). SES: 50% received free SES: 23% received free lunch lunch Family Structure: Family Structure: Approximately 33% lived Approximately 65% lived with both biological with both biological parents. parents. 6-12 Native American students: White students: While approximately one third had never been involved in disciplinary problems (29.1%), 40% received a minimum of 6 and up to 39 disciplinary sanctions. Absenteeism and tardiness were prevalent in this group, about 50% of students were absent from school between 10 and 69 times and 46% of them arrived late at school between 10 and 251 times. While approximately one third had never been involved in disciplinary problems (34.6%), 15% received a minimum of 6 and up to 16 disciplinary sanctions. Absenteeism and tardiness were prevalent in this group, about 54% of students were absent from school between 10 and 120 times and 27% of them arrived late at school between 10 and 223 times. 6-12 Ethnicity Native American M White MCA Math 46.40 51.77 t(627) = -3.391, p < .001 MCA Reading 52.85 59.98 t(873) = -6.551, p < .001 MAP Math 228.32 227.49 Groups are equivalent MAP Reading 212.54 213.96 Groups are equivalent Discipline 4.78 2.56 t(248) = 3.319, p < .001 Absenteeism 12.32 9.10 t(1279) = 4.890, p < .001 Tardiness 15.32 11.20 t(1097) = 2.262, p < .05 Perception of Self 16.35 16.58 Groups are equivalent Perception of Teacher 58.23 60.17 t(1523) = -3.629, p < .001 Perception of Parents 73.86 83.49 t(1293) = -10.214, p < .001 Perception of Own Behavior 64.56 71.98 t(369) = -6.967, p < .001 Perception of School Environment 29.83 30.31 Groups are equivalent Perception of Relationships 55.22 56.49 t(1338) = -2.322, p < .05 Variable t value and Level of Statistical Significance M Descriptive and Qualitative Findings Parents, Teachers, and Students Process of analysis Narrative data Generalized and specific themes to school districts Descriptive and Qualitative Findings: Bagley (Self-Perception Construct) Nearly ½ of children in grades 3-5 do not like reading. 3 times the dislike level of the overall White and AI populations Pre-assessment Leveling Children find math to be very hard and don’t like it. 10% (1/3) higher dislike than the overall White and AI populations in grades 3-5 ½ of students in grades 6-12 identify math as the hardest subject and 7 in 10 teachers agree. Descriptive and Qualitative Findings: Bagley (Teacher Construct) Utilize more technology in the classroom. Only 46% of Bagley teachers agree that Bagley teachers use technology regularly in the classroom versus 68% of the overall teacher population. A majority of students would agree to greater engagement when technology is used. Professional development (technology integration) should occur. Descriptive and Qualitative Findings: Bagley (School Environment Construct) Fear of safety is a concern 14% of Bagley students in grades 6-12 feel unsafe (twice the rate of the overall student population) Only 48% of Bagley students in grades 3-5 feel safe on school buses (10% lower than overall population and 20% lower than overall AI populations) 17% of Bagley students in grades 6-12 do not feel safe on the school bus (nearly double the overall and AI student populations) Descriptive and Qualitative Findings: Bagley (Relationships Construct) 32% of students do not think that their classmates like them (a rate 6 times higher than the overall student population and 3.5 times higher than the overall AI student population) The overall rates of students not liking their principals and principals not liking students based on student and teacher responses is 4 times the rate of the overall population responses. Descriptive and Qualitative Findings: Bagley (Relationships Construct--Continued) 21% of students feel bullied in grades 3-5 and 20% of children in grades 6-12 either feel bullied or aren’t sure if they are bullied. Only 55% of children in grades 3-5 report a responsible adult with whom they can visit if they have problems. This is far lower than the overall and AI populations, 85% and 76%, respectively. Descriptive and Qualitative Findings: COL (Self-Perception Construct) COL students in grades 3-12 identify reading as their least favorite subject (twice the rate of the overall AI population in grades 6-12). 43% of COL students in grades 6-12 view math as their hardest subject. About 1 in 4 students finds math easy, which is similar to overall student population statistics. Descriptive and Qualitative Findings: COL (Teacher Construct) COL students don’t believe teachers are concerned about their academic achievement at rates witnessed in other school districts Only 58% of COL students in grades 3-5 their teachers care about them versus 97% of the overall AI student population. 27% of COL students in grades 6-12 versus 54% of the overall AI student population agree that teachers care about them. 22% of COL students in grades 6-12 disagree that teachers care about them versus 6% of the overall AI student population. Descriptive and Qualitative Findings: COL (Teacher Construct--Continued) COL teachers use technology at a much lower rate than the overall teacher population 26% of students in grades 3-5 agree that teachers used technology versus 91% of the overall grade 3-5 AI student population. This figure increases to 44% in grades 6-12 versus 78% of the overall grade 6-12 AI student population. 46% of teachers at COL agree that they use technology versus 68% of the overall teacher population. Professional development for technology usage and integration should be considered. Descriptive and Qualitative Findings: COL (Parent/Home Life Construct) Lower engagement of parents reported (e.g. checking on homework completion, watches TV rather than spending time with child). Higher rates of physical aggression and yelling in home. High percentage of children uncertain as to where they are staying at night (double the overall AI student population). Descriptive and Qualitative Findings: COL (Student Behaviors Construct) Only 17% of COL students participate in extracurricular activities, compared with 48% of the overall AI and 64% of the Caucasian populations. Drug and alcohol usage is elevated—16% of students in grades 3-5 (4-8 times higher) and 27% in grades 612 (twice the rate of the overall AI student population and more than 5 times the overall rate of the Caucasian student population. Descriptive and Qualitative Findings: COL (Student Behaviors Construct--Continued) 1 in 10 students skip school due to alcohol and drug usage. Are more counselors needed? Descriptive and Qualitative Findings: COL (School Environment Construct) COL students do not like their school schedule as much as the overall student populations. Only 46% of COL students like their schedules as compared with 71% of the overall student population. Descriptive and Qualitative Findings: COL (Relationships Construct) Over 50% of the COL student population are don’t believe or uncertain if teachers like them. 37% of COL children in grades 3-5 feel bullied as compared with 25% of the overall AI and 20% of the Caucasian populations. Descriptive and Qualitative Findings: DL (Student Behaviors Construct) Up to 20% of children do not participate in extracurricular activities due to cost and transportation issues. DL should consider reducing/eliminating participation fees and increasing extracurricular transportation to rural areas, especially for the AI student population. Descriptive and Qualitative Findings: DL (Parents/Home Life Construct) 2/3 of teachers disagree that parents help their children regularly with homework. Students report diminished help in higher grades. Homework should be used sparingly, and if used, students must be able to complete the work independently. AI absence rates are twice that of Caucasian students. DL needs to establish a plan to increase AI student attendance and evaluate the effectiveness of that plan. Descriptive and Qualitative Findings: DL (Mahnomen Self-Perception Construct) Social studies as identified as a difficult, less favorite subject than we find in other schools. Not one Mahnomen teacher identified himself/herself as the most important reason that kids do not succeed. However, 12% of Mahnomen parents identified teachers as the critical factor. Descriptive and Qualitative Findings: DL (Mahnomen Teachers Construct) An elevated percentage of Mahnomen students and parents do not believe that teachers set high expectations for students. There is a sense of tension between some parents and teachers in regard to teacher effectiveness. Mahnomen teachers underutilize technology in the classroom. Descriptive and Qualitative Findings: DL (Mahnomen Student Behaviors Construct) The percentage of Mahnomen students who complete homework on a regular basis is much lower (1 in 3) than is found in other schools. This figure increases to 1 in 2 when those who are “uncertain” are figured into the mix. Homework holds a lower value by students at Mahnomen. Extracurricular activities are more valued by Mahnomen students than is found in other school districts. Concern for student safety is elevated on school buses. Descriptive and Qualitative Findings: DL (Relationships Construct) Classmates appear to have a much greater influence on students than is found in other school districts. Descriptive and Qualitative Findings: Naytahwaush (Self-Perception Construct) Social studies is viewed as a difficult, less favorite subject than is found in most school districts. Reading is popular in grades 3-5, but declines in grades 6-8. Descriptive and Qualitative Findings: Naytahwaush (Teachers Construct) Naytahwaush teachers enjoy a much better relationship with students and parents than is found in other school districts. Naytahwaush teachers appear to do a better job with authentic education than is found in other school districts. Descriptive and Qualitative Findings: Naytahwaush (Parents/Home Life Construct) Between 8% and 14% of Naytahwaush students do not get help with homework. The rate of parents preferring TV over time with their children is several times higher in Naytahwaush than is found in other school districts. Naytahwaush children in grades 6-12 are left at home at a much higher rate than is found in other school districts. Descriptive and Qualitative Findings: Naytahwaush (Parents/Home Life Construct--Continued) Naytahwaush students have even less access to books than the overall AI student population, which has less access to books in the home than the Caucasian population. There is an elevated concern for the home life in Naytahwaush. Descriptive and Qualitative Findings: Naytahwaush (Student Behaviors Construct) The percentage of students not completing homework is higher (1 in 3) in Naytahwaush than is found in most other districts. Very few Naytahwaush students participate in extracurricular activities. Free time usage is a concern. No students in grades 6-8 prefer reading over TV. Descriptive and Qualitative Findings: Naytahwaush (School Environment Construct) Naytahwaush students like school lunch more than students in most school districts. Elevated concern for safety is present in school and on buses. Descriptive and Qualitative Findings: Naytahwaush (Relationships Construct) Naytahwaush students in grades 3-5 reported not liking their classmates or not being liked by classmates at a higher rate than the overall AI and Caucasian student populations. Naytahwaush children report being bullied or not sure if they are bullied at a rate of more than 3 times that of the overall AI and Caucasian student populations. Two-thirds of parents report that students feel bullied at Naytahwaush. Descriptive and Qualitative Findings: Naytahwaush (Relationships Construct--Continued) Classmates influence other students at a rate higher than the overall student population. Adults at Naytahwaush influence students at a rate twice that of the overall AI and Caucasian student populations. Descriptive and Qualitative Findings: Pine Point (Self-Perception Construct) Students, teachers, and parents believe that PP students are good at math. Social studies and science are not viewed as easy for PP students. Descriptive and Qualitative Findings: Pine Point (Teachers Construct) PP teachers, students, and parents seem to enjoy an overall good relationship Teachers at PP seem to do a better job with authentic education than most school districts. Descriptive and Qualitative Findings: Pine Point (Parents/Home Life Construct) Students at PP like homework as a means to learn at a much higher rate than the overall AI and Caucasian student populations. PP children report violent behavior at a much higher rate than children in other school districts. PP children report parental lack of interest in spending time with them at night at a much higher rate than the other school districts. PP children have far less access to books than is found in most other school districts. Descriptive and Qualitative Findings: Pine Point (Student Behaviors Construct) A majority of PP children do not participate in extracurricular activities. PP children spend far more free time watching TV and playing violent video games than do other student populations in this study. Descriptive and Qualitative Findings: Pine Point (School Environment Construct) A larger percentage of students are dissatisfied with their schedules than is found in other school districts. Concern for safety at school and on buses is reported at a higher rate for PP students than found in most other school districts. Descriptive and Qualitative Findings: Pine Point (Relationships Construct) PP students in all grades report not liking their classmates or classmates not liking them at a higher rate than the overall Caucasian and AI student populations. School officials should work to formalize what would be known as hidden curriculum. PP students report feeling bullied and/or not sure if they are bullied at a rate of approximately 3 times that of the overall Caucasian and AI student populations Descriptive and Qualitative Findings: Pine Point (Relationships Construct--Continued) Classmates influence PP students at a rate higher than the overall Caucasian and AI student populations. Adults in school at PP influence student behavior at a rate much higher than the overall Caucasian and AI student populations. Descriptive and Qualitative Findings: WOWE (Self-Perception Construct) Although math is a concern for the overall Caucasian and AI populations, many students at W like math and/or feel that they are good at it. However, about 1/3 of the W student population find math to be hard. Social studies and science are not viewed as easy for children at W. Descriptive and Qualitative Findings: WOWE (Parents/Home Life Construct) Although most students report that their parents help them with homework, up to 1 in 4 students at W would disagree. Far fewer AI children have access to books than do White children. In addition, the overall W student population would appear to have less access to books in the home than do students populations associated with other schools in this study. Descriptive and Qualitative Findings: WOWE (Student Behaviors Construct) The percentage of W students who do not complete homework on a regular basis is much higher than the overall AI and Caucasian populations. The lack of participation in extracurricular activities appears to be tied to limited opportunities, a lack of transportation, and a lack of money. W children prefer spending more time on the subject during the school day than doing homework. Descriptive and Qualitative Findings: WOWE (School Environment Construct) Although a fear for safety is of concern to a limited percentage of the student population, W students report this concern at a higher rate (up to 1/3 do not feel safe or are uncertain) both in school and on buses. A far larger percentage of W children do not like their schedules during the day, as compared with the overall Caucasian and AI student populations Descriptive and Qualitative Findings: WOWE (Relationships Construct) W students do not like their teachers or are uncertain as to whether they like their teachers at the grades 612 levels at a rate (40%) much higher than the overall Caucasian and AI student populations. In addition, around 15% of children report that they do not like paraprofessionals or are not liked by paraprofessionals. Recommendations: Quantitative Data Administrators: Explore differences between the MCA and MAP tests (e.g., structure, content, surrounding events at the time of testing) Be aware that MAP scores do not predict MCA scores Improve academic performance data collection system Ensure immediate access of test results data to teachers and parents Consider developing self-enhancement interventions for students, young students in particular Evaluate the current interventions for absenteeism and tardiness Teachers Elementary Grades: Students in the 3rd grade need reinforcement in math (MCA, partially meeting standards) Students in 3rd and 5th grade need reinforcement in reading (MCA, partially meeting standards) Analyze level of academic challenge (differentiated instruction) as higher levels of disciplinary problems are observed among students with higher reading performances. Adjust expectations. Likewise, higher academic progress in math is observed among students with higher rates of tardiness. Adjust expectations Be aware that disciplinary problems are found among students who are absent more frequently and have a negative perception of relationships in school Be aware that higher academic achieving young students have a negative appraisal of their teachers Reinforce academic performance in the early grades, support transition to middle and secondary grades Develop activities that support the improvement of young student’s perception of own behavior Do not use MAP test as predictor of MCA performance Curriculum-Based Measurements Teachers Middle and Secondary Grades Develop an intervention that impacts on students perception of: Own behavior School environment Relationships (i.e., students with peers, students with teachers, students with administrators) Parents as these are associated with academic performance Place special attention to students in 7th and 12th grades as 2/3 of these students are failing the MCA test in math Be aware that students’ positive perceptions of parents and home environment is associated to better academic performance Be aware that absenteeism and disciplinary problems are more prevalent among students showing lower reading performance Develop awareness of the association that a positive perception of own behavior has with academic performance. To great extent students’ perception of self is influenced by what students hear teachers say about their students. Parents of Young Students: Be aware that even though absenteeism and tardiness at this young age is positively associated with academic achievement, absenteeism and tardiness have a negative association to appraisal of parents and home life Parents of Older Students: Develop awareness of the strong association that a positive parental perception has with academic performance Develop awareness of the association that a positive perception of school has on academic performance. To great extent school perception is influenced by what students hear parents say about school. Be aware that absenteeism, tardiness, and disciplinary problems are observed among students who are in need of significant academic improvement Recommendations: Descriptive and Qualitative Data Recommendations: Qualitative (Self-perception Construct: General) Reading: Pre-assessment is critical at all levels. Leveling Differentiation Math is found to be difficult by many students, and it is the least-liked subject overall. Alternative approaches to teaching math Authentic education and assessment Allow re-takes of summative assessments (applicable to all subjects) AI parents do not view attendance as critical to academic success as do White parents. Work with AI parents to get their children to school on time. Recommendations: Qualitative (Teacher Construct: General) Teacher capacity to explain things was identified as critical. Uniform methodological approach across all grades and subjects. Madeline Hunter’s direct instruction is recommended. Standards (outcomes) Anticipatory Set (Hook) Teaching (input, modeling, checking for understanding) Guided Practice/Monitoring Closure Independent Practice Recommendations: Qualitative (Teacher Construct: General) There must be a cultural shift in thinking among teachers. Teachers do not see themselves as a critical factor in student underachievement. School administrators must stand accountable for teacher implementation of recommendations. Evaluations Professional Development Recommendations: Qualitative (Parents/Home Life Construct: General) All children should have access to breakfast, regardless of free/reduced lunch status. Get books into the homes of American Indian children. Increase teacher/parent communication, especially with American Indian population. Recommendations: Qualitative (Student Behavior Construct: General) Homework should be assigned sparingly Up to 1/3 of students do not complete homework when it is assigned. A majority of teachers would disagree that most parents help their children with homework or check for completion. Extracurricular fees should be reduced or eliminated. Up to 1 in 5 children do not participate due to cost and transportation. Recommendations: Qualitative (School Environment Construct: General) Spend more time on core subjects in school. Children are tired in school. Work with parents to encourage consistent sleep schedules. Students should complete more work in class, under the guidance of a licensed teacher. Many children do not or cannot do homework that is assigned due to various reasons, including poor pedagogy. A majority of students both prefer and feel that they would learn more if they could complete work under the guidance of a teacher as opposed to homework. Final Thoughts Contact Information Ximena Suarez-Sousa, Ph.D., 1104 7th Avenue South Moorhead, MN 56563, 218-477-2095, suarez@mnstate.edu Boyd Bradbury, Ph.D., 1104 7th Avenue South Moorhead, MN 56563, 218-477-2095, bradbury@mnstate.edu