Collegiate Assessment of Academic Proficiency: Mathematics Test Spring 2011 Collegiate Assessment of Academic Proficiency: Mathematics Test Spring 2011 Administration Office of Planning and Assessment, Devin DuPree and Sabrina Sattler, March 2011 Page 1 of 36 Collegiate Assessment of Academic Proficiency: Mathematics Test Spring 2011 Table of Contents Executive Summary....................................................................................................................................... 3 Introduction .................................................................................................................................................. 5 Sample....................................................................................................................................................... 6 Results ........................................................................................................................................................... 9 CAAP Scores Overall .................................................................................................................................. 9 CAAP Scores by Sex, Major, GPA, and Effort .......................................................................................... 13 Additional Questions Scores Overall....................................................................................................... 16 Additional Questions Scores by Sex, Major, GPA, and Effort ................................................................. 17 CAAP and Additional Questions Scores by Core Mathematics Course ................................................... 19 Relationship between CAAP Scores and Additional Questions Scores ................................................... 20 Conclusion ................................................................................................................................................... 22 Appendix ..................................................................................................................................................... 23 Appendix A: Information about the CAAP Mathematics Test ............................................................... 23 Appendix B: Copy of the Additional Questions....................................................................................... 25 Appendix C: Analysis of CAAP Scores ...................................................................................................... 28 Appendix D: Analysis of Additional Questions ........................................................................................ 32 Appendix E: Analysis of CAAP and Additional Questions Scores by Core Curriculum Questions ........... 34 Appendix F: Effort by Student Classification, Sex, College, Major, and GPA .......................................... 36 Office of Planning and Assessment, Devin DuPree and Sabrina Sattler, March 2011 Page 2 of 36 Collegiate Assessment of Academic Proficiency: Mathematics Test Spring 2011 Executive Summary The Collegiate Assessment of Academic Proficiency (CAAP) Mathematics Test is designed to measure skills in mathematics. In coordination with the various Texas Tech University (TTU) Colleges, the Office of Planning and Assessment administered the Mathematics Test in select courses across campus at the beginning of the spring 2011 semester. Courses were chosen to achieve a representative number from each College based on enrollment of juniors and seniors. ACT allows the administering institution to add up to nine additional questions to the chosen CAAP module. TTU chose to add the five Mathematics questions from the Online Senior Assessment (OSA) plus two questions about participants’ completion of their Mathematics Core Curriculum Requirement. A total of 518 TTU students participated in the spring 2011 administration of the CAAP Mathematics Test. The goal was to score tests from 200 juniors and 200 seniors, so 118 answer sheets were excluded due to participants’ answer sheet irregularities, student classification, or college. A final sample of 175 juniors and 225 seniors is included in this analysis. The final sample seems to be a fairly good representation of the population of all TTU juniors and seniors in terms of sex, ethnicity, and college. In comparing scores from the CAAP Mathematics Test, it appears that TTU juniors (mean = 60.7, SD = 4.4) score higher on average than juniors nationally (mean = 58.0, SD = 4.3) and that TTU seniors (mean = 59.6, SD = 4.0) score higher than seniors nationally (mean = 56.8, SD = 4.1). These differences are statistically significant at the 0.05 level. The TTU sample shows statistically significant patterns in CAAP scores by sex, major, GPA, and effort. Male participants score higher on average than female participants. Math majors score higher on average than other majors. Participants with higher GPA’s and participants reporting more effort also score higher on average. These same patterns apply to the additional questions scores by sex, major, and effort. Male participants, Math majors, and participants reporting more effort score higher on average. For the seniors, higher GPA’s also predict higher additional questions scores, but this pattern by GPA was not statistically significant for the juniors. Both participants’ self report of whether they have completed their mathematics core courses and participants’ self report of whether they have completed their mathematics core courses at TTU do not have a statistically significant impact on CAAP scores or additional questions scores. The lack of significant comparisons by mathematics core completion may be due to the small sample sizes of juniors and seniors reporting that they have not completed their mathematics core courses. It might also be due to mistakes in students’ self report of completion. It could also be that the not completed group represents mostly students who have completed one of the two required math courses and that having the extra math course does not improve students’ scores on the CAAP or the additional questions. Office of Planning and Assessment, Devin DuPree and Sabrina Sattler, March 2011 Page 3 of 36 Collegiate Assessment of Academic Proficiency: Mathematics Test Spring 2011 The analysis shows a correlation between the CAAP scores and the additional questions scores of 0.55 (p value < 0.0001). This correlation suggests that there is a significant relationship between the additional questions scores and CAAP scores. This relationship may suggest that the CAAP and the additional questions are measuring similar domains of knowledge or that they are in some other way related. Office of Planning and Assessment, Devin DuPree and Sabrina Sattler, March 2011 Page 4 of 36 Collegiate Assessment of Academic Proficiency: Mathematics Test Spring 2011 Introduction The Collegiate Assessment of Academic Proficiency (CAAP) “is the standardized, nationally normed assessment program from ACT that enables postsecondary institutions to assess, evaluate, and enhance student learning outcomes and general education program outcomes.” 1 The CAAP offers six different modules: Reading, Writing Skills, Writing Essay, Mathematics, Science, and Critical Thinking. 2 As per decision of the University Assessment Committee in fall 2010, the Office of Planning and Assessment administered the Mathematics Test of the CAAP during spring 2011. The Mathematics Test “is a 35-item, 40-minute test designed to measure students' proficiency in mathematical reasoning. The test assesses students’ proficiency in solving mathematical problems encountered in many postsecondary curricula. It emphasizes quantitative reasoning rather than the memorization of formulas.” 3 Please find more information on how the test is constructed and scored in Appendix A. ACT allows the administering institution to add up to nine additional questions to the chosen module. The Core Curriculum Committee (CCC) at Texas Tech University (TTU) has been working on its own instrument to measure student abilities in various core areas, the Online Senior Assessment (OSA). The OSA was designed in 2008 and has been administered to graduating seniors from Texas Tech during the spring semester of 2008, 2009, and 2010. During the summer of 2010 the same assessment was administered to incoming transfer students at Texas Tech as the Transfer Student Assessment (TSA). The instrument has one section for each of the following areas: Humanities, Multicultural, Mathematics, Natural Sciences, Technology and Applied Science, and Social and Behavioral Sciences. The Mathematics section of the OSA is designed to measure abilities that are also measured by the CAAP Mathematics Test. Since the CAAP Mathematics Test is an established instrument it was decided to use the five Mathematics questions from the OSA as additional questions and compare the results. Another two questions were added as additional questions to ask participants if they have completed their Mathematics core requirement courses and if they completed all of these core requirement courses at TTU (see Appendix B for a copy of the additional questions). In coordination with the various Colleges, the Office of Planning and Assessment administered the Mathematics Test in select courses across campus. Courses were chosen to achieve a representative number from each College based on enrollment of juniors and seniors. Juniors and seniors were the target group, so courses with high junior and senior enrollment were selected to participate. Larger courses were chosen from Colleges with higher junior and senior enrollment and smaller courses were chosen from Colleges with lower junior and senior enrollment. Included in the sample were courses from the following Colleges: College of Agricultural Sciences and Natural Resources (AG), College of Architecture (AR), College of Arts 1 http://www.act.org/caap/, accessed 3/2/2011 http://www.act.org/caap/test_modules.html, accessed 3/2/2011 3 http://www.act.org/caap/test_math.html, accessed 3/2/2011 2 Office of Planning and Assessment, Devin DuPree and Sabrina Sattler, March 2011 Page 5 of 36 Collegiate Assessment of Academic Proficiency: Mathematics Test Spring 2011 and Sciences (AS), College of Education (ED), College of Human Sciences (HS), College of Mass Communications (MC), College of Visual and Performing Arts (VP), Edward E. Whitacre Jr. College of Engineering (EN), and Jerry S. Rawls College of Business Education (BA). The CAAP Mathematics Test and the additional questions were administered during the regularly scheduled class time. This allowed most students from each class to participate and helped reduce the bias that may have been created if students had been asked to volunteer to take the survey on their own time instead. Sample A total of 518 Texas Tech University (TTU) students participated in the spring 2011 administration of the CAAP Mathematics Test. The goal was to score tests from 200 juniors and 200 seniors. The Office of Planning and Assessment identified 4 irregularities (e.g., pattern scoring) that were exempt from scoring. Also exempt from scoring were the 97 participants of student classifications other than juniors and seniors. Since there were only 175 juniors in the remaining sample, it was decided to score the results of 225 seniors instead of just 200 to achieve a good campus-wide representation. The remaining 400 answer sheets were scored by ACT. Since the students’ R numbers were used as the student ID on the Mathematics Test, the Office of Planning and Assessment was able to obtain demographic information for the participants from Institutional Research (IR). The demographic information obtained includes sex, ethnicity, student classification, and college. While participants were asked similar questions about their background on the CAAP Mathematics Test, it was decided to use official registration data provided by IR. Using the IR data facilitates comparison to the population numbers and has a greater chance of being accurate. The following graphs compare the final junior and senior samples to the population of all TTU juniors and seniors, respectively, by sex, ethnicity, and college. Sample and Population by Sex: Juniors 60.0% 50.0% 40.0% 30.0% 20.0% 10.0% 0.0% Sample Population Female Male Sample and Population by Sex: Seniors 60.0% 50.0% 40.0% 30.0% 20.0% 10.0% 0.0% Sample Population Female Male Office of Planning and Assessment, Devin DuPree and Sabrina Sattler, March 2011 Page 6 of 36 Collegiate Assessment of Academic Proficiency: Mathematics Test Spring 2011 The junior and senior samples both appear to be good representations of their respective populations in terms of sex. Sample and Population by Ethnicity: Juniors 80.0% 70.0% 60.0% 50.0% 40.0% 30.0% 20.0% 10.0% 0.0% Sample Population Sample and Population by Ethnicity: Seniors 80.0% 70.0% 60.0% 50.0% 40.0% 30.0% 20.0% 10.0% 0.0% Sample Population The junior and senior samples both appear to be good representations of their respective populations in terms of ethnicity. Office of Planning and Assessment, Devin DuPree and Sabrina Sattler, March 2011 Page 7 of 36 Collegiate Assessment of Academic Proficiency: Mathematics Test Spring 2011 Sample and Population by College: Juniors 40.0% 35.0% 30.0% 25.0% 20.0% Sample 15.0% Population 10.0% 5.0% 0.0% AG AR AS BA ED EN HS MC UC VP Sample and Population by College: Seniors 35.0% 30.0% 25.0% 20.0% Sample 15.0% Population 10.0% 5.0% 0.0% AG AR AS BA ED EN HS MC UC VP It appears that the junior sample includes more participants from the Whitacre College of Engineering and fewer participants from the College of Human Sciences than would be expected from the population. The senior sample appears to be a fairly good representation of the population in terms of College. Office of Planning and Assessment, Devin DuPree and Sabrina Sattler, March 2011 Page 8 of 36 Collegiate Assessment of Academic Proficiency: Mathematics Test Spring 2011 Results CAAP Scores Overall Three scores are reported for the CAAP Mathematics Test: total test score (35 items), basic algebra subscore (17 items), and college algebra subscore (18 items). Potential total test scores range from 40 (low) to 80 (high) and potential subscores range from 5 (low) to 25 (high). The following table gives a summary of these three scores for the TTU junior and senior samples. Summary of CAAP Scores n Mean Standard Deviation Minimum Median Maximum Total Test Juniors Seniors 175 225 60.7 59.6 4.4 4.0 49 49 60 59 73 71 Basic Algebra Juniors Seniors 175 225 16.9 16.5 2.4 2.3 9 9 17 16 20 20 College Algebra Juniors Seniors 175 225 16.9 15.9 3.2 3.0 11 10 17 16 24 24 It appears that the TTU juniors scored slightly better on average than the TTU seniors in total test scores (60.7 vs. 59.6), basic algebra scores (16.9 vs. 16.5), and college algebra scores (16.9 vs. 15.9). The differences in total test scores and college algebra scores are statistically significant at the 0.05 level and the difference in basic algebra scores is statistically significant at the 0.10 level (see Appendix C for details). This suggests that TTU juniors score higher on average than TTU seniors on the CAAP Mathematics Test. This pattern of juniors performing better on average than seniors is similar to CAAP participants nationally.4 4 http://act.org/caap/norms/pdf/10Table12.pdf, http://act.org/caap/norms/pdf/10Table13.pdf, accessed 3/2/2011 Office of Planning and Assessment, Devin DuPree and Sabrina Sattler, March 2011 Page 9 of 36 Collegiate Assessment of Academic Proficiency: Mathematics Test Spring 2011 The following graphs show the distributions of total test scores, basic algebra scores, and college algebra scores for our junior and senior samples. CAAP Total Test Scores Percent of Sample 60.0% 52.0% 50.0% 44.0% 40.0% 29.1% 28.0% 30.0% 20.0% 9.1%10.2% 10.0% Juniors 14.9% 8.4% Seniors 1.7%0.4% 1.1%0.9% 0.0% 45 50 55 60 65 70 75 80 Total Test Score The total test score distributions appear similar with a few more seniors scoring between 55 and 60 and a few more juniors scoring between 65 and 70. CAAP Basic Algebra Scores Percent of Sample 60.0% 51.4% 50.0% 41.3% 43.6% 40.0% 28.0% 30.0% Juniors 20.0% 9.7% 9.1% 8.9% 10.0% 1.7% 0.9% Seniors 5.3% 0.0% 0.0% 0.0% 7 10 13 16 19 22 25 Basic Algebra Score The basic algebra score distributions also appear similar with a few more seniors scoring between 13 and 16 and a few more juniors scoring between 16 and 22. Office of Planning and Assessment, Devin DuPree and Sabrina Sattler, March 2011 Page 10 of 36 Collegiate Assessment of Academic Proficiency: Mathematics Test Spring 2011 CAAP College Algebra Scores 35.0% 31.1% 29.1% 27.6% 26.9% 24.9% Percent of Sample 30.0% 25.0% 19.4% 18.9% 20.0% 13.3% 15.0% 10.0% Juniors Seniors 5.7% 5.0% 2.2% 0.0% 0.9% 0.0% 7 10 13 16 19 22 25 College Algebra Score The college algebra score distributions include a few more seniors in each category of scores below 16 and a few more juniors in each category of scores above 16. It appears that the college algebra items contribute more than the basic algebra items to the difference in total test scores for TTU juniors and seniors. The ACT website has a summary for the scores of 12,744 juniors from 122 institutions and 9,402 seniors from 115 institutions who participated in the Mathematics Test in the fall of 2010.5 On average the TTU juniors scored higher than the national sample of juniors in total test scores (60.7 vs. 58.0), basic algebra scores (16.9 vs. 15.3), and college algebra scores (16.9 vs. 15.1).6 On average the TTU seniors scored higher than the national sample of seniors in total test scores (59.6 vs. 56.8), basic algebra scores (16.5 vs. 14.6), and college algebra scores (15.9 vs. 14.3).7 Each of these comparisons is statistically significant at the 0.05 level (see Appendix C for details). This suggests that TTU juniors and seniors score higher on average than juniors and seniors nationally on the CAAP Mathematics Test. 5 http://act.org/caap/norms/, accessed 3/2/2011 http://act.org/caap/norms/pdf/10Table12.pdf, accessed 3/2/2011 7 http://act.org/caap/norms/pdf/10Table13.pdf, accessed 3/2/2011 6 Office of Planning and Assessment, Devin DuPree and Sabrina Sattler, March 2011 Page 11 of 36 Collegiate Assessment of Academic Proficiency: Mathematics Test Spring 2011 The following table shows the number of TTU juniors and seniors who received each total test score and the national percentile for each of these scores.8 For each score, the national percentile shows the percentage of junior or senior participants, respectively, from the national sample that scored at or below that score. The highlighted cells show those TTU participants who scored higher than the 50th national percentile. A majority of the TTU juniors (139 participants, 79%) and a large majority of the TTU seniors (185 participants, 82.2%) scored above the 50th percentile nationally. National Percentiles for CAAP Scores Score TTU Juniors Natl. % TTU Seniors Natl. % 73 2 99 0 99 72 0 99 0 99 71 1 99 1 99 70 0 99 0 99 69 5 99 3 99 68 3 99 2 99 67 5 98 5 99 66 13 97 9 98 65 7 95 10 97 64 14 93 16 96 63 5 90 3 94 62 8 87 9 92 61 17 83 25 90 60 20 75 20 85 59 18 66 26 78 58 21 55 29 68 57 9 46 27 59 56 9 36 15 48 55 6 25 5 35 54 5 19 7 28 53 2 15 4 21 52 1 10 3 15 51 50 49 2 1 1 7 4 2 4 1 1 10 6 3 8 http://act.org/caap/norms/pdf/10Table12.pdf, http://act.org/caap/norms/pdf/10Table13.pdf, accessed 3/2/2011 Office of Planning and Assessment, Devin DuPree and Sabrina Sattler, March 2011 Page 12 of 36 Collegiate Assessment of Academic Proficiency: Mathematics Test Spring 2011 CAAP Scores by Sex, Major, GPA, and Effort The following graphs show the average CAAP total test scores by sex, major, GPA, and participants’ self report of how much effort they gave in taking the assessment (See Appendix C for sample sizes). Average CAAP Scores by Sex 80.0 75.0 70.0 65.0 60.0 55.0 50.0 45.0 40.0 59.8 58.3 61.3 60.8 Juniors Seniors Female Male For both the juniors and the seniors the male participants scored higher on average than the female participants. These differences by sex are statistically significant at the 0.05 level (see Appendix C for details). This suggests that on average male TTU juniors and seniors score higher than female TTU juniors and seniors on the CAAP Mathematics Test. Average CAAP Scores by Major 80.0 75.0 70.0 65.0 60.0 55.0 50.0 45.0 40.0 65.3 63.8 59.5 58.8 Juniors Seniors Math Major Other Major For both the juniors and the seniors the Math majors scored higher on average than the other majors. The Math majors group includes participants majoring in mathematics or other mathematics intense majors (e.g., Physics or Engineering). These differences by major are statistically significant at the 0.05 level (see Appendix C for details). This suggests that on average TTU juniors and seniors with a Math major score higher than TTU juniors and seniors with another major on the CAAP Mathematics Test. Office of Planning and Assessment, Devin DuPree and Sabrina Sattler, March 2011 Page 13 of 36 Collegiate Assessment of Academic Proficiency: Mathematics Test Spring 2011 Average CAAP Scores by GPA 80.0 75.0 70.0 65.0 60.0 55.0 50.0 45.0 40.0 59.0 56.7 59.2 59.8 58.6 59.0 60.3 63.4 59.6 60.9 Juniors Seniors below 2.00 2.01 - 2.50 2.51 - 3.00 3.01 - 3.50 3.51 and above It appears that on average the juniors and seniors with higher GPA’s scored higher than the juniors and seniors with lower GPA’s. These differences in scores by GPA are statistically significant at the 0.05 level (see Appendix C for details). More specifically, the “3.51 and above” group scored significantly higher than the “3.01 – 3.50” group and the “2.51 – 3.00” group for the juniors, and the “3.51 and above” group scored significantly higher than the “2.51 – 3.00” group for the seniors. The results suggest that on average TTU juniors and seniors with higher GPA’s score higher on the CAAP Mathematics Test. Average CAAP Scores by Effort 80.0 75.0 70.0 65.0 60.0 55.0 50.0 45.0 40.0 62.6 61.0 59.8 58.8 57.3 58.3 53.7 53.2 Juniors Seniors tried my best gave moderate effort gave little effort gave no effort For both the juniors and seniors it appears that on average participants who reported more effort on the assessment scored higher. These differences between the groups are statistically significant at the 0.05 level (see Appendix C for details). More specifically, the “tried my best” group scored significantly higher than the other three groups for the juniors and seniors, and the “gave moderate effort” and “gave little effort” groups scored significantly higher than the “gave no effort” group for the seniors. The results suggest that on average TTU juniors and seniors who put forth more effort on the CAAP Mathematics Test score higher. Considering the impact of participants’ effort on CAAP scores, this section of the report will consider if participants’ self report of effort can help explain some of the other patterns found Office of Planning and Assessment, Devin DuPree and Sabrina Sattler, March 2011 Page 14 of 36 Collegiate Assessment of Academic Proficiency: Mathematics Test Spring 2011 in CAAP scores. The following table gives a summary of participants’ self report of effort by other demographic variables considered in this report. For ease of comparison, the percent of participants selecting “tried my best” and “gave moderate effort” are combined in one percentage and the percent of participants selecting “gave little effort” or “gave no effort” are combined in one percentage (see appendix F for more details). Effort by Student Classification (combined groups) Juniors Seniors Effort by Sex (combined groups) Female Male Effort by Major (combined groups) Math Major Other Major Effort by GPA (combined groups) below 2.00 2.01 - 2.50 2.51 - 3.00 3.01 - 3.50 3.51 and above best or moderate effort 92.1% 85.1% Juniors best or little or no moderate effort effort 95.5% 4.5% 89.5% 10.5% best or little or no moderate effort effort 97.1% 2.9% 90.7% 9.3% best or little or no moderate effort effort 80.0% 20.0% 80.0% 20.0% 83.0% 17.0% 100.0% 0.0% 100.0% 0.0% Little or no effort 7.9% 14.9% Seniors best or moderate effort 87.5% 82.9% best or moderate effort 87.9% 84.5% best or moderate effort 100.0% 90.9% 82.4% 87.5% 81.8% little or no effort 12.5% 17.1% little or no effort 12.1% 15.5% little or no effort 0.0% 9.1% 17.6% 12.5% 18.2% The junior participants were more likely to report more effort than the senior participants. This might help explain the higher average score for junior participants. However, the female juniors and seniors were more likely to report more effort than the male juniors and seniors. Since male participants performed better on average than female participants, this would suggest that effort does not help explain the differences in CAAP scores by sex. The junior and senior math majors were more likely to report more effort than the juniors and seniors with other majors. This might help explain the higher average scores for math majors. The patterns in reported effort by GPA seem to be consistent with the patterns in CAAP scores by GPA for the juniors, but not for the seniors. This inconsistency might be due to the small sample sizes included in some of the GPA groups. Office of Planning and Assessment, Devin DuPree and Sabrina Sattler, March 2011 Page 15 of 36 Collegiate Assessment of Academic Proficiency: Mathematics Test Spring 2011 Additional Questions Scores Overall The following table and graph give a summary of the sample’s scores on the additional questions (i.e., the five questions from the mathematics section of the Online Senior Assessment (OSA)). The 19 participants who did not answer any of the additional questions are excluded from this part of the analysis. Summary of Additional Questions Scores Seniors Juniors n 166 215 Mean 55.3% 55.3% Standard Deviation 27.4% 27.2% Minimum 0% 0% Median 60% 60% Maximum 100% 100% Additional Questions Scores 28.3% Percent of Sample 30.0% 24.2% 25.0% 23.3% 19.9% 20.0% 21.7%20.9% 16.3% 13.9% 15.0% 10.2% 10.0% 5.0% 6.0% 11.6% Juniors Seniors 3.7% 0.0% 0% 20% 40% 60% 80% 100% Additional Questions Score It appears that on average the juniors and seniors performed very similarly on the additional questions. Both the juniors and seniors answered, on average, between two and three of the five additional questions correctly, with a low score of zero questions answered correctly and a high score of all 5 questions answered correctly. When considering the scores for the additional questions it is important to note that these questions were answered at the end of the testing period (i.e., after the participants had already taken the 40-minute-long Mathematics Test. This might explain the fact that the overall average score of these questions from this administration (55.3%) is lower than the average score from the last administration of these questions with the OSA (62.3%).9 9 http://www.depts.ttu.edu/opa/assessment/OSA_2010_Mathematics.pdf, accessed 3/2/2011 Office of Planning and Assessment, Devin DuPree and Sabrina Sattler, March 2011 Page 16 of 36 Collegiate Assessment of Academic Proficiency: Mathematics Test Spring 2011 Additional Questions Scores by Sex, Major, GPA, and Effort The following graphs show the average additional questions (AQ) scores by sex, major, GPA, and participants’ self report of how much effort they gave in taking the assessment (See Appendix D for sample sizes). Average AQ Scores by Sex 80.0% 70.0% 60.0% 50.0% 40.0% 30.0% 20.0% 10.0% 0.0% 60.0% 49.5% 63.8% 45.4% Juniors Seniors Female Male For both the juniors and the seniors the male participants scored higher on average than the female participants. These differences are statistically significant at the 0.05 level (see Appendix D for details). This suggests that on average male TTU juniors and seniors score higher than female TTU juniors and seniors on these additional questions from the mathematics section of the OSA. This is the same pattern that was found with the CAAP scores. Average AQ Scores by Major 80.0% 70.0% 60.0% 50.0% 40.0% 30.0% 20.0% 10.0% 0.0% 72.0% 70.6% 50.8% 52.2% Juniors Seniors Math Major Other Major For both the juniors and the seniors the Math majors scored higher on average than the other majors. These differences by major are statistically significant at the 0.05 level (see Appendix D for details). This suggests that on average TTU juniors and seniors with a Math major score higher than TTU juniors and seniors with another major on these additional questions from the mathematics section of the OSA. This is the same pattern that was found with the CAAP scores. Office of Planning and Assessment, Devin DuPree and Sabrina Sattler, March 2011 Page 17 of 36 Collegiate Assessment of Academic Proficiency: Mathematics Test Spring 2011 Average AQ Scores by GPA 80.0% 53.3% 60.0% 40.0% 63.3% 61.7% 60.0% 51.4% 52.6% 68.6% 54.0% 52.3% 37.8% Juniors Seniors 20.0% 0.0% below 2.00 2.01 - 2.50 2.51 - 3.00 3.01 - 3.50 3.51 and above It appears that on average the seniors with higher GPA’s scored higher than the seniors with lower GPA’s. These differences by GPA group for the seniors are statistically significant at the 0.05 level (see Appendix D for details). More specifically, the “3.51 and above” group scored significantly higher than the “3.01 – 3.50” group, the “2.51 – 3.00” group, and the “below 2.00” group for the seniors. The results suggest that on average TTU seniors with higher GPA’s score higher on the additional questions from the mathematics section of the OSA. This is the same pattern that was found for the seniors with the CAAP scores. It appears that on average the juniors from the highest GPA group and the lowest GPA group scored higher than the juniors from the middle GPA groups. This abnormal pattern is likely due in part to the small sample size of juniors in the lowest GPA group (n = 5). These differences by GPA group for the juniors are not statistically significant at the 0.05 level (see Appendix D for details). Average AQ Scores by Effort 80.0% 60.0% 54.9% 60.5% 59.7% 53.8% 40.0% 45.2% 44.0% 27.5% 20.0% Juniors 6.7% Seniors 0.0% tried my best gave moderate effort gave little effort gave no effort The question about effort applied directly to the CAAP Mathematics Test and not the additional questions, but it appears that the responses reflect effort in answering the additional questions as well, especially for the junior participants. It appears that on average the participants who reported more effort on the CAAP scored higher on the additional questions. These differences between the groups are statistically significant at the 0.05 level for the juniors and statistically significant at the 0.10 level for the seniors (see Appendix D for details). More specifically, the “tried my best” and “gave moderate effort” groups scored significantly higher than the “gave little effort” and “gave no effort” groups for the juniors. The results suggests that on average Office of Planning and Assessment, Devin DuPree and Sabrina Sattler, March 2011 Page 18 of 36 Collegiate Assessment of Academic Proficiency: Mathematics Test Spring 2011 TTU juniors and seniors who put forth more effort on the CAAP Mathematics Test also score higher on the additional questions from the OSA. This is the same pattern that was found for the CAAP scores. CAAP and Additional Questions Scores by Core Mathematics Course Also included with the five additional questions from the mathematics section of the OSA were two additional questions asking participant if they had completed their mathematics core courses and if they completed all of their mathematics core courses at TTU. These questions allow us to see if having taken mathematics core courses and if having taken these core courses at TTU seems to make a difference in participants’ CAAP scores and Additional Questions scores. The following graphs show these comparisons. Have you completed your mathematics core courses? Have you completed your mathematics core courses? 80% 70.0 60.0 61.0 59.7 59.5 59.1 Juniors Seniors 50.0 Average AQ Score Average CAAP Score 80.0 40.0 56% 48% 40% Juniors 20% Seniors No Yes Did you complete all of your mathematics core courses at TTU? Average CAAP Score 55% 56% 0% Yes 80% 70 60% 60.6 59.5 60.8 59.8 50 40 No Did you complete all of your mathematics core courses at TTU? 80 60 60% 54% 54% 57% 57% Juniors 40% Juniors Seniors 20% Seniors 0% Yes No Yes No It appears that having completed mathematics core courses does not have a large impact on CAAP or AQ scores. This is true for both the junior and senior participants (see Appendix E for details). It also appears that having completed mathematics core courses at TTU does not have a large impact on participants’ CAAP or AQ scores. This is also true for both the junior and senior participants (see Appendix E for details). The lack of significant findings in CAAP and AQ Office of Planning and Assessment, Devin DuPree and Sabrina Sattler, March 2011 Page 19 of 36 Collegiate Assessment of Academic Proficiency: Mathematics Test Spring 2011 scores by mathematics core completion may be partially due to the small number of juniors (n=28) and seniors (n=20) reporting that they have not completed their mathematics core courses. There may also be problems with the participants’ self report of their core requirement completion. For example, the sample of students reporting that they have not completed their mathematics core courses includes 8 students with math intense majors. It is not likely that students with math intense majors have not completed their mathematics core courses by their junior or senior year. Another possibility is that the not completed group represents mostly students who have completed one of the two required math courses, and that having the extra math course does not create a large difference in CAAP or additional questions scores. Relationship between CAAP Scores and Additional Questions Scores The following table shows how participants’ CAAP scores correlate with their overall additional questions (AQ) scores and with each separate additional question. Correlations with CAAP Scores Correlation p - value n AQ Score 0.5479 < 0.0001 381 AQ1 0.3387 < 0.0001 381 AQ2 0.3209 < 0.0001 381 AQ3 0.3270 < 0.0001 381 AQ4 0.2945 < 0.0001 381 AQ5 0.2755 < 0.0001 381 The correlation between participants’ CAAP scores and participants’ additional questions scores (0.5479) is statistically significant at the 0.05 level. This means that there is a relationship between participants’ CAAP scores and participants’ additional questions scores. This relationship may suggest that the CAAP and the additional questions are measuring similar domains of knowledge or it might suggest that participants’ performance is in some other way related (e.g., participants who do well on tests do well on both the CAAP and the additional questions). Looking at each additional question separately, it appears that correct responses for all of the additional questions were significantly correlated with CAAP scores at the 0.05 level, but that these correlations are not as strong as the correlation with the additional questions score overall. Office of Planning and Assessment, Devin DuPree and Sabrina Sattler, March 2011 Page 20 of 36 Collegiate Assessment of Academic Proficiency: Mathematics Test Spring 2011 The following graph shows how participants’ additional questions scores relate to their CAAP scores. CAAP Scores by AQ Scores 75 70 CAAP Score 65 60 55 50 45 40 0% 20% 40% 60% 80% 100% 120% Additional Questions Score It appears that on average, participants CAAP scores increase as participants’ AQ scores increase. The red trend line shows this relationship. The following table shows a summary of the linear regression model that this trend line represents. Regression Model of CAAP Scores by Additional Questions Scores n F value p value R2 381 162.59 < 0.0001 0.3002 Variable Coefficient Stand Error p value Intercept 55.4 0.41 < 0.0001 AQ Score 8.5 0.67 < 0.0001 The table confirms that participants’ additional questions scores are significantly related to participants’ CAAP scores. The R2 value of 0.3002 reported in the table means that approximately 30% of the variance in CAAP scores can be explained by the variance in additional questions scores. This again suggests that participants’ additional questions scores and CAAP scores are related. Office of Planning and Assessment, Devin DuPree and Sabrina Sattler, March 2011 Page 21 of 36 Collegiate Assessment of Academic Proficiency: Mathematics Test Spring 2011 Conclusion Overall it appears that TTU juniors and seniors score higher on average than juniors and seniors nationally on the Mathematics Test from the CAAP. At TTU, it appears that the juniors score higher on average than the seniors. This pattern is consistent with the national samples of juniors and seniors. The results also suggest that at TTU male juniors and seniors score higher on average than female juniors and seniors. TTU juniors and seniors with a Math major score higher on average than juniors and seniors with another major. Juniors and seniors with higher GPA’s score higher on average than TTU juniors and seniors with lower GPA’s. TTU juniors and seniors who report that they put forth more effort on the assessment score higher. Patterns in participants’ self report of effort may help to explain the patterns in CAAP scores by student classification and major. The results of the additional questions from the mathematics section of the OSA, suggest that male juniors and seniors score higher than female juniors and seniors. Juniors and seniors with a Math major score higher than juniors and seniors with another major. Seniors with higher GPA’s score higher on average than seniors with lower GPA’s. There was not a statistically significant pattern in juniors’ scores by GPA, potentially due to the small number of participants in the lowest GPA group. Although not directly related to the additional questions, juniors and seniors who report that they put forth more effort on the CAAP also score higher on the additional questions. These patterns are consistent with the patterns found in CAAP scores. Participants’ self report of whether they have completed their mathematics core courses and whether they have completed their mathematics core courses at TTU did not have a significant impact on CAAP scores or additional questions scores. The lack of significant comparisons by mathematics core completion may be due to the small sample sizes of juniors and seniors reporting that they have not completed their mathematics core courses, or it might be due to mistakes in students’ self report of completion. It also might be that the not completed group represents mostly students who have completed one of the two required math courses and that having the extra math course does not improve students’ scores on the CAAP or the additional questions. The correlation between additional questions scores and CAAP scores suggests that there is a significant relationship between the two. This relationship may suggest that the CAAP and the additional questions are measuring similar domains of knowledge or that they are in some other way related (e.g., students who do well on tests do well on both the CAAP and the additional questions). The regression model using additional questions scores as the predictor for CAAP scores confirms this relationship. Office of Planning and Assessment, Devin DuPree and Sabrina Sattler, March 2011 Page 22 of 36 Collegiate Assessment of Academic Proficiency: Mathematics Test Spring 2011 Appendix Appendix A: Information about the CAAP Mathematics Test 10 10 Screenshots from http://www.act.org/caap/test_math.html (accessed on 3/2/2011) Office of Planning and Assessment, Devin DuPree and Sabrina Sattler, March 2011 Page 23 of 36 Collegiate Assessment of Academic Proficiency: Mathematics Test Spring 2011 Office of Planning and Assessment, Devin DuPree and Sabrina Sattler, March 2011 Page 24 of 36 Collegiate Assessment of Academic Proficiency: Mathematics Test Spring 2011 Appendix B: Copy of the Additional Questions Student ID: R______________________________ INSTRUCTIONS: Please turn to page 3 in your answer document and find block P, Additional Questions. Each question below corresponds to a column in block P. Read the question and then mark only one circle in each column that best describes your response. For example, if for question A you think answer option 7 is correct, you would bubble in the 7 in column A, etc. QUESTIONS Column A The core curriculum at Texas Tech requires you to have courses in mathematics. For example you could take a course in logic, college level algebra, finite math, statistics, calculus, etc. Have you completed your core curriculum requirement in Mathematics? (For example, this might include such classes at TTU as MATH 1300, MATH 1320, MATH 1330, or PHIL 2310) 0. Yes 1. No 2. I don’t know Column B Did you complete all of your courses for your core curriculum requirement in Mathematics at Texas Tech? (Answering “No” would mean that you took a dual credit class, an advanced placement class, a CLEP exam, or transferred credits from another institution for your Mathematics core curriculum requirement) 0. Yes 1. No 2. I don’t know Column C Office of Planning and Assessment, Devin DuPree and Sabrina Sattler, March 2011 Page 25 of 36 Collegiate Assessment of Academic Proficiency: Mathematics Test Spring 2011 According to the Cable News Network, the number of participants in roller-blade (in-line skating) competitions was 84% larger in 2003 as compared to 1995. The number of rollerblade competition participants from 1995 to 2003… 0. decreased by about 16%. 1. increased by about 16%. 2. almost doubled. 3. almost tripled. Column D Which of the following numbers is largest? 0. 7/12 1. 0.581 2. 0.583 3. 29/50 Column E The population of a western county is 100,000. The local development planners predict that the population will increase by 7% each year for each of the next 10 years. On the other hand, the county electoral board predicts that the population will increase by a constant factor of 7,500 each year for each of the next 10 years. Whose prediction method predicts a larger population for the county at the end of the 10-year cycle? 0. The local development planners’ method. 1. The county electoral board’s method. 2. Both methods predict the same final population at the end of the 10-year cycle. 3. There is not enough information provided to answer the question. Column F Alice is looking to rent an art studio. A realtor has shown her two potential locations. Each location is suitable in terms of lighting, available space and access to local amenities. However, the rental contracts are substantially different. The contract for the first studio, Studio A, stipulates a non-refundable deposit of $1500 with a monthly rent of $450. The contract for the second studio, Studio B, stipulates a non-refundable deposit of only $750 but a monthly rent of $525. If Alice anticipates renting a studio for one year and wants the studio whose total cost for one year is less expensive, which studio contract should she accept? 0. The contract from studio A, because the total contract is less expensive. 1. The contract from studio B, because the total contract is less expensive. 2. Either contract, since the total contract is the same for either studio. 3. There is not enough information provided to answer the question. Column G Office of Planning and Assessment, Devin DuPree and Sabrina Sattler, March 2011 Page 26 of 36 Collegiate Assessment of Academic Proficiency: Mathematics Test Spring 2011 A group of students are thinking of producing a college magazine. They produce a prototype of the magazine and conduct a small survey to compare male and female opinions of it. The following question is asked of a random sample of students: “Would you pay a dollar for this magazine?” Forty females and thirty males replied. The results of the survey are shown in the following graph: Select the answer that most accurately describes the results shown in the graph above. 0. Females are less likely than males to buy the magazine. 1. Males are less likely than females to buy the magazine. 2. Males are more likely to buy the magazine. 3. Females and males are equally likely to buy the magazine. Office of Planning and Assessment, Devin DuPree and Sabrina Sattler, March 2011 Page 27 of 36 Collegiate Assessment of Academic Proficiency: Mathematics Test Spring 2011 Appendix C: Analysis of CAAP Scores CAAP Scores by Student Classification Juniors Seniors n 175 225 Mean 60.7 59.6 St. Dev. 4.4 4.0 t value 2.52 p value 0.0122 Basic Algebra Scores by Student Classification Juniors Seniors n 175 225 Mean 16.9 16.5 St. Dev. 2.4 2.3 t value 1.69 p value 0.0914 College Algebra Scores by Student Classification Juniors Seniors n 175 225 Mean 16.9 15.9 St. Dev. 3.2 3.0 t value 3.04 p value 0.0025 Office of Planning and Assessment, Devin DuPree and Sabrina Sattler, March 2011 Page 28 of 36 Collegiate Assessment of Academic Proficiency: Mathematics Test Spring 2011 CAAP Scores: TTU vs. National Sample (Juniors) TTU National Sample n 175 12,744 Mean 60.7 58.0 St. Dev. 4.4 4.3 t value 8.25 p value < 0.0001 CAAP Scores: TTU vs. National Sample (Seniors) TTU National Sample n 225 9,402 Mean 59.6 56.8 St. Dev. 4.0 4.1 t value 10.13 p value < 0.0001 Basic Algebra Scores: TTU vs. National Sample (Juniors) TTU National Sample n 175 12,744 Mean 16.9 15.3 St. Dev. 2.4 2.6 t value 8.0935 p value < 0.0001 Basic Algebra Scores: TTU vs. National Sample (Seniors) TTU National Sample n 225 9,402 Mean 16.5 14.6 St. Dev. 2.3 2.6 t value 10.86 p value < 0.0001 College Algebra Scores: TTU vs. National Sample (Juniors) TTU National Sample n 175 12,744 Mean 16.9 15.1 St. Dev. 3.2 2.8 t value 8.43 p value < 0.0001 College Algebra Scores: TTU vs. National Sample (Seniors) TTU National Sample n 225 9,402 Mean 15.9 14.3 St. Dev. 3.0 2.5 t value 9.44 p value < 0.0001 CAAP Scores by Sex n Mean St. Dev. t value p value Office of Planning and Assessment, Devin DuPree and Sabrina Sattler, March 2011 Page 29 of 36 Collegiate Assessment of Academic Proficiency: Mathematics Test Spring 2011 (Juniors) Female Male 77 98 59.8 61.3 3.8 4.8 2.26 0.0249 CAAP Scores by Sex (Seniors) Female Male n 105 120 Mean 58.3 60.8 St. Dev. 3.9 3.7 t value 4.90 p value < 0.0001 CAAP Scores by Major (Juniors) Math Major Other Major n 36 139 Mean 65.3 59.5 St. Dev. 3.6 3.8 t value 8.26 p value < 0.0001 CAAP Scores by Major (Seniors) Math Major Other Major n 37 188 Mean 63.8 58.8 St. Dev. 3.0 3.6 t value 8.04 p value < 0.0001 Office of Planning and Assessment, Devin DuPree and Sabrina Sattler, March 2011 Page 30 of 36 Collegiate Assessment of Academic Proficiency: Mathematics Test Spring 2011 CAAP Scores by GPA (Juniors) A) below 2.00 B) 2.01 - 2.50 C) 2.51 - 3.00 D) 3.01 - 3.50 E) 3.51 and above n 8 6 60 48 45 Mean 58.9 58.8 59.2 60.6 63.2 St. Dev. 3.1 5.5 4.2 3.1 4.9 F value 6.82 p value < 0.0001 CAAP Scores by GPA (Seniors) A) below 2.00 B) 2.01 - 2.50 C) 2.51 - 3.00 D) 3.01 - 3.50 E) 3.51 and above n 9 12 92 72 39 Mean 59.6 60.6 58.6 59.6 61.5 St. Dev. 3.8 4.3 3.6 4.0 4.2 F value 4.16 p value 0.0029 CAAP Scores by Effort (Juniors) A) tried my best B) gave moderate effort C) gave little effort D) gave no effort n 70 70 9 3 Mean 62.6 59.8 57.3 53.7 St. Dev. 4.4 3.5 7.0 3.2 F value 10.5 p value < 0.0001 CAAP Scores by Effort (Seniors) A) tried my best B) gave moderate effort C) gave little effort D) gave no effort n 78 93 25 5 Mean 61.0 58.8 58.3 53.2 St. Dev. 3.4 3.8 4.7 2.3 F value 11.04 p value < 0.0001 significant comparisons E>D E>C significant comparisons E>C significant comparisons A>B A>C A>D significant comparisons A>B A>C A>D B>D C>D Office of Planning and Assessment, Devin DuPree and Sabrina Sattler, March 2011 Page 31 of 36 Collegiate Assessment of Academic Proficiency: Mathematics Test Spring 2011 Appendix D: Analysis of Additional Questions Additional Questions Scores by Student Classification Juniors Seniors n 166 215 Mean 55.3% 55.3% St. Dev. 27.4% 27.2% t value 0.02 p value 0.9872 Additional Questions Scores by Sex (Juniors) Female Male n 74 92 Mean 49.5% 60.0% St. Dev. 25.3% 28.3% t value 2.5 p value 0.0133 Additional Questions Scores by Sex (Seniors) Female Male n 100 115 Mean 45.4% 63.8% St. Dev. 24.8% 26.5% t value 5.3 p value < 0.0001 Additional Questions Scores by Major (Juniors) Math Major Other Major n 35 131 Mean 72.0% 50.8% St. Dev. 27.1% 27.1% t value 4.26 p value < 0.0001 Additional Questions Scores by Major (Seniors) Math Major Other Major n 36 179 Mean 70.6% 52.2% St. Dev. 27.3% 26.3% t value 3.81 p value 0.0002 Office of Planning and Assessment, Devin DuPree and Sabrina Sattler, March 2011 Page 32 of 36 Collegiate Assessment of Academic Proficiency: Mathematics Test Spring 2011 Additional Questions Scores by GPA (Juniors) A) below 2.00 B) 2.01 - 2.50 C) 2.51 - 3.00 D) 3.01 - 3.50 E) 3.51 and above n 5 6 58 47 43 Mean 60.0% 53.3% 51.4% 54.0% 63.3% St. Dev. 14.1% 32.7% 27.8% 26.0% 27.6% F value 1.29 p value 0.2768 Additional Questions Scores by GPA (Seniors) A) below 2.00 B) 2.01 - 2.50 C) 2.51 - 3.00 D) 3.01 - 3.50 E) 3.51 and above n 9 12 86 70 37 Mean 37.8% 61.7% 52.6% 52.3% 68.6% St. Dev. 29.1% 27.6% 25.9% 26.0% 28.1% F value 3.95 p value 0.0041 Additional Questions Scores by Effort (Juniors) A) tried my best B) gave moderate effort C) gave little effort D) gave no effort n 69 69 8 3 Mean 54.9% 59.7% 27.5% 6.7% St. Dev. 28.0% 24.1% 30.1% 11.5% F value 6.95 p value 0.0002 Additional Questions Scores by Effort (Seniors) A) tried my best B) gave moderate effort C) gave little effort D) gave no effort n 78 90 23 5 Mean 60.5% 53.8% 45.2% 44.0% St. Dev. 28.6% 26.6% 27.1% 32.9% F value 2.32 p value 0.0772 significant comparisons significant comparisons E>D E>C E>A significant comparisons A>C A>D B>C B>D significant comparisons Office of Planning and Assessment, Devin DuPree and Sabrina Sattler, March 2011 Page 33 of 36 Collegiate Assessment of Academic Proficiency: Mathematics Test Spring 2011 Appendix E: Analysis of CAAP and Additional Questions Scores by Core Curriculum Questions Have you completed your Mathematics Core – Juniors (CAAP) Yes No n 136 28 Mean 61.0 59.5 St. Dev. 4.6 3.5 t value 1.62 p value 0.1069 Have you completed your Mathematics Core – Seniors (CAAP) Yes No n 193 20 Mean 59.7 59.1 St. Dev. 4.0 4.0 t value 0.65 p value 0.5184 Did you complete your Core Courses at TTU – Juniors (CAAP) Yes No n 79 83 Mean 60.6 60.8 St. Dev. 4.2 4.7 t value 0.20 p value 0.8454 Did you complete your Core Courses at TTU – Seniors (CAAP) Yes No n 107 103 Mean 59.5 59.8 St. Dev. 4.1 3.9 t value 0.61 p value 0.5428 Office of Planning and Assessment, Devin DuPree and Sabrina Sattler, March 2011 Page 34 of 36 Collegiate Assessment of Academic Proficiency: Mathematics Test Spring 2011 Have you completed your Mathematics Core – Juniors (AQ) Yes No n 136 28 Mean 54.9% 55.7% St. Dev. 27.8% 25.7% t value 0.15 p value 0.8799 Have you completed your Mathematics Core – Seniors (AQ) Yes No n 193 20 Mean 55.6% 48.0% St. Dev. 27.5% 21.9% t value 1.25 p value 0.2117 Did you complete your Core Courses at TTU – Juniors (AQ) Yes No n 79 83 Mean 53.7% 56.9% St. Dev. 25.7% 29.4% t value 0.74 p value 0.4633 Did you complete your Core Courses at TTU – Seniors (AQ) Yes No n 107 103 Mean 54.0% 57.1% St. Dev. 27.6% 26.9% t value 0.81 p value 0.4162 Office of Planning and Assessment, Devin DuPree and Sabrina Sattler, March 2011 Page 35 of 36 Collegiate Assessment of Academic Proficiency: Mathematics Test Spring 2011 Appendix F: Effort by Student Classification, Sex, College, Major, and GPA Effort by Student Classification Juniors Seniors Effort by Sex Female Male Effort by College Agriculture Architecture Arts and Sciences Business Admin. Education Engineering Human Sciences Mass Comm. University College Visual/Perform. Arts Effort by Major Math Major Other Major Effort by GPA below 2.00 2.01 - 2.50 2.51 - 3.00 3.01 - 3.50 3.51 and above tried my best 42.4% 48.8% tried my best 23.1% 40.0% 47.8% 38.1% 71.4% 67.6% 15.4% 33.3% 100.0% 33.3% tried my best 67.6% 39.8% tried my best 40.0% 60.0% 28.3% 46.5% 70.0% tried my best 46.1% 38.8% Juniors moderate effort 53.0% 40.7% moderate effort 69.2% 60.0% 41.3% 61.9% 28.6% 29.4% 69.2% 33.3% 0.0% 66.7% moderate effort 29.4% 50.8% moderate effort 40.0% 20.0% 54.7% 53.5% 30.0% moderate effort 46.1% 46.3% little effort 3.0% 8.1% little effort 7.7% 0.0% 8.7% 0.0% 0.0% 2.9% 7.7% 22.2% 0.0% 0.0% little effort 2.9% 6.8% little effort 20.0% 20.0% 11.3% 0.0% 0.0% no effort 1.5% 2.3% no effort 0.0% 0.0% 2.2% 0.0% 0.0% 0.0% 7.7% 11.1% 0.0% 0.0% no effort 0.0% 2.5% no effort 0.0% 0.0% 5.7% 0.0% 0.0% little effort 5.9% 12.4% tried my best 37.5% 40.0% tried my best 21.4% 50.0% 42.1% 44.7% 66.7% 40.6% 23.5% 15.4% 33.3% 40.0% tried my best 39.4% 38.7% tried my best 42.9% 36.4% 37.6% 34.4% 66.7% no effort 2.0% 2.5% Seniors moderate little effort effort 50.0% 10.4% 42.9% 14.3% moderate little effort effort 50.0% 21.4% 37.5% 12.5% 43.9% 10.5% 42.1% 10.5% 33.3% 0.0% 46.9% 12.5% 47.1% 23.5% 84.6% 0.0% 33.3% 33.3% 40.0% 20.0% moderate little effort effort 48.5% 12.1% 45.8% 12.5% moderate little effort effort 57.1% 0.0% 54.5% 9.1% 44.7% 15.3% 53.1% 9.4% 33.3% 0.0% Office of Planning and Assessment, Devin DuPree and Sabrina Sattler, March 2011 Page 36 of 36 no effort 2.1% 2.9% no effort 7.1% 0.0% 3.5% 2.6% 0.0% 0.0% 5.9% 0.0% 0.0% 0.0% no effort 0.0% 3.0% no effort 0.0% 0.0% 2.4% 3.1% 0.0%