October 21, 2009
RIVERSIDE CITY COLLEGE/ACADEMIC SUPPORT
Data Analysis of JumpStart Program
Summer 2009
1
Data provided by Dr. Myung Hwa Koh | Academic Support
October 21, 2009
RIVERSIDE CITY COLLEGE/ACADEMIC SUPPORT
Table of Contents
Page
Overall result of data analysis of the JumpStart students……………………………….
3
1. JumpStart Survey………………………………………………………………..
3
Participants ……………………………………………………………………….
3
Q1. Factors impacting students’ confidence about Accuplacer test ……………..
3
Q2. Confidence Level Changes between the First and the Second Accuplacer Tests……
3
2. Accuplacer Test Changes……….....………......………......………......………...
4
3. English Challenge Test Score….....………......………......………......………....
6
What differences were there between students who participated in the JumpStart
program and students who didn’t participate in the JumpStart Program?.......................
6
1. JumpStart Survey………………………………………………………………..
6
Participants ……………………………………………………………………….
6
Confidence Level Changes between the First and the Second Accuplacer Tests………..
6
2. Accuplacer Test Changes……….....………......………......………......………...
7
Appendix..........................................................................................................................
10
Accuplacer/PTESL Placement Grid ...………......………......………......………...
10
Table A1. English Assessment Placement Level Changes ...………......………….
11
Table A2. Reading Assessment Placement Level Changes ...………......…………
11
Table A3. Math Assessment Placement Level Changes ...………......…………….
12
Table A4. English Challenge Test Score...………......…………………………….
12
2
Data provided by Dr. Myung Hwa Koh | Academic Support
October 21, 2009
RIVERSIDE CITY COLLEGE/ACADEMIC SUPPORT
1. Overall Results of the Data Analysis of JumpStart Students
1. JumpStart Survey Analysis
Participants
Twenty nine students participated in the JumpStart Program. Of these, 23 students completed the
JumpStart Survey.
Q1. To what extent did the following factors increase your confidence about your ability to get a
higher Accuplacer test score on your second try?
The survey used a five-point Likert-type scale (ranging from 1= “Not at all” to 5 = “A significant
extent”) to examine what impact the following seven factors had on increasing students’
confidence about their ability to get a higher Accuplacer test score on their second try.
As shown in Table 1 below, the top four factors were 1) enrolled in Jump Start Program, 2)
participated in SI session, 3) studied on my own, and 4) participated in a study group.
Table 1. Factors impacting students’ confidence about Accuplacer test
N
Not at
all
Little
A little
Enrolled in Jump Start program.
21
1
2
Participated in SI sessions.
19
3
1
Studied on my own.
20
1
2
3
Participated in a study group.
13
3
1
Reviewed the Accuplacer Sample Test
15
1
1
4
questions on the RCC website.
Received tutoring assistance at RCC.
12
1
3
3
Received tutoring assistance outside
8
2
1
RCC.
Note: Scale: 1= Not at all; 2= Little; 3= A little; 4= A lot; 5= A significant extent;
A lot
Mean (Std)
8
8
4
3
2
A significant
extent
10
7
10
6
7
1
2
4
3
3.33 (1.44)
3.38 (1.44)
4.29 (.845)
4.00 (1.05)
4.00 (1.26)
3.92 (1.26)
3.87(1.30)
Q2. Confidence Level Changes between the First and the Second Accuplacer Tests
The survey used a five-point Likert-type scale (ranging from 1= “Strongly Disagree” to 5 =
“Strongly Agree”) to assess student confidence level changes between their first and second
attempts on the Accuplacer Test. As shown in Table 2 and Figure 1 below, we can conclude that
there is a statistically significant confidence level difference between the first and the second try.
3
Data provided by Dr. Myung Hwa Koh | Academic Support
October 21, 2009
RIVERSIDE CITY COLLEGE/ACADEMIC SUPPORT
Table 2. Confidence Level Changes
N
The first Accuplacer Test
Min Max Mean (p1)
The second Accuplacer Test
N Min
Max
Mean (p2)
P2-P1
I tried harder to achieve a higher
1
5
3.52
3
5
4.48
0.96
score*.
(1.504)
(.750)
I felt more confident about my
1
5
3.00
2
5
4.38
1.38
preparation for the Accuplacer test.*
(1.643)
(.921)
Note: Values significant p<.01 are indicated by *. The Paired t-statistic test was used to determine the statistical
significance.
Figure 1.
5
4.48
4.5
4
4.38
3.52
3.5
3
3
2.5
First time
2
Second time
1.5
1
0.5
0
I tried harder to achieve a I felt more confident
higher score*.
about my preparation for
the Accuplacer test.*
2. Accuplacer Test Score Changes
Twenty nine students participated in the JumpStart Program. Among them, 27 students
completed the two Accuplacer Tests.
Overall, 5 of 27 (18.5%) students’ English test scores improved in the second attempt on the
Accuplacer test, 9 (33.3%) students’ Reading test scores improved, and 12 (44.4%) students’
Math test scores improved (see Figure 2.) Table A1, A2, and A3 located in the Appendix
provide English, Reading, Math assessment placement level changes.
4
Data provided by Dr. Myung Hwa Koh | Academic Support
October 21, 2009
RIVERSIDE CITY COLLEGE/ACADEMIC SUPPORT
Figure 2. The percentage of students who got a higher test score in the second attempt on the Accuplacer
Test.
50.0%
45.0%
40.0%
35.0%
30.0%
25.0%
20.0%
15.0%
10.0%
5.0%
0.0%
44.4%
33.3%
18.5%
English
Reading
Math
As Table 3 and Figure 3 explain below, the mean English, Math, and Reading post-test scores
were higher than the corresponding mean pre-test scores (see Table 4). The Appendix also shows
the relation between placement test scores and courses taken in English, Reading, Math, and ESL.
English. The mean English pre-test score was 10.00, which is the equivalent of English 60A.
The mean English post-test score was 12.59, which falls between English 60A and 60B. The
mean English test score was improved by 2.59 points.
Reading. The mean Reading pre-test score was 12.22, which falls between REA-81 and REA-82,
and the mean Reading post-test score was 18.15, which is closer to the equivalent of REA-82.
The mean Reading test score was greatly improved by 5.93 points.
Math. The mean Math pre-test score was 10, which falls in Math-63 and the mean Math post-test
score was 14.07, which is closer to Math-64. The mean Math test score was also improved by
4.07 points.
Table 3. Descriptive Statistics
N
Pre_English_Score
Post_English_Score
Pre_Reading_Score
Post_Reading_Score
Pre_Math_Score
Post_Math_Score
5
Minimum
27
27
27
27
27
27
5 (ENG-60A/ESL)
5 (ENG-60A/ESL)
10(REA-81)
10(REA-81)
10(MAT-63)
10(MAT-63)
Maximum
20(ENG-50)
30(ENG-1A)
30(REA-83)
40(College level)
10(MAT-63)
30(MAT-35)
Data provided by Dr. Myung Hwa Koh | Academic Support
Mean
10.00
12.59
12.22
18.15
10.00
14.07
Std. Deviation
2.402
7.767
5.064
11.107
.000
5.377
October 21, 2009
RIVERSIDE CITY COLLEGE/ACADEMIC SUPPORT
Figure 3. English, Reading, and Math Placement Test Score Changes between the First and Second
Attempts on the Accuplacer Test
20
18.15
15
12.59
14.07
12.22
10
10
Pre-test
10
Post-test
5
0
English
Reading
Math
A regression analysis was used to test for a relationship between test score changes and factors
impacting student confidence about the Accuplacer test. We found that there was no significant
relationship between test score changes and confidence factors.
Finally, a paired t-test was used to check for a statically significant test score difference between
the first and second attempts on the Accuplacer Test. We found that the Reading and Math
Placement tests had statistically significant score differences between the pre-tests and post-tests
at alpha=.01 (0.002, 0.001 respectively).
3. English Challenge Test Score
Twenty out of 29 students took a English challenge test. Among them, 19 (79.17%) students’
English test scores improved in the English challenge test (see Table 4 and Table A4).
Table 4. English Challenge Test Results
The First English
Placement level
ENG-60A
ENG-60A/PTESL
English Challenge Test Level
Frequency
Percent
ENG-60A
5
21.7%
ENG-60B
16
69.6%
ENG- 50
2
8.7%
Total
ENG- 50
23
1
100.0%
100.0%
2. What differences were there between students who participated in the JumpStart
program and students who didn’t participate in the JumpStart Program?
6
Data provided by Dr. Myung Hwa Koh | Academic Support
October 21, 2009
RIVERSIDE CITY COLLEGE/ACADEMIC SUPPORT
1. JumpStart Survey Analysis
Participants
Twenty-eight (28) students participated in the JumpStart Survey. All participants were divided
into either the JumpStart group or the Non-JumpStart group according to their participation in
the JumpStart program. Twenty-three (23) were in the JumpStart group while 5 students were in
the Non-JumpStart group.
Changes in the Level of Confidence between First time and Second time Accuplacer Tests
The survey used a four-point Likert-type scale (ranging from 1= “Strongly Disagree” to 5 =
“Strongly Agree”) to assess student confidence levels with their preparation for the Accuplacer
Test. As shown in Table 5 below, we found that the JumpStart Group had statistically significant
confidence level changes between the first and the second attempts on the Accuplacer test.
However, the Non-JumpStart group evidenced no statistically significant confidence level
changes.
Table 5. Comparisons of Confidence Level Changes between JumpStart Group and Non-Jump Start
Group.
JumpStart Group
Non- JumpStart Group
N
Min
Max
Mean
N
Min
Max
Mean
I tried harder to achieve a higher score the
first time I prepared for the test.
21
1
5
3.52
(1.50)
5
3
5
4.40
(0.89)
I tried harder to achieve a higher score the
second time I prepared for the test.
21
3
5
4.48
(0.75)
5
4
5
4.60
(0.55)
I felt more confident about my preparation for
the Accuplacer test the first time I took it.
21
1
5
3.00
(1.64)
5
2
5
2.80
(1.30)
I felt more confident about my preparation for
the Accuplacer test the second time I took it.
21
2
5
4.38
(0.92)
5
2
5
4.20
(1.30)
2. Accuplacer Test Score Changes
Twenty nine students participated in the JumpStart Program. Among them, 27 students
completed the two Accaplacer Tests. The following statistics look at difference between the
twenty-seven students who completed the Accuplacer test (denoted as the JumpStart group) and
13 other students who did not participate in the JumpStart program (the Non-JumpStart group).
Overall, 5 out of 27 (18.5%) JumpStart students’ English test scores improved in the second
attempt on the Accuplacer test, while 5 out of 13 (38.5%) non-JumpStart students’ English test
scores improved. Nine (33.3%) JumpStart students’ Reading test scores improved, while 6 non7
Data provided by Dr. Myung Hwa Koh | Academic Support
October 21, 2009
RIVERSIDE CITY COLLEGE/ACADEMIC SUPPORT
JumpStart students’ Reading test scores improved. Twelve (44.4%) students’ Math test scores
improved, while 3 (23.1%) non-JumpStart students’ Math test scores improved (see Figure 4.)
Table A1, A2, and A3 located in the Appendix provide English, Reading, Math assessment
placement level changes.
Figure 4. The percentage of students who got a higher test score in the second attempt on the Accuplacer
Test.
50.0%
46.2%
45.0%
40.0%
44.4%
38.5%
33.3%
35.0%
30.0%
23.1%
25.0%
20.0%
JumpStart Group
18.5%
Non-JumpStart Group
15.0%
10.0%
5.0%
0.0%
English
Reading
Math
As Table 6 below shows, the mean English, Math, and Reading post-test scores were higher than
their pre-test scores for both of the two groups.
Table 6. Paired Samples Statistics
Group
N
JumpStart
Group
Pair 1 Pre_English_Score
Mean
Std. Deviation
20(ENG-50)
10.00
2.402
27 5 (ENG-60A/ESL)
30(ENG-1A)
12.59
7.767
27
10(REA-81)
30(REA-83)
12.22
5.064
27
10(REA-81) 40(College level)
18.15
11.107
27
10(MAT-63)
10(MAT-63)
10.00
.000
27
10(MAT-63)
30(MAT-35)
14.07
5.377
13
10(ENG-60A)
10 (ENG-60A)
10.00
.000
Post_English_Score
13
10(ENG-60A)
30(ENG-1A)
15.38
8.530
Pair 2 Pre_Reading_Score
13
10(REA-81)
20(REA-82)
12.31
4.385
13
10(REA-81) 40(College level)
21.54
14.051
13
10(MAT-63)
10(MAT-63)
10.00
0.00
13
10(MAT-63)
30(MAT-35)
13.08
5.965
Post_English_Score
Post_Reading_Score
Pair 3 Pre_Math_Score
Post_Math_Score
Pair 1 Pre_English_Score
Post_Reading_Score
Pair 3 Pre_Math_Score
Post_Math_Score
8
Max
27 5 (ENG-60A/ESL)
Pair 2 Pre_Reading_Score
NonJumpStart
Group
Min
Data provided by Dr. Myung Hwa Koh | Academic Support
October 21, 2009
RIVERSIDE CITY COLLEGE/ACADEMIC SUPPORT
Figure 5 shows English, Reading, and Math placement mean differences between pre-tests and post-tests
across the two groups. In this study, the Jump Start Group had a larger mean Math difference than the
Non-JumpStart Group.
Figure 5. English, Reading, and Math Score Changes
10
9.23
9
8
7
5.93
6
5.38
5
4
3
JumpStart Group
4.07
3.08
2.59
Non-JumpStart Group
2
1
0
English Score
Change
Reading Score
Change
Math Score
Change
A paired t-test was used to check for statically significant test score differences between students’
first and second tries at the Accuplacer Test. We found that students in the JumpStart Group had
statistically significant Reading and Math test score differences between their pre-tests and posttests at alpha=0.01(p=.002, p=.001 respectively). In the Non-JumpStart Group, there were
statistically significant English and Reading test score differences between the pre-tests and
post-tests at alpha=0.05 (p=.044, p=.033 respectively)
We also tested to see if there was a statistically significant test score difference between the first
and second attempts on the Accuplacer Test across the JumpStart group and Non-JumpStart
Group. However, we found that there was no statistically significant score differences between
pre-tests and post-tests. While there were statistically significant differences within group and
within disciplines, there were no statistically differences across both groups. In this study, we
also found that the test score change between the pre-tests and post-tests in the Non-JumpStart
Group was more varied than was the case in the JumpStart Group. Because of the difficulty of
recruiting students in the Non-JumpStart group and individual difference, we did not find the
answer. However, we plan to analyze the question in the future.
9
Data provided by Dr. Myung Hwa Koh | Academic Support
October 21, 2009
RIVERSIDE CITY COLLEGE/ACADEMIC SUPPORT
Appendix
Accuplacer/PTESL Placement Grid (For tests taken after July 01, 2001)
CPT-4 (Reading)
CPT-5(Writing)
CPT-1,2 or 3 (Math)
40-College Level
30-English 1A
70- Math 1A
30-Reading Skills 83 20-English 50
60- Math 10
20-Reading Skills 82 15-English 60B
50- Math 25, 36
10-Reading Skills 81 10-English 60A
40- Math 4, 5, 11, 12
5- English 60A/PTESL 30- Math 53, 35
20- Math 52
15- Math 64
10- Math 63
10
Data provided by Dr. Myung Hwa Koh | Academic Support
PTESL
50- ESL55
40- ESL54
30- ESL53
20- ESL52
10- ESL51
October 21, 2009
RIVERSIDE CITY COLLEGE/ACADEMIC SUPPORT
Table A1. English Assessment Placement Level Changes between the First and Second Attempts on the
Accuplacer Test
Group
JumpStart Group
The First English
Placement Level
The Second English
Placement Level
Frequency
ENG-50
ENG-60B
1
100.0%
ENG-60A
ENG-60A/PTESL
2
8.3%
ENG-60A
17
70.8%
ENG-60B
1
4.2%
ENG-50
4
16.7%
Total
Non-JumpStart
Group
Percent
24
100%
ENG-60A/PTESL
ENG-60A/PTESL
2
100.0%
ENG-60A
ENG-60A
8
61.5%
ENG-60B
1
7.7%
ENG-50
4
30.8%
13
100%
Total
Table A2. Reading Assessment Placement Level Changes between the First and Second Attempts
on the Accuplacer Test
Group
JumpStart Group
The First Reading
Placement Level
REA-81
The Second Reading
Placement Level
Frequency
REA-81
17
77.3%
REA-83
5
22.7%
22
100.0%
REA-83
2
50.0%
CL
2
50.0%
Total
4
100.0%
REA-83
REA-83
1
100.0%
REA-81
REA-81
6
60.0%
REA-82
1
10.0%
CL
3
30.0%
10
100.0%
REA-81
1
33.3%
REA-83
1
33.3%
CL
1
33.3%
Total
3
100.0%
Total
REA-82
Non-JumpStart
Group
Total
REA-82
11
Percent
Data provided by Dr. Myung Hwa Koh | Academic Support
October 21, 2009
RIVERSIDE CITY COLLEGE/ACADEMIC SUPPORT
Table A3. Math Assessment Placement Level Changes between the First and Second Attempts on the Accuplacer
Test
Group
JumpStart Group
The First Reading
Placement Level
MAT-63
The Second Placement
Level
Frequency
MAT-63
15
55.6%
MAT-64
4
14.8%
MAT-52
7
25.9%
MAT-35
1
3.7%
27
100.0%
MAT-63
9
75.0%
MAT-64
1
8.3%
MAT-52
1
8.3%
MAT-35
1
8.3%
12
100.0%
1
100.0%
Total
Control Group
MAT-63
Total
MAT-64
Percent
MAT-64
Table A4. English Challenge Result
The First English Placement
Level
ENG-60A
English Challenge Test
level
Frequency
Percent
ENG-60A
5
21.7%
ENG-60B
16
69.6%
2
8.7%
23
100.0%
1
100.0%
ENG-50
Total
ENG-60A/PTESL
12
ENG-50
Data provided by Dr. Myung Hwa Koh | Academic Support