CAAP Spring 2006-7 Report
On
Juniors
Introduction
The Collegiate Assessment of Academic Proficiency (CAAP) Test was given early spring
term 2006-7 to a representative sample of junior students from all AUB faculties. With
the help of the Registrar’s Office and the Banner system, suitable times for administration
were scheduled for the sample. Despite continuous reminders and urging, only 235 of the
selected 397 (59%) junior students took it though we rescheduled some extra sessions.
We also tried to motivate them by providing cafeteria lunch vouchers for LL 5,000 each.
Faculties, in turn, contacted selected sample and urged them to take it as a faculty
requirement. Unfortunately, we could not get full response rate. Examining the sample
representativeness (Table 1), reveals that it is quite proportional to original sample with
slight over representation of FEA and OSB and slight under representation of FAFS and
FAS. Each student who took the test was given the Critical Thinking (CT) component of
the test and was allowed a test of his/her choice from Mathematics Reasoning (MR),
Science Reasoning (SCR), Reading (R), and Writing Skills (W).
The tests were administered following CAAP standardized administration procedures.
Completed forms were sent to ACT for scoring and a month later reports were received.
There was an institutional report, in addition to individual student reports. Each student
received a report detailing his/her performance on tests that were taken, giving score and
percentile rank compared to AUB students and also compared to American national
norms of comparable 4-year institutions. In addition, students who got ≥50th percentile
received a Certificate of Achievement.
Table 1. Representativeness of CAAP Sample Spring 2007
CAAP Sample
College
%
AG
13
AS
35
EA
21
HS
4
NU
2
SB
24
FM
TOTAL
#
53
140
82
17
7
95
2
396
TOOK CAAP
College
%
AG
11
AS
33
EA
24
HS
3.4
NU
2
SB
27
59
#
25
77
56
8
5
64
235
Results
Results of the various CAAP tests for the whole sample are reported in Table 2.
Comparison with 2003-6, and with American national norms is also provided. AUB
students’ best performance is still on MR and SCR followed by Writing, though sample
is small, while their lowest performance is on R. Compared to national norms; AUB
performance is significantly higher on MR and SCR only, while there are no significant
differences on other subjects. With respect to previous performance on CAAP, this year
performance is highest in all subjects. Figure 1 also provides a figural representation of
1
the results. This finding needs to be checked against candidates’ GPA to check if this
group is academically similar or weaker than that of previous years. Examining GPA data
reveals that this year’s sample has a higher GPA than last year as 51% of them reported a
GPA of ≥ 3.01 vs. 44% in 2006, though slightly lower than 2004/5 (55%). However,
2007 performance is still higher than 2004/5. On examining their actual GPA, we find
that average of whole sample required to take CAAP is 78, while average of those who
took it is 80 and those who did not take it is 75, so, sample is slightly biased in terms of
good students.
Table2. Comparison of CAAP Results with National Norms and with 2003-5
Year
N
CT
SCR
R
MR
W
2006/7
235
63.1
65.0
62.4
65.7
65.6
N=235
N=63
N=35
N=123
N=13
2005/6
245
61.9
60.5
59.9
64.9
63.4
2004/5
403
62.7
62.9
62.5
64.6
63.0
2003/4
736
62.3
61.2
59.7
64.2
60.0
Norms
39,463
62.7
61.4
62.6
58.1
64.2
Results were further examined by comparing CAAP scores for students with similar GPA
in both years. Table 3 reports breakdown of scores by GPA for both years. Most
noticeable increases were in all groups in SC and in 2.5-3.0 group in R, in the 2.0-2.5 in
MR, ≥ 3.0 group in CT (highlighted in blue). So, in general the higher means in 2007 in
most areas represent higher competence.
Table 3. Breakdown of CAAP Results by GPA
GPA
N
%
CT
SC
06
07
06 07
< 2.00
3
2.0-2.50
37
62
16 61
61 66
2.51-3.00 53
61
23 61
58 64
3.01-3.50 85
36 62
64
61 65
≥ 3.51
34
67
15 64
66
No
23
64
10 60
63
response
R
06
07
06
MR
07 06
58
58
62
64
59
63
64
63
62
64
65
69
66
65
64
67
68
66
62
62
Performance on CAAP tests was also compared by gender, GPA, major and whether
English was a first language or not. With respect to gender, examining Table 4 reveals
that performance was quite similar on all tests, and where there were differences (W and
R) they could not be significant because of small sample size. Gender results are reported
graphically in Figure 2.
2
Figure 1. CAAP Test Results for 2004-6 and in Comparison with Norms
66
64
62
2006/7
2005
2004
Norms
60
58
56
54
CT
SC
R
MS
Table 4. CAAP Results by Gender
Gender
N
CT
SCR
110
Male
124
Female
W
R
MR
W
63
65
64
66
67
(n=110)
(n=20)
(n=8)
(n=75)
(n=7)
63
65
62
65
64
(n=124)
(n=42)
(n=27)
(n=48)
(n=6)
Figure 2. CAAP Results by Gender
68
67
67
66
66
65 65
65
64
65
64
64
Male
63 63
63
Female
62
62
61
60
59
CT
SCR
R
MR
W
As to age differences on the CAAP tests, all who took the test were juniors and there
should not have been large differences in age level nor in results due to age.
3
With respect to differences resulting from English being a first language or not, there
were differences as noted in Table 5 mainly on MR with group whose native language is
not English scoring higher. The difference in Writing performance is not significant due
to small sample size. Differences were noted, however, on CAAP test scores by GPA. In
general the higher the GPA the higher the CAAP test sore. Table 3 provides summary
results by GPA, while figure 2 highlights differences graphically.
Table 5. CAAP Results by Native Language
English
N
CT
SCR
R
First
Language
68
Not First
Language
166
MR
W
63
66
62
64
67
(n=68)
(n=17)
(n=18)
(n=25)
(n=7)
63
65
63
66
64
(n=166)
(n=46)
(n=17)
(n=97)
(n=6)
Figure 3. CAAP Score Differences by GPA
70
65
CT
SC
60
R
55
MR
50
2.01-2.50
2.51-3.00
3.01-3.50
≥ 3.51
Differences by major were also noted, students from biological sciences got highest
scores on all tests, and except MR and Engineering and Social Sciences got highest in
MR and second highest in CT. Business was second highest in MR. Table 6 presents
CAAP test results by major, while figures 4-5 present differences in CT and MR by
major and in comparison with 2006 (in blue). In CT, all majors improved with exception
of business and nutrition who maintained previous year’s scores. Improvement is noted in
biology, engineering and physical science majors. As for MR, social sciences and
business revealed noticeable improvement.
4
Table 6. Results by Major
Major
N
CT
Agriculture
5
60
Architecture
1
Biol. Sc.
35
66
Business
58
62
Marketing
3
Communications
3
Community Service
1
Comp. Sc.
7
Education
4
Engineering
SCR
R
67
64
61
MR
W
64
63
64
55
64
68
Health Sc.
10
61
61
Nutrition
17
61
64
Letters
2
Math
5
Philosophy
1
Physical sc.
10
62
Social sc.
10
64
No response
8
60
63
65
68
Figure 4 CT Scores by Major, Comparison with 2006
(blue)
66
65
64
63
62
61
60
59
58
57
BY
BUS
CSC
ENG
5
Nutrition
PYSC
SOSC
Figure 5 Math Reasoning by Major, Comparison with 2006 (blue)
68
66
64
62
60
58
56
BUS
CSC
Eng
SOSC
Results also revealed important information for Writing, Mathematics, and Reading in
terms of sub scores. Table 7 provides sub scores for each of these tests, in addition to a
comparison with national norms and with 2003-6. In Writing, AUB students consistently
do better on usage/mechanics than on rhetorical writing, though they have improved this
year in later. With respect to Reading, they performed quite same on social science
readings and on arts/literature, though usually they perform better on social science.
Improvement in Writing is noticeable over the years and we are approaching the
American norms. With respect to math, they do very well on both sections and much
higher than the norms.
Table 7. Writing, Reading, and Math Sub score Results, 2003-7
Test
N
2007
2006
2005
2004
Writing: Usage/Mechanics
13
18.1
16.8
17.2
15.6
Writing: Rhetorical
13
17.6
16.6
16.0
14.7
Reading: Arts/literature 35
16.2
14.3
15.5
14.5
Reading: Social sciences
35
15.9
15.5
16.5
15.2
Essay
3.4
3.0
3.2
Math: Basic algebra
123 18.3
18.0
17.6
17.8
Math: College algebra
123 19.2
20.2
20.4
20.0
2003
15.9
14.9
14.3
15.0
3.4
16.6
18.2
Norms
17.2
17.3
15.7
16.5
3.3
15.3
15.2
Certificates of Achievement
A good number of students obtained Certificates of Achievements indicating that they
achieved ≥50th %ile of the normative sample. Table 8 provides the number and
percentage of certificates obtained in different subjects and in comparison with 2004-6.
Percentages in 2007 are higher than previous years and are closer to 2005 figures.
Increase in nearly all subjects though to a lesser extent in CT. Figure 6 provides the
figures graphically by subject. Some students obtained certificates in one subject, others
in two. Table 9 details this information in comparison with 2005. The highest percentage
of certificates was obtained in Math Reasoning followed by Science Reasoning.
6
Table 8 Distribution of Certificates of Achievement by Subject
Subject
N Certificates %Certificates %Certificate %Certificate %Certificate
07
07
06
05
04
Critical
235
123
52
49
55
51
Thinking
Math
124
124
100
94
81
88
Reasoning
Science
63
49
78
45
72
50
Reasoning
Writing
13
9
69
55
41
14
Skills
Reading
35
20
57
33
52
31
Figure 5. Number of Certificates by Test
49
9
Math
124
20
Critical Thinking
Reading
Science
Writing Essay
123
Writing
Table 9. Frequency of Number of Distributions, Comparison with 2005-6
N
%Certificates %Certificates %Certificates
%
(06)
07
06
05
Total
07
# who got
79
39
54
45
34
certificates
in 1 subject
# who got
123
61
46
55
52
certificates
in 2
subjects
Total
202
100
100
100
86
%
Total
06
44
%
Total
05
39
38
47
82
86
The percentage of students who took certificates is similar to that of 2005. Similarly, the
percentage of students who were awarded two certificates went up to 52%. Figure 6
provides graphic distribution of certificates.
7
Figure 6. Distribution of Certificates
33
No certificate
Certificate in 1
Subject
123
79
Certificate in 2
Subjects
Results by Demographic Variables
It was interesting to study performance on CAAP with student other information like
actual GPA, schools they came from, in addition to their entry level scores like SAT and
Composite score. Appendix provides scores on different tests by school and schools with
highest averages on different tests. Of interest is that students with highest average in CT
were coming from schools outside Lebanon.
Conclusion
Administering the CAAP is very useful as it provides AUB with an indicator of the level
of its students in basic general education skills and competencies that include thinking
critically, reasoning and written communication. The information provides us with
information regarding skills needing improvement and the changes over time. It is a very
important and serious outcome that needs to be maintained and even encouraged.
We still have problems with students taking the CAAP. Despite urging and promises and
threats, we had 59% participation rate. We need to work more on this side by finding
ways to motivate all juniors to take the tests and to put their best effort while doing so.
This year’s results showed improvement across all subjects but the sample, as usual, was
slightly biased towards more competent students, as revealed in the report. This,
however, has been the case for years but we hope that improvement noted this year will
be maintained.
8
Appendix
School Averages on Various Tests
HIGH SCHOOL
A.U.B.
A/Dhabi Int'l School,UAE
Abdul Aziz M. Al-Khalifah Sec.
Ahliah Sch,Wadi Abu Jmil
Al Ghazali H Sch, Hay Sillum
Al -Jamil Al-Ahliya School
Al Kawthar School, Beirut
Al-Hussan National
Sch.Dammam
Al-Mustafa High Sch.,Nabatieh
Amer.Comm.School, Beirut
American School of Kuwait
Antonins Col/Frers,Baabda
Arizona State University
Arndt-Oberschule(Gymnasium)
Ashbal Al Sahel,H/Hureik
Athenee de Beyrouth.
Bahrain Bayan Sch.
Balamand University
Baptist Sch,Amman
Beirut Arab University
Beirut Babtist Sch/Mazraa
Beirut Comm.Sch/Hadath
Beirut Evang.Sch.for girls&boy
British Int'l Sch.of Jeddah,KS
Brummana High Sch.
Cairo Amer.Col,Egypt
Christian Teach.Institute,Hors
City International School,Beir
Col.de La Sagesse,Achrafieh
Col.des Freres,Deddeh
Col.Melkart,Hazmieh
Col.Prot.Francais, Beirut
Col.St.Coeur,Hadath
Col.St.Coeurs,Sioufi
Coll.Al-Mostapha Al-Hoch,Tyr
Coll.Notre Dame Du Balamand
College du Sacre Coeur, Beirut
College Louise Wegmann, Beirut
College MARISTE Champville
College Saint Georges, Hadath
College St. Joseph,Antoura
W
Count Avg
0
1
68
0
0
0
0
0
1
62
0
0
0
1
0
1
0
0
0
0
0
0
0
0
0
0
0
2
0
0
1
0
0
0
1
0
0
0
0
0
0
0
0
0
64
71
63
65
67
9
MR
Count Avg
0
0
0
1
64
0
1
69
0
0
0
0
1
0
1
0
0
0
2
0
1
1
0
0
0
2
0
0
0
1
0
1
2
1
5
1
1
1
0
0
2
6
1
0
65
69
68.5
65
70
60.5
68
67
69
66
64.4
59
68
67
66
64
61
R
Count Avg
0
1
65
1
66
0
0
0
0
0
0
0
2
0
0
0
1
0
0
1
0
0
2
0
0
0
1
0
0
1
0
0
0
0
0
0
0
0
0
1
0
0
0
0
60.5
67
69
63.5
67
50
69
CT
Count
1
2
1
1
1
1
1
1
1
1
4
1
1
1
1
1
2
1
1
1
2
1
1
2
1
4
1
2
1
2
3
1
7
1
1
3
1
2
2
6
1
1
Avg
63
60
69
61
62
60
59
58
61
60
63.8
69
63
71
66
61
61.5
71
66
66
66
61
49
61
71
61
58
66.5
63
66
64.3
68
63.7
50
60
65
54
60
65
63.7
56
61
SCR
Count Avg
1
65
0
0
0
1
64
0
1
65
0
1
1
1
0
0
0
0
1
0
0
0
0
0
1
1
0
0
2
1
0
0
1
1
0
1
0
0
2
1
1
0
0
0
1
63
65
71
65
65
54
62.5
65
63
66
66
66
61
61
65
Convent Sch.Jesus &
Mary,Rabwe
Damascus Comm.Sch.
David Karam Educ.Cent,Koura
DePaul University,Chicago,IL
Doha College, Qatar
Ecole Elite, Msaytbeh
Ecole Francaise de Damas
Evange.Sec.Sch,Tripoli
Friends Sc,Ramallah
GrandLyceeFrancolibanais,bey
Greater Beirut Evangelical Sch
Greenfield Coll,Beirut
Hariri High School II, Beirut
Hillcrest Sc,Nijeria
ICARDA Int'l Sch.Of Aleppo
Imane Pilot Sc,Zarif
Int'l College, Ras Beirut
Int'l Sch,Koura
Int'l Sch.of Choueifat, Al Ain
Int'l Sch.of Choueifat,Choueif
K.A.I.S.King AbdulAziz Int'l S
King Saud Int'l. School, Riyad
Leb Evan Sch for B&G,Louaizeh
Leb. Am. University,Ras Beirut
Leb.Int'l Sc,Beirut
Leb.Prepar.School, Beirut
Leb.Rawdah High School
Lycee Abdul-Kader, Beirut
Lycee Francais Int'l Jeddah
Lycee Francee,Doha, Qatar
Lycee Franco Libanais, N.Ibr
Lycee Franco Libanais,Verdun
Lycee Nat,Shweifat
Manar Modern Sc,R.Matn
N.D.de Jamhour College
N.D.Nazareth Col., Beirut
N.D.Nazareth Col.,Beirut
N.D.University, Louaize
Najd National School, Riyadh
Nat.Evangl.Sch,Nabatiyeh
Nat.Prot.Col,Kfarshima
National Orthodox Sch.,Jordan
New English Sc,Jordan
Notre Dames des Apotres,KabEli
Off Sec Sch Majdel Baana
Off. Manara Sec.Sc,Bekaa
Off.Sec.Sch,Rayak
Off.Sec.Sch./Girls,Sidon
Official Sec.Sch.,Bchamoun
0
0
0
0
0
0
0
0
0
0
1
0
0
0
1
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
70
65
10
1
0
2
1
0
1
1
1
1
6
0
0
0
0
0
2
12
0
1
7
1
1
1
1
1
1
0
2
1
1
1
3
1
1
5
1
0
1
1
0
1
1
0
0
1
0
1
0
0
67
70.5
57
60
64
66
61
66.2
69
64.5
68
64.9
57
70
68
62
70
66
68.5
67
59
61
65.7
68
58
67.4
70
66
67
64
57
65
71
1
0
0
0
0
0
0
0
0
2
0
0
2
1
0
0
5
0
0
1
0
0
0
3
0
0
1
0
0
0
0
0
0
0
1
1
0
0
0
0
0
0
1
0
0
1
0
0
0
72
66.5
65
67
60.4
64
58.7
57
65
57
56
61
3
1
2
1
1
1
1
1
1
9
1
1
2
1
1
2
23
1
1
10
1
1
3
4
1
1
1
3
1
1
1
5
1
1
6
2
1
1
1
1
2
1
1
1
1
1
1
1
1
68.7
68
70
56
60
49
62
68
62
64.6
72
61
66
72
67
63.5
64.6
68
66
62.7
62
63
64.3
59.3
62
68
56
59.3
71
63
57
66.4
66
58
63.3
64
64
49
61
65
60.5
49
59
59
62
63
63
58
54
1
1
0
0
1
0
0
0
0
1
0
1
0
0
0
0
6
1
0
2
0
0
2
0
0
0
0
1
0
0
0
2
0
0
0
0
1
0
0
1
1
0
0
1
0
0
0
1
1
63
68
64
62
61
68.2
68
63.5
67.5
61
69
68
66
64
60
65
60
Omar Farroukh Sec.Sch.
Omar Mukh.Edu.Cen.Bekaa
Our Lady of Grace School, Kfar
Qalaa Sec.School, Sidon
Rafic Hariri High School,Sidon
Ras Beirut Int'l Sch.,AirPort
Rawafed Priv.School
Rawdah H.Sch., Beirut
Sagesse High School, Ain Saade
Sagesse St.Jean Brazila,Baabda
Shouf National College, Baakli
Srs. Ste. Famille, Hoche-Hala,
St.Coeur,Kfarhabab
St.Famille Maronite,Tripoli
St.Famille,Fanar
St.Francois Cappucins, Beirut
St.Joseph l'Apparition, Z.Blat
St.Joseph,Jbeil
St.Joseph,K/Shahwan
St.Louis Univ./Madrid Campus
St.Mary Orthodox Col,Beirut
Sultan's School, Oman
Tannokh.Nat.Sch,Aley
Thanawiyyat Tarbieh,Sidon
The Int'l Sch of Choueifat, Sh
Walther-RathenauOberschule,Be
0
0
0
0
0
0
0
1
1
0
0
0
0
0
0
0
0
0
0
1
0
0
0
0
0
59
68
68
0
0
0
0
2
0
1
1
7
3
1
0
1
0
1
0
1
1
1
1
0
3
0
1
1
1
0
11
66
70
63
67.7
68.7
71
71
61
55
64
65
62
69.3
70
63
67
0
0
0
0
0
0
0
1
1
0
1
0
0
0
0
0
0
0
0
0
0
0
0
0
1
0
59
58
61
65
1
2
1
2
3
1
2
10
6
1
1
1
1
1
1
1
1
1
4
1
5
1
1
1
2
60
64
59
62
63.7
65
58.5
61.9
64.3
68
63
61
63
62
65
56
56
65
62.3
69
65.4
63
54
57
67.5
1
2
1
0
3
0
1
1
0
0
0
0
1
0
1
0
0
0
3
0
2
1
0
0
0
60
67.5
68
1
67
1
66
66
63
64
66
63
66.3
64
65
Highest Ten Schools in Subjects
HIGH SCHOOL
Arizona State University
Greater Beirut Evangelical Sch
A.U.B.
Sagesse High School, Ain Saade
St.Louis Univ./Madrid Campus
Col.Prot.Francais, Beirut
City International School,Beir
ICARDA Int'l Sch.Of Aleppo
American School of Kuwait
Brummana High Sch.
HIGH SCHOOL
Off.Sec.Sch,Rayak
Sagesse St.Jean Brazila,Baabda
Srs. Ste. Famille, Hoche-Hala,
David Karam Educ.Cent,Koura
Tannokh.Nat.Sch,Aley
Baptist Sch,Amman
King Saud Int'l. School, Riyad
Leb.Int'l Sc,Beirut
N.D.Nazareth Col., Beirut
Ras Beirut Int'l Sch.,AirPort
HIGH SCHOOL
Amer.Comm.School, Beirut
Lycee Franco Libanais,Verdun
Int'l College, Ras Beirut
Damascus Comm.Sch.
Int'l Sch,Koura
N.D.Nazareth Col.,Beirut
Our Lady of Grace School, Kfar
Leb Evan Sch for B&G,Louaizeh
Omar Mukh.Edu.Cen.Bekaa
St.Joseph,K/Shahwan
HIGH SCHOOL
Greater Beirut Evangelical Sch
Hillcrest Sc,Nijeria
Arizona State University
Bahrain Bayan Sch.
British Int'l Sch.of Jeddah,KS
Lycee Francais Int'l Jeddah
David Karam Educ.Cent,Koura
A/Dhabi Int'l School,UAE
American School of Kuwait
St.Louis Univ./Madrid Campus
Avg. WRITING
SKILLS
71
70
68
68
68
67
65
65
64
63
Avg. MATH
REASONING
71
71
71
70.5
70
70
70
70
70
70
Avg. SCIENCE
71
69
68.2
68
68
68
68
67.5
67.5
66.3
Avg. CRITICAL
THINKING
72
72
71
71
71
71
70
69
69
69
12