July 16, 2013
CLA Exam: Preliminary Analysis
Percentile Scores: FR vs. SR
The percentile ranking of freshmen who took the CLA in Fall 2011 and Fall 2012 was slightly higher than
seniors who took the exam in Spring 2012 and Spring 2013.
Performance
Task
Analytic
Writing
Make and
Argument
Critique an
Argument
52.2
54.1
53.5
54.0
43.0
51.1
49.4
52.2
Freshman (2011FA and
2012 FA)
Seniors (2012SP and
2013SP)
Writing Scores
Freshman and seniors writing ability are virtually identical to peers across the USA.
Seniors (N=85)
Analytic Reasoning
and Evaluation
Writing
Effectiveness
Writing
Mechanics
Problem Solving
NU
All
NU
All
NU
All
NU
All
Performance
Task
3.55
3.4
3.55
3.5
3.7
3.7
3.4
3.3
Make-anArgument
3.9
3.6
4
3.7
3.85
3.8
Critique-anArgument
3.45
3.4
3.75
3.5
3.95
3.9
Freshmen (N=151)
Performance
Task
Analytic
Reasoning and
Evaluation
Writing
Effectiveness
NU
All
NU
All
NU
All
NU
All
2.95
2.95
3
2.9
3.3
3.2
2.85
2.7
Writing
Mechanics
Problem Solving
July 16, 2013
Make-anArgument
3.4
3.25
3.45
3.25
3.55
3.4
Critique-anArgument
2.95
2.8
3
2.85
3.55
3.4
Value-Added Score
Value added is calculated at the expected increase in score between freshmen to seniors. Value added
scores are calculated as Z scores (number of standard deviations above/below the mean). The writing
improvement between the freshman group and the senior group is modest and at the expected level in
all categories. A negative score means less improvement was observed, not that skills decreased. There
appears to be a significant variability from year-to-year. However, while we are firmly in the “near
expected improvement” category being well below the 50th percentile in all cases is not where we want
to be.
2011-12
2011-12
Average
Level
Value Added
Score
%-ile
Value Added
Score
%-ile
Value Added
Score
%-ile
Total CLA
Score
near
-0.51
29
-0.23
39
-0.37
34
Performance
Task
near
-0.05
47
-0.51
25
-0.28
36
Analytic
Writing
near
-0.51
25
0.29
60
-0.11
42.5
Make an
Argument
near
-0.28
36
0.42
62
0.07
49
Critique an
Argument
near
-0.84
15
0.25
61
-0.295
38
Input Variables: Expected Academic Ability
Scores were collected for Performance Task, Analytic Writing, Make-an-Argument, and Critique-anArgument. Student performance is evaluated based on EAA (entering academic ability). Data are scaled
into quintiles with labels ranging from “well above average” to “well below average.” Some students
had more than one score, but no students were evaluated on every score. 40% of freshmen scored well
above or above average, compared to 43% of seniors. At the other extreme, a larger fraction of
freshmen (50%) scored at below or well below expected, while only 39% of seniors scored at this level.
July 16, 2013
well above expected
above expected
near expected score
Below expected
well below expected
N scores
Freshmen
13%
17%
19%
27%
23%
270
Seniors
17%
24%
21%
17%
21%
126
Summary
This two-year study has three principal conclusions:
1. The writing and critical thinking skills for NU students is very comparable to their national peers.
2. The improvements in critical thinking skills (the “value added”) for NU students is virtually
identical to what is expected.
3. We now have a reasonable baseline to compare the impact that the NSP program will have on
writing and critical thinking. (FR are all under NSP requirements; SR are all under old core
requirements).