ANALYZING THE CONTENT OF CAAP
Learning College Conference
February 26-27, 2009

COLLEGIATE ASSESSMENT OF ACADEMIC
PROFICIENCY (CAAP)






ACT product
Use four modules college wide
Each region gives only 2 of the 4
Rotation schedule ensures that regions administer a different pair each academic year
Required of all associate degree graduates
Administered in capstone courses

PURPOSE OF CAAP

To measure 4 of the College’s 8 general education outcomes




Demonstrate critical and creative thinking.
Apply basic scientific concepts in a variety of settings.
Communicate effectively in written, oral and symbolic forms.
Exhibit quantitative literacy.

PURPOSE OF CAAP

To identify areas of strength and weakness as compared to graduates of other public community colleges

Results reviewed by curriculum committees, regional academic officers, central academic affairs staff

To inform changes designed to improve student learning

RESULTS CURRENTLY REPORTED







Descriptive statistics
Mean scores by module
Mean scores by subcategory
Percentile ranks
Comparison of mean scores to national norms
Level of effort student reported giving the module
Mean scores and level of effort by region

CAAP CONTENT ANALYSIS REPORTS


Enable analysis of specific content areas:
Writing Skills

Punctuation





Basic Grammar and Usage
Sentence Structure
Strategy
Organization
Style

CAAP CONTENT ANALYSIS REPORTS

Mathematics

Prealgebra





Elementary Algebra
Intermediate Algebra
Coordinate Geometry
College Algebra
Trigonometry

CAAP CONTENT ANALYSIS REPORTS

Science


Analysis
Generalization

Understanding

Critical Thinking

Analysis of Arguments


Evaluation of Arguments
Extension of Arguments

ACT’S PROCESS



Students (ITCC and nationally) are classified into proficiency groups

Top 25%


Middle 50%
Bottom 25%
For each student, percentage of items answered correctly is calculated
For each proficiency group, students’ percentcorrect scores are averaged and displayed using bar graphs

WRITING SKILLS: PUNCTUATION http://www.act.org/caap/tests/writing.html

COMPARISONS AND DIFFERENCES



Reports differences in percentages between Ivy
Tech and national groups


<5% difference considered negligible
5%-10% difference considered moderate

>10% difference considered substantial
Negative differences indicate areas where Ivy
Tech students “had more difficulty with content category items than did the normative group”
Positive differences indicate that Ivy Tech students found items easier than did the norm

WRITING SKILL COMPARISON
N=2627

GROUP TASKS – 15 MINUTES



Divide into teams and identify a “reporter”
Take results of one module
Discuss and determine the following:

In relationship to students at other community colleges, where are our students’ strengths?
….where are their weaknesses?


What should the College do to improve students’ ability in the content area being assessed?
What can your program do to improve students’ ability in the content area being assessed?

REPORTS

In relationship to students at other community colleges, where are our students’ strengths?
….where are their weaknesses?

What should the College do to improve students’ ability in the content area being assessed?

What can your program do to improve students’ ability in the content area being assessed?

ANALYZING THE CONTENT OF CAAP
Cherry Kay Smith
Executive Director of Academic Policy and Assessment
317.916.7810
csmith@ivytech.edu