Collegiate Learning Assessment Report Excerpt from the
Annual Assessment Report
2014-2015 Academic Year Report
Office of Accreditation Assessment and Learning
The Office of Accreditation, Assessment and Learning is responsible for directing the university’s
assessment process, and for coordinating planning and implementation strategies for the assessment of
major fields of study, general education, and academic programs. The office also serves as the primary
liaison to the university's regional accrediting body, the Higher Learning Commission.
University Assessments
One of the main responsibilities of the Office of
Accreditation, Assessment and Learning (AAL)
is the coordination, administration, and
communication of key assessments for the
university. AAL currently oversees the
administration of seven assessments, which
(1) gauge student learning and practices
associated with student success across
students’ educational experiences,
(2) assess faculty and staff experiences related to
work and campus climate, and
(3) provide accountability and demonstrate
compliance in external reporting and
accreditation processes.
The purpose of conducting these assessments is
to collect the data necessary to continuously
improve the Kent State student and employee
experience.
communication skills using performance tasks
and questions, and also gauges overall student
growth in these skills (e.g., analysis and problem
solving, critical reading and evaluation, and
writing mechanics and effectiveness). The
CLA+ was developed and is overseen by the
Council for Aid to Education.
The following assessments are currently
overseen by AAL:
The Collaborative on Academic Careers in
Higher Education (COACHE) Faculty Job
Satisfaction Survey gauges faculty job
satisfaction and experiences, and helps to
identify “drivers of faculty vitality”. Institutions
enter into a three-year COACHE membership to
aid in transforming actionable survey data into
practices and/or policies. COACHE and the
COACHE survey were developed and are
overseen by the Harvard University Graduate
School of Education.
The Beginning College Survey of Student
Engagement (BCSSE) assesses incoming first
year students’ high school experiences and
expectations for participating in educational
practices that have been shown to be linked with
learning and student success during their first
year as undergraduates. The BCSSE was
developed and is administered by the Indiana
University Center for Postsecondary Research.
The Kent State University Graduated
Student Survey (GSS) provides feedback
regarding the quality of recent graduates’
undergraduate education and primary activities
(e.g., employment, education, family) following
graduation. The survey also allows for the
identification of individuals who positively
impacted graduates’ college experience. The
GSS is administered by the Kent State
University Survey Research Lab.
The National Survey of Student Engagement
(NSSE) assesses key aspects of first year and
senior students’ undergraduate experience,
including students’ engagement in educational
practices that have been shown to be linked with
learning and student success. The NSSE was
developed and is administered by the Indiana
University Center for Postsecondary Research.
The Great Colleges to Work For program was
designed to recognize institutions that have
created great workplaces. Employees’ responses
to the ModernThink Higher Education Insight
Survey provide insight into multiple dimensions
of workplace experience and quality. The Great
Colleges to Work For program was developed
and is overseen by The Chronicle of Higher
Education and ModernThink.
The Collegiate Learning Assessment (CLA+)
measures critical thinking and written
The Kent State University Alumni Surveys
provide feedback from alumni regarding the
Office of Accreditation, Assessment and Learning | Division of Academic Affairs | Office of the Provost | Kent State University
Kent State University
AY 2014-2015 Annual Assessment Report
quality of their education, and are administered
in conjunction with academic program review.
The Alumni Surveys are administered by the
Kent State University Survey Research Lab.
The number of assessments administered during
a given academic year varies, as some
assessments are administered annually, while
others are administered at less frequent intervals
(e.g., every three years, only once, in conjunction
with academic program reviews).
Table 1 displays the seven assessments
coordinated and administered by AAL. The first
column shows the name of each assessment,
followed by the groups sampled and occurrence
of each assessment in the second and third
columns, respectively.
Table 1. Assessments Conducted by Accreditation, Assessment and Learning (AAL)
Kent State University Assessments
Assessment
Group(s) Sampled
Occurrence; Years Administered*
Beginning College Survey of Student
Engagement (BCSSE)
First Year Students
Every three years prior to Fall
semester; 2010, 2013
National Survey of Student
Engagement (NSSE)
First Year and Senior Students
Every three years during Spring
semester; 2011, 2014
Collegiate Learning Assessment
(CLA+)
First Year Students (Fall);
Senior Students (Spring)
Every year during Fall and Spring
semesters
Kent State University Graduated
Student Survey (GSS)
Recently graduated (UG)
students who graduated during
the preceding year
Every year during Summer session(s)
Collaborative on Academic Careers in
Higher Education (COACHE) Faculty
Job Satisfaction Survey
Faculty
Spring 2015; Assessment is
administered during first year of threeyear membership
Great Colleges to Work For
University Employees
(Faculty and Staff)
Every year during Spring semester
Kent State University Alumni Surveys
Recently Graduated Students
Varies; Occurs in conjunction with
Program Reviews
*subject to revision as needed
Moving Assessment Forward in AY 2014-2015
Enhancing the coordination, administration, and
communication of Kent State’s university-level
assessments was a key priority for AAL in the
2014-2015 academic year. This priority, which
will be an ongoing one, took the form of multiple
initiatives that moved the university’s
assessments forward this year.
Some of these initiatives include:
Enhancing Communication of Existing
Assessments and Assessment Findings
A central focus during the 2014-2015 academic
year was to enhance awareness about the
assessments that AAL coordinates on behalf of
the university, and to communicate related
assessment findings. To do this, AAL staff
attended university committee meetings
throughout the year, giving presentations about
the assessments, making announcements
regarding upcoming survey initiatives, and
providing handouts summarizing information
about and key findings from a selection of
recently administered assessments. The
following report is one such example of these
efforts to enhance communication about
university assessments and assessment findings.
Office of Accreditation, Assessment and Learning | 2
Kent State University
AAL staff also updated the assessment portion of
its website (www.kent.edu/aal/assessment),
including updating its overall layout and
assessment-specific report materials. AAL staff
also added several new “snapshot” handouts.
These results snapshots are intended to provide
accessible, concise, and visually appealing
findings at-a-glance for internal and external
stakeholders. Additional snapshot documents
will be developed and shared during the coming
academic year.
Enhancing Assessment Processes and
Procedures
Another key focus during the 2014-2015
academic year was to enhance the processes and
procedures used to coordinate and administer
university assessments. AAL staff - together
with the input of newly formed faculty and staff
assessment-specific committees (e.g., KSU
COACHE team, GSS planning group) and other
university staff (e.g., Research Compliance) worked to enhance the methods associated with
each assessment administered, carefully
examining each step of the assessment process.
AAL staff look forward to continuing these
enhancement efforts with each subsequent
survey administration.
Enhancing Assessment Participation
An additional focus of the current academic year
was to enhance participation in university
surveys and assessments in the form of increased
response rates. AAL staff was pleased to achieve
this goal, likely due in part to the
aforementioned enhancements to assessment
communication and administration procedures.
For example, accompanying revisions and
enhancements to the survey recruitment and
administration process, students’ participation in
AY 2014-2015 Annual Assessment Report
the CLA+ during this 2014-2015 academic year
reflected an increase of 30 or more additional
students each semester, compared to the
preceding academic year’s participation - a 30.6%
and 47.6% increase over the 2013-2014 academic
year’s Fall and Spring participation, respectively.
Additionally, employees’ participation in the
Great Colleges to Work For faculty/staff survey
this 2014-2015 academic year marked an
increase of 40 participants compared to last
academic year’s participation - a 30.5% percent
increase over last academic year’s participation.
Efforts to enhance assessment participation will
continue in the coming academic year.
Enhancing Support for University Needs and
Initiatives
An additional objective of the 2014-2015
academic year was to continue to enhance
support of university needs and initiatives. For
the current academic year, this took the form of
various efforts including: providing assessment
data to support Academic Affairs Strategic Plan
metrics; updating and enhancing reporting
through Kent State University’s College Portrait
(Voluntary System of Accountability) together
with Institutional Research (IR; formerly,
RPIE), including new Student Achievement
Measure reporting; developing and
administering two surveys to assess the
assessment management system software needs
of both academic and administrative units; and
creating reports needed as part of Provost’s
Office initiatives.
The primary purpose of conducting university
assessments is to collect the data necessary to
continuously improve the Kent State student and
employee experience, an effort that AAL staff
look forward to continuing to support through
efforts like these in the upcoming academic year.
Report Overview
During the current 2014-2015 academic year,
four key assessments were administered: (1) the
Collegiate Learning Assessment (CLA+), (2) the
Kent State University Graduated Student
Survey (GSS), (3) the Great Colleges to Work
For survey for faculty and staff, and (4) the
Collaborative on Academic Careers in Higher
Education (COACHE) Faculty Job Satisfaction
Survey. An annual update, including overviews
and any available recent findings for each of the
four assessments administered during the 20142015 academic year, is provided below.
Office of Accreditation, Assessment and Learning | 3
AY
2014-2015
UNIVERSITY
ASSESSMENTS
Collegiate
Learning
Assessment
(CLA+)
Kent State University
AY 2014-2015 Annual Assessment Report
Collegiate Learning Assessment (CLA+)
Overview and AY 2014-2015 Update
The Collegiate Learning Assessment (CLA+) is a
computer-based test of undergraduate students’
critical thinking and written-communication
skills using performance tasks and questions.
The assessment, which Kent State University
began administering in 2009, was developed and
is coordinated by the Council for Aid to
Education (CAE).
The test helps the university learn more about
students’ higher-order skills (e.g., critical
thinking), and how much students’ undergraduate
education experience may help to enhance these
skills. Also, a central measure of the CLA+ (an
overall “value-added” score, which captures how
close Kent State students’ actual performance on
the CLA+ is to what their scores are expected to
be) serves as a metric in the Academic Affairs
Strategic Plan, helping the university gauge the
extent to which it is on track for shorter and
longer term goals of enhancing academic
excellence and innovation. Moreover, by
administering the CLA+, the university meets
one of the stipulations of Voluntary System of
Accountability (VSA) membership: that each
member institution conduct an assessment of
student learning outcomes using one of the
currently four approved instruments/methods.
The CLA+ is administered every year, twice a
year - to first time, full-time Kent campus
freshman in the Fall and to Kent campus
graduating seniors in the Spring. Students who
elect to complete the CLA+ do so on a computer
in a proctored testing location on campus. The
assessment takes up to approximately 90 minutes
to complete. Incentives are provided to eligible
students for participation.
The assessment is comprised of two parts: (1) a
Performance Task, for which the student reviews
documents provided and then prepares a written
response to a hypothetical situation, and (2)
Selected-Response Questions that the student
answers after reviewing documents provided. The
performance task assesses analysis and problem
solving, writing effectiveness, and writing
mechanics, while the selected-response questions
assess scientific and quantitative reasoning,
critical reading and evaluation, and critiquing an
argument.
AY 2014-2015 Administration
The CLA+ was administered twice during the
2014-2015 academic year – to eligible first year
students from September 29 – October 31, 2014
in the Fall, and to eligible seniors from February
27 – April 17, 2015 in the Spring. Results from
these assessments will be posted on the
Accreditation, Assessment and Learning website
when available, and will be included in next
academic year’s Annual Assessment Report.
Increased Participation
Accompanying revisions and enhancements to the
survey process, including the survey recruitment
process, students’ participation in the CLA+
during this 2014-2015 academic year reflects an
increase of 30 or more additional students each
semester, compared to last academic year’s
participation. 158 eligible first year students took
the CLA+ in the Fall of this academic year, and
93 eligible graduating seniors took the CLA+ in
the Spring semester – a 30.6% and 47.6% increase
over last academic year’s Fall (n=121) and Spring
(n=63) participation, respectively.
AY 2013-2014 Results
The following are findings from CLA+ administration during the 2013-2014 academic year. During the
2013-2014 academic year, the CLA+ was administered twice – to entering freshman students in the Fall
of 2013 (n=121) and to graduating seniors in the Spring of 2014 (n=63).
Student Characteristics
Most respondents reported that they identify as female (First Year [FY]: 79%, Senior [SN]: 78%), and
as white and non-Hispanic (FY: 74%, SN: 86%). The majority of respondents indicated that English is
their primary language (FY: 95%, SN: 98%), and the most frequently reported level of parents’
Office of Accreditation, Assessment and Learning | 5
Kent State University
AY 2014-2015 Annual Assessment Report
education was a Bachelor’s degree (FY: 38%, SN: 38%). The most frequently reported field of study was
“Helping/Services” (FY: 37%, SN: 29%), and the majority of seniors were non-transfer students (90%).
Table 2 below shows the student respondent characteristics in greater detail. Percentages for first year
and senior students are shown separately.
Table 2. Respondent Characteristics, AY 2013-2014
First Year (FY) n=121; Senior (SN) n=63
Gender
FY % SN %
Female
79
78
Male
19
22
Decline to State
2
0
Item totals
100
100
Race/Ethnicity
African-American/Black, nonHispanic
American Indian/Alaska
Native/Indigenous
Asian
Hispanic or Latino
Native Hawaiian or other Pacific
Islander
Other Race/Ethnicity
White, non-Hispanic
Decline to State
Item totals
FY %
11
Primary Language
English
Other Primary Language
Item totals
SN %
5
2
6
2
0
0
3
2
0
1
74
4
100
2
86
3
100
FY %
95
5
100
SN %
98
2
100
Parent Education
FY %
Less than High School
0
High School
21
Some College
18
Bachelor’s Degree
38
Graduate or Post-Graduate Degree
22
Item totals
100
SN %
0
17
21
38
24
100
Transfer Status
Non-Transfer Student
Transfer Student
Item totals
FY %
----
SN %
90
10
100
Field of Study
Business
Helping/Services
Humanities & Languages
Science & Engineering
Social Sciences
Undecided/Other/N/A
Item totals
FY %
4
37
12
21
15
11
100
SN %
10
29
13
21
16
13
100
Summary CLA+ Results
Table 3 provides an overview of Kent State students’ CLA+ performance relative to the performance of
students at other participating institutions.
Kent State first year students performed well
relative to peers at other institutions, earning an
Table 3. Summary Results, AY 2013-2014
overall mean CLA+ score (1091) placing them in
First Year (FY) n=121; Senior (SN) n=63
the 80th percentile (i.e., the percentage of
FY
SN
institutions whose CLA+ scores were lower than
Mean Score
Total CLA+
1091
1144
Kent State’s students). Kent State senior students
Mean Score
Score
80th
55th
Percentile Rank
earned a total mean CLA+ score (1144) placing
them in the 55th percentile. The table shows
Effect Size v. FY
-0.4
similar patterns of performance among mean
Performance Mean Score
1070
1123
Performance Task scores (FY:1070, SN:1123),
Mean Score
Task
70th
52nd
Percentile Rank
placing Kent State students in the 70th and 52nd
percentiles for first years and seniors, respectively.
Effect Size v. FY
-0.31
Highest among all other Kent State first year
Mean Score
Selected1112
1164
th
rd
mean scores, KSU first years responded well to
Mean Score
Response
85
63
Percentile Rank
Selected-Response Questions, earning a mean
Questions
score (1112) placing them in the 85th percentile.
Effect Size v. FY
-.33
Seniors earned a mean score (1164) for the
Mean Score
Entering
1102
1108
Selected-Response Questions placing them in the
Mean Score
Academic
74th
70th
63rd percentile. Entering Academic Ability (EAA)
Percentile Rank
Ability
represents Kent State students’ prior achievement
Effect Size v. FY
---
Office of Accreditation, Assessment and Learning | 6
Kent State University
AY 2014-2015 Annual Assessment Report
on either the SAT (Math and Critical Reading combined) or the ACT Composite. An ACT-SAT
crosswalk is used to facilitate the comparison of scores. Kent State students mean EAA scores (FY:
1102, SN: 1108) were relatively comparable across classes, placing first years and seniors in the 74th and
70th percentiles, respectively.
The effect sizes in Table 3, reported in standard deviation units, are calculated by first subtracting first
year mean scores from senior mean scores, then dividing these values by the standard deviation of first
year scores. These values (i.e., 0.4 for total mean CLA+ scores, 0.31 for mean Performance Task scores,
and 0.33 for mean Selected-Response Question scores) represent growth estimates, gauging student
performance and learning outcomes growth across classes and over time. Because each of the effect
sizes is a positive value, they represent improved performance (as opposed to an effect size of zero, for
instance, which would indicate no improvement or difference in performance).
Mastery Levels
Table 4 shows Kent State students’ level of mastery on the CLA+. There are four possible Mastery
Levels that both individual students and
institutions (by class) can attain. These levels are:
Table 4. Mastery Levels, AY 2013-2014
“Below Basic”, “Basic”, “Proficient”, and
First Year (FY) n=121; Senior (SN) n=63
FY
SN
“Advanced”. The Mastery Levels that students
receive on individual student score reports are
Mean Mastery Level
Basic
Proficient
based off of their individual total CLA+ scores.
Percent Below Basic
12
6
The Mastery Levels that institutions receive are
Percent Basic
45
29
based off of each class’s (i.e., first year, senior)
Percent Proficient
41
62
mean total CLA+ score.
Percent Advanced
2
3
Table 4 shows that Kent State first year students
were assigned a CLA+ Mastery Level of Basic,
based on the class mean total CLA+ score. Kent
State seniors’ mean total CLA+ score, on the other
hand, translates to a Mastery Level designation of
Proficient for the class, indicating a greater degree
of mastery relative to KSU first year students still
early in their undergraduate careers.
*denotes mean mastery level
100%
75%
SN*
FY*
50%
FY
SN
25%
FY
SN
FY SN
0%
% Below % Basic
%
%
As the table also shows, 86% of Kent State first
Basic
Proficient Advanced
year students were designated as Basic or
Proficient in terms of CLA+ mastery, with a fairly
balanced distribution across the two Mastery Levels (Basic: 45%, Proficient: 41%). 91% of KSU seniors
were designated as having Basic or Proficient levels of mastery, with the majority falling into the
Proficient designation (62%).
Value-Added Scores Estimates
Table 5 and Figure 1 show value-added score estimates for Kent State students. Value-added scores
are intended to capture how students’ undergraduate education experiences contribute to their higherorder skills like critical thinking. Value-added scores are calculated using hierarchical linear modeling
(HLM) and capture the difference between Kent State University’s actual senior mean CLA+ scores and
expected senior mean CLA+ scores. Expected mean scores reflect typical senior CLA+ performance at
institutions testing similar student samples, which are identified based on senior ACT/SAT scores and
mean first year CLA+ performance.
Performance levels based on value-added scores are defined by the CAE as the following:
Above +2.00:
+2.00 to +1.00:
+1.00 to -1.00:
-1.00 to -2.00:
Below -2.00:
“well above expected”
“above expected”
“near expected”
“below expected”
“well below expected”
Office of Accreditation, Assessment and Learning | 7
Kent State University
AY 2014-2015 Annual Assessment Report
Table 5. Value-Added Estimates, AY 2013-2014
First Year n=121; Senior n=63
Value-Added
Score
Total CLA+ Score
-0.28
Performance
Level
Percentile
Rank
Near
34th
-0.96
0.4
Confidence Interval Bounds
Lower
Upper
Performance Task
-0.34
Near
35th
-1.05
0.37
Selected-Response
Questions
0
Near
43rd
-0.75
0.75
Observed CLA+ Score
Table 5 shows Kent State students’ scores for three types of value-added estimates: an overall or total
CLA+ value-added score, a Performance Taskspecific value-added score, and a SelectedFigure 1. Value-Added Estimates, AY 2013-2014
Response Questions-specific value-added score.
Observed vs. Expected CLA+ Scores
For all three value-added scores (-0.28, -0.34, and
0, respectively), Kent State students achieved a
performance level of “near expected”, with their
strongest performance on the Selected Response
Questions placing them in the 43rd percentile (i.e.,
the percentage of institutions whose CLA+
Selected-Response value-added scores were lower
than Kent State’s students). The 95% confidence
intervals listed point to the precision of these
estimates. Providing context for the value-added
scores with confidence intervals is important,
given that the scores are expectations - or
estimates - rather than observations.
Expected CLA+ Score
Key:
x Kent State University
x All 4 year CLA+ Colleges and Institutions
e Observed performance equal to expected
Drawing on these models, Figure 1 contextualizes
Kent State students’ performance - in the form of
their total value-added score - relative to the
performance of all other four-year colleges and
institutions participating in the CLA+. The
scatterplot, provided by the CAE, shows all
participating institutions’ expected versus
observed total CLA+ scores. The yellow diagonal
line represents where observed scores are equal to
expected scores. Those institutions with points
above the yellow line have observed scores that
exceeded expected CLA+ scores. Conversely,
those institutions with points below the yellow
line have observed scores that did not exceed
expectations with respect to CLA+ scores.
Office of Accreditation, Assessment and Learning | 8
Kent State University
AY 2014-2015 Annual Assessment Report
Spotlight: Update - Academic Affairs Strategic Plan and the CLA+
Data from the CLA+ serve as one of multiple
metrics measuring Enhancing Academic
Excellence and Innovation - one of six goals
that comprise the Academic Affairs Strategic
Plan. The plan spells out an aim to increase
Kent State’s overall performance level on the
CLA+.
Overall CLA+ performance is captured by
value-added scores, which are indicators of the
extent to which undergraduate education
experiences contribute to student growth in
higher-order skills like critical thinking. As
noted above, these scores capture how close
Kent State students’ actual performance on the
CLA+ is to what their scores are expected to be.
The table and figure below show four value-
added scores. The Baseline 2013 score,
highlighted in the Academic Affairs Strategic
Plan, captures Kent State students’ actual total
value-added score for the 2012-2013 academic
year and a starting point for Strategic Plan
projections regarding the CLA+. The -0.7
score indicates that students’ growth during
the 2013 Baseline year was “near expected”,
according to CAE-defined performance levels.
The second column shows Kent State students’
actual total value-added score for the 20132014 academic year. This -0.28 score –
midway between the Baseline and
Intermediate values highlighted in the
Academic Affairs Strategic Plan - is classified
as “near expected”. The score indicates gains
in the university’s total value-added score,
Academic Affairs Strategic Plan Goals for CLA+ Metric
Total ValueAdded Score
1.25
1
0.75
0.5
0.25
0
-0.25
-0.5
-0.75
-1
Baseline 2013
AY 2013-2014
Intermediate 2015
Target 2018
-0.7
-0.28
0.5
1
Total Value-Added Score
Baseline
AY 2013- Intermediate Target Goal
Actual 2013 2014 Actual Goal 2015
2018
compared to the previous year, suggesting
some movement in the direction of Strategic
Plan goals. Finally, the third and fourth
columns show Intermediate (2015) and Target
(2018) score goals for future CLA+
performance (0.5 and 1, respectively) laid out
in the Plan.
AAL will continue to track progress annually
against these intermediate and target aims for
CLA+ performance as one means to assess the
extent to which Kent State is Enhancing
Academic Excellence and Innovation in line
with Academic Affairs Strategic Plan goals.
Office of Accreditation, Assessment and Learning | 9