Maximizing Your
Assessment by Combining
Student Survey Results:
EBI, NSSE, and CIRP
Jillian Kinzie, Ph.D.
Associate Director, Center for Postsecondary Research and National Survey of Student
Engagement (NSSE) Institute
John Pryor, M.A.
Director, Cooperative Institutional Research Program, Higher Education
Research Institute
Darlena Jones, Ph.D.
Director of Research and Development, Educational Benchmarking
Session Objectives




Introduction
Overview of CIRP, NSSE, and EBI
Data Triangulation
Using Studies to Measure…







Retention
Integration to Campus
First-Year Program Improvement
Learning Outcomes
Using Multiple Data Sources
Other Data Considerations
Small Group Discussion
Introduction
CIRP: Cooperative Institutional Research Program
What is CIRP?
Introduction

CIRP surveys have been
used for decades to
understand and illustrate
the impact of college.

Can use the surveys as
cross-sectional surveys,
but the key advantage of
CIRP surveys is that they
are longitudinal.
Astin’s I – E – O Model
Environments
YFCY/CSS
(e.g., place of residence
during college, interactions
with peers and faculty, curricular
and co-curricular experiences)
Inputs
CIRP Freshman Survey
(e.g., academic performance
in high school, financial concerns
prior to college entry,
expectations for college,
degree aspirations,
self-concept in high school)
Outcomes
YFCY/CSS
(e.g., satisfaction with college,
retention, gains in college, postcollege plans)
Higher Education Research Institute at UCLA
Cooperative Institutional
Research Program
Freshman
Survey
YFCY
Faculty Survey
CSS
Funded
Research
• Ford Foundation
• Templeton Foundation
• National Institutes
of Health
CIRP Freshman Survey (TFS)

Largest and longest-running national study of
American college students…2008 is 43nd
administration

Initiated in 1966 at the American Council on
Education; Housed at HERI (UCLA) since 1973
CIRP Freshman Survey (TFS)

Three types of input items:
 Pretests
on possible outcomes measures
 Self-predictions about possible future
outcomes
 Personal characteristics that might effect
the propensity to attain certain outcomes
1966
2007
206,865
respondents
374,261
respondents
251
Colleges
and
Universities
536
Colleges
and
Universities
Total over 42 years:
13,408,942 students
1,708 colleges and universities

Your First College Year
(YFCY)

Launched in 2000
 208,157 students at 378
schools

Designed to assess academic
and personal development over
the first year of college

Developed in collaboration with
the Policy Center on the First
Year of College
Your First College Year Survey (YFCY)

Mission: Longitudinal assessment of student development
and institutional/programmatic impact during the first year of
college

Study Launch: Pilot Studies in Spring 2000 & 2001; National
Assessment launched in Fall 2002

Data Collected: Since its inception, surveyed nearly 150,000
first-year students at nearly 250 institutions nationwide

Administration: Disseminated at end of the first year in an
on-campus administration format; paper and/or web versions;
overall response rates 35-82%.
YFCY…


Comprehensive in content to assess academic & social
experiences as well as cognitive & affective development
Designed as a follow-up instrument:



40% of items are direct post-tests to questions on the
CIRP Freshman Survey
Survey Content
 Academic achievement & engagement
 Learning strategies and pedagogical practices
 Residential & employment experiences
 Interactions with family, peers, faculty & staff
 Patterns of behavior
 Student values and life goals
 Satisfaction, self-concept, & feelings of personal
success
 Plans for the next academic year
Space for 20 questions of local relevance
YFCY Deliverables and Data…
Standard Deliverables
 Institutional Profile
 Longitudinal Profile
 Electronic Data File
 Reports on
Spreadsheet
Additional Data Services
 Institutional Data in
PowerPoint Executive
Summary
 Peer Group Reports
 Data Merges
Using YFCY Data to Enhance
Campus Assessment Efforts







Purposes
Self-study reports &
strategic planning
Retention studies
Examining group differences
among students
Programmatic assessment
Measuring student
development & institutional
impact
Feedback to campus
constituents
Creation of a student
information system






Methodologies
Descriptive analyses with
campus data
Comparative analyses
 Institutional subgroups
 Institutional vs. national data
 Aspirant group or consortium
Measures of association
Factor analyses
Trends analyses
Multivariate analyses
NSSE: National Survey of Student Engagement
Who is NSSE?
NSSE Stats…

Launched in 2000, supported by institutional participation fees

Parent survey CSEQ (1979)

National design team created survey

Administered by Indiana University Center for Postsecondary
Research

More than 1,300 different baccalaureate colleges/universities from
50 states, Puerto Rico, and Canada (CCSSE is for 2-year
institutions)

Consortia can add up to 20 questions (e.g., women’s colleges,
HBCUs, state systems)

Spring administration to first-year & senior students

Third party, standardized administration; average 40% response
rate, random sample, and select comparison groups.
NSSE Purposes

Provide reliable indicators of “good
educational practices”

Support institutional improvement and
accountability efforts

Foster comparative & consortium activity

Refocus conversations about quality in
undergraduate education

Direct measure of frequency of participation
in educational practices associated with
learning and development.
Survey Instrument
College student
survey that
assesses the extent
to which students
engage in
educational
practices
associated with
high levels of
learning and
development
Two Components of Student Engagement
students do – time and
energy devoted to
educationally purposeful
activities
 What
institutions do – using
effective educational
practices to induce students
to do the right things
 What
NSSE Reports

Comparative data (National, Carnegie Class, Select
Comparison Groups, and/or Consortium)
 Respondent
 Frequency
 Mean
Distributions
Comparisons
 Benchmark
 Multi-year
 Pocket
Comparisons
Benchmark Comparisons
Guide Report (for prospective students)
 Executive

Characteristics
Snapshot Report
Data File [so you can link to other data!!]
BCSSE – Beginning College Survey of Student Engagement

Based on requests for related precollege measures

Measures selected high school
experiences and students interest in and
expectations for participating in
educationally purposeful activities during
college

Designed to be combined with spring
NSSE data for “pre- and post”- views of
the first-year experience; Use to examine
gap between expectations and
engagement and study effect of students’
background on NSSE scores

Can also be used as stand alone
assessment

Pilots in 2005, 2006; Officially launched
in 2007
NSSE Possibilities




Merging NSSE data with
school records
Descriptive displays of
engagement patterns by
any number of student
characteristics
Prediction models for
retention, degree
attainment, grades, other
outcomes
Tracking student
engagement year to year





Comparisons against
aspirational, regional,
and mission-related
institutions
Program assessment
Accreditation reporting
Consortium and
system data sharing
Scholarly research
EBI: Educational Benchmarking, Inc.
Who is EBI?
EBI Stats…

Founded in 1994 by Joseph Pica, Ed.D and Glenn Detrick (retired
in 2002)

Over 1500 Colleges and Universities (U.S.A. and 15 other
countries like Australia, Mexico, Spain, and Egypt) have
participated in EBI’s studies

Surveyed over 12 million people

Nearly 100 Assessments in 10 areas of higher education

Over 500 custom assessments for schools like MIT, Ohio State,
UCLA, University of Georgia, and University of Florida

Produced over 17,000 customized reports

Creation of WESS
EBI’s Partners (alphabetical order)
AACN (American Association of Collegiate Nursing)
ACUHO-I (The Association of College and University
Housing Officers International)
ACUI (Association of College Unions International)
AFA (Association of Fraternity Advisors)
Ball State University –
MHLI (Military Housing and Lodging Institute)
OTC (Outside the Classroom)
Policy Center on the First Year of College
Relevant Assessments…
EBI First-Year Initiative (FYI)




Mission: To assess the effectiveness of first-year courses /
seminars.
Survey
Development: Created in conjunction with the Policy
f
Center for the First Year of College. Initial funding of project by the
Pew Charitable Trusts. Currently funded by institution participation.
Study Stats: Pilot Study in Spring 2001; National Assessment
launched in Fall 2001. 169 colleges and universities have submitted
over 185,000 responses.
Why Participate: Provides institution information on course
effectiveness. Provides instructors’ their class results for individual
improvement.
Relevant Assessments…
ACUHO-I/EBI Resident Assessment




Mission: To assist college and universities in assessing resident
satisfaction of residence hall functions and the effectiveness of
residence hall life.
Survey Development: Created in conjunction with ACUHO-I.
Housing professionals collaborated with EBI to design survey
questions.
Study Stats: National Assessment began Spring 1998. Since
then, 567 colleges and universities worldwide have submitted over
2.6 million responses.
Why Participate: Provides evidence of student satisfaction of their
on-campus living experience and student learning outcomes linked
to CAS Standards.
Relevant Assessments…
MAP-Works




Mission: An early warning indicator of retention and student
success
Survey Development: Created in conjunction with Ball State
University. Sherry Woosley from BSU collaborated with EBI to
design survey questions.
Study Stats: Study piloted in 2006 – full national study in 2007.
Over 40 institutions are utilizing MAP-Works in 2008
Why Participate: Provides reporting directly to each first-year
student regarding their behaviors benchmarked against their firstyear cohort. Provides information directly to advisors, hall
directors, and FYE instructors regarding their students so early
intervention can occur.
Relevant Assessments…
FoE/EBI Foundation of Excellence Student
and Faculty/Staff Assessments



Mission: Assist campus professionals in the self-study of their
first-year program in support of the Foundations of Excellence
Survey Development: Created in conjunction with the Policy
Center on the First Year of College. Randy Swing, John Gardner,
and Betsy Barefoot collaborated with EBI to design survey
questions.
Study Stats: Studies piloted Summer 2005. National
Assessment launched Fall 2005. Since 2005, 67 schools have
participated in the 4 year study and 39 schools have participated
in the 2 year study.
Assessment Features…
General Features…



Professional Survey Development: EBI’s survey development
team working in partnership with professional organizations
Data Collection: Paper surveys (where available) or online
WESS: State-of-the art data collection, reporting, and distribution
system. Programmed and maintained by EBI software engineers.



View Reports Online: Provides visualizations of results and ability to
download responses and/or calculated data for higher-level analysis
Share Reports Online: Ability to allow access to online reporting to others
Written Analysis: Analysis notebooks with full detailed descriptive analysis
(factor and question means, frequency distributions, etc.) Analysis also
available online in PDF format.
Assessment Features…
Comparisons…



Inter-Institutional Comparisons: Provide comparative
information for high level decision-making and resource allocation
Intra-Institutional Comparisons: The ability to code units
(areas/hall/floors, chapters, or course sections) for internal
comparison to support internal improvement
Longitudinal Comparisons: Provide trend data to assess the
impact of initiatives
Customization…


Statistical Analysis: Provides statistical testing and information
for resource allocation
Institutional Specific Questions: Institutions may add up to 10
questions (5 can be open-ended if using our online surveying
system, WESS)
Data Triangulation
What Is Data Triangulation?

Application and combination of several data points or
sources to overcome weaknesses of single-measure
studies
Why Use Data Triangulation?



Increase confidence in findings through convergence
of different perspectives
The point at which perspectives converge is seen to
represent reality
Builds a rich data resource
Astin’s I-E-O Model
Environments
CSS, YFCY, EBI, NSSE
(e.g., place of residence,
interactions with peers and faculty,
engagement, programmatic features,
curricular & co-curricular experiences)
Inputs
CIRP Freshman, BCSSE,
MAP-Works
(e.g., academic performance
in high school, financial concerns
prior to college entry,
expectations for college,
degree aspirations,
self-concept in high school)
Outcomes
YFCY, CSS,
EBI, NSSE
(e.g., post-college aspirations,
satisfaction with college,
academic and social adjustment,
degree completion rates)
Similarities and Differences
Comparing these instruments, we’ll be answering
the questions...
 What are the unique features of these instruments?
 What are the similarities between these instruments?
 What are the differences between these instruments?
 How do these instruments complement each other?
Next, how these instruments address…
 Retention
 Integration
 Learning Outcomes
 First Year Program Improvement
Retention
NSSE and Retention…
Links between NSSE & Retention Goals:

NSSE is a stimulus for reflection on what the institution
does well & areas for improvement

Focuses attention on ultimate goal of persistence –
improved educational experiences (not just retention)

Identifies what distinguishes the institution, where


Student experience matches espoused mission

Student experience falls short
NSSE results can be connected to institutional data to
build persistence models
NSSE and Retention…
NSSE items and Tinto’s Model of Student Persistence
 Students engagement in effective educational practices
as indication of level of academic & social integration
(1a-v; 6a-f; 9a-g etc.)
 Institutional environment items indicate the level of
commitment to student success (10a-g)
 Student intentions to engage in educational activities
before graduating (7a-h)
NSSE and Retention…

Retention Indicators




Monitor satisfaction
Examine intent to engage in enriching educational
experiences
Identify problematic student behaviors (e.g., % preparing
for class; % preparing 2 or more drafts; frequency of
contact with faculty outside of class)
Retention Studies



Study relationship between engagement and first-year
retention
Compare persisters vs. non-persisters
Study program impact (e.g., first-year learning
communities; course-based service experience)
NSSE and Retention at one institution…
Retention to Second Year by Engagement
100%
90%
93%
91% 92%
84%
79%
95%
91%
88%
84%
84%
91%
92%
91%
86%
85%
93%
89%
88%
87%
82%
93%
92%
89%
85%
82%
80%
70%
60%
50%
40%
30%
20%
10%
0%
Level of Academic
Challenge
Academic and
Collaborative
Learning
Student Interaction
w ith Faculty
Enriching Educational
Experiences
Supportive Campus
Environment
Very Low
79%
84%
85%
82%
82%
Low
84%
88%
86%
87%
85%
Average
91%
91%
91%
88%
89%
High
93%
95%
92%
89%
92%
Very High
92%
84%
91%
93%
93%
CIRP and Retention…




CIRP Freshman Survey asks students to predict the
likelihood that they will transfer, stop-out, or leave
YFCY asks if they have transferred from a different
institution; CIRP TFS pre-test data will be merged with
the YFCY responses even if the pre-test was
administered at a different institution
YFCY asks students to share their plans for their
sophomore year, i.e., intent to re-enroll
Data can be merged with information from the registrar
to compare predicted vs. actual retention rates
CIRP and Retention…


One of the biggest problems with studying retention is
that it is inextricably linked with who you admit (precollege characteristics and performance)
Longitudinal assessment (e.g. CIRP Freshman
Survey and YFCY) allows institutions to control
potentially-biasing background characteristics when
assessing first-to-second year retention rates
“Many of the so-called highly productive institutions turn out to
be underproductive when the caliber of their entering students
is taken into account.” Alexander Astin, 2003
CIRP and Retention…
Consideration of “inputs” and “environments” in retention
analyses identifies populations of at-risk students and
programs/experiences that facilitate or inhibit persistence


Ex. What entering characteristics positively predict first-tosecond year enrollment? Emotional health is a positive
predictor, but recent cohorts of students are exhibiting more
stress & greater levels of depression
Ex. How do service-learning, first-year seminars, and
learning communities affect the decision to re-enroll?
Service learning is a positive predictor on its own; First-year
seminars and learning communities are effective when taken in
combination
EBI and Retention…
Enrollment
Management/R
etention
Academic
Affairs

Who is responsible for student
success on your campus?




Student
Affairs
Enrollment Management/
Retention?
Student Affairs?
Academic Affairs?
What information do you know
about this first-year student?
Student ID: YD252952
HS GPA: 3.93
SAT Verbal: 29
Location: In state
Gender: Female
Race: African American
Age: 18
Major: Undecided
Do you really know them?
EBI and Retention… I’m thinking
Paradigm Shift

What would happen
if…
Enrollment
Management /
Retention
Academic
Affairs
ALL faculty/staff were
First-Year
Seminar
responsible for
Instructor
student success?
I’m really
 YOU knew
Financial
homesick
student was
Aid
struggling?
Could you do something
Minority
about it before it was too
I don’t think I can Student
Affairs
late?
afford college


about
transferring
I’m struggling
in my math
class
Student
Affairs
Residence
Hall Staff
Academic
Advisor
Academic
Department
Heads
My roommate
and I argue all
the time
MAP-Works Process
• Expectations
• Behaviors
• Social Norming
• Expectations
• Campus
Resources
• Student Profile
• Institution Profile
• Campus Resources
• Student
Summary
• Scan Students
Integration to Campus
NSSE and Engagement…
What is the level of student engagement on
your campus?

NSSE data demonstrates what students do and what
students believe the institution emphasizes

Results can be used to shape new student behavior

Criterion reference benchmarking to compare against
predetermined value (e.g., 20% FY students worked
with peers on assignments outside of class – should this
be higher??)
CIRP and Student Involvement
Astin’s Involvement Theory

The amount of student learning and development associated with
any educational program is directly proportional to the quantity
and quality of student involvement in that program

The effectiveness of any educational policy or practice is directly
related to the capacity of that policy or practice to increase
student involvement
CIRP and Academic Involvement
CIRP
TFS
CIRP
YFCY
Studied > 6 hours/week
49%
68%
Look up scientific research articles and resources
23%
29%
Revise your papers to improve your writing
54%
53%
Seek feedback on your academic work
49%
42%
Ask questions in class
59%
34%
Been bored in class
38%
37%
Asked professor (teacher) for advice after class
28%
15%
“A” average
38%
21%
CIRP and Co-Curricular Involvement
CIRP
TFS
CIRP
YFCY
Performed volunteer work
89%
63%
Attended religious services
79%
53%
Socialized with Friends >6 hrs/week
70%
79%
Student Clubs or Groups > 3 hrs/week
40%
31%
Online Social Network Sites > 3 hours/week
43%
56%
EBI and Interaction…
First-Year Initiative Assessment

Course Improved Connections with Faculty

Course Improved Connections with Peers

Sense of Belonging and Acceptance
AFA/EBI Fraternity/Sorority Assessment

“Fraternity/Sorority Enhanced Interpersonal Relationship Skills”

Top predictor of Overall Program Effectiveness
EBI and Interaction…
FoE/EBI Foundations of Excellence Student
Assessment

Top two predictors of Overall Evaluation of Program:


Overall Evaluation: Transition Support
All Students: Campus Environment
ACUHO-I/EBI Resident Assessment

“Interaction with Others in the Hall” is a top predictor
of Overall Evaluation of Program for 90% of schools
EBI and Interaction…
MAP-Works – all about Interaction…

Levels of Personal One-on-One Interaction





Student to Faculty/Staff (like FYE Instructor / Advisor / Hall Director)
Faculty/Staff interactions with other Faculty/Staff
Upper Administration interaction with Faculty/Staff
Upper Administration interaction with Student
Aggregate Study of Student Interaction

The factor, “Social Integration” is top predictor of a successful social
transition to college
First-Year Program
Improvement
NSSE and FYP Improvement…


Identify strengths and areas for growth
 Ex. FY students frequently engage in active learning
(+); but are involved in enriching educational
experiences at low levels (-)
Assess effectiveness of FY interventions
 Ex. Low student faculty interaction scores at small
college prompts institution to establish mentoring
program, and involve students in undergraduate
research; 2 years later NSSE results rise on this
benchmark
NSSE and FYP Improvement…

Sewanee: University of the South: Concern about FY
students’ low active and collaborative learning results led
Sewanee faculty to rejuvenate first-year seminar course,
increase active pedagogies & collaborate with student affairs to
create rich co-curricular elements.

Worcester Polytechnic Institute: NSSE results showed FY
students less engaged than seniors. Created new FY
interdisciplinary, inquiry-based seminars; better integration of
disciplines; engaging introductory courses. Associate Dean
appointed to Office for the First Year. Assessment plan for the
FY in development with NSSE indicators as key component.

UW Green Bay: Used NSSE items in Freshman Seminar Pilot
Study to test impact of revised seminar.
CIRP and FYP Improvement…

Several common first-year programs are included on
the survey

First-year seminars
 Service learning
 Student-centered pedagogies
 Special Interest Housing
 Tutoring
 Remedial education
 Diversity/multicultural experiences
 Student-faculty research partnerships

Measures satisfaction with numerous campus
services and facilities
CIRP and FYP Improvement…
Using YFCY Data to Improve Programs

Identify patterns of participation in specific programs

Measure levels of satisfaction with services/programs

Conduct sub-group analyses of involvement in programs to
examine differential involvement


Ex. First-year seminars and involvement with faculty
Identify the impact of programs and services on key outcomes of
the first year individually and in combination
EBI and FYP Improvement…
First-Year Initiative Assessment - Mission of study is the










improvement of the first-year course/seminar. Pertinent factors
include “Course Improved…”
Study Strategies
Academic and Cognitive Skills
Critical Thinking
Connections with Faculty
Connections with Peers
Out-of-Class Engagement
Knowledge of Campus Policies
Knowledge of Academic Services
Managing Time and Priorities
Knowledge of Wellness
EBI and FYP Improvement…
ACUHO-I/EBI Resident Assessment
AFA/EBI Fraternity/Sorority Assessment
ACUI/EBI College Union Assessment and
Student Activities Assessment


All studies contain class standing as a categorical question.
Institutions can filter results to first-year students to understand the
impact of their programs
All assessments’ mission is program improvement by
understanding students’ perceptions of their program
EBI and FYP Improvement…
Foundations of Excellence Student and
Faculty/Staff Assessments


Both studies measure the institutions’ delivery of the first year of
college from the first-year students’ perceptions and the
faculty/staff’s perceptions
Assessments are integrated into a full self-study of the
institution’s first year of college overseen by the Policy Center.
EBI and FYP Improvement…
MAP-Works!


Study measures first-year students’ expectations of their firstyear and their perceptions of their environment during that initial
transition
Online reports are sent to each student and their advisor, hall
director, and/or first-year course instructor.



Students’ reports focus on the top areas of transition (i.e. study time, selfmanagement, making friends, residence hall transition, etc.)
Faculty/staff’s reports focus on areas of risk self-reported by student (i.e.
no support system, few expected study hours, expected graduation, etc.)
Institution’s reports focus on areas of low performance by large segments
of population to support improvement of first-year programs.
Learning Outcomes
Self-Reported Outcomes…
Can be trusted so long as…

Questions are clear, with singular meanings and
interpretations.

The information requested is known to respondents.

Respondents believe the questions merit a thoughtful
response.

Response options are complete and appropriate.

Questions avoid risk to the respondent, and do not
encourage the respondent to respond in socially
desirable ways.
NSSE and Learning Outcomes…
Provides student learning process and outcomes
indicators

To what extent are FY students developing
their writing skills?
% FY writing papers > 5 pgs.
 % FY students who prepare 2 or more drafts
 % FY reporting substantial gains in writing
effectively
 % FY worked on paper requiring integrating ideas,
sources

NSSE and Learning Outcomes…
NSSE & Educational and Personal Gains (% “very much” or “quite a bit”)
First-Year
Students
Seniors
Thinking critically and analytically
81%
87%
Acquiring a broad general education
82%
86%
Working effectively with others
66%
78%
Writing clearly and effectively
72%
77%
Learning effectively on your own
70%
77%
Using computing and information technology
65%
76%
Acquiring job or work-related knowledge and skills
57%
72%
Speaking clearly and effectively
60%
72%
Understanding yourself
60%
66%
Analyzing quantitative problems
55%
65%
Solving complex real-world problems
49%
58%
Understanding people of other racial/ethnic backgrounds
50%
52%
Voting in local, state, or national elections
24%
23%
Self-Reported Educational and Personal Gains
from College
CIRP and Learning Outcomes…
Using TFS & YFCY: Academic & Cognitive Indicators



Grade point average
 Goes down by ½ point during first year
Longitudinal change in self-ratings of skills
 Writing ability (+8%)
 Mathematical ability (-3%)
 Academic ability (-5%)
Self-perceived changes
 General knowledge (21% report much stronger)
 Critical thinking skills (19% report much stronger)
 Analytical/problem-solving skills (17% report much stronger
CIRP and Learning Outcomes…
Using TFS YFCY: Other Indicators




Identity development
 +4% in personal rating of self-understanding
Interpersonal skills
 +13% report their ability to work as part of a team
Civic engagement
 +8% in goal to keep up to date with political affairs
Awareness and acceptance of diversity and
multiculturalism
 +3 in goal to improve understanding of other
countries/cultures
EBI and Learning Outcomes…

Academic Assessments are linked to professional learning
outcomes standards…





Undergraduate Business Exit Assessment (AACSB Standards)
Nursing Exit Assessment (AACN Standards)
Teacher Exit Assessment (NCATE Standards)
Engineering Exit Assessment (ABET Standards)
Student Affairs studies are linked to professional learning
outcomes standards, for instance…

ACUHO-I/EBI Resident Assessment (ACUHO-I, CAS, and
NASPA/ACPA Learning Reconsidered)

AFA/EBI Fraternity/Sorority Assessment (AFA, CAS, NASPA/ACPA
Learning and Leadership Reconsidered)


ACUI/EBI College Union Assessment (ACUI and CAS Standards)
ACUI/EBI Student Activities Assessment (Learning Reconsidered,
Leadership Reconsidered, CAS Standards)
Using Multiple Data
Sources
Linking Data…


In-house surveys
National surveys (a growing list)

CIRP








TFS/ YFCY / CSS
NSSE
CSEQ / CSXQ
EBI Benchmarking surveys
Noel Levitz Student Satisfaction Inventory
ETS Major Field Tests
ACT Collegiate Assessment of Academic
Proficiency
Institutional data: GPA, financial aid,
transcripts, retention, certification
tests, etc.
NSSE, EBI, and Engagement…
Washington State University study: What is the
relationship between residence hall environment and
student engagement? Is there a difference by gender?
(Robert Tattershall, Director, Housing and Conference Services, Washington State
University)
 Used
EBI data to characterize those residence hall
floors with high and low interaction
 Identified correlations between NSSE’s
engagement measures and the residence hall
interaction levels
 Found gender differences
NSSE, CIRP, and Retention…



Disaggregated NSSE results from seniors by those
who started at the institution as first-year students and
those who entered as transfer students
Better understand the transfer student experience.
Combines with CIRP to develop a fuller portrait of the
transfer student experience.
CIRP, NSSE, and FYP Improvement at Elon…

Elon University




High institutional ethos of assessment
Building a student database
Value-added assessment methodology guides studies
Both CIRP/YFCY and NSSE…

Lead to additional research
 Are utilized for longitudinal study & benchmarking
 Provide valuable measures of cognitive skills
 Assess student interaction with faculty
 Help to understand overall educational experience
CIRP, NSSE, and FYP Improvement at Elon…

Key uses of YFCY







Time allocation
Benchmark for issues of campus community
Cognitive and skill changes
Measuring use and satisfaction with campus programs and
services
Facilities evaluation (classrooms, labs, library)
Social activities and behaviors
Primarily used by Student Life &, more recently,
General Education Office
CIRP, NSSE, and FYP Improvement at Elon…

Key uses of NSSE

Activities inside the classroom (e.g., speaking up,
presentations, working with students)
 Activities outside the classroom (e.g., plays, museums,
events)
 Evaluation of various functions (e.g., administrators,
academic advising)
 Academic rigor

Primarily used by Student Life & Academic Affairs
Survey Administration Cycles…
Some Examples…

Administer all surveys to all students and merge them together
with campus data to create an extensive student information
system

NSSE every 3 years, alternate CIRP, EBI institutional surveys

Administer NSSE and EBI two years in a row to establish
baseline data and assess needs for first-year learning
communities

Administer NSSE to a random sample of first-year students
and YFCY to those remaining

Administer CIRP TFS and YFCY and/or BCSSE and NSSE for
program participation patterns and outcomes;
supplement more detailed programmatic information
from EBI
Q&A
Contact Us…
Cooperative Institutional Research Program (CIRP)
Higher Education Research Institute at UCLA www.heri.ucla.edu
National Survey of Student Engagement (NSSE)
Center for Postsecondary Research, Indiana University
www.nsse.iub.edu
Educational Benchmarking (EBI)
EBI Service Center, Springfield, MO
www.webebi.com