How we let them drive:
Developing dashboards to deliver multiple years of
NSSE/FSSE data
Our Process
Southern Association of Institutional Research
2010 Conference - New Orleans
Monday, September 27th
4:15 p.m. - 5:00 p.m.
Robin Logan, Director of Institutional Research
Earl Harmsen, Faculty H-E-B School of Business
Michael Schulte, Graduate Research Assistant & Doctoral Candidate
4 year, Private, Catholic
Hispanic Serving Institution
Established 1881 by the
Sisters of Charity of the Incarnate Word
Carnegie: Master’s L
San Antonio: 7th largest city in the US
UIW Demographics
UG
GR
PR
INTL
TOT
5,464
1,241
509
494
7,708
Main
ADCaP
VU
INTL
TOT
69%
15%
10%
6%
100%
HISP
MIN
WHI
INTL
UNK
TOT
49%
11%
21%
11%
8%
100%
Geri Hockfield Malandra. (2008) Accountability and learning assessment in the
future of higher education. On the Horizon, 16(2), 57-71. Retrieved September 10,
2010, from proQuest Education Journals. (Document ID: 1484475591).
Accountability and assessment
have become prominent topics of
institutional and public discussion
Gonyea, R. M., & Kuh, G. D. (Eds.). (2009). Editor’s note: Using NSSE in
institutional research. New Directions for Institutional Research, 141, 1-4. DOI:
10.1002/ir.282
...“increased visibility and importance
of institutional research offices staffed
by highly skilled and competent
professionals who can provide campus
leaders with objective, trustworthy
data about student and institutional
performance”
multiple years of data
NSSE 3 years + 2010
FSSE 2 years + 2010
"still in the binders"
various committees and assessment teams
needed “meaningful” information
too much to comprehend all at one time
the challenge
•
multi-campus institution
•
20 questions
referenced in the QEP
•
QEP population for NSSE required filters to present the
correct views with FSSE data
•
colleges requesting data specific to them
evaluate what had
already been done
narratives...
lots of words!
lots of actions!
unclear
methodologies
data quality
NSSE Multi-Year Data Analysis Guide. (2010). Indiana University Center for
Postsecondary Research. Retrieved September 23, 2010 from
http://nsse.iub.edu/2010_Institutional_Report/pdf/NSSE%20MYDAG.pdf
1. Identify and focus on specific questions
2. Select and employ appropriate methods of
analysis
3. Attend to data quality for each year in the
analysis
4. Take into account changes in NSSE items
and reports across years
5. Merge multiple years of data
how does IR...?
•
deliver what multiple
stakeholders need
•
move towards “plug-and-play”
•
keep it simple
•
keep our sanity
Hughes, C. (2009). Framing the activities of institutions and academic
development units in support of assessment. Innovations in Education and
Teaching International, 46(2), 123-133. Retrieved September 10, 2010, from
Research Library. (Document ID: 1796476001).
•
Universities demonstrate their
commitment to enhancing and
assuring the quality of assessment
through the employment of a
diverse array of initiatives.
NSSE ALL
FSSE ALL
NSSE UNIT
QEP Combined File
Tableau - known!
Interactive - for UNIT - known!
Data source created in MSExcel
One file or multiple files?
NSSE ALL
stacked data for multi-year comparison
check alignment of questions
add identifier columns (YR, FY/SY, GROUPS)
decision about experimental items from prior years
•
Means and
Frequencies
•Means
...
well-intentioned, but not
enough to present to the
community
•Frequency
distributions...
referenced for all but one
benchmark in the QEP
XLSX: NSSE ALL DATA
But what about question
response rate?
But what about knowing
which was used in the
QEP?
But what about ...
XLSX: NSSE ALL DATA
NSSE UNIT
•
much easier
•
same basic steps
•
data for all responses - requires CALCS!
•
match to campus data for campus and college
**not necessary with 2010 because it was institutional
column
XLSX: NSSE UNIT LEVEL DATA
•
What do we include?
filters on the dashboard
QEP notations
benchmark notations
XLSX: NSSE UNIT LEVEL DATA
•
What do we include?
means and frequencies
response rate specifically for the
question
XLSX: NSSE BENCHMARK DATA
•
What do we include?
•
static tables from NSSE ALL
•comparison
Institutions
to NSSE, Master’s, and Peer
FSSE ALL
•
Three concepts to consider:
those questions chosen for QEP
benchmark comparisons to NSSE
with
without
XLSX: FSSE DATA
•
Going beyond the frequencies
•
*for those with NSSE crosswalks
XLSX: FSSE DATA
Incorporating the QEP goals
XLSX: NSSE AGG QEP CALCS
Crosswalk comparison
XLSX: FSSE CROSSWALK
•
Methodology
•
FSSE: Main Campus faculty
only
•
NSSE: Data filtered to Main
Campus respondents
QEP - Should it be its own workbook?
YES!
Respondent Characteristics
XLSX: NSSE RESPONDENTS
XLSX: PEER INSTITUTIONS
End Result
•
NSSE ALL
•
•
almost too big
NSSE UNIT
•
split into three interactive
workbooks because of
size
FSSE
split into four
workbooks because
of size
QEP
limited to 20 questions
End Result
Worksheets:
Data source: MSExcel
started with multiple files
transferred to a single
multi-tabbed file
NSSE Benchmarks
NSSE ALL Data
NSSE UNIT Data
Peer Institutions
NSSE Respondents
FSSE Crosswalk
FSSE Data
Moore, J. (2005). Seven recommendations for creating sustainability
education at the university level: A guide for change agents. International
Journal of Sustainability in Higher Education 6(4), 326-339. Retrieved
September 10, 2010, from Research Library (Document ID: 929185721).
Recommendation: integration of
university plans, decision-making
structures and evaluative
measures and the integration of
the research, service and teaching
components of the university
Gonyea, R. M., & Kuh, G. D. (Eds.). (2009). Editor’s note: Using NSSE in
institutional research. New Directions for Institutional Research, 141, 1-4. DOI:
10.1002/ir.282
...“ascendance of student
engagement and other process
indicators that serve as both proxy
measures for institutional quality
and actionable information to
inform improvement efforts.”
Robin Logan, Director of Institutional Research
Earl Harmsen, Faculty H-E-B School of Business
Michael Schulte, Graduate Research Assistant & Doctoral Candidate