Running head: STUDENT LEARNING OUTCOMES FACTOR ANALYSIS
University of Minnesota, Twin Cities Student Learning Outcomes Factor Analysis Report
Krista M. Soria
Analyst
Office of Institutional Research
University of Minnesota Twin Cities
1
STUDENT LEARNING OUTCOMES FACTOR ANALYSIS
2
The purpose of this report is to describe the methodology used to develop factor scores
for the seven University of Minnesota, Twin Cities Student Learning Outcomes utilizing the
Student Experience in the Research University (SERU) survey. Rather than rely upon one survey
item to indirectly measure students’ development within the seven Student Learning Outcomes
(SLOs), I created seven factors comprised of several survey items. The Student Learning
Outcomes developed at the University of Minnesota state that, at the time of receiving a
bachelor’s degree, students:

Can identify, define, and solve problems

Can locate and critically evaluate information

Have mastered a body of knowledge and a mode of inquiry

Understand diverse philosophies and cultures within and across societies

Can communicate effectively

Understand the role of creativity, innovation, discovery, and expression across
disciplines

Have acquired skills for effective citizenship and life-long learning.
The SERU survey is a census survey and was administered from March 2014 to June
2014 to all qualifying undergraduate students at the University of Minnesota, Twin Cities (n =
28,300). The response rate (students who answered at least one survey item) was 29.44% (n =
8,332). Of the responders, 70% were randomly assigned to complete a wildcard module
comprised of items specific to the University of Minnesota, Twin Cities. Of the respondents to
the wildcard module, 4,400 students answered all items that were used to create the factor scores
for the seven student learning outcomes.
STUDENT LEARNING OUTCOMES FACTOR ANALYSIS
3
Of the responders (n = 4,400), the sample was 59.5% female (n = 2,620), 74.3% White (n
= 3,268), 1.0% American Indian or Alaskan Native (n = 45), 9.9% Asian (n = 436), 3.0% Black
(n = 133), 0.3% Hawaiian (n = 13), 2.9% Hispanic (n = 129), 7.8% international (n = 343), and
0.8% were of unknown ethnicity (n = 33). The respondents’ average age was 21.76 years (SD =
5.05) and their mean cumulative grade point average was 3.31 (SD = 0.49). Transfer students
constituted 26.5% of the sample (n = 1,167). By year and term of registration, the sample
included 31.4% first-year students (n = 1,385), 24.81% second-year students (n = 1,092), 20.64%
third-year students (n = 917), 16.45% fourth-year students (n = 724), and 6.41% fifth-year and
beyond (n = 282).
Methodology
Items were chosen through two separate research stages to examine validity and
reliability: (1) the student mapping project and resulting confirmatory factor analysis using the
University of Minnesota’s 2010 SERU data and (2) the examination of the factor structures and
reliability coefficients for the items identified as representative of the Student Learning
Outcomes using SERU 2014 data.
Historical Analyses: Student Item Mapping Project
In 2010, the Office of Institutional Research examined the validity of inferences that
seven SLOs would be manifested by items on student surveys. The findings of this study, in part,
informed the present analyses as the results guided survey item placement into Student Learning
Outcomes factors. In 2010, 199 students were asked to map 110 SERU items to each of the
seven Student Learning Outcomes. In this confirmatory factor analysis, I used both core survey
items and occasionally, module survey items if they were mapped to a particular Student
Learning Outcome by approximately 50% of the students through the mapping process. While it
STUDENT LEARNING OUTCOMES FACTOR ANALYSIS
4
would have been ideal to analyze the core items only, there were not enough indicators to
estimate a solution in some cases. The dataset analyzed was the 2010 SERU survey file for
University of Minnesota students.
Through the confirmatory factor analysis (CFA) process, all items mapped to a particular
Student Learning Outcomes by approximately 50% of students were included in the original
model. Brown (2006) suggested looking at modification indices to locate problematic items that
were unduly influencing model fit. Brown recommended that items with M.I. indices greater
than 3.84 suggests that freeing a parameter to be estimated may enhance model fit. Because
items were selected that enhanced model fit and the proportion of variance explained by the
latent construct (or particular Student Learning Outcome), Brown (2006) also recommended that
the models created should be considered as preliminary. The evaluative criteria for item fit
statistics can be seen in Table 9.
The Present Analysis: Student Learning Outcome Factor Structures
The second stage of our factor analysis identification consisted of estimating Cronbach’s
[1951] reliability coefficient using the SERU 2014 core and wildcard module. Alpha values of
.75 or higher were considered acceptable and the items were further analyzed though exploratory
factor analysis. Items with factor loadings greater than 0.4 and were included in the analysis. To
obtain factors from our survey items, I employed principle components factor analysis
procedures using SPSS 21.0 (IBM, 2012). I also used varimax (orthogonal) rotation and fixed the
number of factors extracted to one. The factors were computed using the regression method and
standardized with a mean of 0.0 and a standard deviation of 1.0.
In the following tables, the superscript of “+” indicated the items came from a set that
created a good Student Learning Outcome model fit using CFA. The superscript of “m” indicates
STUDENT LEARNING OUTCOMES FACTOR ANALYSIS
5
that while the item did not produce a good model fit using CFA, they were mapped by the
majority of University of Minnesota students as representing best reflecting that particular SLO.
Please refer to table 8 to peruse item (a) scales, (b) selection through CFA analysis of 2010
SERU, and (c) mappings to particular Student Learning Outcomes by the majority of students
that participated in the mapping project.
Results
The results of the seven analyses are located in Tables 1-7 below. The results were
remarkably similar to last year’s analyses, with similar factor loadings and alpha reliability
values for each of the seven factors. There were a number of items removed from the core of the
survey that impacted some of the factors; for example, items assessing students understanding of
the importance of personal social responsibility, self-awareness, computer skills, and internet
skills were removed from the core and consequently had to be removed from factor seven (the
Student Learning Outcome related to acquiring skills for effective citizenship). I replaced one of
those missing items with an item in which students were asked to assess the extent to which their
experiences on campus helped them to understand the viewpoints of others from different
societies. Adding this particular item improved the reliability of the factor.
STUDENT LEARNING OUTCOMES FACTOR ANALYSIS
6
Appendix A:
Student Learning Outcomes Factors, Item Loadings, and Reliability
Table 1
Factor Analysis for “Can Locate, Define, and Solve Problems”
Item
Creatively identified and solved a problem related to academic worka
Creatively identified and solved a problem in your personal lifea
Can identify, define, and solve problemsb
Understand the role of creativity, innovation, discovery, and
expression across disciplinesb
Create or generate new ideas, products or ways of understandingc
Explain methods, ideas, or concepts and use them to solve problemsc,+
Current ability level-Analytical and critical thinking skillsd,+
Factor Loading
(α =.757)
.829
.773
.711
.705
.572
.476
.444
Table 2
Factor Analysis for “Can Locate and Critically Evaluate Information”
Item
Examined how others gathered and interpreted data and assessed the
soundness of their conclusionse,+
Judge the value of information, ideas, actions, and conclusions based
on the soundness of sources, methods and reasoningc,+
Reconsidered your own position on a topic after assessing the
arguments of otherse,m
Used facts and examples to support your viewpointe
Critically evaluated an information source (e.g. reviewing a website
for accuracy, critiquing authors’ claims, thinking about the validity of
an argument, etc.)a,m
Break down material into component parts or arguments into
assumptions to see the basis for different outcomes and conclusionsc,+
Current ability level-Other research skillsd
Current ability level-Library research skillsd,m
Current ability level-Ability to read and comprehend academic
materiald,+
Current ability level-Analytical and critical thinking skillsd,m
Can locate and critically evaluate informationb
Extensively revised a paper before submitting it to be gradedf
Factor Loading
(α =.827)
.727
.710
.661
.670
.632
.626
.577
.521
.511
.500
.499
.425
STUDENT LEARNING OUTCOMES FACTOR ANALYSIS
Table 3
Factor Analysis for “Have Mastered a Body of Knowledge and Mode of Inquiry”
Factor Loading
Item
(α =.744)
Felt as though you have mastered major concepts related to your
.720
academic major (e.g. academic portfolio, extended research project,
etc.)a
Current ability level-Understanding of a specific field of studyd,+
.656
b
Have mastered a body of knowledge and a mode of inquiry
.638
Incorporated ideas or concepts from different courses when
.635
e
completing assignments
Found a course so interesting that you did more work than was
.623
f,+
required
Brought up ideas or concepts from different courses during class
.600
f
discussions
Current ability level-Ability to read and comprehend academic
.575
d
material
Table 4
Factor Analysis for “Understand Diverse Philosophies and Cultures Across Societies”
Factor Loading
Item
(α =.759)
Current ability level-Ability to appreciate cultural and global
.842
diversityd,+
Current ability level-Ability to appreciate, tolerate and understand
.826
d,+
racial and ethnic diversity
Understood the viewpoints of others from different societiesa
.724
Understand diverse philosophies and cultures within and across
.658
societiesb
7
STUDENT LEARNING OUTCOMES FACTOR ANALYSIS
8
Table 5
Factor Analysis for “Can Communicate Effectively”
Item
Interacted with faculty during lecture class sessionsg,m
Contributed to a class discussionf,m
Communicated an idea, project, or research to a broader audience in
an effective waya
Made a class presentationf,m
Talked with the instructor outside of class about issues and concepts
derived from a courseg,m
Helped a classmate better understand the course material when
studying togetherg,m
Current ability level-Interpersonal (social) skillsd,+
Worked on class projects or studied as a group with classmates outside
of classg,m
Can communicate effectivelyb
Felt confident in your ability to complete writing assignments
successfully? f
Current ability level-Ability to be clear and effective when writingd,+
Factor Loading
(α =.805)
.725
.662
.629
.621
.597
.586
.581
.574
.489
.474
.467
Table 6
Factor Analysis for “Understand the Role of Creativity, Innovation, Discovery, and Expression
Across Disciplines”
Factor Loading
Item
(α =.794)
a
Believed that research activity is important in any academic discipline
.731
Felt as though you have mastered major concepts related to your
academic major (e.g. academic portfolio, extended research project,
.730
etc.) a
Believed that you were prepared to engage effectively as a citizena
.717
Understand the role of creativity, innovation, discovery, and
.621
expression across disciplinesb
Found that writing activities and assignments helped you to think
.515
critically and/or creatively about course content?h
Current ability level-Other research skillsd
.598
Create or generate new ideas, products or ways of understandingc,+
.538
d
Current ability level-Library research skills
.522
Found a course so interesting that you did more work than was
.508
requiredf
STUDENT LEARNING OUTCOMES FACTOR ANALYSIS
9
Table 7
Factor Analysis for “Have Acquired Skills Necessary for Effective Citizenship and Life-Long
Learning”
Factor Loading
Item
(α =.708)
Gained the skills to ensure that your learning and development will
.848
continue formally and informally throughout your entire lifea
Understood the viewpoints of others from different societiesb
.775
b
Have acquired skills for effective citizenship and life-long learning
.539
Current ability level-Leadership skillsd
.768
Table 8
Survey Item Strings
Superscript
String
a
Since starting at the University of Minnesota, how often
have you engaged in the following behaviors?
b
To what extent do you feel that your experiences on this
campus have contributed to your learning and
development in the following areas?
c
Thinking back over your coursework this academic year,
how often were you REQUIRED to do the following?
d
Please rate your level of proficiency in the following
areas when you started at this institution and now.
e
Thinking back this academic year, how often have you
done the following?
f
During this academic year, how often have you done
each of the following?
g
How frequently have you engaged in these activities so
far this academic year?
h
During this school year, across all of your courses, how
frequently have you…?
+
m
Majority of students mapped item to SLO and
confirmatory factor analysis results suggest good model
fit
Majority of students mapped item to the SLO
Original Coding
1 = never to 6 =
very often
1 = very little to 4 =
very much
1 = never to 6 =
very often
1 = very poor to 6 =
excellent
1 = never to 6 =
very often
1 = never to 6 =
very often
1 = never to 6 =
very often
1 = never to 6 =
very often
STUDENT LEARNING OUTCOMES FACTOR ANALYSIS
10
Table 9
Confirmatory factor analysis and fit statistics with recommended model fit evaluative criteria
Fit Assessment
Absolute Fit
Parsimony
Correction
Comparative Fit
Fit Statistic
Weighted Root Mean Square Residual
(WRMR)
Root Mean Square Residual (WRMR)
Comparative Fit Index (CFI) and TuckerLewis Index (TLI)
Recommended Cut-off
≤ 1.0
≤ 1.0
≥ 0.95
STUDENT LEARNING OUTCOMES FACTOR ANALYSIS
References
Brown, T.A. (2006). Confirmatory factor analysis for applied research. New York: Guilford
Press.
Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika,
16(3): 297–334.
IBM Corp. (2012). IBM SPSS Statistics for Windows, Version 21.0. Armonk, NY: IBM Corp.
11