NSSE 2014:
Accolades and Action Items
Faculty Senate Nov. 20, 2014
Patrick Barlow, Ph.D., Assessment Coordinator
• Administered to First Year (828) and Senior Students (674)
• Survey had 3 major sections: Standard items (87), Standard
Demographics (22) UW Consortium (23), Experiences with
Writing Module (13)
• 3 Comparison Groups: UW system peers (8), Carnegie Class
(264), entire 13/14 NSSE group (983)
• Accolades:
•
•
•
•
•
Access to High Impact Practices
Supportive campus environment
Improvement in Engagement Indicators FY to SR year
Improvement in Writing Experiences
Overall Satisfaction and Desire to Return.
Accolades
Engagement Indicators: FY
Engagement Indicators: FY
• 62% of FY students experience at least one HIP
(primarily service learning)
• 96% of SR students experience at least one HIP.
HIPs
HIPs
Campus support: FY
Campus Support: SR
Percentage Rating Their Overall
Experience as "Excellent" or "Good"
First-year
UW-L
Percentage Who Would "Definitely" or
"Probably" Attend This Institution Again
First-year
UW-L
93%
UW Comprehensives
90%
UW Comprehensives
87%
85%
Senior
Senior
UW-L
UW-L
94%
UW Comprehensives
UW Comprehensives
89%
0%
25%
50%
75%
91%
100%
86%
0%
25%
Satisfaction & Desire to Return
50%
75%
100%
Most Common to Least Common Writing Tasks: First Year
(based on frequency count of Most or All assignments)
Assignments/Student Behaviors
1 Analyzed or evaluated something you read, researched, or observed
Received feedback from a classmate, friend, or family member about a draft
2
before turning in your final assignment
3 Summarized material you read, such as articles, books, or online publications
4Argued a position using evidence and reasoning
5 Gave Feedback to a classmate about a draft for outline
6Addressed a real or imagined audience
Talked with a classmate, family member, friend to develop your ideas before
7
starting assignment
8 Wrote in the style and format of a specific field
%
56
9 Described your methods or findings related to data you collected
27
10 Explained in writing the meaning of numerical or statistical data
17
49
46
42
39
35
33
30
Instructor Behaviors
1Provided clear instructions describing what she or he wanted you to do
83
2Explained in advance the criteria used to grade your assignment
3Explained in advance what he or she wanted you to learn
78
61
Experiences with Writing: First Year
Most Common to Least Common Writing Tasks: Seniors
(based on frequency count of Most or All assignments)
Assignments/Student Behaviors
1Analyzed or evaluated something you read, researched, or observed
2Wrote in the style and format of a specific field
3Summarized material you read, such as articles, books, or online publications
4Argued a position using evidence and reasoning
5Described your methods or findings related to data you collected
6Addressed a real or imagined audience
Received feedback from a classmate, friend, or family member about a draft
7
before turning in your final assignment
Talked with a classmate, family member, friend to develop your ideas before
8
starting assignment
9Explained in writing the meaning of numerical or statistical data
10Gave Feedback to a classmate about a draft for outline
%
70
62
56
48
40
39
33
30
29
24
Instructor Behaviors
1Provided clear instructions describing what she or he wanted you to do
85
2Explained in advance the criteria used to grade your assignment
3Explained in advance what he or she wanted you to learn
82
64
Experiences with Writing: Seniors
• Action Items:
•
•
•
•
Low Engagement in Learning Strategies (SR&FY)
Low Engagement in Diversity Experiences (SR&FY)
Low Student Faculty Interaction (primarily FY)
Concerns on Access to General Education Courses and
Advising (FY)
• Writing Experiences: Nature of Tasks (FY)
• Addressing Personal Obstacles to Academic Progress (SR)
Action Items
UW-L
Learning
Strategies
Learning
Strategies
UW Comprehensives
Carnegie Class
Mean
Mean
Effect
size
FY
37.7
36.3 *
.10
39.7 *** -.14
39.5 *** -.13
SR
37.8
37.2
.04
41.0 *** -.21
40.3 *** -.17
Mean
Effect
size
NSSE 2013 & 2014
Mean
Effect
size
Notes: Results weighted by institution-reported sex and enrollment status (and institution size for comparison groups); *p<.05, **p<.01,
***p<.001 (2-tailed); Effect size: Mean difference divided by pooled standard deviation; Symbols on the Overview page are based on
effect size and p before rounding.
Learning Strategies (SR % shown)
UWL
Discussions with Diverse
Others
Discussions with Diverse
Others
UW
Comprehensives
Effect
size
Carnegie Class
Mean
Effect
size
NSSE 2013 & 2014
Mean
Effect
size
Mean
Mean
FY 36.6
35.9
.04
40.3 *** -.23
40.9 *** -.27
SR 36.9
37.1
-.01
41.5 *** -.28
41.8 *** -.31
Notes: Results weighted by institution-reported sex and enrollment status (and institution size for comparison groups); *p<.05, **p<.01, ***p<.001
(2-tailed); Effect size: Mean difference divided by pooled standard deviation; Symbols on the Overview page are based on effect size and p before
rounding.
Discussions with Diverse Others (SR % shown)
UW-L
Engagement
Indicator
Student-Faculty
Interaction
Mean
17.8
Your first-year students compared with
UW Comprehensives
Carnegie Class
NSSE 2013 & 2014
Mean
Effect
size
19.9 *** -.15
Mean
Effect
size
20.0 ***
-.15
Mean
Effect
size
20.3 *** -.17
Notes: Results weighted by institution-reported sex and enrollment status (and institution size for comparison groups); *p<.05, **p<.01,
***p<.001 (2-tailed); Effect size: Mean difference divided by pooled standard deviation; Symbols on the Overview page are based on
effect size and p before rounding.
Student Faculty Interaction: FY Results
UW-L
Engagement
Indicator
Student-Faculty
Interaction
Your senior students compared with
UW Comprehensives
Carnegie Class
NSSE 2013 & 2014
Mean
Mean
Effect
size
26.1
24.8 *
.08
Mean
Effect
size
23.2 ***
.18
Mean
Effect
size
23.7 *** .15
Notes: Results weighted by institution-reported sex and enrollment status (and institution size for comparison groups); *p<.05, **p<.01,
***p<.001 (2-tailed); Effect size: Mean difference divided by pooled standard deviation; Symbols on the Overview page are based on
effect size and p before rounding.
Student Faculty Interaction: Senior Results
UW Consortium Item wording or description
My advising interactions help me make better
decisions about my academic goals
UW-L Mean
UW Mean
Effect
size
3.0 ▽
3.1**
-.13
2.7 ▽
2.9***
-.26
2.3 ▲
2.0***
.32
1.7 △
1.6**
.13
(18% Disagree)
How satisfied have you been with the
availability of courses needed to fulfill
general education requirements?
(32% Dissatisfied)
Difficulties getting the courses you need
(45% minor, 27% mod, 10% major)
Lack of good academic advising
(30% minor, 12% mod, 4% major)
Barriers for Academic Progress: FY
Please rate the following as obstacles to your
academic progress during the current academic
year
Lack of personal motivation
(38 % Minor, 18 % Mod, 6% Major)
Poor academic performance
(28% Minor, 9 % Mod, 3% Major)
Personal health issues, physical or mental
(27% Minor, 16% Mod, 6% Major)
UW-L
UW Comprehensives
Effect
size
1.9 △
1.8***
.17
1.6 △
1.4***
.20
1.8 △
1.6**
.13
Barriers to Academic Progress: SR
• First Year Responses to items on writing indicated concerns.
• UWL experience contributed to Writing clearly and effectively
(below Carnegie and NSSE groups)
• Experiences with Writing Module indicated concerns:
• Analyzed or evaluated something you read, researched, or
observed
• Described your methods or findings related to data you
collected in lab or field work, a survey project, etc.
• Argued a position using evidence and reasoning
• Provided clear instructions describing what he or she wanted
you to do
• Explained in advance what he or she wanted you to learn
Writing Experiences: FY concerns
1. For all students, engaging them in more academic and cocurricular activities that expose them to multiple perspectives and
interactions across difference would help address some of the
diversity concerns.
2. Finding paths for first year students to connect with faculty would
be advantageous and would build on aspects of our Firm Footing
project like Eagle Alert and the advising taskforce.
3. Ongoing review of our approaches to writing instruction and the
nature of writing assignments appears warranted. This may be best
started by looking at what is taking place in the first year.
4. Course access for lower division students as an obstacle to progress
may prove to be a good area for review as we are already aware of
some issues for access to science courses and the growing national
interest on the need to monitor and report on graduation rates.
Recommendations
UW-L
Response
rate
Sampling
errorb
35%
First-year
UW
NSSE
Compreh Carnegie 2013 &
ensives
Class
2014
25%
22%
22%
+/- 2.8% +/- 1.5% +/- 0.3% +/- 0.2%
Senior
UW-L
36%
UW
NSSE
Compreh Carnegie 2013 &
ensives
Class
2014
30%
27%
26%
+/- 3% +/- 1.2% +/- 0.3% +/- 0.1%
a. Comparison group response rate and sampling error were computed at the student level (i.e., they are not institution averages).
b. Also called “margin of error,” sampling error is an estimate of the amount the true score on a given item could differ from the estimate
based on a sample. For example,
if the sampling error is +/- 5.0% and 40% of your students reply "Very often" to a particular item, then the true population value is most
likely between 35% and 45%.
Response Rates
Representativeness
Female
Full-time
First-time, first-year
a
Race/ethnicity
Am. Indian or Alaska Native
Asian
Black or African American
Hispanic or Latino
Native Hawaiian/Other Pac.
Isl.
White
Other
Foreign or nonresident alien
Two or more races/ethicities
Unknown
First-year
Respondent % Population %
Senior
Respondent Population %
%
65
100
85
56
99
79
65
95
N/A
58
95
N/A
0
2
1
3
0
2
1
3
0
1
1
3
0
2
1
3
0
87
0
3
3
0
0
87
0
3
4
0
0
91
0
1
2
1
0
90
0
1
3
0
Representativeness: Race/Ethnicity
• First Year ACT scores
• Population Avg/SD= 24.37/2.93
• Respondent Avg/SD=24.67/3.04
• t (3144) = 2.45, p < (.01), Cohen’s d = .10
• Senior ACT scores
• Population Avg/SD= 24.81/2.95
• Respondent Avg/SD=25.03/2.98
• t (2494) = 1.55, p > (.12)
Representativeness: ACT