Evidence
and
Assessment of Student Learning
April 25, 2008
Satellite Student Union
Shoshone Room
8:50 a.m. to 1:00 p.m.
Today’s Agenda
8:45 to 9:10
Networking
9:10 to 9:30
The Why and How of Assessment—Harry Hellenbrand
9:30 to 9:45
More Can Be Less: Assessing Academic Competencies Common to
All Undergraduates and Specific to Programs—Vicki Pedone
9:45 to 10:10
Student Engagement and Learning—Bettina Huber
10:10 to 10:20
Assessment and WASC—Beth Say
10:20 to 10:30
Stretch Break!
10:30 to 11:15
Team/Table Discussions: Table Assignments by College Teams
11:15 to 11:30
Recorders/Reporters prepare team report by area
11:30 to 12:00
Reports from teams to entire group
12:00 to 12:05
Wrap Up—Harry Hellenbrand
12:05 to 1:00
Lunch and Networking
The Why and How of
Assessment
Harry Hellenbrand
Provost and Vice President for Academic Affairs
WE HAVE
HIDDEN
LEARNING
BEHIND
METAPHOR
AND
AUTHORITY
—A BLACK
BOX.
ASSESSMENT
DISCLOSES
LEARNING; OR
DOES IT?
TRENDS, CHANGE
CULTURE
W
A
S
C
IL
KNOW
THYSELF
FSL
THY
STUDENT
THY TESTS
SL
DIRECT, INDIRECT;
TRIANGULATION OF
EFFECTS, NOT
ESSENCE
1.Ns
2.TRAINING
3.GE AND MAJOR
4.NAEP, CLA, LOCAL
5.ENTRY
6.EXIT
?
?
More Can Be Less:
Assessing Academic Competencies
Common to All Undergraduates and
Specific to Programs
Vicki Pedone
Interim University Assessment Coordinator
and
Associate Dean
College of Science and Mathematics
Problems with assessment of SLOs
developed by each program
Redundancy
Lack of expertise in assessment
Lack of common database
Inability to compare data across programs
Standardized vs. Campus-based
Measures
CSU accountability measures include
nationally standardized CLA and NSSE.
These do not replace assessment of
student performance in our programs.
But we need to do it more efficiently and
effectively.
University Assessment Task Force
The charge of the Task Force is to identify
the abilities and intellectual traits that all
students are expected to gain through
their educational career at CSUN.
Two categories of characteristics:
• academic core competencies
• personal core values
Can more assessment be less?
Increase faculty
workload?
Replace program
assessment?
Core Competency Miracle Cure
A multidisciplinary workgroup will determine:
• SLOs of a particular core competency
• assessment instruments
• student populations to assess over time
Assessment in a variety of settings:
• GE Basic Skills • Gateway courses in majors • WPE
• Upper-division GE • Capstone courses in majors
Program instruments of same competency
include basic components--and more.
Advantages: making more less
• Programs can focus on SLOs fundamental
to their discipline.
• Faculty can improve/replace homegrown
assessment tools and strategies.
• Longitudinal measurement is facilitated.
• Differences between student groups can
be used to focus programs.
• Student learning is evaluated “holistically.”
• Implementation can be slow and deliberate.
Challenges
Agreement to measure a subset of common
SLOs using common instruments.
Common data management system for
assessment a MUST
Who will analyze data? Who will make
recommendations to improve student
learning?
Resources needed?
Thanks to Assessment Task Force
Tami Abourezk
Kim Badrkhan
Kira Bracero
Kathy Dabbour
Paula DiMarco
Jordan Eickman
Hilda Garcia-Putzel
Melissa Giles
Steve Graves
Bob Lingard
Leah Marcel
Renee Martinez
Vicki Pedone
Student Engagement
and Learning
at Cal State Northridge
Bettina J. Huber
Director of Institutional Research
NSSE
National Survey of Student Engagement
Engagement: teaching and learning activities
fostering intense involvement with one’s studies
Eighth annual survey: Spring 2007
Entering and exiting students at
523 colleges and universities participated
1,903 Cal State Northridge students participated
(1,040 first time freshmen & 863 graduating seniors)
NSSE
National Survey of Student Engagement
Relied on two comparison groups:
Respondents from six other participating
CSU campuses
(Bakersfield, Dominguez Hills, Fresno, Long Beach, Los
Angeles, and San Francisco)
Respondents from 51 “Carnegie Peers”
(large, public, and primarily non-residential)
Figure 1. Percentage of Respondents Beginning College
Elsewhere by Respondent Group
100.0
90.0
73
80.0
62
51
60.0
50.0
40.0
30.0
9
20.0
10.0
4
Percentage
70.0
2
0.0
Freshmen
CSU Campuses
Seniors
Cal State Northridge
Carnegie Peers
Table 1. Percentage of Respondents Belonging to Different Age
Groups by Respondent Group
Freshmen
Other
CSUs
19 or younger
Seniors
Carnegie
CSUN
Peers
94.0
97.6
88.0
4.9
1.1
2.1
0.3
100.0
(No. of respondents) (1,944)
20 - 23
24 or older
Total
Other
CSUs
Carnegie
CSUN
Peers
7.8
4.3
0.3
40.0
0.0
43.7
0.4
59.0
59.7
56.3
40.6
100.0
100.0
100.0
100.0
100.0
(866)
(20,276)
(2,585)
(707)
(24,634)
Table 2. Percentage of Respondents Belonging to Different
Racial and Ethnic Groups by Respondent Group
Freshmen
Other
CSUs
African American
Seniors
Carnegie
CSUN
Peers
Other
CSUs
Carnegie
CSUN
Peers
6.0
8.1
7.0
6.0
7.1
6.2
Asian
Latina/o
23.6
33.6
12.5
38.2
10.6
10.1
19.7
27.5
13.1
23.5
7.0
9.5
Native American
White
0.4
25.7
0.6
26.6
0.7
63.5
0.9
30.4
0.6
32.9
0.7
67.9
Other
5.5
14.0
3.9
4.9
22.8
3.8
Total
100.0
100.0
100.0
100.0
100.0
100.0
(No. of respondents) (2,434)
(1,040)
(23,272)
(3,023)
(863)
(27,568)
Figure 2. Percentage of Respondents Reporting That They
A. Do Not Live Within Walking Distance of Campus.
B. Spend at Least 6 hours per Week Commuting to Class
88
100.0
90.0
75
80.0
85
B
A
65
60.0
50.0
46
29
34
36
40.0
41
41
41
Percentage
63
70.0
30.0
20.0
10.0
0.0
Freshmen
Seniors
CSU Campuses
Freshmen
Cal State Northridge
Carnegie Peers
Seniors
Figure 3. Percentage of Respondents Reporting That They
A. Work Off Campus for More Than 10 Hours per Week
B. Care for Dependents on a Weekly Basis
100.0
90.0
57
43
47
32
41
32
40.0
53
54
50.0
36
Percentage
60.0
B
64
56
A
70.0
65
80.0
30.0
20.0
10.0
0.0
Freshmen
Seniors
CSU Campuses
Freshmen
Cal State Northridge
Carnegie Peers
Seniors
Figure 4. Percentage of Respondents Reporting That They
Participate in Co-Curricular Activities on a Weekly Basis
(e.g., campus organizations, student government, sports,
fraternities/sororities)
100.0
90.0
80.0
46
53
60.0
50.0
35
34
40.0
36
Percentage
70.0
37
30.0
20.0
10.0
0.0
Freshmen
CSU Campuses
Seniors
Cal State Northridge
Carnegie Peers
Figure 5. Percentage of Respondents Saying
A. My entire educational experience at this institution has
been excellent or good.
B. I would definitely or probably attend the same institution if
I were starting college over again.
81
80
B
78
82
83
80
83
82
83
80.0
A
79
86
90.0
84
100.0
Percentage
70.0
60.0
50.0
40.0
30.0
20.0
10.0
0.0
Freshmen
Seniors
CSU Campuses
Freshmen
Cal State Northridge
Carnegie Peers
Seniors
Figure 6. Percentage of Respondents Saying That, Overall,
the Quality of the Academic Advising They Have Received
Has Been Excellent or Good
100.0
73
75
Percentage
64
70.0
64
60.0
50.0
40.0
30.0
20.0
10.0
0.0
Freshmen
CSU Campuses
Seniors
Cal State Northridge
Carnegie Peers
62
80.0
73
90.0
Figure 7. Percentage of NSSE Respondents Saying
A. Contact Among Students of Differing Socio-Cultural Backgrounds
Receives Substantial Encouragement on Their Campus
B. They Frequently Have Serious Conversations with Students Belonging to
Racial and Ethnic Groups Differing From Their Own
C. They Frequently Include Diverse Perspectives in Coursework
100.0
B
C
40.0
30.0
20.0
10.0
Cal State Northridge
Carnegie Peers
rs
ni
o
Se
hm
en
Fr
es
rs
ni
o
Se
hm
en
Fr
es
rs
ni
o
Se
hm
en
0.0
CSU Campuses
68
60
62
53
50
64
74
44
50.0
58
50
51
53
58
60.0
56
63
Fr
es
Percentage
70.0
62
71
80.0
70
A
90.0
Figure 8. Percentage of NSSE Respondents Saying They
A. Wrote One or More Long Papers (i.e., 20 pages or longer) in 2006-07
B. Frequently Prepared Several Drafts of a Paper Before Submitting It
100.0
A
90.0
B
77
80.0
64
50
54
46
47
47
50.0
57
57
60.0
40.0
30.0
20
15
20.0
17
Percentage
70.0
10.0
0.0
Freshmen
Seniors
CSU Campuses
Freshmen
Cal State Northridge
Carnegie Peers
Seniors
Figure 9. Percentage of NSSE Respondents Frequently
A. Making Class Presentations
B. Working with Others on Projects During Class
C. Working with Others on Projects Outside of Class
100.0
A
90.0
B
C
80.0
68
62
33
27
34
36
46
43
39
40.0
55
52
53
52
47
50.0
57
60
56
56
60.0
30.0
20.0
10.0
CSU Campuses
Cal State Northridge
Carnegie Peers
rs
ni
o
Se
hm
en
Fr
es
rs
ni
o
Se
hm
en
Fr
es
rs
ni
o
Se
hm
en
0.0
Fr
es
Percentage
70.0
Figure 10. Percentage of NSSE Respondents Saying Their College
Education Has Contributed Substantially to Their Ability to:
A. Speak Clearly & Effectively B. Write Clearly & Effectively
C. Solve Complex Real-World Problems
D. Understand People of Other Racial & Ethnic Backgrounds
100.0
A
90.0
80.0
71
C
B
77
73
D
75
69
61
60
60.0
50.0
40.0
30.0
20.0
10.0
CSU Campuses
Cal State Northridge
Carnegie Peers
Se
ni
or
s
en
Fr
es
hm
Se
ni
or
s
en
Fr
es
hm
Se
ni
or
s
en
Fr
es
hm
Se
ni
or
s
en
0.0
Fr
es
hm
Percentage
70.0
65
NSSE Responses by College:
To Whom Do the Respondents Belong?
•
•
•
•
•
•
•
Arts, Media, & Communication
Business & Economics
Engineering & Computer Science
Health & Human Development
Humanities
Science & Mathematics
Social & Behavioral Sciences
112
194
73
131
131
80
125
NSSE Responses by College:
What Do The Tables Cover?
•
•
•
•
•
•
•
Table 1 : Student Satisfaction with Their Education
Table 2: Writing-Intensive Curriculum
Table 3: Active and Collaborative Learning
Table 4: Emphasis on Abstract Thinking Skills
Table 5: Diversity
Table 6: Job-Relevant Experiences
Table 7: Students’ Self-Assessments of Their Learning
Items Discussion Might Focus On
• Aspects of diversity that we are not strong on (e.g.,
conversations with different values or opinions, understanding others’ point
of view)
• Aspects of writing that we are not strong on (e.g., short and
mid-sized papers, integrating ideas from several sources)
• Aspects of active and collaborative learning that we
are not strong on (e.g., discussing ideas from readings, contributing to
class discussion)
• Abstract thinking skills
•
•
•
•
Analyzing the Basic Elements of Ideas or Theories
Applying Theories or Concepts to Practical Problems
Making Judgments About the Value of Arguments or Methods
Synthesizing and Organizing Information or Experiences
• On-the-Job Experience
AAC&U Employer Survey
• Commissioned in mid-2007
• Peter D. Hart Research Associates interviewed 301
employers in November/December period.
• respondents’ companies have at least 25 employees
• 25% or more of new hires have baccalaureate degree
• respondents included CEOs, presidents, and vice presidents
• Issued Report in late January 2008
(How Should Colleges Assess and Improve Student Learning?:
Employers’ Views on the Accountability Challenge)
Table 3. Employers' Views of Various Forms of
Student Assessment
Very or fairly effective in
Ensuring
Evaluating
Worth
Student
Potential
Devoting
Preparedness for Success Resources to
Supervised/evaluated internship/community-based
project (application of college learning in real-world setting)
83%
67%
50%
Advanced comprehensive senior project demonstrating depth of knowledge in major and problemsolving, writing, and analytic reasoning skills
79%
61%
31%
Essay tests to evaluate problem-solving, writing, and
analytic thinking skills
60%
54%
35%
Electronic portfolio of student’s college work
56%
56%
32%
32%
29%
5%
(includes accomplishments in key skill areas and faculty
assessments)
Multiple-choice tests of general content knowledge
Figure 11. Percentage of NSSE Respondents Planning to
A. Acquire On-the-Job Experience
B. Do Community Service or Volunteer Work
C. Complete a Culminating Senior Experience
D. Work with a Faculty Member on a Research Project
100.0
A
90.0
76
80.0
76
73
70.0
D
68
64
60.0
47
50.0
40.0
32
30.0
20.0
10.0
CSU Campuses
Cal State Northridge
Carnegie Peers
Se
ni
or
s
en
Fr
es
hm
Se
ni
or
s
en
Fr
es
hm
Se
ni
or
s
en
Fr
es
hm
Se
ni
or
s
en
0.0
Fr
es
hm
Percentage
C
B
28
Table 4. 2005-06 Cross-Sectional CLA
Administration
Fall 2005 Freshmen
Score
Level
Spring 2006 Seniors
Score
Level
Number of student participants
270
126
Mean SAT score
910
978
2005-06 Value Added
Score
Level
CLA Segments
Performance Task
977
Above
1068
At
Analytic Writing
Make an Argument
Break an Argument
1043
1049
1019
At
1285
1298
1265
Well Above
Overall Score
Expected
Actual
987
1031
At
At
At
1123
1177
Well Above
Above
Above
91
242
249
246
136
146
Below
Above
Above
Above
At
At
Expected
Above
Expected
At
Expected
Table 5. 2005-07 Longitudinal CLA Administration
Fall 2005 Spring 2007 2005-07 Value Added
Score
Score
Score
Level
Number of student participants
80
Mean SAT score
953
CLA Segments
Performance Task
1035
1060
26
At
Analytic Writing
Make an Argument
Break an Argument
1068
1093
1032
1112
1122
1093
44
29
62
At
1058
1082
1093
35
At
Overall Score
Expected
Actual
At
At
Table 6. Two Freshmen CLA Cohorts
Fall 2005 Admin.
Score
Level
Fall 2007 Admin.
Score
Level
Number of student participants
270
157
Mean SAT score
910
924
CLA Segments
Performance Task
977
Analytic Writing
Make an Argument
Break an Argument
1043
1049
1019
Overall Score
Expected
Actual
987
1031
Above
At
At
At
At
973
1087
1105
1058
978
1025
At
Well Above
Well Above
Above
Above
The Learning Habits Project
• Inaugurated in late Fall 2007
• Purpose: to track newly enrolled students likely to succeed at
CSUN in an effort to gain insight into the characteristics and
practices of the most effective among them – that is, we seek to
find out about their learning habits.
• Initial Cohort: 82 freshmen entering in Fall 2007
• High School GPA of 3.5 or higher and/or
• Fully proficient in Mathematics and English at entry
• Will track this cohort, and two subsequent ones, over a
4-6-year period
The Learning Habits Project –
Data Being Gathered
•
•
•
•
Selected class assignments
Course-taking patterns and performance
Demographic information
End-of-term responses to a brief set of open-ended
questions (e.g., characteristics of courses that provided
particularly good learning experiences)
• In-depth interviews in which participating students are
asking to reflect on their learning
Assessment and WASC
Elizabeth Say
Dean
College of Humanities
Let’s Take a Break!
Team/Table Assignments
Wrap Up