Quality in Undergraduate Education
QUE
Susan Albertine
The College of New Jersey
Nevin Brown
Education Trust
Ron Henry
Georgia State University
09/20/02
1
Roles
• Communication specialist – Nevin Brown
• Project director – Susan Albertine
• Standards process experts – Education Trust
– Ruth Mitchell
– Patte Barth
• Funders
– Pew Charitable Trusts – Ellen Wert
– ExxonMobil Foundation – Truman Bell
• Project evaluators - PSA
09/20/02
2
Roles
• Critical friends – disciplinary consultants
–
–
–
–
–
–
–
–
–
Spencer Benson – biology – U.Maryland
Jay Labov – biology – NRC
Gordon Uno – biology – U. Oklahoma - AIBS
Lendol Calder – history – Augustana College
Noralee Frankel – history – AHA
Mills Kelly – history – George Mason
Kathleen Blake Yancey – English – Clemson
Bernie Madison – mathematics – MAA
Jerry Sarquis – chemistry – Miami Univ. Ohio - ACS
09/20/02
3
Language
[NPEC]
‘Learning outcome’ - the knowledge (facts,
concepts, principles) and skills (processes,
strategies, methods) to be learned
‘Standard’ - a predetermined criterion of a level
of student performance
‘Assessment’ - the process of collecting
data/evidence about student learning outcomes
09/20/02
4
Context of our work
• Taxonomy of learning outcomes includes the
following objectives:
Outcomes for a course
Outcomes for a program
• General Education
• Discipline
09/20/02
5
Context of assessment work
Taxonomy of assessment includes the following
activities:
•
•
•
•
•
•
•
Assessment for student in a course
Assessment for student in a program
Assessment of a course
Assessment of faculty instruction in course
Assessment of a program
Assessment of a department
Assessment of an institution
09/20/02
6
Context of our work
Taxonomy of assessment includes the following
activities:
Assessment for student in a course
• Formative
• Summative
Assessment for student in a program
• Developmental
• Summative
09/20/02
7
Why we are here - Objectives
• Assessment – the heart of the matter –
Standards in practice –
Scoring guide development
• To develop teaching strategies for assisting
students in achieving standards
09/20/02
8
Agenda
Cluster Groups: Friday after dinner
 Discussion of successes, barriers; faculty/department
buy-in; two-year/four-year collaboration
Disciplinary Groups: Saturday 8:30-10:30 am; Saturday
1:00-3:30 pm; 3:45-5:00 pm; Sunday 9:00-11:00 am
 How are we accomplishing valid and reliable
assessments?
 How do you know they work?
 Designing rubrics; Using Understanding by Design
 Practicing, modeling, using rubrics to score student
work
 Discussion of successes, barriers; faculty/department
buy-in
09/20/02
9
Agenda
• Plenary Session: Saturday 10:45-noon
Kathleen Blake Yancey
On Evaluating Student Work
09/20/02
10
Some items to ponder
The essential purpose of professional development should be the
improvement of student learning, not just the improvement of
the instructor who is involved in the professional development
activity.
Professional development should be designed to develop the
capacity of instructors to work collectively on problems of
practice, within their own institutions and with practitioners
from other institutions, as much as to support the knowledge
and skill development of individual educators.
09/20/02
11
Some items to ponder
•
•
•
•
Other assessment initiatives:
CUSE – evidence-based learning in intro science
courses; indicators for assessment of program;
evaluation of instruction
ECS – How do institutions move towards a
competency-based system that ensures that all
students acquire AKS?
NGA – How to develop online data bases that
measure course-level quality?
Psychology
09/20/02
12
QUE Deliverables
• Department and campus draft learning
outcomes, performance descriptions,
collections of student work, and assessments
of student learning
09/20/02
13
QUE Milestones
• Stage 1: Learning outcomes: What should
students know, understand, and be able to
do?
– Learning outcomes for level 14
– Learning outcomes for level 16
– Disciplinary contributions to General
Education learning outcomes
09/20/02
14
QUE Milestones
• Stage 2: Assessment: What is acceptable
evidence that students have attained desirable
understandings and proficiencies?
– Aligning assignments with standards
– Developing scoring guides or rubrics – analytical
and holistic
– Constructing performance standards for a learning
outcome
– Scoring student work
09/20/02
15
QUE Milestones
• Stage 3: Practical ideas for learning
experiences and instruction
– Coping with large numbers of students
– Using electronic portfolios
• Moving to program level
– Gap analysis or Super-matrix
09/20/02
16
Super-matrix or gap analysis
Course 1
Course 2
Course 3
Course 4
Course 5
Total
Outcome 1
1
4
4
0
4
13
Outcome 2
2
1
2
0
2
7
Outcome 3
1
2
0
2
0
5
09/20/02
17
Super-matrix or gap analysis
Major (4): Outcome is fully introduced, developed
and reinforced throughout the course. Students
demonstrate an “application knowledge” or
“understanding.”
Moderate (2): Outcome is introduced and further
developed and reinforced in course. Students
demonstrate a “working knowledge” of the outcome.
Minor (1): Outcome is introduced in course. Students
have a “talking knowledge” or “awareness” of the
outcome.
Not at all (0)
09/20/02
18
Super-matrix or gap analysis
For the matrix of courses within program,
comparing program outcomes:
Does the course add significantly to the learning
of the program outcome?
Does the course add significantly to the
assessment of the program outcome?
09/20/02
19
QUE Objectives
• Development and use of standards for lower
division to facilitate the transition to upper
division within 4-year institutions and for
transfer from 2-year to 4-year institutions
• Development and use of standards for
graduation from college
• Levels 14 and 16 represent performance-bound
learning [not the time it takes to get there]
• Learner-centered learning, not time-specific or
place-specific learning
09/20/02
20
Next meetings
Bi-coastal meetings in February/March 2003
Spring breaks
Major association meetings
Focus on design and student work
Potential speaker – Grant Wiggins
National meetings in fall 2003 and spring
2004
09/20/02
21
QUE web site
• QUE Web site is at
Http://www.gsu.edu/que
• private section
– user name standards
– password standards
09/20/02
22
Quality in Undergraduate Education
QUE
Susan Albertine
The College of New Jersey
Nevin Brown
Education Trust
Ron Henry
Georgia State University
09/20/02
23