Boston University Center for Excellence and Innovation in Teaching
Common Definitions
Activities undertaken by teachers –
and by their students in assessing themselves—
that provide information to be used as feedback to
modify the teaching and learning activities in which
they are engaged.
Black, P.J. & Wiliam, D. (1998) Inside the Black Box: Raising standards through classroom assessment. King’s
College, London.
(see http://en.wikipedia.org/wiki/Assessment_for_Learning)
Boston University Center for Excellence and Innovation in Teaching
Types of Assessment
•
•
•
•
•
•
•
Assessment for learning
Assessment as learning
Assessment of learning
Evaluation
Pre-assessment
Formative assessment
Summative assessment
Boston University Center for Excellence and Innovation in Teaching
Assessment for Learning
• Comprises two phases.
• Pre-assessment.
• Formative assessment.
• Is based on a variety of information sources.
• Portfolios.
• Works in progress.
• Teacher observation.
• Conversation.
• Provides feedback (verbal or written) to the student that is descriptive and
emphasizes strengths, identifies challenges, and points to next steps.
• Involves instructor adjustment of instruction to keep students on track.
• Does not provide grades or scores; record-keeping is anecdotal and
descriptive.
• Occurs throughout the learning process, from the outset of the course of
study to the time of summative assessment.
Boston University Center for Excellence and Innovation in Teaching
Assessment as Learning
• Requires student awareness of goals of instruction and criteria for
performance.
• Involves goal-setting, monitoring progress, and reflecting on results.
• Implies student ownership and responsibility for moving his or her
thinking forward (metacognition).
• Occurs throughout the learning process.
Boston University Center for Excellence and Innovation in Teaching
Assessment of Learning
• Is accompanied by a grade (summative)
• Compares one student’s achievement with standards
• Produces results that can be communicated to the student
• Occurs at the end of the learning unit
Boston University Center for Excellence and Innovation in Teaching
Evaluation
A judgment made on the basis of a
student’s performance.
Boston University Center for Excellence and Innovation in Teaching
Pre-Assessment
• Determines what a student does and does not know about a topic
• Determine a student's learning style or preferences used to determine
how well a student can perform a certain set of skills related to a
particular subject or group of subjects
• Occurs at the beginning of a unit of study
• Is used to inform instruction
• Makes up the initial phase of assessment for learning
Boston University Center for Excellence and Innovation in Teaching
Formative and Summative Assessment
Formative assessment
• Determines a student’s knowledge and skills, including learning gaps as
they progress through a unit of study.
• Is used to inform instruction and guide learning.
• Occurs during the course of a unit of study.
• Makes up the subsequent phase of assessment for learning.
Summative assessment
• Is done at the end of a unit of study to determine the level of
understanding the student has achieved.
• Grades against an expected standard.
Boston University Center for Excellence and Innovation in Teaching
Levels of Analysis
Assessment can be conducted at various levels:
•
•
•
•
•
Student
Course
Major
Degree
Program
Boston University Center for Excellence and Innovation in Teaching
Association of American Colleges and Universities:
AAC&U
VALUE (Valid Assessment of Learning in Undergraduate Education) assumes that:
•
To achieve a high-quality education for all students, valid assessment data are needed to guide
planning, teaching, and improvement;
•
Colleges and universities seek to foster and assess numerous essential learning outcomes
beyond those addressed by currently available standardized tests;
•
Learning develops over time and should become more complex and sophisticated as students
move through their curricular and cocurricular educational pathways toward a degree;
•
Good practice in assessment requires multiple assessments, over time;
•
Well-planned electronic portfolios provide opportunities to collect data from multiple
assessments across a broad range of learning outcomes while guiding student learning and
building self-assessment capabilities;
•
e-portfolios and assessment of work in them can inform programs and institutions on progress
in achieving expected goals.
http://www.aacu.org/value/index.cfm
Boston University Center for Excellence and Innovation in Teaching
Example:
AACSB Assurances of Learning
Fundamentals
• Introduced outcomes assessment in 1991 standards
• Changed in 2003 to reflect need for improved
accountability measures.
• Emphasize direct assessments of student
learning.
• Requires formulation of specific learning goals.
• Support two principles: accountability and
continuous improvement.
Boston University Center for Excellence and Innovation in Teaching
Outcome Assessment
The systematic collection, review, and use of
information about educational programs undertaken
for the purpose of improving student learning and
development.
Assessment essentials: planning, implementing, and improving
assessment in higher education, Palomba and Banta, 1999.
Boston University Center for Excellence and Innovation in Teaching
Process
1. Define student learning goals and objectives.
2. Align curricula with the adopted goals.
3. Identify instruments and measures to assess
learning.
4. Collect, analyze, and disseminate assessment
information.
5. Use assessment information for continuous
improvement.
Boston University Center for Excellence and Innovation in Teaching
Process: Another Form
1. What will our students learn in our program? What
are our expectations?
2. How will they learn it?
3. How will we know they
have learned it or not?
4. What will we do if they have not learned it?
Boston University Center for Excellence and Innovation in Teaching
Program Focus
The Focus is the Program
• Course-level assessments are not the focus
• Learning goals should reflect:
• Broad educational expectations for each
degree program, regardless of major.
• Major intellectual and behavioral competencies
a program intends to instill in its students due
to the total educational experience across a
given program.
Boston University Center for Excellence and Innovation in Teaching
Program Focus
“Learning goals should express expectations that
reflect the expected depth and breadth of student
knowledge and skills that are the sustainable
foundations for life-long learning in support of their
professional and personal development.”
Boston University Center for Excellence and Innovation in Teaching
Learning Goal Examples
Our graduates will understand the importance of
behaving ethically in their professional lives (i.e., have
an ethical perspective).
Boston University Center for Excellence and Innovation in Teaching
Learning Goal Examples
Corresponding Objectives:
• Our students will know the professional code of conduct
within their discipline.
• Our students will identify the activities/issues in their chosen
profession that may present ethical challenges, and will
articulate the consequences associated with unethical
behavior.
• Our students will identify an ethical dilemma in a scenario
case and apply an ethics model or framework to propose and
defend a resolution.
Boston University Center for Excellence and Innovation in Teaching
Learning Goal Examples
Our graduates will demonstrate problem solving skills,
supported by appropriate analytical and quantitative
techniques.
Boston University Center for Excellence and Innovation in Teaching
Learning Goal Examples
Corresponding Objectives:
• In a case setting, students will use appropriate analytical
techniques to identify a business problem, generate and
compare alternatives, and develop a solution.
• In a case setting, students will recognize and analyze ethical
problems, choose, and defend a solution.
Boston University Center for Excellence and Innovation in Teaching
Learning Goal Examples
Our graduates will be effective communicators.
Corresponding Objectives:
• Our students will produce professional quality business
documents.
• Our students will deliver a professional quality
presentation accompanied by appropriate technology.
• Our students will demonstrate effective interpersonal
communication skills in a team setting.
Boston University Center for Excellence and Innovation in Teaching
Assessment Measures Examples
Learning Objective: Students will demonstrate competency
in a second language to qualify for admission.
Assessment Measure: Acceptable score on standardized
examination of a second language is required for admission
to a business program.
Boston University Center for Excellence and Innovation in Teaching
Assessment Measures Examples (selection)
Learning Objective: Students will demonstrate competency
in statistical analysis and reasoning skills.
Assessment measure: A placement exam is given to measure
competency as part of the admissions process.
Boston University Center for Excellence and Innovation in Teaching
Assessment Measures Examples (course embedded)
Learning Objective: Our students will produce professional
quality business documents.
Assessment measure: Students’ work from a case analysis
assigned in a senior capstone course is assessed for writing
effectiveness.
Boston University Center for Excellence and Innovation in Teaching
Assessment Measures Examples (course embedded)
Learning Objective: Our students will demonstrate
appropriate abilities to integrate knowledge across business
disciplines in a case solution.
Assessment measure: A case analysis presentation in a
business strategy class is assessed for developing a strategic
recommendation that spans functional areas.
Boston University Center for Excellence and Innovation in Teaching
Indirect Measures
Examples: alumni surveys, graduating student surveys,
and/or employer surveys.
Information from indirect measures can complement
the direct assessment processes, but they are not
acceptable substitutes for direct assessment of learning.
Boston University Center for Excellence and Innovation in Teaching
Course Grades Don’t Cut It
•
Course grades are aggregate measures on examinations,
research papers, other projects, homework, presentations,
class participation, etc. and, therefore, cannot measure
students’ performance or achievement of a specific skill or
knowledge set.
•
Professors set the criteria and standards for their courses
and these may not be completely aligned with programlevel learning goals because of the focus on learning
related to the specific business discipline for which the
faculty member is responsible.
Boston University Center for Excellence and Innovation in Teaching
Generalizability?
•
Course level: instructors want to know whether their
students are getting the “key points” of the course.
•
Major
•
•
Some majors have “disciplinary” expectations.
•
Department chairs/deans want to know whether their
students are achieving the learning goals for the major.
Program
•
Most programs have accreditation requirements.
•
Provosts want to know whether students are achieving the
learning goals for the program
Boston University Center for Excellence and Innovation in Teaching
Misconceptions About Program Assessment
1.
The results of assessment will be used to evaluate faculty performance.
No. Faculty awareness, participation, and ownership are essential, but
assessment should not be used to evaluate or judge individual faculty
performance. Program assessment findings are used to improve
programs.
2.
Our program is working well; we don’t need to bother with assessment.
Program assessment findings are used to improve programs – and there’s
always room for improvement.
3.
One person can do this; too many faculty will only complicate things.
Having all faculty involved means bringing in different perspectives and
ideas for improving the academic program – and helps with buy-in.
From University of Central Florida: http://oeas.ucf.edu/doc/acad_assess_handbook.pdf
Boston University Center for Excellence and Innovation in Teaching
Misconceptions About Program Assessment
4. Program assessment might be used to eliminate some programs.
Program assessment is generally intended to be formative evaluation.
5. Assessment is a waste of time and does not benefit the students.
Assessment “forces” articulation of objectives and learning outcomes in
order to improve student learning.
Is this a waste of time?
6. Not so bad; we’ll come up with a good plan and keep using it.
Program assessment must be an ongoing and continuous process.
7. Okay – but it’s time-consuming and complex.
Effective program assessment will take some of your time and effort, but
there are ways to make it manageable.
Boston University Center for Excellence and Innovation in Teaching
AACSB FAQs
Must all students be assessed?
Sampling is okay as long as there is a high degree of
confidence that the data are representative, valid, and
reliable.
Boston University Center for Excellence and Innovation in Teaching
AACSB FAQs
What is the minimum performance standard?
The goal for overall student performance on any given
learning goal should be determined by each school
consistent with its mission, degree programs, and
student profile.
Boston University Center for Excellence and Innovation in Teaching
AACSB FAQs
Must all graduates meet the expected standard on all
learning goals?
No, but if students are not achieving the learning goals
at acceptable levels, action must be taken to strengthen
the curriculum for future students.
Boston University Center for Excellence and Innovation in Teaching
AACSB FAQs
Can a faculty member be responsible for the
assessment of a learning goal within his/her class?
Yes, but the learning goal, operational objective, and
supporting rubrics for evaluating performance must be
used consistently and follow the collective decisions of
the faculty as a whole regarding what constitutes
acceptable student performance.
Boston University Center for Excellence and Innovation in Teaching
AACSB FAQs
Must learning goals be measured separately?
Each learning goal should have its own performance
standard, but a common method or activity can be used
to gather data on more than one learning goal. For
example, a case analysis may be useful for assessing
analytical thinking as well as writing skills or a
presentation may be used for evaluating oral
communication skills and business disciplinary
competence.
Boston University Center for Excellence and Innovation in Teaching
AACSB FAQs
Is benchmarking against other institutions required?
No.
Should student learning be assessed at multiple points
in the curriculum?
This is not required – but it’s a good idea!
Boston University Center for Excellence and Innovation in Teaching
AACSB FAQs
If specific students do not perform well on assessments,
is remediation required?
No, but again, if many students have problems, the
curriculum should be addressed and evidence of that
should be available.
Boston University Center for Excellence and Innovation in Teaching
AACSB FAQs
Can the collective work of student teams be used for
assessment?
Not for individual performance, but the collective work
of the team may provide a basis for assessing
performance as a team member.
How many learning goals are needed?
Four to 10 learning goals for each degree program.
There is no limit, but this is the guidance in order to
keep the assessment program manageable.
Boston University Center for Excellence and Innovation in Teaching
AACSB FAQs
Must all learning goals be assessed each year?
No, but a systematic process is needed to insure all
goals are assessed to support meaningful curricula
change and development. Normally, each goal should
be evaluated at least twice over a five-year AACSB
review cycle.
Boston University Center for Excellence and Innovation in Teaching
AACSB FAQs
What documentation must be maintained?
Schools should maintain copies of instruments, courseembedded assignments, scoring grids or rubrics,
summary of data and analyses, samples of student
products used, documentation that the data was used,
and documentation of the curricula actions that were
taken based on assessment results.
Boston University Center for Excellence and Innovation in Teaching
AACSB FAQs
What are the most popular AACSB learning goals?
• Effective communication.
• Skills.
• Ethics.
• Knowledge of all business disciplines
• Critical thinking.
• Effective decision making.
Boston University Center for Excellence and Innovation in Teaching
AACSB FAQs
What are the most popular AACSB learning goals
(continued)?
• Problem solving ability.
• Ability to integrate across business disciplines.
• Global perspective.
• Team skills.
• Competency in the major.
Boston University Center for Excellence and Innovation in Teaching
Questions/Discussion?
Boston University Center for Excellence and Innovation in Teaching