Innovative Educators Webinar
December 1, 2010
Jan Norton, Presenter









Terminology
The Black Box
Timetables
Data/Measurement
Data/Measurement Cautions
Program Outcomes & Examples
Student Outcomes & Examples
Closing & Questions









Visions
Missions
Values
Goals
Objectives
Outcomes
Program vs. Student Outcomes
Learning Outcomes

.






Actions leading to specific, observable results
What students can expect to learn
How students or programs will be different as a result of what happens in the box
What knowledge, skills, and abilities will be improved
What measurable changes occur in tutoring programs

 Scope of assessment
 ≥ 6 years
 2 years
 1 year
 1 semester
 1 course
 1 assignment
 1 tutoring session

 Planning for Assessment
 Continual processes
 Multiple times per semester
 Once or twice per semester
 Two or three times per year
 Once per year
 Every two years
 Every 4 years

 Why Measure
 What to measure
 Before (Benchmarks)
 During
 After (Outcomes)

 How to measure
 Program-generated data
 Course-specific data
 Campus data
 Campus surveys
 Standardized tools
 Locally developed tools

 More about locally developed tools
 Surveys
 Likert-type scales
 Explicated scales
 Behavioral rubrics
 Self assessment by student
 Tutor assessment
 Observer assessment







Direct vs. Indirect Measures
Data access and consistency
Realistic limitations
Cooperation
The Baseline Challenge
Time & money
(and staff and comfort and ……)

 Possible Areas of Assessment Focus
 Establishing baselines
 Usage
 Satisfaction or quality
 Level/variety of services
 Staff development
 Reaching certification or meeting standards





Outcome Statement
 As a result of the new marketing strategy, the number of students attending group tutoring sessions will increase by 10% in Spring 2011.
Timetable
 One semester
Starting Measure (Benchmark)
 Average of past attendance
Ending Measure
 Workshop attendance records





Outcome Statement
 During 2010-11, the number of tutors who earn
CRLA Level I certification will increase by 5%.
Timetable
 One academic year
Starting Measure (Benchmark)
 Number of tutors achieving Level I in previous year(s)
Ending Measure
 Number of tutors achieving Level I in 2010-11





Outcome Statement
 Outside reviewers using the NADE self-study process will score the tutoring program at 85% or above on all sections.
Timetable
 One year
Starting Measure (Benchmark)
 Internal review & self-study
Ending Measure
 External review





Outcome Statement
 As a result of tutor training sessions, tutors will demonstrate an improved understanding of Bloom’s
Taxonomy.
Timetable
 One semester or year, depending on training schedule
Starting Measure (Benchmark)
 Pre-test
Ending Measure
 Post-test

TYPES
 Affective
 Build confidence
 Improve attitude
 Reduce anxiety
 Cognitive
 Mastery of material
 Conscious strategies use
MEASURES
 Qualitative
 Interviews & focus groups
 Opinion surveys
 Observations or self reports
 Quantitative
 Assignment & course grades
 Instrument scores
 Observation rubrics





Outcome Statement
 After being tutored for at least one hour, clients will be more confident in their ability to accomplish the academic task they focused on in tutoring.
Timetable
 One tutoring session
Starting Measure (Benchmark)
 Client self-report
Ending Measure
 Client self report





Outcome Statement
 Students who work with the study skills tutors will demonstrate an increased ability to use memorization strategies.
Timetable
 One tutoring session
Starting Measure (Benchmark)
 Tutor or observer checklist
Ending Measure
 Tutor or observer checklist





Outcome Statement
 75% of the students who work with a reading tutor for at least two sessions will score at least
75% on the main ideas reading test.
Timetable
 Within three weeks
Starting Measure (Benchmark)
 Reading quiz at start of tutoring
Ending Measure
 Chapter reading test in class





Outcome Statement
 Students who attend at least 10 group tutoring sessions for Algebra Review 090 will earn an average grade of B or above in the course.
Timetable
 One semester, length of course
Starting Measure (Benchmark)
 Placement score into course
Ending Measure
 Course grade





Outcome Statement
 After attending mandatory tutoring for all 5 of their developmental education courses, students will be retained until the start of their junior year at a rate at least 25% higher than developmental students for whom tutoring was not required.
Timetable
 Two years
Starting Measure (Benchmark)
 Developmental course placement
Ending Measure
 Enrollment level





Outcome Statement
 Students attending SI for World Religions will “[apply] previously understood information, concepts, and experiences to a new situation or setting.” (CAS)
Timetable
 Within one semester
Starting Measure (Benchmark)
 Pre-survey
Ending Measure
 Post-survey






Check with your institutional research or assessment office
Review early drafts of outcomes statements with your supervisor
Pre-test any locally created measurement tool
Be prepared for outcomes that are not met or wildly exceeded
Remember the black box: we do not own or control the box, and our ability to see into it is very limited