Measuring Your Success in
Course Redesign
Sandi Connelly
Gosnell School of Life Sciences, RIT, Rochester, NY
Sandra.Connelly@rit.edu or RITGenBio@gmail.com
S
The Dirty Words in Education
1. Success
2. Efficacy
3. Assessment
4. Evaluations
Our Topics for Today
1. How will you measure success?
2. Will you use historical data (common exams, etc.) to
support the efficacy of your assessments (MyLab &
Mastering)?
S
Mastering platform in Sciences similar to MyLab platform in
other disciplines
3. MyLab & Mastering assignments will contribute what
percentage to a students final course grade? =
assessment
4. What do you do with the information evaluated?
Today’s Plan
My course structure
Implementation of a few simple techniques
Quick ways to measure success
How to use MyLab / Mastering to maximize student
learning
How do other data support MyLab / Mastering use?
How much is too much (or too little!) for a grade item?
How to use MyLab / Mastering for Course Assessment
My Course
General Biology (year-long sequence)
Typical # of students: 250-340 (taught in two sections)
This year 408
Typical # of different majors: 55 from all RIT Colleges
~ 50/50 science/non-science
Freshmen -> grad students
Current grade calculation (2013 – 2014):
65-70% exams; 15% Mastering Bio HW; 10% Mastering Bio
quizzes; 5-10% in class participation
Before you measure success…
Are Students Prepared to Succeed?
Preparation (lack of!) comes in many forms
Reading / writing comprehension
English as a Second Language (or 3rd, 4th, ….)
Learning disabilities
Fundamental knowledge in sciences
Study skills
Dedication to the course
Dedication to the degree / college
Etc……
Preparing Students for Success
Homework & Quiz
Assign immediately on Day 1
Provide timely feedback on the assignment
“Suggestions” from instructor do hold water!
Require regular assignments
In Class Response Activities
Every day! (“mock quiz” type)
Immediate or timely feedback critical
Preparing Students for Success
Be clear about the goals of pre-course activities
E.g. pretests, diagnostic tests, etc.
Benefit to students
Not enrolled in a class in which they cannot succeed
Decreases DWF rates
Increase retention – intangibles
Benefit to you / course
Less time on “catch up”, More time on application
Less stresses associated with DWF, office hours, etc.
So you have
“prepared” students
… Now what?
S
Measuring Success –
How? Of what?
Retention (D/W/F)
Common exams?
Shared assessments across sections?
Final exam scores
Final course grades
MyLab / Mastering
Alternative learning initiatives
Measuring Success –
How? Of what?
What works for …
You
Your students
Your discipline
Your assessment office?
Your supervisor?
Measuring Success –
How? Of what?
Let’s focus on MyLab / Mastering as a
success tool …
We can add in the other things later …
MyLab / Mastering = X %
2008-2009: Not much – trial only; assignments optional
3 attempts per question; unlimited time
2009-2010: Not much – trial only; assignments optional
3 attempts per question; unlimited time
2010-2011: 5% of grade; ~20% never did any; nearly all
students who completed the assignments had a 100% avg
3 attempts per question; unlimited time
2011-2012: 15% of grade; all complete at least most; avg scores
88-93%
2 attempts per question; unlimited time
MyLab / Mastering = X %
2012: 20% of grade HW and 5% on quizzes; all students
complete nearly all (> 97% completion rate)
HW = 2 attempts per question; unlimited time
Quiz = 1 attempt per question; 60 min time limit
2012 Example
Mitosis/Meiosis HW: 96 questions; avg time spent = 44
minutes; avg score = 85.9%
Homework due before lecture
Result?
No time in class on process – instead, theory!
MyLab / Mastering = X %
2013 – 2014: 15% of grade HW and 10% on quizzes;
all students complete nearly all (> 98% completion rate)
HW = 2 attempts per question; unlimited time; weekly
Quiz = 1 attempt per question; 60 min time limit;
weekly
Food for thought … is this % too much?
Will students cheat, copy, etc.?
Everything has
challenges …
S
What is your biggest challenge?
Time is the untamed obstacle for everyone ..
Take 5 minutes
Sit down! (or better yet – while you are shampooing!)
What information do you really want your students to
retain?
WHY?
What do you want your students to leave your class
knowing?
WHY?
Challenges to Assessment
Things that I have considered and toyed with …
How much do I want the students to do outside
of class on MyLab/Mastering?
Limited to “Time Estimated” = 1 hour
Less if possible! I want their attention!
Challenges to Assessment
Things that I have considered and toyed with …
What is the value of a long assignment if the
students don’t do it, copy it from another student,
just use all of the hints, etc.?
For students who are failing exams, but have 100%+ on
the homeworks, I look at their time spent in MyLab /
Mastering
Discuss issue with them in person and make note in
the Institute Reporting
Challenges to Assessment
Things that I have considered and toyed with …
Are the students just memorizing answers, or are
they applying the information and retaining it?
Have stopped assigning reading / test bank questions
Implement more low stakes quizzes
Forced repeated recall of information in class
You have data
… Now what?
S
Student Data Assessment
Why assess?
My general interest
Are the students learning what I think they are learning?
How can I improve this course?
University call to assess Gen Ed courses (Accreditation)
Assessment needs?
My interest
Exam and homework scores before and after MyLab / Mastering
implementation
University Guidelines: 1 – 4 scale of comprehension
Student Data Assessment
What was assessed?
Kept it simple to start!
Exam averages
Add as I have interest / time!
Participation in MyLab / Mastering
Scores on MyLab / Mastering
Time spent on MyLab / Mastering
Student Data Assessment
Exam grades (presented herein)
All stored in ExcelTM since 2007
Mastering (not presented)
Export grades & time spent from Mastering to ExcelTM
Processing, Compilation, and Statistical Analyses
ExcelTM
Exam Score Trends
80
75
70
Plant
Animal
65
60
55
50
2008
2009
2010
2011
2012
2013
Err = St Err
Avg n = 244
Talk to me data!
Higher exam averages every year – but leveling out
Significant differences only in plants – but trends are
obvious
Too many students from too many majors to see huge jumps
in scores despite all efforts
Tightening of exam scores
Standard deviations and ranges decreasing
Increasing amount of “average time spent” on MyLab /
Mastering assignments
Assessment Challenges
Why do exam grades change?
Things I have considered
What changes from year to year other than
MyLab / Mastering?
Students?
Accessibility of tutors?
Me?
Order of topics covered?
Lab instructors?
Assessment Challenges
Why do exam grades change?
Things that I want to consider
Students
Time spent on studying (other than MyLab / Mastering)?
Major?
College entrance exams? AP Bio?
DWF rates by major and biology “experience”
Me
Lectures improved?
Lecture structure change with online assignments?
“Napkin Notes”
General observations:
Increased time spent on assignments after 1st exam of term
Increased # questions in office hours on assignments
Increased time spent “off topic” in lecture – not a bad thing!
Changes implemented to date:
Lecture time is more interactive
Participation activities with the students (worksheets, group work,
think-pair-share, flipped lectures, etc.)
Modifications to assignments and quizzes to “challenge” the
students in preparation for class and exams
Advice for Instructors Looking to
Assess Success
Don’t assess while you are teaching the course!
Assessment can be done any time!
Store the data, save it for later
Even if you have time … subconsciously not good for your
teaching!
Don’t try to assess everything at once!
Pick one exam in your course, and compare the average ±
standard deviation (standard error) between years
Even simpler?
Compare exam averages between class sections. TA DA! You just
did an assessment!
Advice for Instructors Looking to
Assess Success
Don’t expect MyLab / Mastering to make your students in to
geniuses!
These are tools! And how the students implement the tools (accept
the tools!) will be highly variable.
You may not see immediate changes in exam grades
Accept the “intangible” as a result!
You can not grade student enthusiasm (which means you can’t
assess it – sorry!)
Student enthusiasm is contagious – not just to other students – but
also to you!
Advice for Instructors Looking to
Assess Success
What can you do with your data?
FLAUNT IT!
White papers
Peer-reviewed papers
Teaching portfolio
Promotion dossier
Assessment office
Final thoughts …
S
What else is assessable?
I got my class back!
Not spending class time on giving quizzes, answering questions
about homework format, etc.
What do we do then? Talk about Biology!
AND … most of my assessment is built in to MyLab /
Mastering and my course management system!
Don’t reinvent the wheel!
Download grades, assign your assessment strategy, .. TA DA!
What else is assessable?
HUGE time savings in my classes
I have “recovered” 35 minutes per week in class (of 150 min) with
no quizzes and homework things in class
Prepared students means < 45 min lecture!
> 105 min biology exploration!
Time saving is so huge … how huge is it?
Enough that I am completely “flipping” one section
DATA! DATA! DATA! DATA! DATA! DATA! DATA! DATA!
But ..
The Most Important Thing to Me?
What the Students Have to Say!
"Mastering Bio helps a lot! The more you have us do,
the more I learn!”
"I wish that our exams were as interactive as Mastering!
I love learning that way!"
"(Favorite part about Mastering) aspects of the
assignments helped me to better grasp the materials
through the videos. I would not have done as well on
those parts of the test without Mastering.”
"More Mastering - 'nuf said!"
Let’s Recap
1. How will YOU measure success?
2. Will YOU use historical data (common exams, etc.) to
support the efficacy of your assessments (MyLab &
Mastering)?
3. MyLab & Mastering assignments will contribute what
percentage to YOUR students final course grade? =
assessment
4. What do YOU do with the information evaluated?
The Dirty Words in Education
1. Success
2. Efficacy
3. Assessment
4. Evaluations
THOUGHTS!?
Sandi Connelly
Gosnell School of Life Sciences, RIT, Rochester, NY
Sandra.Connelly@rit.edu or RITGenBio@gmail.com
S