Answers to your questions
Barbara McNeice-Stallard, Director
Annel Medina Tagarao & John Barkman
Educational Research Assessment Analysts
Research & Institutional Effectiveness
LLRC Retreat, January 2016
Ways to evaluate impact: We can help
• Focus
Groups/Interviews
• Surveys
• Databases (Banner,
Library, Lab)
Focus Groups/Interviews
How do
statistician’s minds
work?
Why do they read
books such as How
to Lie with
Statistics?
What was I thinking?
Why?
Focus Group/Interview Essentials
• 6-8 people per group (or one-on-one for interviews)
• Questions have to be answered with more than yes/no
• Record (type like a crazy person; write on flip chart key points)
• Guide the discussion, but let deviations happen for awhile
• Analysis: Combine themes to re-look at what they are saying (e.g.,
communications –
what about it? Good or bad?
Suggestions for Change?)
• Use of Results
http://www.mtsac.edu/administration/research/pdf/tips/ResearchTips%20v1n3%20focus%20groups.pdf
Rate the plant
arrangement’s color.
(1=hideous;
5=magnificent)
Surveys/Questionnaires
•
•
•
•
Frequency
Opinions
Beliefs
Attitudes
How much do you
love, love, love
your plants?
(1=lots but not
enough; 5=very
much)
Survey Essentials
• makes sense to respondent
• Is concrete (e.g., What do you like the most about this course?)
• uses important time periods (e.g., within the last week)
• uses grade 9 reading level
• Use fewest number of words uses few words
• avoids biasing (e.g., What do you think of this useful legislation?)
• avoids bundling (e.g., using ‘and’ & ‘or’)
• avoids negative phrasing (e.g., Why you would not like a poke in the
eye?)
http://www.mtsac.edu/administration/research/pdf/tips/ResearchTips%20v1n1%20designing%20surveys.pdf
Data From Data: What is happening?
essential and unique to your
program (Luckily…many are
already created!)
• Library activity
• Lab attendance
• Banner (Student
Information System)
• TracDat (outcomes)
Database Essentials
• Quality counts: Garbage in; Garbage out
• Ways to summarize (counts, average, %,
???)
• Group together Look at details
• Standardization: Connect databases for
greater insight
• Your database impact on students and
the college:
•
•
•
•
Grades
Overdue books
Lab attendance
Student Learning Outcomes
http://www.mtsac.edu/administration/research/pdf/tips/Keeping_a_Clean_Local_Database.pdf
Activities (15 minutes)
One per group…
1.Focus Group
2.Survey
3.Database
3 groups to report out
(2 minutes
each…seriously!)
Petrified Wood
Pretend you have your work completed –
all of it.
• If you get the results you expect, will it
actually prove the point?
• What if the findings are something you
didn’t expect? What would it mean?
• How will you use your findings?
• If the above makes you uneasy, go
back and re-look at your design. Time
invested now will save you tons of
time (and heartache) later.
Thank you! research@mtsac.edu