Using an On-line Survey Tool to Streamline

advertisement
Phillip R. Rosenkrantz, Ed.D., P.E.
Industrial & Manufacturing Engineering Department
California State University, Pomona





Introduction
Examples of Survey Use
Design & Deployment Features
Analysis & Sharing Features
Questions/Comments




SurveyMonkey on-line surveys are being used
for a variety of applications
Motivated by past experiences with data
collection and analysis for ABET visits
SurveyMonkey used for a variety of
assessment jobs
Other companies offer similar products (e.g.
Zoomerang)





College Level Assessment
Program Level Assessment
Course Level Management & Assessment
First Year Experience
Capstone Course Sequence

Cal Poly Pomona Project Symposium
Feedback
◦ Industry evaluators for senior project presentations
surveyed for overall assessment of outcomes vs.
importance
◦ Gap analysis was easily created using summarized
survey data

College of Engineering Assessment Interest
Survey
◦ College faculty and administrators involved with
preparing the self study surveyed to identify
assessment committee priorities

IME Department 2005 Alumni Survey –Every
three years to:
◦ Assess progress on objectives
◦ Feedback on changing curriculum needs.

Student Advising Survey
◦ Comprehensive survey to explore student attitudes
and practices
◦ Revised advising program based on actual student
feedback.

IME Basic Knowledge Survey –
◦ Assesses student retention of knowledge and
concepts from five selected lower division courses.
◦ Valuable in finding out where to put place efforts to
increase learning.
◦ Validity and Reliability were considered

ME Summer Class Needs Survey
◦ A quick survey of ME students regarding summer
course offerings
◦ resulted in higher-than-expected summer course
attendance

IME Outcomes Assessment Survey

IME 312 Statistics Knowledge Survey

EGR 403 On-line Introductory Survey

EGR 403 Team Member Evaluations

IME 301 Project Proposal
◦ Primary outcomes assessment tool for a-k outcomes in key courses
◦ Looks for improvement & weaknesses to target for special
attention.
◦ Pre and Post knowledge survey to assess student learning
◦ Uses Bloom’s Taxonomy.
◦ Find out student interests, strengths, and weaknesses for forming
project teams
◦ Student assessment of team members
◦ Used midway through and at the end of the project
◦ Proposal form for student data collection and analysis projects.
◦ Faster and more efficient than Blackboard survey tool

Engineering First Year Experience Survey
◦ Assess the results of the EGR 100 first year
experience course

IME 112 Team Evaluation Survey

IME 112 Time Use & Management Survey
◦ Introductory course team evaluation survey.
◦ Input time management assignment results.
◦ Summarized feedback about student habits and
practices.

IME 112 Outcomes Survey
◦ Assessment of specific course features and
outcomes

Sample Senior Project Report Assessment

Sample Senior Project Presentation Assessment

Senior Project Report Assessment

Reflective Piece Results

Industry Supervisor Senior Project Assessment
◦ Students use to practice scoring written reports as part of a
Senior Project preparation course.
◦ Students use to practice scoring video tapes of oral report
◦ Faculty input assessment of written senior project reports
◦ Evaluator asked to input assessment of a reflective piece
◦ Written by students after completion of the senior project
◦ Covers three a-k soft skill outcomes scored by a rubric.
◦ Industry feedback on professionalism of Senior Project
students.
◦ The survey link is sent in an email message.

Relatively inexpensive for the features provided
◦ SurveyMonkey and Zoomerang monthly “Pro” packages for
about $19/month
◦ Annual packages available for $200/year that will handle
amount needed

Ease of creating a new survey or starting with a
previous survey
◦ User friendly
◦ Old surveys can be imported as templates for new surveys

Color schemes and customized page design
◦ Many color schemes and formats to choose from
◦ Customized designs can be created with a little more
effort

Paging and end of survey options
◦ Can be one page long or broken into separate pages
◦ Responders can be sent to various “thank you” or
other pages

Can embed survey links in email & web pages
and send reminders
◦ Links created for embedding in email or web pages
◦ Can send to email list for tracking and reminder
messages
◦ Reminder feature does produce good results

Surveys can be opened or closed for data
collection as necessary

Sharing survey results

Ability to filter the results by response alternative
(e.g., demographic or specific question choice
◦ Unfiltered survey results can be shared with others
◦ Various levels of sharing interaction allowed
◦ Can be password protected
◦ Surveys can be reused
◦ Results can be filtered to a specific demographic
combination
◦ Avoids need to create a new survey for each application
◦ Can name and save filters

Ability to collect surveys with options required for
IRB approval
◦ Can create various levels of anonymity and confidentiality
◦ IP address, email address, and other info can be excluded

Can create crosstabs for up to five choices for
any question
◦ Powerful, but limited option

Data downloadable in various forms depending
how needed
◦ Data can be downloaded to a spreadsheet as raw data
◦ Can be summarized various ways

Virtual database can be created by carefully
choosing questions
◦ Demographic questions can be chosen to create a data
base allowing for analyzing data over time
◦ Useful feature for outcomes assessment


Phillip R. Rosenkrantz, Ed.D., P.E.
rosenkrantz@csupomona.edu
IME Outcomes, Objectives & Assessment
http://www.csupomona.edu/~ime/objectives.htm
Download