Surveys as Student Learning
Measures
CLAS Unit Head Meeting
September 27, 2006
Maria Cimitile, CLAS Associate Dean
Priscilla Carreras, CLAS Graduate Assistant
Julie Guevara, Accreditation and Assessment Officer
What are surveys?

Systematic efforts to collect information
about people by asking them to respond
to specific questions.



Helpful in gaining backgrounds, experiences,
plans for the future, opinions, and attitudes.
Can use paper, phone, or internet format.
Provide insight into student attitudes about
programs, assignments, courses, exams etc.
Survey Planning and Conceptualization





Should you use a survey?
What do you want to learn?
Who is the sample or audience?
How will you distribute the survey?
Sampling techniques – interviews vs.
individual survey.
Survey Planning and Conceptualization






Obtain representative sample.
Determine time and people power.
Evaluate survey process when done.
Timeline – when it’ll be created, returned.
IRB Institutional Review Board.
Divide items into two categories: “have to
know” and “nice to know”, then omit “nice
to know”.
Designing the Survey Questions
Close-ended Questions


Closed ended questions are those questions, which can
be answered finitely. Quick and require little time
investment. Can also provide incomplete responses. By
definition, these questions can be restrictive.
Examples: Multiple choice
Rankings
Rating Scale
Ecosystem Rating Scale
Likert Scale
Multiple Choice

Quickly answered and tallied. No room for
unexpected answers.
Chemistry Example:
- Goal: Graduate students who are well
equipped to enter the job market.
- Objective: Technical emphasis majors will enter
the workforce and apply their skills and
knowledge.
- Measure: develop and send employer surveys
to companies that hired our graduates 6-12
months after they’ve been hired.
Rankings
Tedious.
 Assume respondents feel differently about each
response.
 Do not provide size of feeling between ranked
items.
 AVOID THEM!!!
Example: on a scale where “10” means you have a
great interest in a subject and “1” means you
have none at all, how would you rank your
interest in each of the following subjects?

Rating Scale
Another type of multiple choice question.
Example: States conclusions based on data
collection.
 Excellent
 Good
 Fair
 Poor

Eco-system Rating Scale



Ask for two ratings. The second gives
information on the environment in which the
first was made
Observes difference between growth obtained
and growth desired.
Measures attainment of goals and objectives.
Example: Ability to conduct a psychosocial assessment
Importance: Not Critical __1 __2 __3 __4 __ 5 Vital
Preparation: Poor
__1 __2 __3 __4 __ 5 Excellent
Likert Scale
(Agreement Scale)
Efficient
 Permits comparison
 Can be used in place of ranking
Example:

Employees exhibit requisite safety skills in the lab.
Strongly Agree
Agree
Disagree
Strongly Disagree
NA
Numeric Open-ended Questions
Example:
In the past two years (1/04 – 1/06) how
many of our graduates have you hired?
Text Open-ended Questions



Participants compose their own answers.
Provide infinite response.
 Develop trust, are perceived as less
threatening, allow an unrestrained or free
response, and may be more useful with
articulate users.
Not popular in surveys because:
 Lengthen time required to complete survey.
 Require more mental energy.
Examples: How can we achieve the objectives
of this course?
Writing Good Survey Questions
Keep It Short and Simple (KISS)





Keep questions short.
Each item should ask only one question.
Use simple vocabulary – NO JARGON!
Make definitions, assumptions, and
qualifiers clear.
Do not make significant memory
demands.
Writing Good Survey Questions





Do not ask for broad generalizations of
attitudes “Do you consider yourself a good
student?”
Do not ask questions which are
uncomfortable to answer honestly.
When possible, give the option of not
knowing; Not applicable.
Avoid asking leading questions
 Ex. Why are GVSU students your best
employees?
Define terms, qualifiers, assumptions, etc.
Maximizing Participation Rate

Professional appearance of survey is
important.


People are more likely to complete and return
a survey if it looks like someone cares about
it.
Be considerate of your participants.



Include self-addressed & stamped envelope.
Give adequate time for response.
Remember that they are doing you a favor.
Telephone Survey





Survey should be no more than 10
minutes.
Use simple format, with easy to follow
directions.
Avoid open-ended questions.
Pre-test survey by reading it to people
without looking at them.
Document attempts to reach participants.
Employer Surveys


Before completing employer survey, inform
alumni.
Skills


Importance of skills in particular field/work
place.
Skills employee attained at university or in
specific program.
References






Creative Research Systems (2006). The Survey System.
http://surveysystem.com/sdesign.htm
Introduction to Developing Surveys. Georgia Tech Office for
Assessment.
http://www.assessment.gatech.edu/eReports/slide_shows/Short_Gui
de_to_Survey_Research.pdf#search=%22developing%20surveys%
22
Palomba, C.A. and Banta, T.W. (1999). Assessment essentials. San
Francisco, CA : Jossey-Bass.
Richardson, Dr. John V. (2002). Open versus closed ended questions
in the reference environment.
http://polaris.gseis.ucla.edu/jrichardson/dis220/openclosed.htm
Survey How To (2006). University of Texas at Austin Division of
Instructional Innovation and Assessment.
http://www.utexas.edu/academic/diia/assessment/iar/how_to/meth
ods/survey.php
Suskie, L. (2004). Assessing student learning: A common sense
guide. Bolton, MA : Anker Publishing.
USE
SPARINGLY
When
there are more than 6 possible
answers.
When
asking for occupation.
When
structured item may bias
responses by steering participants in
one direction.
When
asking a high interest question,
such as “How would you
change/improve/etc.”
Student Surveys


Surveys of graduating seniors.
Longitudinal Surveys of students attending
university, enrolled in specific program.
Alumni Surveys


Employment related surveys.
Programmatic.


Departmental mission
Skill attainment
Who to Survey?



Students
Alumni
Employers