Non-experimental Methods
Observation and
Survey Research
© 2001 Dr. Laura Snodgrass, Ph.D.
1
Observational Methods
• Naturalistic Observation
• Roots in Anthropology
• Field Research
– settings
© 2001 Dr. Laura Snodgrass, Ph.D.
2
Observational Data
• Qualitative
– video or audio tape
– immerse in situation
– describe
– interpret
– explanatory themes and frameworks for understanding
© 2001 Dr. Laura Snodgrass, Ph.D.
3
Observational data
• Quantitative
– coding of events
– develop coding systems
– e.g. SYMLOG
– often requires expectations
– limits and directs observations
– easier to analyze
© 2001 Dr. Laura Snodgrass, Ph.D.
4
Issues in Observational Methods
• Participation
– observation vs. participation
– degree of objectivity
– degree of direct experience
• Concealment
– reactivity
– ethics
– nature of group and setting
© 2001 Dr. Laura Snodgrass, Ph.D.
5
Issues continued
• Scope
– What behaviors are important?
– Negative case analysis
• events that do not fit explanatory themes
– sampling of subsets of behavior
© 2001 Dr. Laura Snodgrass, Ph.D.
6
Issues continued
• The Observer
– biased observations
• expectations
• pre-existing feelings about groups
– reliability
• training
• measures of inter-rater reliability
© 2001 Dr. Laura Snodgrass, Ph.D.
7
Survey Techniques
• Questionnaires
– individuals may not understand or take seriously
– group administration is fast
– mail surveys can give low response rates
• expect about 20%returns
– internet surveys - control over users
© 2001 Dr. Laura Snodgrass, Ph.D.
8
Survey Techniques
• Interviews
– rapport
– interviewer bias
– face-to-face
– telephone
– focus groups
© 2001 Dr. Laura Snodgrass, Ph.D.
9
Problems with Answers
• Response set
– social desirability
– oppositional people
• Lack of variation
– all one end
– all the middle
© 2001 Dr. Laura Snodgrass, Ph.D.
10
Addressing the problems
• Emphasize seriousness and value of data
• Anonymity of respondent
• Multiple questions on same issues and direction of
answers
© 2001 Dr. Laura Snodgrass, Ph.D.
11
Constructing Questions
• Before you start
• Types of questions
• Wording
• Responses
• Pretest
© 2001 Dr. Laura Snodgrass, Ph.D.
12
Before You Start
• Keep objective clear
– no extraneous questions
• Plan data analysis first
– will the data answer your questions
– how will each question be analyzed
© 2001 Dr. Laura Snodgrass, Ph.D.
13
Types of Questions
• Attitudes and Beliefs
• Facts and Demographics
• Behaviors
© 2001 Dr. Laura Snodgrass, Ph.D.
14
Wording
• Keep it simple
– grade school level
– know your population
• Avoid “double-barreled” questions
• Avoid “loaded” questions
– emotional wording
• Avoid negative wording
© 2001 Dr. Laura Snodgrass, Ph.D.
15
Responses
• Open- vs. closed-ended
– ease of analysis
– missing options (choices)
• Number of response alternatives
– usually 5 to 9
• Rating scales
– e.g. strongly agree to strongly disagree
– not at all to very much
© 2001 Dr. Laura Snodgrass, Ph.D.
16
Responses continued
• Numeric scales
– give range and explain end points
• Semantic differentials
– bipolar opposites
• good to bad
• happy to sad
– usually three basic dimensions
• evaluation
• activity
• potency
© 2001 Dr. Laura Snodgrass, Ph.D.
17
Responses continued
• Non-verbal scales
– e.g. faces for kids
• labeling scales
– usually good to label alternatives
– may have different kinds of choices
• e.g. recommendation scales all on upper end
© 2001 Dr. Laura Snodgrass, Ph.D.
18
Pretest
• Show friends or small sample
– talk out loud
– ask questions
– alternative interpretations
• Refine questions
– clarify
– expand options as necessary
© 2001 Dr. Laura Snodgrass, Ph.D.
19