Different Approaches for
Data Collection/Data Analysis
•
•
•
•
•
•
Experiment
Survey
Quasi-Experiment
Secondary Data Analysis
Content Analysis
Historical Comparative
Methods (Archival
Analysis )
•
•
•
•
•
•
Participant Observation
Ethnography
Individual Qualitative
Interview
Textual Analysis
Focus Group Discussion
Del-Phi Method
Secondary Data Analysis
Secondary Data Analysis
• A type of research in which data collected by others are
reanalyzed.
Primary Data Analysis
• Original analysis of the data collected in a study.
Meta-Analysis
• “Analysis of analysis”
• Quantitative procedure for summarizing or integrating the
findings obtained from a literature.
• Uses the results of individual research projects on the
same topic as data points for a statistical study of the
topic.
Secondary Data Analysis
Advantage
• Saves cost for data collection.
• Saves time required for data collection.
• May obtain data otherwise impossible.
Disadvantage
• Data collected may not be suitable for the researcher’s purposes.
(Validity)
• All necessary data may not be available in existing data.
• Requires time to search for the data set.
• Original data set may not be accurate.
• Existing data sets may contain more variables than what the
researchers need. Need time to sort out the data.
Content Analysis
General usage of the term
•
Refers to any of several research techniques used to
analyze the content of written, spoken, or pictorial
communication such as books, newspapers, television
programs, or interview transcripts.
Specific usage of the term
•
Refers to a quantitative technique that attempts to
quantify the meaning of the content of the written, spoken
or pictorial communication.
Content Analysis
1.
2.
3.
Determine “Unit of Analysis”
Determine “Indicators”
“Code” the data
Example
Research Question: What values and beliefs were expressed
by advocacy coalition about Social Security?
Unit of Analysis: Congressional testimony on Social
Security reform from 1983 to 2004.
Indicators: Values---Statement on what they like.
Beliefs---Statement on what things should be.
Content Analysis
1. Issue of “Sampling”
e.g. Randomly sample 20 congressional testimony each
month.
2.
•
•
Issue of “Coding”
Coding Sheet
Intercoder Reliability
Exercise:
Content Analysis of Course Syllabus
•
Content:
PA410U Syllabus
•
Coding:
Civic Capacity (CCI ) Initiative Matrix
Historical Comparative Methods
(Archival Analysis)
Focus on identifying historical trends or patterns.
• Focus on comparing the historical trends or
patterns.
Quantitative approach:
•
•
•
Secondary Data analysis.
Content analysis of archival data.
Qualitative approach:
•
•
Oral history.
Case study.
Qualitative Research Methods
•
•
•
•
•
•
•
Examples:
Participant Observation
Ethnography
Individual Qualitative
Interview
Textual Analysis
Focus Group Discussion
Del-Phi Method
Key features of
Qualitative Research
1. Collection primarily of qualitative rather than quantitative data
Qualitative methods emphasize observations about natural behavior and
artifacts that capture social life as it is experienced by the participants
rather than the numerical representations of the categories predetermined
by the researcher.
2. Exploratory research question.
Qualitative researchers typically begin their projects seeking to discover
what people think and how they act, and why, in some social setting.
3. Inductive reasoning (Reasoning that moves from more specific
kinds of statement to more general ones)
Only after immersing themselves to many observations, do qualitative
researchers try to develop general principles to account their observations.
Key features of
Qualitative Research
4. A focus on human subjectivity.
Qualitative methods emphasize the meanings that participants attach
to events and that people give to their lives.
5. Reflexive research design.
In the qualitative methods, the research design may need to be
reconsidered or modified in response to new developments, or to
changes in some other component as research progresses.
6. Sensitivity to the subjective role of the researcher.
Qualitative researchers should be sensitive to the role they play in
the process of data collection. “Researcher as an instrument”
Ways to collect qualitative data
1.
2.
3.
Participant Observation
Ethnography
Individual Interview
a)
b)
4.
5.
6.
Semi-structured interview
Unstructured interview
Textual Analysis
Focus Group Discussion
Del-Phi Method
Participant Observation
•
Sometimes referred to as “fieldwork” in
anthropology.
•
A method in which natural social processes are
studied as they happen.
•
Researcher takes a role of “participant”
Ways to collect qualitative data
1.
2.
3.
Participant Observation
Ethnography
Individual Interviews
Individual Interviews
Semi-structured interview
• Interviews with an interview guide containing
primarily open-ended questions that can be
modified for each interview.
Unstructured interview
• The interviewer starts with only a general sense of
the topics to be discussed and creates questions as
the interaction proceeds.
Interview Guide
•
A list of topics to cover and the order in which to
cover them that can be used to guide less
structured interviews.
•
The questions must be judged its appropriateness
for each interview, and may be re-ordered, and reworded if necessary.
Types of Questions
Open-ended questions vs. Close-ended question
Open-ended
•
“How would you describe the performance of the new Japanese Prime
Minister”?
Closed-ended
•
“Do you think the new Japanese Prime Minister is doing a good job?”
Probes
Questions that allow the interviewee to further expand on
their earlier comments.
•
•
“Can you elaborate on what you mean by . . .”
“Why is that so?”
Ways to collect qualitative data
1.
2.
3.
4.
5.
6.
Participant Observation
Ethnography
Individual Interview
Textual Analysis
Focus Group Discussion
Del-Phi Method
Focus Groups
•
•
•
•
A type of group interview.
Focuses on group interaction on a topic selected
by the researcher.
Ideally 4- 12 participants.
The interaction is directed by a moderator who
asks questions and keeps the discussion on the
topic.
Reference:
Dr. David Morgan (Institute on Aging, PSU)
• “Focus groups as qualitative research”(1995). Sage.
• Annual Review of Sociology 22, p. 129-153 (1996)
Ways to collect qualitative data
1.
2.
3.
4.
5.
6.
Participant Observation
Ethnography
Individual Interview
Textual Analysis
Focus Group Discussion
Del-Phi Method
Different Purposes for Research
•
•
•
•
Pure Research : Discovery of knowledge for the sake of
knowledge.
Applied Research: Conducted with some practical
purposes in mind.
Exploratory Research: Conducted in order to
explore the phenomena.
Explanatory Research : Conducted in order to
explain the phenomena.
• Evaluation Research: Conducted in order to develop and
“evaluate” programs.
Evaluation research
•
Research specifically designed to assess the
impact of a specific program, policy or legal
change.
•
Often the focus of an evaluation is whether the
program, policy, or law has succeeded in effecting
intentional or planned change.
Developing and Evaluating a Program
Research
Program
Needs assessment
Design a program
Implement a program
Process Evaluation
Outcome Evaluation
Redesign a program
Or Terminate a program
Types of Evaluation
1. Needs Assessment
•
An Analysis of whether a problem exists, its severity, and an
estimate of essential services.
2. Process Evaluation/Formative Analysis
•
•
Research that monitors a program or policy to evaluate the
process of implementation of the program.
Provides the program staff with evaluation of the current
program and suggestions for improving the program design
and service delivery.
Types of Evaluation
3. Outcome evaluation (Impact analysis,
summative analysis)
•
Research that is designed to “sum-up” the effects of program,
policy, or law in accomplishing the goal or intent of the program,
policy or law.
4. Cost-Benefit Analysis
•
Research that compares a program’s costs to its benefits.
•
A study designed to weigh all expenses of a program (its costs) against
the monetary estimates of the program’s benefits (even putting dollar
values on intangible benefits.)
Selecting a study design
Quasi-Experiment
• An experiment design that is missing one or more
aspects of the classic controlled experiment.
Panel Study (Longitudinal Study)
A study design in which data are collected about
one sample at least two times and all variables are
measured, not controlled by the researcher.
Selecting a study design
Cross-sectional Study
• A study design in which data are collected once
for all the variables of interest using one sample.
•
Case Study
A research strategy that focuses on one case (an
individual, a group, an organization, and so on)
with in its social context at one point in time, even
if that one time spans months or years.
Summary
Group Exercise