GS/PPAL 6200 3.00 Section N
Research Methods and Information
Systems
January 13, 2015
Professor Brenda Spotton Visano
Office: 130 McLaughlin
Voice Mail: (416) 736-2100 ext. 20470
E-mail: spotton@yorku.ca
Agenda
• Introduction to Data Sources
– Guest Speaker: Mr. Walter Giesbrecht, Librarian,
Scott Library, York University
• Review of Last Class
• Introduction to Research Methods (cont’d)
– Criteria for Evaluating Research
– Research Orientation (Methods)
– Research Design
– Checklist for Reviewing Research
Review of First Class
• Objectives of First Class:
– Brief Introduction to Course
– Introduction to Each Other’s Policy Interests
– Motivating the Study of Policy Research Methods
• Achieving the Objectives
– Information contained on Course Outline
– Group discussion of sample survey questions to illustrate and
motivate issues in ensuring data integrity and ethical use
– Short videos to illustrate how common understanding and
supporting evidence depends on currency of data and
perspective
– Deductive and Inductive Reasoning; Sample Fallacies to be
avoided
Results of First Class Survey
Total number of Respondents, N=17
Topics of Interest for Policy Research:
–
Justice:
•
•
–
–
–
Social justice: 6
Criminal Justice/Access: 2
Judiciary/Legislation: 2
Other: Child Welfare, Public Dental Health, Privacy Issues
Undecided: 4
Research Methods of Interest for Policy Research:
–
–
–
–
Qualitative: 5
Quantitative: 3
Both Qualitative and Quantitative (including meta-analysis): 5
Undecided: 4
Some Familiarity with Basic Statistics:
–
–
Yes: 11 (includes hesitant answers such as “I think so”)
No: 6
Some Familiarity with Basic Excel:
–
–
Yes: 14
No: 3
Introduction to Excel: https://support.office.com/en-us/article/Excel-2013-training-courses-videos-and-tutorials-aaae974d-3f47-41d9-895e97a71c2e8a4a?ui=en-US&rs=en-US&ad=US
Parting Question #1 from Last Class
• What do we know? How do we know what we
know? How do we discover what we don’t know?
• The positivist view: knowledge is only
phenomena confirmed by the senses, subjected
to and supported by empirical testing, and
“value-free” – an external, objective reality
• The post-positivist view: knowledge includes the
subjective meanings – interpreting versus
explaining or, interpreting for explaining, and
social reality is constructed, mutable, and created
by our actions
Parting Question #2 from Last Class
• Does interpretation play a role in research? Is the
researcher independent of the research subject
or does the background (education, beliefs,
values) of the researcher influence what is
observed?
• Values can influence: choice of research areas,
formulation of research questions, choice of
methods, formulation of research design and
data collection, analyses, interpretation and
conclusions.
Research in Public Policy
Excerpted from Morçöl and Ivanova (2010)
Categories of Methods
Quantitative Orientation
Qualitative Orientation
Empirical Inquiry - Design
Methods
Experimental, Crosssectional, Longitudinal
Case study
Empirical Inquiry - Data
Collection Methods
Surveys, Secondary Data
Qualitative (long, in-depth,
or semi-structured)
Interviews
Empirical Inquiry - Data
Analysis Methods
Statistical, Regression, or
Time-series Analyses
(Computer-assisted)
Qualitative Data Analyses
Empirical Inquiry Combined Methods
Game Theory, Simulations,
Systems Analysis, MetaAnalyses, Network
Analyses
Case study, Legal Analyses,
Archival, Ethnography,
Grounded Theory
Methods of Decision
Making and Planning
Cost-benefit, Decision
Analyses, Linear
Programming
Brainstorming, Delphi
Criteria for Evaluating Research
• Reliability/Replicability: Would the same
results be achieved by the same measurement
technique applied to the same research
subject by the same or different researchers?
Is the study able to be replicated with the
same results?
• Validity: are the research conclusions “valid”?
Validity - Basics
• Are we testing what we claim to be
measuring? E.g., Do IQ tests measure
intelligence? (construct validity)
• Is there a causal relationship between the
dependent and independent variables tested?
(internal validity)
• Can we generalize the cause-effect beyond the
research study to other people/places/times?
(external validity)
Research Design
• Experimental design: random assignment to
control and experimental groups, manipulate
isolated independent variable, observe outcomes
• Quasi-experimental design: omit random
assignment, observe outcomes from “natural”
experiments, e.g.
• Cross-sectional design e.g., unstructured
interviews: observations taken at a point in time,
• Longitudinal design: track observations over time
• Case study design: detailed analysis of a single
case (person, program, etc)
Research Design, Orientation, Validity
Design
Quantitative
Orientation
Experimental Clinical trials
Qualitative
Orientation
Internal
Validity
External Validity
Not
applicable
High
High if random
assignment and
low if social
research involves
deception
Problematic
since
independent
variable
cannot be
manipulated
High if sample
random
Crosssectional
design
Survey,
Interviews,
structured
Focus groups
observation,
content analysis,
regression
analysis
Longitudinal
design
Time series
analysis
Ethnographies Problematic,
ibid
High if initial
sample random
Case study
Survey
Interviews
Low
Checklist for Reviewing Research
Case study: “Analyzing E-Government Research…”
Heeks and Bailur (2006)
• Subject: E-government research
• For discussion in class:
–
–
–
–
–
–
–
–
–
Research Question:
Philosophy:
Research Design:
Data Collection Method:
Data Analysis Method:
Sample Size and Representativeness:
Conclusions:
Reliability/Replicability:
Validity: