Variables, Validity, Theory, Hypotheses
Bernard, 2013, ch2; Cresswel, 2014, ch3 & 7
dimitris.christopoulos@modul.ac.at
www.dimitriscc.wordpress.com
Variables
▪ Unidimensional Variables
▪
▪
▪
▪
Height & weight
Birth order
Age
Marital status
▪ Multidimensional Variables
▪
▪
▪
▪
Stress
Wealth
Political orientation
Religiosity
Variables
▪ Dependent Variables
▪ The variable impacted by the independent variable(s)
▪ Example: Longevity
▪ Independent Variables
▪ Logically prior to dependent variables
▪ Example: Sex, Education, Occupation, Diet, Smoking
▪ Example: How long one lives is impacted by, or ‘dependent’ on,
these many variables. However, living to be 90 does not change
your gender or your occupation earlier in life because they stand
‘independent’ of longevity.
Defining Variables
▪ Conceptual Definitions
▪ Abstractions, articulated in words, that facilitate understanding.
▪ The kind of definitions you find in a dictionary.
▪ Operational Definitions
▪ Consist of a set of instructions on how to measure a variable that
has been conceptually defined.
▪ Example: Measure ‘extroversion’ in a person by having them take
the Myers-Briggs Type Indicator (MBTI) and record their
extroversion score.
Variables: Level of Measurement
▪ 1. Nominal: An exhaustive, mutually exclusive, list of
names.
▪ Example: Ethnic groups, or religious identifications
▪ 2. Ordinal: Same as above, plus they can be rank
ordered.
▪ Example: Socioeconomic classes, or levels of agreement
Variables: Level of Measurement
▪ 3. Interval: Same as above, plus each unit has a
meaningful quantitative distance from each other.
▪ Example: Degrees Fahrenheit, or points on an IQ score
▪ 4. Ratio: Same as above, plus there is a true zero point,
measuring the absence of the phenomenon being
measured.
▪ Example: Age, or income in dollars
▪ Note: 80 degrees F, is not twice as hot as 40 degrees.
▪ But 80 years old, is twice as old as 40.
Consider and Respond to these two questions
▪ How many companies in the world have a woman as top
manager or CEO?
▪ 2%
▪ 10%
▪ 18%
▪ In 1990, 58% of the world’s population lived in low-income
countries. What is the share today?
▪ 9%
▪ 37%
▪ 61%
Consider and Respond to these two questions
▪ How many companies in the world have a woman as top
manager or CEO?
▪ 2%
▪ 10%
▪ 18%
▪ In 1990, 58% of the world’s population lived in low-income
countries. What is the share today?
▪ 9%
▪ 37%
▪ 61%
Gapminder
www.gapminder.org
What is the value of statistical analysis?
What is the value of statistical analysis?
▪ https://www.ted.com/talks/hans_rosling_shows_the_best_st
ats_you_ve_ever_seen
Lowest Level Rule
▪ Always try to collect data on the lowest level unit of
analysis possible.
▪ You can always aggregate data collected on individuals,
but you can never disaggregate data collected on
groups.
▪ Ecological Fallacy: Drawing conclusions about the
wrong unit of analysis.
▪ Example: Drawing conclusions about ‘neighbors’ from data on
‘neighborhoods.’
Instruments
▪ Precision: The power of measurement, or the ‘number
of decimal points.’
▪ Example: Is the scale measuring to the nearest pound or to the
nearest ounce?
▪ Accuracy: The correct calibration of the instrument; it is
not ‘off.’
▪ Example: The scale always measures 2 lbs too heavy.
Instruments: Reliability
▪ Reliability: Whether or not you get the same answer by
using an instrument to measure something more than
once.
▪ Example: Does the scale give you the same weight each time you
step on it (assuming you have not eaten or worked out between)?
Instruments: Validity
▪ Face Validity: On the ‘face of it’ do the operational
indicators of the concept make sense?
▪ Example: An instrument measuring educational aptitude asks
questions about study habits and eye color. Which one does not
have face validity?
▪ Content Validity: When an instrument has appropriate
content for measuring a complex concept/construct.
▪ Example: An instrument measuring ‘cultural identity’ that only asks
questions about food and clothing style preferences, but neglects
content on language and historical traditions. The content is not
broad enough to sufficiently represent ‘cultural identity.’
Instruments: Validity
▪ Construct Validity: A close fit between the construct it
supposedly measures and actual observations made with
the instrument.
▪ Example: A personality assessment lacks construct validity if it
identifies you as ‘introverted’ when you and your many friends
would say the opposite.
▪ Criterion Validity: A close fit between the measures it
produces and measures produced by some other
instrument that is known to be valid.
▪ Example: A personality assessment that identified you as
‘extroverted’ when the Meyers Briggs assessment did too, has
criterion validity.
Ockham’s Razor
▪ The Principle of Parsimony:
“Don’t make things more complicated
than they need to be.”
- William Ockham (1285-1349)
In science, simpler explanations, theories, or measures are
preferred over more complicated ones.
Correlation
▪ When two variables are related they are said to co-vary.
This is also called correlation or simply association.
▪ Many independent variables may each only contribute a
little to the outcome of the dependent variable, but the
correlation may still be quite direct and causal.
▪ Example: Having personally known someone who died of AIDS is
only one of many independent variables that is known to have a
causal impact on the choice of whether to use a condom during
sexual activity.
Spurious Variables
▪ Two variables are spurious when their correlation (no
matter how strong) is caused by a third variable.
▪ Example: There is a correlation between the number of firefighters
at a fire, and the amount of property damage caused. Should we
assume that the firefighters are causing greater damage?
What are theories?
What are scientific theories?
Theory
▪ Theory: A mechanism that explains the association
between two or more correlated variables.
▪ Theories are simply ideas about how things work.
Frames to help make sense of the world.
▪ We all hold ‘informal theories’ or ‘axioms’ about many
things. Formal theory, however, can be tested and
revised.
The Use of Theory
▪ The literature determines what theories may be used to
examine the research questions
▪ Quantitative analysis test theories as explanations
▪ Qualitative analysis may generate the theory
▪ Mixed methods studies may have no theories at all or a
theoretical framework in which both quantitative and
qualitative data are collected
Theories in Quantitative Research
▪ Here a theory is a scientific predication or explanation of
what the Researcher expects to find out about a set of
interrelated variables.
▪ When a Researcher uses a theory to predict the
anticipated outcome of a study he/she has created the
hypothesis.
▪ As the Researcher conducts the study and puts forward
an explanation for his/her predictions and study
findings, he/she is presenting his/her theory.
Examples of theories
The world is flat
Aliens regularly visit Earth
The cosmos is finite
Relations matter to human happiness
Planet Earth has warmed up since the industrial revolution
Price inflation is associated to growth rates
Corruption is associated to political culture
Levels of innovation are associated to levels of spending in
primary research
▪ Success of start-ups is associated to availability of venture
capital
▪
▪
▪
▪
▪
▪
▪
▪
Theories in Quantitative Research
▪ Micro-level theories: limited to a small segment of time,
place or number of people: i.e. reciprocity, trust, balance etc
▪ Meso-level theories: link micro-level and macro-level
theories with the use of innovative theoretical constructs: i.e.
complexity theory or agent based theories
▪ Macro-level theories: attempt to explain processes at a wider
scale, for example at the level of society: i.e. feminism regards
patriarchy as a causal variable to the production of gender
inequality
Forms of Theories in Quantitative Research
• Theory
– Interrelated set of constructs formed into propositions that
specify the relationships among variables
– Describes how and why variables are related
• Forms include
– Set of hypotheses: These are interconnected and demonstrate a
process
– Series of if-then statements: These explain why one would
expect an independent variable to influence or cause a
dependent variable
– Visual model: These allow the reader to visualize the
interconnections between the variables
Quantitative Theoretical Perspective
1.
2.
3.
4.
Examine discipline specific literature (psychological, sociological or
a combination of both).
Examine the existing body of knowledge on the topic and related
topics.
Look for connections between the independent and the
dependent variables.
Script out the theory section. (See Example 3.1)
•
•
•
•
Name the theory you plan to use
State the central hypothesis of the theory
Present information about who has used the theory and its applicability
Present an adoption of the theory to the variables of your study using ifthen logic
Qualitative Theory Use
▪ Provides broad explanations: Themes provide the context for
hypothesis to be tested from the literature.
▪ Provides theoretical lens for the examination of issues of
marginalized groups: This allows for a transformative perspective
as it influences the questions asked, the data collected and analyzed
and the research outcomes.
▪ The theory becomes the end point: Most qualitative research is an
inductive process building from the data to the broad themes to a
generalized model or theory.
▪ No explicit theory: Some qualitative studies build from detailed
descriptions of phenomenon.
Alternative study topics: humor
https://www.youtube.com/watch?v=twCnSD03PoA
Qualitative Theoretical Perspectives
▪
▪
▪
▪
▪
Feminist perspective
Race discourse
Critical theory
Queer theory
Disability inquiry
Endpoint: a theory that is generated
Researcher may also choose not to employ theory in a qualitative study
Tips for Using Theory in Qualitative Study
▪ First decide if you will use a theory in your study.
▪ If you decide to use a theory, identify the theory that will be
used using the transformative or advocacy lens.
▪ Place the narratives about the theory early in the research
proposal or at the end of the proposal.
Locating Theory in a Qualitative Study
1. Studies based on cultural themes or theoretical
lens: The theory is placed in the opening
passages.
2. Studies based on emerging design or
qualitative inquiry: The theory appears in the
beginning and is modified as the study
progresses.
3. Studies based on grounded theory or reciprocal
relationships: The data determines the theory
that is used.
Use of Theory in Mixed Methods
▪ Mixed methods studies may:
▪ Include theory deductively (theory testing and validity)
▪ Include theory inductively (an emerging theory or pattern)
▪ Use a theoretical lens or perspective to guide the study
Social Science Theory Use
In Social Science research the theory is the overarching
framework:
1. The theory is placed at the beginning of the study and
guides the questions/hypothesis.
2. The theory is named and described and it informs the
explanation of the major variables in the study.
3. The theory may be diagrammed that indicate the causal
links between the primary variables or concepts.
4. The theory guides the data collection process.
5. The theory informs the findings of the study and
provides a basis for comparison with other studies.
RESEARCH QUESTIONS AND HYPOTHESES
Creating a Research Question 1/3
▪ Write down a research question on a topic of your choice
Creating a Research Question 2/3
▪ Write down a research question on a topic of your choice
▪ Compare your question to one prepared by one of your
colleagues
Research Questions and Hypotheses
▪ Investigators place signposts to carry the reader through a
plan for a study.
▪ An important signpost is the research question or the
hypothesis that narrows the purpose statement to predictions
about what will be learned or questions to be answered in the
study.
Qualitative Research Questions
▪ Qualitative researchers pose research questions
▪ Not objectives
▪ Not hypotheses
▪ Two types of qualitative research questions to focus a
study's purpose:
▪ Central question
▪ Broad question that asks for exploration of the central phenomenon
▪ Subquestions
▪ Questions that narrow the focus of the study
Writing Qualitative Research Questions
▪ Begin with "what" or "how” to convey an open or
emerging design
• Focus on a single phenomenon or concept, what is the one single
concept, that you want to explore?
• Use exploratory verbs as nondirectional rather than directional
words, like affect, influence, impact, determine, cause and relate
• Use open-ended questions without reference to the literature or
theory
• Specify the participants and research site (unless stated previously)
A Script for Writing a Qualitative Central Question
▪ (How or What) is the (“story for” for narrative research;
“meaning of” the phenomenon for phenomenology;
“theory that explains the process of ” for grounded theory;
“culture-sharing pattern” for ethnography; “issue” in the
“case” for case study) of (central phenomenon) for
(participants) at (research site).
Quantitative Research Questions and Hypotheses
▪
Quantitative researchers pose research questions or hypotheses and
objectives to focus the study's purpose
▪
Quantitative research questions:
▪ Questions about the relationships among variables that the investigator
seeks to know
▪
Quantitative hypotheses:
▪ Predictions that the researcher makes about the expected relationships
among variables
▪ Predictions about the population values that the researcher will estimate
based on data from a sample
▪
Quantitative objectives:
▪ Indicate a study's goals
▪ Used frequently in proposals for funding
Writing Quantitative Research Questions and Hypotheses
▪ Consider 3 approaches to the variables for a question or hypothesis:
▪ Compare groups
▪ Relate variables
▪ Describe responses
▪
▪
▪
▪
Specify questions and hypotheses based on theory if possible
Measure the independent and dependent variables separately
Generally use demographic information as intervening variables
Use consistent words and ordering for independent and dependent
variables
Scripts for Writing Quantitative Research Questions
and Hypotheses
▪ Quantitative research question describing outcomes:
What is the frequency and variation of scores on ____________ (name the
variable) for ______________(participants) in the study?
▪ Quantitative research question focused on examining the
relationship among variables:
Does _________ (name the theory) explain the relationship between _________
(independent variable) and _________ (dependent variable), controlling for the
effects of _________ (control variable)?
▪ Quantitative null hypothesis:
There is no significant difference between _________ (the control and
experimental groups on the independent variable) on _________ (dependent
variable).
Forms for Writing Quantitative Research Questions
and Hypotheses
▪ The use of variables in research questions or hypothesis
if one wants to do:
An experiment or group comparisons
A survey that correlates variables
A descriptive study
▪ Write research questions and hypothesis that logically
follow from relationship among variables in a theory
▪ Research questions or hypothesis may indicate cause
and effect logic
▪ Research questions and hypothesis should have no
redundancies, do not write both
Forms for Writing Quantitative Research Questions
and Hypotheses
▪ If writing hypotheses, use a consistent form:
▪ Null hypotheses (predict no difference or no relationship)
▪ Alternative hypothesis or directional hypotheses (predict direction of
difference or relationship)
▪ Nondirectional hypotheses (predict a difference or relationship, but not
its direction)
▪ Use non demographic variables
▪ Use the same pattern of word order in the questions or hypotheses
to enable a reader to easily identify the major variables
▪ If writing research questions:
▪ First, specify descriptive questions for each important variable
▪ Next, state inferential questions that relate variables or compare groups
▪ Finally, add questions in which variables are controlled
Creating a Research Question 3/3
▪ Write down a research question on a topic of your choice
▪ Compare your question to one of your colleagues
▪ Is there a theory or a set of assumptions behind the
question?
▪ Is the research question exploratory or explanatory?
▪ Is there a logical sequence to the question?
▪ What type of evidence would refute the premise?
Mixed Methods Research Questions and Hypotheses
▪ Advance both qualitative and quantitative research
questions (or hypotheses) in order to narrow and focus
the purpose statement
▪ Use guidelines for writing good qualitative and quantitative
questions and hypotheses
▪ Order questions to match the mixed methods design
▪ In a two-phase design, order to match the phases
▪ In a one-phase design, order according to the method given the
most weight
▪ Include a mixed methods research question that:
▪ Conveys the methods and procedures
▪ Conveys the content of the study
▪ Combines the methods and content
Different Ways to Write Questions and
Hypotheses into a Mixed Methods Study
▪ Write separate qualitative questions and quantitative questions or
hypotheses:
▪ At the beginning or as they emerge in phases
▪ This places emphasis on the two approaches
▪ Write separate questions or hypotheses followed by a mixed
methods question:
▪ This highlights the two approaches as well as their combined
strength
▪ Write only a mixed methods question:
▪ This emphasizes the integration and not the individual parts
Thank you for your attention
A Null Hypothesis
Writing Qualitative Research Questions
▪ Ask 1-2 central questions, these should be broad and ask
for an explanation of the central phenomenon or
concept in the study. Here the intent is to explore the
general factors relevant to the central phenomenon.
▪ Ask no more than 5-7 subquestions these will narrow the
study but leave open the questioning.
▪ Relate the central question to the strategy of inquiry:
▪ Ethnography, questions verify accuracy of data
▪ Critical ethnography, questions build on an existing body of
literature
▪ Phenomenology, questions broadly stated without specific
reference to existing literature
▪ Grounded theory, questions generate a theory
A Model for Descriptive Questions and Hypotheses
Hypotheses and Research Questions
in a Mixed Methods Study