Some Central Issues in Social
Research
•
Description versus Explanation (notes follow Punch,
Ch. 2)
• Goal of scientific inquiry is not just to describe
phenomena, but to explain them and to make
predictions based on the patterns that are uncovered
and their putative causes or at least their
antecedents
• The purpose of description is to give an account of
the phenomenon in order to improve understanding
and reduce complexity by extracting the features
necessary to achieve understanding. The “what”
• The purpose of explanation is to locate the causal
underpinnings of events; to create a “story” in which
the reasons for things or the enabling or antecedent
conditions which give rise to them are laid out. The
“why” and/or the “how”.
Description versus Explanation
•
Presumably if we know what happened, why it
happened, and how it happened, we will be able to
•
•
•
Predict under what circumstances it might happen again
and
Possibly prevent/encourage it from happening again by
removing/encouraging motivating or enabling
conditions.
Although generally speaking purely descriptive
studies have less appeal to journal editors,
descriptive research is important
•
•
•
When the phenomena of interest are new and
unexplored, or
When the researcher is attempting to isolate causal
factors for testing with confirmatory methods
Description is of course the essential work of
ethnography
Description versus Explanation,
continued
•
How does explanation relate to theory? Theory is a
systematic, sometimes axiomatic effort to explain a
phenomenon or group of phenomena. Both descriptive and
explanatory studies have a role in the development of
theory.
•
Theory verification vs. theory generation
•
•
•
Theory verification: from theory we derive hypotheses which
are then tested. More likely to be quantitative
Theory generation: theory derived from the data we have
collected and described, using a defined strategy such as
grounded theory. More likely to be qualitative
Punch presents a model of the structure of scientific
knowledge based on a “nomothetic” view (nomothetic
refers to disciplines characterized by the search for
universally applicable scientific laws).
•
In the model empirical generalizations are based on observed
regularities in raw data, and the generalizations are “covered”
by an explanatory theory.
Question-Method Connection
•
The choice of research method or combination of methods
is tied to the type of question asked (although in practice
the question is often shaped to the available resources
including the researcher’s training and area of expertise,
e.g., the “law of the hammer”, see also “methodolatry”)
•
•
•
Questions like “what sorts of messages are conveyed in ads
featuring female athletes” are exploratory in nature and call
for qualitative methods.
A question like “what is the relationship between gender of
celebrity endorser and product type” calls for quantitative,
correlational methods
A question like “which type of celebrity (athlete, musician,
actor) is most effective in promoting a Democratic senatorial
candidate” calls for quantitative, experimental methods with
random assignment of subjects to conditions and systematic
rotation of endorser type across messages
Question-Method Connection,
continued
•
•
•
Other questions such as “what is the process by which
celebrity endorsement leads to product purchase”
might be best pursued through interview methods
with structured recall leading to a narrative account of
the trajectory from exposure to purchase.
The important thing to remember is that the
questions come first and the methods follow (although
methods can sometimes limit what can be studied)
The hard part is to figure out what the question is,
precisely, that you want answered. When the question
has been properly formulated the way to study it
becomes much more clear.
Tight versus Loose Structure
•
How much structure is applied to the research ahead of
time, with respect to
•
Research questions
•
Research design
•
Data
•
•
•
Can range from explicitly stated hypotheses to research
questions to completely exploratory work with no formulated
research questions and only a loose conceptual framework, if
any, for guidance
Can range from experimental designs with planned
comparisons among conditions and control of potentially
confounding variables, to quasi-experimental design to case
studies and ethnographies
Can range from ratio level measurement where the obtained
data is coded into pre-established categories based on valid,
reliable, established measures to inductive categorization
based on categories which emerge from the researcher’s
immersion in the data
Tight versus Loose Structure,
continued
•
•
•
Typically a quantitative study will introduce the most
explicit forms of these three sorts of structure
(research questions, research design, data) , at the
beginning or very early in the research process
Typically a qualitative study will seek to keep
structure to a minimum, at least at the beginning, so
that less obvious and more elusive patterns and
structures can emerge
Similarly, theory verification research will tend toward
the highly structured and theory generation research
will be more loosely structured
Introduction to Research Methods
•
•
•
Notes from Williams and Monge, Chapters 1,2,3
A research method “refers to the strategy, plan and
activities undertaken to accomplish the research”
Often both narrative/descriptive or qualitative
approaches and quantitative methods using statistics
can be profitably employed to provide answers to
questions about human communication.
•
In a study of “The Telegarden” done some years back one of our
questions was how the members dealt with people who violated
the social norms of the group which had formed. Both quantitative
methods involving numerical comparisons of coded message posts
and historical/descriptive analysis involving a critical incident
helped us to further our understanding.
Uses of Quantitative Research
•
It doesn’t make much sense to talk about one approach as
being superior to the other.
•
•
•
Both have their uses and can complement each other and lead
to further interesting questions when the methods yield
conflicting findings.
The important thing is to decide what circumstances might
make application of quantitative methods appropriate.
Quantitative methods are appropriate when measurement
can offer a useful description
•
We can say that someone is 6 feet tall, or we can say that she
is “the tallest in the group”, or that she is “one of the tall ones”
(interval, ordinal, nominal), or we could say that she is an
“Amazon”, or someone who “towers over this researcher” It
depends on what we want to do with the information.
Sometimes we want to use very precise metrics, and
sometimes we want to make comparisons, and sometimes we
want to provide a subjective impression of an object of study
in reference to ourselves.
Is Measurement Relevant, Useful,
Possible?
•
Is measurement relevant and possible?
•
One of the questions we have to ask is at what level it might
be possible to measure something.
•
Suppose we have a group of ten cat owners. What
measures would we use to sort them out by how much
they love their cats? Amount spent on vet bills? Time per
day spent petting, grooming?
•
Could we order them from 1 to 10, where number 10 loves
her cat ten times as much as number 1?
•
Could we order them but say only that number ten loves
her cat more than number 9 by the same amount that
number 2 loves her cat more than number 1?
•
Could we merely order them, but not say anything about
how much they differ, pairwise?
•
Could we merely classify them into high, low and moderate
love? Would a narrative description gained from interviews
and/or observations be more useful?
Are there Generalizations to be Made
and or Hypotheses to be Tested?
•
Quantitative methods are appropriate when there are
statistical generalizations to be made/hypotheses to be
tested
•
One of the reasons for using statistics is that we want to be
able to generalize about the attitudes, beliefs, and behaviors of
people on the basis of a set of observations that will probably
be limited by various constraints including financial ones.
• So we have to find out the answers to various questions,
such as,
•
•
•
Am I able to describe precisely the population to which I want
to generalize?
Do I have confidence that I can draw a sample that is large
enough and representative enough to stand in for the
population?
Is the variability within the sample so great that it doesn’t
make sense to talk about central tendencies?
Are there Observable Differences?
•
Another reason for using statistics that we want to answer
questions like, “is the tendency I observed in this group
something distinctive, different from what one would
expect just by chance?” or, are these two, or three, or
some larger number of groups different in ways that
exceed chance expectations?
•
•
Suppose the question were about the efficacy of a new drug. What
sort of confidence would you have to have that the difference
between a treatment and a control group not taking the drug was
not due to chance before you would recommend the drug? Ten
percent? Five percent? One percent? Less than that?
How much confidence would you have if the report about the drug’s
efficacy were based on methods of interviews and participant
observation (anecdotal reports of “feeling better”)? Would
statistical hypothesis testing increase your confidence? Would it
depend on how “feeling better” was measured?
Some Problems with Quantitative
Studies
• Typical problems in quantitative
studies:
• Validity; we may not be measuring what
we think we are measuring. Subjects
don’t see it as we do
• Reliability; we may not have confidence
that our “yardstick” is measuring in a
consistent way
• Convenience samples
Common Terms in Quantitative
Research
•
Some terms that reflect the application of the scientific method in
communication research practice:
•
•
•
•
Phenomenon: the object of study; the behaviors, beliefs, attitudes,
characteristics, and their purported inter-relationships that we seek to
describe, explain, and/or predict (e.g. How competent people are at
using computers)
Variable: an observable property or characteristic that can be
measured (operationalized) and is expected to vary across cases or
observational instances, such as what people report about their ability
to use computers, or, what computer skills people can demonstrate,
etc.
•
•
Independent variable
Dependent variable
Measurement: a plan for operationalizing the variable, such as using
a particular scale or observational technique (e.g. Durndell and Haag
self-report measure of computer self-efficacy)
Data: the obtained values which the researcher ascribes to individual
measured instances of the variable (the numbers researchers assign to
the answers study subjects give to items on the self-efficacy scale)
Terms in Quantitative Research,
continued
•
•
•
•
Population: the totality of cases which constitute the sphere
within which the phenomenon is to be observed. Could be people
in general, people in the US, college students, children under 12,
could also be states, cities, animal shelters, department stores,
countries, etc.
Sample: some portion of the population which is believed to be
representative
Descriptive statistics: statistically derived values that represent
the central tendencies and variability with a body of data
Sampling statistics: values used to make inferences about the
characteristics of the population from which they were drawn,
including the variation of the sample characteristics from
corresponding population parameters
Elements of a Quantitative Study: The
Problem
•
Typical sort of quantitative study as reported in a
journal will feature the following elements:
•
Literature review leading to problem statement
leading to research question or hypothesis
•
•
•
Research obliged to “follow the conversation”, summarize
what is known, what is controversial or not known, and
what remains to be done.
May formulate a general question or make specific
predictions about what relationships among variables may
be found.
Hypothesis ordinarily will be stated in terms of a specific
operationalization of the variables of interest as well as the
nature of their relationships
Elements of a Quantitative Study:
Method
•
Method, including subjects, measures and procedures
• Descriptive methods, where variables and their
•
•
relationships are observed and described (seniors who
have high scores on computer self-efficacy have high
scores on social support)
Quasi-experimental (e.g. seniors who take a computer
class have higher scores on social support after one
year than those who don’t sign up (no random
assignment, based on self-selection; no experimenter
manipulation)
Experimental methods, where one or more of the
variables are manipulated by the researcher to
provide for systematic examination of their
interrelationships. (same as above except half of the
seniors have been randomly assigned to a six-month
long computer class and half have not)
•
Importance of control, avoiding of confounding factors
Elements of a Quantitative Study,
Method, continued
•
•
•
Subjects (Respondents, Participants); who constitutes a “case”
in the study. Some studies may have other cases such as web
sites, tv episodes, etc.
Materials: questionnaires, interview protocols, coding
schemes, hardware, environmental manipulations, etc.
measures, operationalizations, material circumstances of
experimental conditions.
Procedures: Instructional set, repeated measures design,
debriefing, etc.
Elements of a Quantitative Study:
Results
•
Results
•
Reliability and validity assessments
•
•
•
Reliability : internal consistency of measure;
alternatively, consistency of measurement over time
with the same subject, case, instance. Test-retest;
alpha coefficient
Validity: does the measure really assess what it claims
it does? Internal validity (does the research design
represent what it says it does?) External validity (can
the study’s results be said to apply to the real world?
Face, concurrent, predictive, construct (convergent
validity and discriminant validityare two subtypes of
construct validity-measures that should be related are,
and that should not be related, theoretically, are not).
Validity implies reliability; but reliability does not imply
validity
Elements of a Quantitative Study:
Results, continued
•
Data analyses (performing calculations on obtained data,
conducting statistical hypothesis testing).
•
Researcher is assumed to have made decisions beforehand as
to the level of statistical error that will be tolerated, what sort
of “tail” (one or two) the test will have if appropriate, etc.
Elements of a Quantitative Study:
Discussion
• Discussion
• Conclusions
• Limitations
• Implications for future research
Fundamental Assumptions of
Quantitative Research
Steps in the research process (notes from Kendrick)
• Specify research goals and devise one or more
research questions
• Review the literature
• Formulate hypotheses: reduce problem to be studied
to a testable and preferably falsifiable statement of a
set of relationships among variables of interest
• Measure and record
• Analyze the data
• Invite scrutiny
Sampling from Populations
•
Populations, elements of populations, and units of analysis
•
•
•
•
•
•
Population --all registered voters in US
Elements within populations – the individual
registered voter
Units of analysis or units of observation—the
individual, or perhaps the individual’s household,
spouse, etc.
Sample: subset of population selected to represent the
population from which you draw it
Probability sample: selected so that each element in the
population has a known probability of being included in the
sample
Sampling frame: list of elements in the population (e.g.
list of all persons paying property taxes in Los Angeles
County in 2008)
Sampling from Populations, cont’d
•
•
Types of sampling
• Simple random sampling (random number table,
computer program)
• Systematic sampling with a random start; only the
first element is selected at random; then every nth
Multistage sampling processes
• Stratification; partitioning the sampling frame by
factors such as gender, ethnicity etc before random
or systematic sampling
• Clustering, for ex., in absence of good sampling
frame, obtain list of clusters of elements in the
frame, e.g., neighborhoods, zip codes, city blocks,
then randomly select clusters, then randomly select
within clusters
Variables and their Relationships
•
Variables: properties that vary from person to person, such
as age, gender, attitudes toward politics, self-monitoring,
etc., or that vary within the same element or unit of
analysis over time.
•
•
•
•
Independent variable: assumed to be causal (logic, time
order, ascribed vs. achieved characteristics)
Dependent variable: assumed to be an effect or result of one
or more other variables
Variables have categories (e.g., M/F; Caucasian nonHispanic/African-American, Hispanic, etc.;) and numbers or
values can be assigned to these
Relationships between variables
•
•
Control variables; some variable other than the independent or
dependent variable which is thought to influence their
relationship; e.g., effect attributable to ethnicity may actually
be due to the correlation of religion and ethnicity
Symmetrical relationship; variables are correlated but it is not
possible to specify the direction of a causal relationship
Independent and Dependent
Variables: Which is Which?
 Independent variables must be chronologically prior to
dependent variables
 Ascribed characteristics (inherited, properties over
which one has little/no control, such as height or eye
color or ethnicity are usually independent variables
(although not always)
 Achieved or acquired characteristics (attitudes,
values, beliefs, etc) are frequently dependent
variables, though not always (consider health
behaviors, for example, in which attitudes and beliefs
play a major role in health information seeking,
compliance with physician’s instructions, etc.)
Levels of Measurement
•
•
•
Categories, Values, Data (e.g., Category =Gender: Values:
1=Female, 2=Male; Data: respondent #22=2)
Levels of Measurement (Kendrick, Chapter 2; Williams and Monge,
Chapter 3). The level at which a variable is measured determines
the kinds of statistics which can be used to describe or make
inferences about the variable. Generally, researchers use numbers
to classify or categorize, to rank or order, or to assign a score or
rating
Discrete vs. Continuous Variables
•
•
Discrete: variables countable in quantities that can’t be reduced to
smaller units, such as no. of children, jobs you hold, movies you attend
weekly.
Continuous: countable in quantities that can be reduced indefinitely,
such as time. A variable may be theoretically continuous such as age
but in practice it will be measured as discrete because of available
metrics
•
(smallest age measurement usually in hours or days); same with
attitudes or other ratio type variables that are treated in practice as
discrete
Levels of Measurement, continued
•
Categorical vs. Numerical
•
Categorical: categories predetermined by researcher
•
•
•
•
Exclusivity: answer categories should not overlap
Exhaustivity
Examples (demographic (ethnicity, marital status, income categories),
Likert-type attitude scales, semantic differential), frequency measures
(once a week, once a month, etc.)
Types of categorical variables
•
•
•
Nominal: no rank order implied; categories simply discrete (gender, ethnicity,
religion, etc). Considered weakest level of measurement
Ordinal: rank order implied, for example, when age, income, education level,
etc. are measured not numerically but in categories where one end of the
range is “higher” for the researcher’s purposes. There is no information about
the magnitude of differences. Often ordinal scaled data like Likert or semantic
differential data is treated in practice as if the intervals between response
categories were equal so more complex statistical analysis can be applied
Numerical: direct measurement in terms of numbers, amounts,
frequencies with no pre-conceived categories
•
Examples: hours spent on the Internet weekly, absolute salary, age in
years, miles between cities, scores on some aptitude, IQ and
personality tests, etc.
Levels of Measurement, continued
•
Types of numerical variables:
•
•
Ratio level; numbers are assigned to identify ordered
relations with respect to some property ; there is an
absolute zero amount of the property; proportions are
meaningful (e.g. travel in miles between two points, where
zero means the two points are the same, and 500 miles is
twice as far as 250 miles, thus being isomorphic with
relations in the real, physical world). (But- on a test, 0%
correct may not mean no amount of the property
measured by the test unless the test has a true random
sample of all possible indicators of the property)
Interval level; numbers are assigned to identify ordered
relations with respect to some property, but no absolute
zero; proportions not meaningful, but distances are
assumed to be equidistant between scale intervals. For
many communication variables, it is not reasonable to
think of “zero” amount of the property. What is zero
amount of self-monitoring? Of shyness? etc
Levels of Measurement, cont’d
•
•
•
Categorical data can be treated numerically if the
researcher believes the categories are roughly
equidistant (e.g. strongly agree, agree distance is
same as agree, neutral, and same as strongly
disagree, disagree)
Numerical data can be treated as categorical. For
example, income, age, hours watching TV data can be
collected categorically for various reasons
The level of measurement determines the type of statistical tests
and procedures which can be applied to the data. Statistics
appropriate for numerical data (ratio, interval) are not
appropriately applied, for example, to nominal or ordinal data. But
data can be “scaled back” (e.g. taken back to a lower level of
measurement) and statistical procedures appropriate to, say,
ordinal data can be applied (e.g. change hours worked per week to
categories such as none, some, lots, many)
Hypotheses and Research
Questions
•
Hypotheses: hunches or expectations that social scientists
have about relationships between or among variables,
commonly but not always expressed as the expectation that
variation in an independent variable will “cause” or be
associated with variation in a dependent variable. Some
examples from recent papers:
•
•
•
“Local Mexican Web pages will reflect a higher level of
collectivism than the Mexican Web pages of American
companies”
“Internet users' concern for privacy online negatively influences
their trust of a commercial Web site”
Hypotheses look different from research questions:
•
RQ1. Does subject gender interact with medium in the
development of relational intimacy, and if so, how do these
variables interact?