Paper-Based Questionnaire Surveys
in U.S. ranch-management literature
Kristin Gangwer
February 22, 2010
GEOG 5161-002
Overview of Questionnaire Surveys
• Method of generating primary data for quantitative
or mixed-methods studies
• “Provides a quantitative or numeric description of
trends, attitudes, or opinions of a population by
studying a sample of that population.”
• Data then used to calculate statistical information
Considerations
• Questionnaire design
– Open, closed
• Sample
– Population, size, demographic characteristics, selection of individuals,
stratification
• Delivery medium
- Face-to-face, postal, telephone, email
• Cross-sectional or longitudinal
• Response bias
• For the rest of the presentation I will be speaking specifically
about: Paper-based, postal questionnaires that are self
administered and cross-sectional.
General Strengths & Weaknesses
Strengths
• Increases potential sample size
• Reaches geographically-disperse respondents
Weaknesses
• Lower response rates (especially without return
postage)
• Incomplete responses
• Increased costs
• Can’t clear up misunderstandings
Advantages for Ranch-Management Studies
•
•
•
•
•
Aging population
Geographic dispersion
Access to contact information
Sample size
Popular for studying natural resource issues
(Didier and Brunson 2004, 331)
1.
2.
3.
4.
5.
Population parameters can be quantified
Analytical tools are well defined and can be used to provide generalizations about the
population under study
Variance of key parameters and the degree of confidence associated with hypotheses
can be estimated
Replication and validation are relatively easy when standardized procedures are used
Survey method generally accepted by the scientific community
Examples
1. “Socioeconomic behavior of cattle ranchers, with
implications for rural community development in the West”
(Smith and Martin 1972)
2. “Motivation of Colorado ranchers with federal grazing
allotments” (Bartlett et al. 1989)
3. “To ranch or not to ranch: Home on the urban range?”
(Liffmann, Huntsinger and Forero 2000)
4. “Classifying federal public land grazing permittees” (Gentner
and Tanaka 2002)
Socioeconomic Behavior of Cattle Ranchers, with
Implications for Rural Community Development in the West
Arthur H. Smith and William E. Martin
American Journal of Agricultural Economics, 1972
• Sought to understand cattle ranching and ranchers
by viewing the ranch resource as generating both
production and consumption outputs.
• 89-person random sample of Arizona ranch owners
• 33-question questionnaire
• Economic and attitudinal variables
Socioeconomic Behavior of Cattle Ranchers, with Implications for
Rural Community Development in the West
Arthur H. Smith and William E. Martin
American Journal of Agricultural Economics, 1972
•
Factor analysis
– 11 factors (explaining 69.2 percent of the variance of the original variables)
– 4 basic groups (not mutually exclusive)
•
Discriminant analysis
– Used to determine if a significant difference in goals and attitudes existed between ranchers
willing to sell their ranches and those who weren’t
•
Final discriminant function
– Used 3 of the original factors, which were
found to significantly explain differences
between groups (i.e., keep or sell)
•
Predict keep or sell with 80% accuracy
Motivation of Colorado Ranchers with Federal
Grazing Allotments
E.T. Bartlett, R.G. Taylor, J.R. McKean and J.G. Hof
Journal of Range Management, 1989
•
•
•
•
•
•
•
•
Determined ranchers’ willingness to sell their ranches and determined which
factors were important in their decision to ranch.
Postal survey
Questions developed from literature
Random sample of the 1,530 ranchers who had summer federal grazing permits in
CO in 1983, and who had cattle
Excluded other seasons and grazing for sheep from sample
Pre-survey of 238 to see if population understood and responded well to the
format and questions. Also to estimate response variance and response rate to
determine the required sample size. Result: shortened survey and clarified some
questions.
Improved survey sent to 1,000 ranchers, with one additional mailing to
nonrespondents. Pooled pre-survey and survey responses.
313 useable surveys returned.
Motivation of Colorado Ranchers with Federal
Grazing Allotments
E.T. Bartlett, R.G. Taylor, J.R. McKean and J.G. Hof
Journal of Range Management, 1989
• Simple correlation analysis didn’t appear promising
• Cluster analysis -- 4 groups identified
– 7 variables relating to continuing ranching examined
– K-mean clustering
– Nonhierarchical
To Ranch or Not to Ranch: Home On the Urban
Range?
Robin H. Liffmann, Lynn Huntsinger and Larry C. Forero
Journal of Range Management, 2000
•
•
•
•
•
•
•
•
Ranchers in 3 CA counties (2 areas) surveyed about effects of increasing development,
land use change, and attrition of the ranching community on their commitment to
ranching -- and also to assess land conservation program acceptability.
Questions: practices, reasons for ranching, and what influences ranching’s future.
Sample randomly selected from compilation of U.C. Extension’s Farm Advisor list, NRCS
list, and an emergency feed program list.
Respondents had to 1) graze livestock in the study areas and 2) be the main decision
maker.
Postal questionnaire, 4-wave mailing technique, winter of 1993/94
Eligible and usable questionnaires:132 of 178 in Tehama County, and 113 of 204 in
Alameda/Contra Costa County.
Booklet form, 19 pages long, 48 questions and groups of questions (topically based)
Likert-scale, 1-5 ranking of importance (1=not at all important, 4=very important,
5=does not apply
To Ranch or Not to Ranch: Home On the Urban
Range?
Robin H. Liffmann, Lynn Huntsinger and Larry C. Forero
Journal of Range Management, 2000
• Statistical tests performed:
– Chi-square: significant differences in
categorical responses between
ranchers in the 2 study areas
– Student’s t-test: compare grouped
continuous variables (ex: respondent
age, how long ranch owned).
– Alpha level = .1 (p<.1 discussed as
significant)
Classifying Federal Public Land Grazing Permittees
Bradley J. Gentner and John A. Tanaka
Journal of Range Management, 2002
•
•
•
•
•
•
•
Identified the characteristics and attitudes of public land ranchers (U.S. Forest
Service and BLM).
Duplicate addresses and institutional permits removed from permittee lists,
leaving total individual ranch-operator population of 21,018.
Random sample of 2,000 operators
4 mailings spread over 8 weeks
53.5% useable response rate
<3 items left blank, responses kept and missing values imputed from rest of data
Follow-up telephone survey of 100 randomly selected non-respondents: the
means of key demographic characteristics not significantly different between
respondents and non-respondents. Data can be interpreted to accurately
represent the population of all public land permittees.
Classifying Federal Public Land Grazing Permittees
Bradley J. Gentner and John A. Tanaka
Journal of Range Management, 2002
• Cluster analysis -- 8 groups
identified
– Observational units
– Rancher attributes
• Previous literature (Bartlett et
al. 1989; Smith and Martin
1972)
• Focus group interviews in
Oregon and New Mexico
helped hone list
– K-means algorithm
– Validation process
Limitations
• Inhibits discovery
– Pre-determined questions and appropriate range of responses
• Inflexible
– Can’t adapt throughout research process
• Focus on aggregates rather than individuals
– Heterogeneity of ranchers (difficult to make generalizations, etc.)
• Can’t account for unanticipated outcomes
• Misses detailed, ranch-level analysis and complexity (Eakin
and Conley 2002, 272)
• Unable to explore rancher decision-making frameworks (Sayre
2004, 668)
Other Methods
•
Other surveys
– Face-to-face, structured interviews
– Web, telephone questionnaires
•
Qualitative methods
•
– In-depth interviews (Eakin and Conley 2002)
– Semi-structured interviews (Knapp and Fernandez-Gimenez 2009)
– Grounded theory: inductive approach where researcher begins with observations and
then identifies patterns (Didier and Brunson 2004, 331)
– “The authors are currently planning a study in Arizona to examine these “impact”
relationships in detail and to explain the viability or lack of viability of small rural
communities and the ranches nearby. In doing so, the authors do not feel they can use
aggregate methods such as input-output analysis or economic base analysis, traditionally
used by economists in impact studies. Such methods are too mechanical and gloss over
the complex social interrelationships so important in the business of small rural
communities. The authors feel they must take an almost anthropological view of the
communities’ inhabitants in order to examine the detailed interactions involved” (Smith
and Martin 1972, 8).
Mixture of interviews and questionnaire (Rowe, Bartlett and Swanson, Jr. 2001)
Sources
•
•
•
•
•
•
•
•
Bartlett, E.T., R.G. Taylor, J.R. McKean, and J.G. Hof. 1989. Motivation of Colorado ranchers
with federal grazing allotments. Journal of Range Management 42: 454-457.
Creswell, J.W. 2009. Research design: qualitative, quantitative, and mixed methods
approaches. Thousand Oaks, CA : Sage.
Didier, E.A. and M.W. Brunson. 2004. Adoption of range management innovations by Utah
ranchers. Journal of Range Management 57: 330-336.
Eakin, H., and J. Conley. 2002. Climate variability and the vulnerability of ranching in
southeastern Arizona: a pilot study. Climate Research 21: 271-281.
Kitchin, R. and N.J. Tate. 2000. Conducting research in human geography: theory,
methodology and practice. London: Prentice Hall.
Liffmann, R.H., L. Huntsinger, and L.C. Forero. 2000. To ranch or not to ranch: Home on the
urban range? Journal of Range Management 53: 362-370.
Rowe, H.I., E.T. Bartlett, and L.E. Swanson, Jr. 2001. Ranching motivations in 2 Colorado
counties. Journal of Range Management 54: 314-321.
Smith, A.H. and W.E. Martin. 1972. Socioeconomic behavior of cattle ranchers, with
implications for rural communities. Journal of Agricultural Economics: 1-9.
Questions?