Basic issues in
measuring gender
attitudes
Ko Oudhof
Statistics Netherlands
What will I tell you?
• Just for a start: your own contribution
• Subjective indicators
• What are attitudes
• Measurement issues
• Analytic issues
• Here and there: gender/ international
comparability
workshop UNDP/UNECE gender statistics 2004
Before I tell you anything
• Think of one short statement on
the role of women or men in
decision making that according to
yourself would make it possible to
distinguish respondents into
advocates and opponents of
gender equality by looking at their
(dis)agreement with your
statement
workshop UNDP/UNECE gender statistics 2004
Introductory vocabulary
• Indicator
• One digit, evaluating, goalrelated
• Cognitive
• About seeing, knowing and
thinking
• Evaluative
• About good or bad
• Affective
• About like or dislike
• Item
• Statement or question
• Scale
• List of items
workshop UNDP/UNECE gender statistics 2004
Policy and role of indicators
Selection policy goals
Assessing policy process
 Evaluate policy results
workshop UNDP/UNECE gender statistics 2004
Subjective indicators and policy-1
 Selection policy goals
 what do people (not) want? (Worries, aspirations,
satisfactions)
 what do people need or get rid off? (immaterial
needs, happiness)
 Assessing policy process
 Public support (trust, support)
 Assess course of policy (predictions, prognosis)
 Evaluate policy results
 Goal attainment (health, inequality, perceived
safety, xenophobia)
workshop UNDP/UNECE gender statistics 2004
Subjective indicators and policy-2
 No ‘objective’ observation?
 Subjective condition real policy objective
 Direct measurement
 Both subjective and objective indicators depart
from implicit assumptions on each other in some
implicit psychological model on behavior!
 Vague? Limits to aggregation!
 Measuring all possible wrongs?
 Indicators with a large mandate needed
 Statistical weaknesses




No money and no counting
Monetary value or size of subjective condition?
Specific measure and methodology
Experts needed
workshop UNDP/UNECE gender statistics 2004
Subjective indicators
• Policy-relevance (issues)
• Need- or Behavior-related (predictability)
• Variability (daily fluctuations versus almost
invariable states)
– Now – indicators
– In these times – indicators
– Long term perspective - indicators
workshop UNDP/UNECE gender statistics 2004
Subjective conditions and the world
•
•
•
•
•
•
•
•
Needs and wants
Emotions
Perception
Experience
Learning
Motives
Goals
Etc.
Now response
Structured
• Fysical
environment
stable
behavior?
Now feedback
workshop UNDP/UNECE gender statistics 2004
• Social
environment
Attitudes (common elements in most
definitions)
• Oriented on object, person, institution
or event
• Evaluative component
• Cognitive component
• Affective component
• Stable condition or construct
• Intermediary between object stimulus
and behavioural response:
consistency
workshop UNDP/UNECE gender statistics 2004
Relatives with likeness
 Opinions (now)
 stability less
 more cognitive and not always evaluative
 behavioral relation weaker
 Values (long term)
 general and less object-oriented
 stability higher
 behavioral relation more indirect
 Norms (derivative)
 prescription of behavior
stability higher
 behavioral relation stronger and more direct
 less cognitive and less affective
workshop UNDP/UNECE gender statistics 2004
Relation subjective elements
Abstraction
value
attitude
opinion
Time
workshop UNDP/UNECE gender statistics 2004
Model Theory Planned Behaviour
(Ajzen)
social norm
attitude
behavioural
intention
perceived
behavioural
control
workshop UNDP/UNECE gender statistics 2004
behaviour
General Model (Van der Pligt & De Vries)
cognitive
responses/
considerations
weight
attitude
(evaluative
response)
attitudeobject
affective
responses/
considerations
weight
workshop UNDP/UNECE gender statistics 2004
behavioural
response
Relation attitude – behavior ( reasoned action approach in 2004)
Ajzen & Fishbein, 2004
workshop UNDP/UNECE gender statistics 2004
Attitudes and gender policy
 Hardly any NSI
 Why gender attitudes?
 Attitude change as objective?
 Defensive in discussion?
 Same question elsewhere?





Macro-economy: confidence consumers/producers
Business world: marketing
Politics : voting behavior
Health: perceived health
Crime: feeling of insecurity
workshop UNDP/UNECE gender statistics 2004
Gender attitude research and tools
in practice
Mainly academic or ad hoc research
Few international research projects
Gender role (labor market or household)
main topic
Hardly any standardisation
Example: attitudes on female decision
making
Support preferential policies
Attitudes among decision makers
Acceptance of female management
Effects of leadership styles
workshop UNDP/UNECE gender statistics 2004
Engendering attitudes
•
•
•
•
•
•
Objects
Explaining behavior
Measurement tools
Analysis
Interpretation
Presentation
•
•
•
•
•
•
Gender Issues
Engendered concepts
Gender validity
By sex or more*?
By sex or more*?
By sex or more*?
More = differences compared to other non-gendered
research as consequence of earlier steps
workshop UNDP/UNECE gender statistics 2004
Measurement of attitudes
 Explicit measurement
(under conscious control respondent)
 one item
 multi-item
Implicit measurement
(without conscious control respondent)
 observation of behavior (non-obtrusive)
 bodily response
 response latency
 Academic research and less relevant for
statistical offices etc.
workshop UNDP/UNECE gender statistics 2004
Quality of measurement - reliability
– equal outcomes of tool when measuring the
same?
– random error
– inter-items reliability
– test-retest / split-half
– interobserver reliability
– quality measure versus external factors
workshop UNDP/UNECE gender statistics 2004
Quality of measurement - validity
– Similar results from other tools when measuring
the same
– Systematic error
– Construct validity
– convergent validity – what should
– divergent validity - not what should not
– Predictive validity
– Multitrait-multimethod matrix as solid validitytesting design
workshop UNDP/UNECE gender statistics 2004
Survey?
• Insight in own attitude/opinion
• Can they express the attitudes/opnions:
– personal conditions (e.g. ability)
– situational conditions (e.g. individual interview?)
• Plausibility true answering
– personal conditions (e.g. strategic response)
– situational conditions (e.g. interviewer interaction)
• Alternative informants/ assessing documents
• General considerations on survey design
workshop UNDP/UNECE gender statistics 2004
Single item or multi-item measurement?
• Quick  Cheap
• Response time  Expensive
• all or nothing, also in timeseries
• Shortening scale generally
possible
• one-dimensional
• Multidimensional
• sometimes quite high and
reliable
• Scale properties can be
assessed
• how do you assess
psychometric properties
• International comparability and
standardisation of scales (or
subscales)
workshop UNDP/UNECE gender statistics 2004
Multi-item variants
Osgood scale
Thurstone scale
Likert scale
Guttman scaling
multi-object
measuring
• General dimensions
• Pretested dichotomous scaled items
• Addition of multi-point (3-100) items
• Scaled statements
• Conjunct / dominance/ similarity
monetary methods • Simulated markets/ hedonic price
analysis/ contingent valuation (CV) or
(WTP)
ranking (CR)
workshop UNDP/UNECE gender statistics 2004
Likert scale
• Rather simple
• List of items expressing positive and negative
opinions on attitude object
• Selection of relevant items by content
• Choice of answering categories
–
–
–
–
Number
meaning of scores
middle category
don’t know: yes or no
• Scale rating by summing item values (after recoding)
• Self-made or standard?
workshop UNDP/UNECE gender statistics 2004
Selection of items
•
•
•
•
•
•
•
•
•
•
Relevant for all groups (e.g. young + old)
Clear and unequivocal interpretation
No multiple question items
No double negations
No questions but statements (response set)
No confirmation bias  pos + neg
Time spans: now/these days/whole life
Suggestive expression (most people…)
Biased or suggestive answering categories
Personalised or public statements (Hakim)
workshop UNDP/UNECE gender statistics 2004
More possible interferences
•
•
•
•
•
•
•
•
Character of survey (crime or labour?)
Interviewer
Order of topics in questionnaire
Introduction of scale
Interference of different topics in one scale
Order of items
No repeats or redundancy
Social desirability  overreporting or
underreporting
workshop UNDP/UNECE gender statistics 2004
So you’ve got your data
•
Assessing or reassessing quality of scale?
1. Reliability aspects
2. Validity aspects
•
Deciding what to do considering
–
–
–
–
Objectives (employer/ supervisor)
Tools (standards?)
Methodological explanations
Explanation of results
workshop UNDP/UNECE gender statistics 2004
Item and scale analysis
• Assessing reliability of scale as given
– Depending on design
– Without any validity analysis of scale
• (re)assessing items + scale(s)
–
–
–
–
linearity and other assumptions?
multidimensional?
dropping items possible?
selection of techniques to assess scale
• Consistency/ homogeneity items
• Analysis content via Princ.Comp./ factoran./
scaling
workshop UNDP/UNECE gender statistics 2004
Scale ratings
• Which ratings should be used?
– sum
– weighted sum (only part of items needed?)
– factor scores
• To be used for what?
– is level relevant? (breakdowns or time series)
– is level confusing? (comparability)
– nature of audience (general public or scientists)
workshop UNDP/UNECE gender statistics 2004
Gender & international
•
•
•
•
Which issue or topic?
Which concept?
Which measurement tool?
Main problem for both: validity
– reduction or prevention of systematic error
• Analysis: extra = validity analysis
• Interpretation = plus restraint by validity
• Presentation = including reserves by limited
validity?
workshop UNDP/UNECE gender statistics 2004
More to learn
• In hand-out suggestions for further reading
• Standard handbooks for students social
psychology
• Look on the internet by using searching
machines: attitude, gender, survey (e.g.
Ajzen)
• Search for sites on international surveys
(e.g. European Social Survey) and research
databases
workshop UNDP/UNECE gender statistics 2004
Evaluating both scales
• Gender dimension
• Inter-item consistency? Homogeneity?
• Valid multidimensionality?
• Quality of separate items?
• Scale quality
• Etc.
workshop UNDP/UNECE gender statistics 2004