SC971 Week 2: Stratified and Multistage Sampling
We will build upon the basic concepts of survey sampling introduced last week to discuss the main features of practical
sample designs:
Proportionate Stratification
Systematic Sampling
Disproportionate Stratification
Variable Sampling Fractions
Multistage Sampling
Probability Proportional to Size (PPS) Sampling
Components of Design Effects
Proportionate Stratified Sampling
Think of the population as being divided into I subsets (i = 1, ... I), with Ni units in the ith subset (𝑁 = ∑𝐼𝑖=1 𝑁𝑖 ).
Proportionate sampling involves:
- Selecting a sample independently from each subset (so we call the subsets sampling strata);
- Selecting the same proportion, n/N, of units from each stratum
Hence, the sample size in stratum i will be
𝑛𝑁𝑖
𝑁
Motivation: to ensure (unlike SRS) that the sample proportion from any particular stratum equals the population
proportion.
Effect: This will increase precision if strata are correlated with survey measures.
Standard Errors for Stratified Sampling
In general: 𝑉𝑎𝑟(𝑥̅ ) =
𝑁𝑖2 𝑆𝑖2
𝐼
∑𝑖=1 2 (1
𝑁 𝑛
𝑖
−
𝑛𝑖
𝑁𝑖
),
- (2.1)
where 𝑆𝑖2 = 𝑉𝑎𝑟𝑖 (𝑥) is the variance of 𝑥 within stratum i
Note:
1. Differences between strata do not contribute to 𝑉𝑎𝑟𝑖 (𝑥).
2. Consequently, sampling variance will be reduced if strata are homogeneous (small 𝑆𝑖2 ).
3. In the case of proportionate sampling,
1
𝑛
𝑛
𝑁
𝑉𝑎𝑟(𝑥̅ ) = (1 − ) ∑𝐼𝑖=1 𝑁𝑖 𝑆𝑖2
𝑛𝑖
𝑁𝑖
𝑛
= , so
𝑁
- (2.2)
Example
Recall the example from last week of a population of 6 individuals with associated measures. Now suppose that we know in
advance which are men and which are women, so we treat these as two sampling strata:
Men
A
2
B
6
Women
D
10
C
8
E
10
F
12
Now, design is to sample one from each stratum and estimate mean.
There are now only 9 possible samples of 2 from 6 (cf. 15 with SRS)
Sample
1
2
3
4
A
E
A
F
B
C
5
6
7
B
E
B
F
C
D
8
9
D
E
D
F
Members of
sample
A
B
A
C
A
D
Values in
sample
2
6
2
8
2 2 2 6
10 10 12 8
6 6 6 8 8 8 10 10 10
10 10 12 10 10 12 10 12 12
Sample mean
4
5
6
8
6
7
7
B
D
8
9
9
C
E
9
C
F
E
F
10 10 11 11
Example continued
As before, we can calculate the variance of this sampling distribution:
𝑥̅ = 72⁄9 = 8
Sample
Mean
xi
Frequency
5
6
7
8
9
10
11
1
1
2
1
2
1
1
N=9
Mean
x freq
5
6
14
8
18
10
11
72
Deviation
Squared deviation
from mean (8)
from mean
xi  x
 x i  x 2
-3
-2
-1
0
1
2
3
9
4
1
0
1
4
9
𝑆𝐸 2 = 30⁄8 = 3.75 𝑐𝑓. 64⁄14 = 4.57 (𝑆𝑅𝑆)
Freq. x
Sq.dev.
 x i
 x
9
4
2
0
2
4
9
30
2
Design Effect due to Proportionate Stratified Sampling
So, 𝑆𝐸𝐶2 = 3.75.
2
Recall from last week, 𝑆𝐸𝑆𝑅𝑆
=
64
14
= 4.57.
Proportionate stratified sampling has reduced sampling variance by a factor of 3.75⁄4.57 = 0.82, compared to SRS.
This is the design effect due to stratified sampling: 𝐷𝑒𝑓𝑓(𝑥̅ ) = 0.82. [and 𝐷𝑒𝑓𝑡(𝑥̅ ) = 0.91]
Stratum Construction for Proportionate Stratification
Choose strata that are homogeneous in terms of 𝑥
In other words, we want an association between 𝑥 and 𝑖, i.e. some variance between strata, not only variance within
strata
In general:
- a larger number of strata will be associated with greater precision gains;
- strata defined by cross-classification of multiple factors better than same number of strata defined by many
categories of one factor (e.g. 4 age bands x gender x 4 regions vs. 32 age bands);
- stratum boundaries should be chosen carefully for continuous factors.
Implicit and Explicit Stratification
Stratified sampling: sorting (stratifying) the sampling frame prior to selection.
There are two ways of doing this:
Explicit Stratification involves sorting the population list (frame) into distinct strata and then sampling independently
from each stratum (as in above example)
Implicit Stratification involves sampling systematically from an ordered (stratified) list
It is possible (and often desirable) to combine explicit and implicit stratification - i.e. to stratify implicitly within explicit
strata
Systematic Sampling
Involves sampling at a fixed interval down a list
If the list is ordered in some meaningful way, this has the effect of (proportionate) stratification
It also has the advantage of being easy to implement
Procedure is to calculate the required interval (I), then generate a random start (R). The sampled units are then the Rth,
(R+I)th, (R+2I)th etc. on the list.
I = N/n, where N is the total number of units on the list, and n the desired sample size. R is a random number between 1
and I.
Note that I need not be an integer. E.g. if desired n is 500 and N = 10,679, using I = 21.36 will give exactly n = 500, but
rounding to I = 21, will give n = 508. Do not use I = 21 and then stop once 500 are sampled: biased!
Stratified vs. Quota Sampling
Imposing quotas has similar effect to stratification - namely to reduce sampling variance
But, quota sampling also has inherent bias towards more accessible and more willing population members
This may manifest itself as a bias in the survey measures
Thus, quota sample estimates could have relatively high precision, but be biased and therefore have low accuracy (see
last week’s lecture)
So, if it is important to ensure correct representation in terms of a particular variable… generally better to do this
through stratification and random selection than through the use of quotas and subjective selection
Disproportionate Stratified Sampling
Recall (last week) that probability sampling does not require all units to have an equal probability of selection
Disproportionate sampling involves:
- Selecting a sample independently from each stratum;
- Not selecting the same proportion of units from each stratum, but instead allowing the sampling fraction, 𝑓𝑖 =
𝑛𝑖
𝑁𝑖
, to
vary between strata
Motivation 1: to increase sample size from particular strata of interest without unduly increasing overall sample size.
Effect 1: to increase precision for estimation within that stratum but, usually, to reduce precision for total sample
estimation.
Motivation 2: to over-sample strata with particularly high variance.
Effect 2: to increase precision for total sample estimation.
Disproportionate Stratified Sampling: Estimation
For unbiased estimation, we can no longer use the direct sample analogue of the population parameter. Instead, we
should use the Horvitz-Thompson estimator, which in the case of a mean is:
𝑛
∑
𝑤 𝑙 𝑥𝑙
𝑋̅̂ = ∑𝑙=1
𝑛
𝑙=1 𝑤𝑙
- (2.3)
Where 𝑤𝑙 is the design weight (or sampling weight) assigned to sample unit 𝑙.
Design weights should be proportional to the inverse of the selection probability: 𝑤𝑖𝑙 =
𝑁𝑖
𝑛𝑖
.
Design Effect due to Disproportionate Stratified
Sampling
Recall expression (2.1) (ignoring f.p.c.): 𝑉𝑎𝑟(𝑥̅ ) =
𝑁𝑖2 𝑆𝑖2
𝐼
∑𝑖=1 2
𝑁 𝑛
𝑖
In the case of motivation 1 for disproportionate stratification, it will often be the case that 𝑆𝑖2 varies little, so it is
instructive to consider consider 𝑉𝑎𝑟(𝑥̅ ) when 𝑆𝑖2 = 𝑆 2 . Then, 𝑉𝑎𝑟(𝑥̅ ) =
So, 𝐷𝑒𝑓𝑓 (𝑥̅ ) =
=
2
𝑁𝑖
𝐼
∑
𝑁2 𝑖=1 𝑛
𝑛
𝑖
𝑛
𝑁2
And 𝑛𝑒𝑓𝑓 (𝑥̅ ) =
∑ 𝑛𝑖 (𝑤𝑖2 )
- (2.4)
𝑛
𝐷𝑒𝑓𝑓(𝑥̅ )
=∑
𝑁2
𝑛𝑖 (𝑤𝑖2 )
- (2.5)
𝑆2
𝑁𝑖2
𝑁
𝑛𝑖
∑𝐼𝑖=1
2
Example
Example: Suppose I = 2; N1 = N2; 𝑆12 = 𝑆22 = 𝑆 2 . Consider two alternative sampling schemes:
a) Proportionate allocation:
𝑛𝑖
𝑁𝑖
=
𝑛
𝑁
;
b) Disproportionate allocation, where
𝑛1
𝑁1
=2
𝑛2
𝑁2
; i.e. 𝑛1 =
Then, with design a), from (2.1) we have:
𝑉𝑎𝑟(𝑥̅ ) =
2
𝑆2
𝑁𝑖
𝐼
∑
𝑖=1
𝑁2
𝑛
𝑖
=
2
2(𝑁⁄2)
( 𝑛 )
𝑁2
⁄2
𝑆2
=
𝑆2
𝑛
(of course!)
And with design b), we have :
𝑉𝑎𝑟(𝑥̅ ) =
2
(𝑁⁄2)
( 2𝑛
𝑁2
⁄3
𝑆2
So, 𝐷𝑒𝑓𝑓 (𝑥̅ ) =
9
8
+
2
(𝑁⁄2)
𝑛⁄ )
3
= 1.125
=
9 𝑆2
8 𝑛
[and 𝐷𝑒𝑓𝑓 (𝑥̅ ) = 1.06]
2𝑛
3
; 𝑛2 =
𝑛
3
Deff due to Disproportionate Stratified Sampling, ctd.
For designs with variable sampling fractions, it may be reasonable to assume 𝑆𝑖2 ≅ 𝑆 2 , so we can use the simplified
expression (2.4), 𝐷𝑒𝑓𝑓 (𝑥̅ ) =
𝑛
𝑁2
∑ 𝑛𝑖 (𝑤𝑖2 ), to approximate the impact on variance of estimates.
In general, it will be found that:
- a larger range of sampling fractions (weights) will result in a larger 𝐷𝑒𝑓𝑓 (greater loss of precision);
- over-sampling a large subgroup will result in a greater loss of precision than over-sampling a small subgroup;
- when the main aim is to produce estimates for subgroups, equal sample sizes per subgroup will be an efficient design;
when the main aim is to produce estimates for the total population, equal sampling fractions will be efficient.
Graphical Illustration of Approximate Design Effect
The graph shows the relationship between the proportion of the sample taken from a stratum with a relatively high
sampling fraction (x-axis) and the consequent loss of precision, as measured by the design effect (y-axis).
- The three lines relate to three different possible relative weights: 2:1, 4:1 and 10:1 (i.e. w1=1 in all cases).
- Obviously, these examples are all for the simple case of I = 2.
- It demonstrates the first two bullet points on the previous page
3.4
DEFF VSF
3
2.6
2.2
1.8
1.4
1
0
0.2
0.4
0.6
0.8
n1/n
w2=2
w2=4
w2=10
1
Approximate Design Effect
The simplified expression (2.4) for the design effect due to variable sampling fractions (disproportionate stratified
sampling) in commonly used in the situation of motivation 1 (see earlier), i.e. a desire to over-sample a small subgroup
in order to increase its representation
It is also used in another situation where variable sampling fractions arise, though not (explicitly) from stratification,
namely when the sampling frame or method gives us no choice, e.g.
 When the frame contains duplicates that can only be identified as such at the data collection stage (example:
survey of motorcyclists);
 In certain multi-stage sampling situations, where there are constraints on the number of selections that can be
made at the final stage (example: address-based sampling, with one person selected per address).
Over-Sampling Variable Strata
Motivation 3 for using disproportionate stratified sampling:
Sometimes, we can identify strata that have high population variances. Over-sampling these strata will tend to increase
the precision of total-sample estimates (reduce standard errors).
We can only do this if we have advance estimates of stratum variances.
Generally only used in rather specialist situations, e.g. business or agricultural surveys with large and predictable
variation in stratum variances.
Example
Example: Suppose I = 2; N1 = N2; and that we predict 𝑆12 = 2𝑆22 (= 1.33𝑆 2 𝑖𝑓 𝑋̅1 = 𝑋̅2 ).
Consider two alternative sampling schemes:
𝑛
𝑛
a) Proportionate allocation: 𝑛1 = ; 𝑛2 = ;
2
2
b) Disproportionate allocation: 𝑛1 = 0.58𝑛; 𝑛2 = 0.42𝑛
Then, with design a), from expression (2.1) we have:
𝑉𝑎𝑟(𝑥̅ ) =
𝑁𝑖2 𝑆𝑖2
𝐼
∑
𝑁2 𝑖=1 𝑛
1
𝑖
=
2
(1.33𝑆 2 +0.66𝑆 2 )(𝑁⁄2)
(
)
𝑛⁄
𝑁2
2
1
=
And with design b), we have :
𝑉𝑎𝑟(𝑥̅ ) =
2
1.33𝑆 2 (𝑁⁄2)
(
𝑁2
0.58𝑛
1
+
2
0.66𝑆 2 (𝑁⁄2)
)
0.42𝑛
= 0.971
So, 𝐷𝑒𝑓𝑓 (𝑥̅ ) = 0.971 [and 𝐷𝑒𝑓𝑓 (𝑥̅ ) = 0.986]
𝑆2
𝑛
𝑆2
𝑛
(of course!)
Optimum Allocation
Sampling variance for total sample estimates can in principle be minimised by following the optimum allocation rule:
𝑛𝑖
𝑆𝑖
𝛼
𝑁𝑖 √𝐶𝑖
Where 𝐶𝑖 is the unit cost of data collection for a unit in stratum 𝑖. So, if data collection costs do not vary between strata,
this simplifies to:
𝑛𝑖
𝛼 𝑆𝑖
𝑁𝑖
and if stratum variances are equal, it further simplifies to:
𝑛𝑖
𝛼 𝐾
𝑁𝑖
demonstrating than an equal probability selection method is optimum in the situation where variances and data
collection costs are equal in all strata (other things being equal).
Practical Limits to Stratification
In multistage sampling contexts, stratification is often only possible at PSU level (e.g. household surveys)
Correlation between strata and survey variables is typically modest
Multi-purpose nature of surveys: optimal stratification for one estimate may produce no benefit for another
Typically there is a lack of information about stratum variances
Example of Stratification
The Health Survey for England 1996 (DH)
Stage 1: Postcode Sectors stratified by:
o
o
o
o
o
The 14 Regional Health Authorities (1st-level explicit strata)
Proportion of adults with limiting long-term illness, in three bands (2nd-level explicit strata)
Proportion of households with "non-manual" head, in two bands (3rd-level explicit strata)
Proportion of households with no car, in two bands (4th-level explicit strata)
Proportion "non-white" (5th-level stratification: implicit)
720 sectors were sampled systematically
Stage 2: Within each sector, addresses are in postcode order, and selected systematically. This provides some
geographical stratification.
Multi-Stage Sampling
The units in the population are arranged hierarchically
A 3-stage design would entail:
Primary sampling units (PSUs)
Secondary sampling units (SSUs)
Sample elements
It would be necessary to assign every element uniquely to one SSU and every SSU uniquely to one PSU
Stage 1: select sample of PSUs
Stage 2: select sample of SSUs within each selected PSU
Stage 3: select sample of elements within each selected SSU
PSUs, SSUs, Elements
Example: general population survey:
PSUs might be postcode sectors
SSUs might be households
Elements might be persons
Example: business survey:
PSUs might be companies
SSUs might be workplaces
Elements might be employees
There could be any number of stages; 2, 3 or 4 are common
Why Multi-Stage Sampling?
No frame of elements available, but frame of PSUs available (example: national sample of school pupils, where schools
could be PSUs)
Cost of data collection (example: general population sample involving face-to-face interviewing)
Access to elements may only be via “gatekeepers” (examples: students, employees, trainees)
Data quality (example: in the case of face-to-face interviewing, field work can be better supervised if in clusters)
Design Choices (clustering): Some General Points
Larger sample sizes per cluster (𝑛𝑖𝑗 ) will generally result in larger design effects due to clustering (see later)
But larger 𝑛𝑖𝑗 will also generally result in larger cost savings (e.g. field interviewers, gatekeepers)
Necessary to make an appropriate compromise: i.e. where cost saving outweighs loss in precision, to produce higher
overall accuracy per unit cost (cf. last week’s lecture)
Selection Probabilities for Multistage Sampling
With multi-stage sampling, the selection probability of each element is the product of the (conditional) selection
probabilities at each stage
e.g. probability of sampling unit 𝑙 in SSU 𝑘 in PSU 𝑗 is 𝑝𝑗𝑘𝑙 = 𝑝𝑗 x 𝑝𝑘|𝑗 x 𝑝𝑙|𝑗,𝑘
For unbiased estimation, we need to weight each sampled element 𝑗𝑘𝑙 by 𝑤𝑗𝑘𝑙 = 1/𝑝𝑗𝑘𝑙
So, it is important to control and record the selection probabilities at each stage.
Other things being equal, it is desirable to keep selection probabilities equal for all elements. With multi-stage sampling,
there are many ways to do this.
For example, in 2-stage sampling, whatever we set 𝑝𝑗 to be, 𝑝𝑙|𝑗 can be set proportional to 1⁄𝑝𝑗 and this will produce an
epsem (self-weighting) design.
Selection Probabilities for Multistage Sampling ctd.
There are three intuitive alternative ways to set selection probabilities:
a) select PSUs with equal probabilities and then a fixed number of elements within each (𝑛𝑗 = 𝑛1 ∀ 𝑗). This results in
unequal selection probabilities and is therefore undesirable because it will generally cause a loss in precision compared
with an epsem design.
b) select PSUs with equal probabilities and then a variable number of elements within each (𝑛𝑗 𝛼 𝑁𝑗 ), to give equal
overall selection probabilities. This design has practical problems. The elements in one PSU typically form one
interviewer workload, so large variation in 𝑛𝑗 is undesirable. There is additionally a (usually modest) loss of precision
associated with variation in 𝑛𝑗 . And, the sample size is not fixed in advance - it is a random variable!
c) select PSUs with PPS and then a fixed number of elements within each. This overcomes the problems associated with
a) and b), but it depends on the availability of an accurate measure of the number of elements in each PSU (and SSU).
We now discuss this design further.
Probability Proportional to Size (PPS) Selection
How it works: A 2-stage design.
We set 𝑝𝑗 proportional to 𝑁𝑗 (the number of elements in the population in PSU j). So 𝑝𝑗 = C 𝑁𝑗 .
We then select the same number of elements, D, from each sampled PSU, so 𝑝𝑙|𝑗 = D/ 𝑁𝑗 .
Then, 𝑝𝑙 = 𝑝𝑗 x 𝑝𝑙|𝑗 = C 𝑁𝑗 x D/ 𝑁𝑗 = CD, which is the same for every element.
Implementation:
We do not need to calculate the selection probabilities at each stage in order to make the selection.
We create a cumulative total down the list of PSUs and then sample systematically down that list of totals, including each
PSU within which the interval falls.
Example of PPS 2-Stage Sample Selection
Example: Selection of 3 PSUs from 10 with PPS and 25 units from each selected PSU, so that n=75.
PSU
1
2
3
4
5
6
7
8
9
10
Size
1000
900
800
1200
1500
1300
1100
500
1000
700
Cum. size
1000
1900
2700
3900
5400
6700
7800
8300
9300
10000
Selection
*
*
*
Now, N = 10,000 and n = 3 (PSUs).
To select systematically (see earlier), I = 3,333 and R needs to be a random number between 1 and 3,333. Suppose we
generate R = 1,050.
Then, we sample the PSUs that contain elements 1,050, (1,050 + 3,333) and (1,050 + 2 x 3,333), i.e. PSUs 2, 5 and 7
Some Limitations of PPS Sampling of PSUs
We may have only imperfect estimates of the number of elements in each PSU (the size measure):
We could then adjust the sample size within each PSU to keep overall probabilities equal or we might simply weight by
𝑤𝑙 =1/𝑝𝑙
The sampling interval might be smaller than the number of elements in some PSUs. (This will only happen if the sampling
fraction of PSUs is large and/or the size of PSUs is highly variable.) Those PSUs will be certain to be sampled, and could be
sampled more than once.
We might place these PSUs in a separate stratum and include them with certainty (𝑝𝑗 = 1). We might also increase
their sample size of elements, to keep overall probabilities equal, or we might weight.
Design Effect due to Clustering
Clustering tends to increase sampling variance (but this may be offset by the fact that a larger sample size can be obtained
for any given cost). This is because units within a cluster tend to be more homogeneous than units as a whole.
Clustering is therefore tending to have the opposite effect to stratification.
The design effect due to clustering takes the form:
𝐷𝑒𝑓𝑓𝑐𝑙 = 1 + (𝑏 − 1)𝜌
- (2.6)
where 𝑏 is sample size per cluster (in practice 𝑏 may vary – see note on next page), and 𝜌 (roh) is the intra-cluster
correlation.
𝜌 = 0: randomly sorted clusters
𝜌 = 1: perfectly homogeneous clusters
Note: 𝜌 is a population characteristic relating to the chosen definition of PSU; 𝑏 is chosen by the researcher as part of the
sample design
e.g. 𝑏 = 10: if 𝜌 = 0 then 𝐷𝑒𝑓𝑓𝑐𝑙 = 1; if 𝜌 = 1 then 𝐷𝑒𝑓𝑓𝑐𝑙 = 10; more realistically, if 𝜌 = 0.05 then 𝐷𝑒𝑓𝑓𝑐𝑙 = 1.45
A Note about Cluster Sample Size
Expression (2.6) strictly holds only when there is no variation in cluster sample size, i.e. 𝑛𝑗 = 𝑏 ∀ 𝑗. For complex surveys,
where 𝑛𝑗 may vary and, additionally, unequal selection probabilities may be used, the design effect due to clustering is:
𝐷𝑒𝑓𝑓𝑐𝑙 = 1 + (𝑏 ∗ − 1)𝜌
𝐽
where 𝑏 ∗ =
𝑛𝑗
2
∑𝑗=1(∑𝑙=1 𝑤𝑗𝑙 )
𝑛𝑗
2
∑𝐽𝑗=1 ∑𝑙=1 𝑤𝑗𝑙
- (2.7)
.
2
Note that for an epsem design, this gives 𝑏 ∗ =
∑𝐽𝑗=1(𝑛𝑗 )
𝐽
∑𝑗=1 𝑛𝑗
In some situations, notably when variation in 𝑛𝑗 is small, mean cluster size, 𝑏̅ = 𝑛̅𝑗 , may provide an adequate approximation.
But often it is a poor approximation: see Lynn & Gabler (2005)
Lynn P & Gabler S (2005) Approximations to b* in the prediction of design effects due to clustering, Survey Methodology, 31, 101-104
Example of Intra-Cluster Correlations
From the British Social Attitudes Survey:
𝜌̂
𝑏
̂
𝐷𝑒𝑓𝑡
̂ if 𝑏 = 10
𝐷𝑒𝑓𝑡
Household size
Owner-occupier
Has telephone
Asian
Roman Catholic
0.070
0.231
0.102
0.334
0.037
16.6
16.5
16.5
8.3
16.4
1.45
2.14
1.61
1.86
1.25
1.28
1.75
1.38
1.53
1.15
Not racially prejudiced
Extra-marital sex wrong
Dodging VAT is OK
0.021
0.044
0.021
8.4
8.3
8.2
1.08
1.15
1.07
1.03
1.08
1.04
Variable
Note: 𝜌̂ is low for attitudinal variables, so design effects small. But 𝜌̂ is large for variables related to ethnicity and
housing type. Thus, the most effective degree of clustering might be greater for an attitude survey (fewer clusters, with
larger 𝑛𝑗 ) than for a housing survey.
http://courses.essex.ac.uk/sc/sc971/