Confidence intervals surrounding poverty estimates
The poverty estimates presented in the Scottish Households Below Average
Income publication series are based on a sample survey and therefore there
is some degree of statistical error, or uncertainty, around the estimates
produced. In other words, when it is reported that 16 per cent of individuals
are living in relative poverty before housing costs, this should be understood
not as an exact figure but as a best estimate.
Two different random samples from one population are unlikely to give exactly
the same survey results, which are likely to differ again from the results that
would be obtained if the whole population was surveyed. The level of
uncertainty around a survey estimate can be calculated and is commonly
referred to as sampling error.
We can calculate the level of uncertainty around a survey estimate by
exploring how that estimate would change if we were to draw many survey
samples for the same time period instead of just one. This allows us to define
a range around the estimate (known as a “confidence interval”) and to state
how likely it is that the real value that the survey is trying to measure lies
within that range. Confidence intervals are typically set up so that we can be
95 per cent sure that the true value lies within the range – in which case this
range is referred to as a “95 per cent confidence interval”.
Confidence intervals provide a guide to how robust the estimates are. Tables
1 to 4 below provide confidence limits around the key poverty estimates.
For instance, Table 1 shows that the best estimate for the number of
individuals in relative poverty before housing costs in 2012/13 was 16 per
cent, with a lower confidence limit of 14 per cent and an upper confidence
limit of 17 per cent. This means that we can be 95 per cent confident that the
percentage of individuals in relative poverty lies between 14 and 17 per cent.
Similarly, the lower confidence limit for the number of people in relative
poverty was 740 thousand and the upper confidence limit was 900 thousand.
So we can be 95 per cent confident that the true number lies between those
two figures.
The width of the confidence limits surrounding poverty estimates, when
compared to the magnitude of change between years, suggests that caution
should be exercised when making year on year comparisons.
When calculating the difference in poverty rates between two years, the same
methodology can be used to calculate a 95 per cent confidence interval for
this change. As such, if the entire confidence interval surrounding the change
is greater than zero (that is, the confidence interval does not contain zero),
then we can be 95 per cent that the year-on-year change is greater than zero
and is therefore statistically significant.
1
Calculating Confidence Intervals
In previous years, confidence intervals have been calculated using a formula
that sought to account for the difficulties associated with the complicated
sample design of the survey. Also, poverty thresholds are based on an
estimate of the UK median income. When calculating errors for poverty rates
it needs to be acknowledgement that median income is itself an estimate from
the survey and therefore has its own uncertainty.
While the previous method took these facts into consideration, and provided a
good estimate of errors, the methodology has been improved this year by
using a method called the bootstrap.
In the bootstrap, multiple new samples of the dataset are created, with some
samples containing multiple copies of one case with none in another.
Exploring how an estimate would change if we were to draw many survey
samples for the same time period instead of just one sample allows us to
generate confidence intervals around the estimate.
This has the advantage that it replicates the sample design and recalculates
both the median income and the poverty rate, therefore accounting for errors
associated with both calculations.
2
Table 1. Individuals in relative poverty before housing costs with 95%
Confidence Intervals
Estimate
Percentage
Lower
Upper
confidence limit confidence limit
Number (thousands)
Lower
Upper
Estimate
confidence limit confidence limit
Before Housing Costs
All individuals
2003/04
2004/05
2005/06
2006/07
2007/08
2008/09
2009/10
2010/11
2011/12
2012/13
18
17
17
17
17
17
17
15
14
16
17
16
16
15
16
16
16
14
12
14
20
18
19
18
18
18
18
16
15
17
910
860
870
840
870
860
870
770
710
820
850
790
800
770
800
790
790
700
640
740
970
920
950
910
940
940
940
840
780
900
Children
2003/04
2004/05
2005/06
2006/07
2007/08
2008/09
2009/10
2010/11
2011/12
2012/13
23
21
21
22
20
21
20
17
15
19
20
18
18
19
17
19
17
15
13
16
26
23
24
24
22
24
22
20
18
22
240
210
210
210
190
210
200
170
150
180
210
180
180
180
170
180
170
140
120
150
270
240
240
240
220
240
230
200
180
220
Working Age
Adults
2003/04
2004/05
2005/06
2006/07
2007/08
2008/09
2009/10
2010/11
2011/12
2012/13
16
15
15
14
15
16
16
14
13
15
14
14
14
13
14
14
15
12
11
13
17
16
16
16
16
17
18
15
14
17
480
460
460
440
470
490
500
440
410
480
440
420
420
400
420
450
460
390
370
420
520
500
500
490
510
550
550
480
460
540
Pensioners
2003/04
2004/05
2005/06
2006/07
2007/08
2008/09
2009/10
2010/11
2011/12
2012/13
22
20
20
19
21
16
16
16
14
15
19
18
17
16
19
14
14
14
12
12
24
22
22
21
23
18
18
18
16
18
200
180
180
180
210
160
170
160
140
150
180
160
160
160
180
140
140
140
120
130
220
210
210
210
230
180
190
190
170
190
Source: HBAI dataset, DWP.
3
Table 2. Individuals in relative poverty after housing costs with 95%
Confidence Intervals
Estimate
Percentage
Lower
Upper
confidence limit confidence limit
Number (thousands)
Lower
Upper
Estimate
confidence limit confidence limit
Before Housing Costs
All individuals
2003/04
2004/05
2005/06
2006/07
2007/08
2008/09
2009/10
2010/11
2011/12
2012/13
20
19
20
19
19
19
19
17
16
19
19
18
18
17
17
17
17
16
15
17
22
21
21
20
20
20
20
19
18
21
1010
960
980
940
960
960
970
900
860
1000
940
890
890
860
880
890
900
820
780
890
1080
1030
1060
1020
1040
1050
1050
980
940
1120
Children
2003/04
2004/05
2005/06
2006/07
2007/08
2008/09
2009/10
2010/11
2011/12
2012/13
26
25
24
25
24
26
24
21
19
22
24
22
21
22
21
23
22
18
16
19
29
28
27
28
26
29
27
24
22
26
270
250
240
250
240
250
240
210
190
220
240
220
210
220
200
220
220
180
160
190
300
280
270
280
270
290
280
240
220
260
Working Age
Adults
2003/04
2004/05
2005/06
2006/07
2007/08
2008/09
2009/10
2010/11
2011/12
2012/13
18
18
18
18
18
19
19
18
17
21
17
17
17
16
17
18
18
16
15
18
20
20
20
19
20
21
21
20
19
23
560
560
560
550
570
600
600
570
550
660
510
520
520
500
520
550
550
520
490
570
600
610
610
600
620
660
660
620
610
760
Pensioners
2003/04
2004/05
2005/06
2006/07
2007/08
2008/09
2009/10
2010/11
2011/12
2012/13
20
16
16
15
15
11
12
12
12
11
17
14
14
13
13
9
10
10
10
9
23
18
18
17
18
13
14
14
14
14
180
150
150
140
150
110
120
120
120
120
160
130
130
120
130
90
100
100
100
90
210
170
170
160
170
130
140
140
140
150
Source: HBAI dataset, DWP.
4
Table 3. Individuals in absolute poverty before housing costs with 95%
Confidence Intervals
Estimate
Percentage
Lower
Upper
confidence limit confidence limit
Number (thousands)
Lower
Upper
Estimate
confidence limit confidence limit
Before Housing Costs
All individuals
2003/04
2004/05
2005/06
2006/07
2007/08
2008/09
2009/10
2010/11
2011/12
2012/13
19
17
17
16
16
16
15
15
15
17
17
16
16
15
15
15
14
14
14
15
20
18
18
17
18
17
17
16
17
18
930
860
860
800
830
820
790
770
780
880
870
800
800
730
770
750
730
710
710
790
990
920
930
860
900
880
860
830
870
970
Children
2003/04
2004/05
2005/06
2006/07
2007/08
2008/09
2009/10
2010/11
2011/12
2012/13
23
21
21
21
19
20
18
17
17
20
21
18
18
18
16
18
16
15
14
17
26
24
23
23
21
23
20
20
19
23
240
210
210
210
190
200
180
170
170
200
210
180
180
180
160
170
150
150
140
170
270
240
240
230
220
230
210
200
200
230
Working Age
Adults
2003/04
2004/05
2005/06
2006/07
2007/08
2008/09
2009/10
2010/11
2011/12
2012/13
16
15
15
14
14
15
15
14
14
16
15
14
14
13
13
14
14
12
13
14
17
17
17
15
16
17
16
15
16
18
490
460
460
430
450
470
470
440
460
520
450
420
420
390
410
430
420
390
410
450
530
510
510
470
490
520
520
480
520
580
Pensioners
2003/04
2004/05
2005/06
2006/07
2007/08
2008/09
2009/10
2010/11
2011/12
2012/13
22
20
19
17
20
15
15
16
16
16
20
18
17
15
18
13
13
14
13
14
24
22
21
19
22
17
16
18
18
19
200
180
180
170
190
150
150
160
160
170
180
160
160
150
170
130
130
140
130
140
230
210
200
190
220
170
170
180
180
200
Source: HBAI dataset, DWP.
5
Table 4. Individuals in absolute poverty after housing costs with 95%
Confidence Intervals
Estimate
Percentage
Lower
Upper
confidence limit confidence limit
Number (thousands)
Before Housing Costs
All individuals
2003/04
2004/05
2005/06
2006/07
2007/08
2008/09
2009/10
2010/11
2011/12
2012/13
20
18
18
17
17
18
17
17
18
21
19
17
17
15
16
16
16
16
17
19
21
20
19
18
19
19
19
19
20
23
990
920
900
840
880
900
890
900
950
1110
920
850
830
780
810
830
820
830
870
1000
1060
980
980
910
950
980
970
970
1040
1240
Children
2003/04
2004/05
2005/06
2006/07
2007/08
2008/09
2009/10
2010/11
2011/12
2012/13
26
24
23
22
22
24
22
21
22
26
23
21
20
20
19
21
20
18
19
22
29
26
25
25
24
26
25
24
25
29
260
240
230
220
220
240
220
210
220
250
230
210
200
200
190
210
190
180
190
220
300
270
260
250
250
270
250
240
250
290
Working Age
Adults
2003/04
2004/05
2005/06
2006/07
2007/08
2008/09
2009/10
2010/11
2011/12
2012/13
18
18
18
16
17
18
18
18
19
22
17
16
16
15
16
17
17
16
17
20
19
19
19
18
19
20
20
20
20
25
550
540
540
500
530
570
570
570
590
720
510
500
500
460
480
520
520
510
530
630
590
590
590
550
580
620
620
620
660
820
Pensioners
2003/04
2004/05
2005/06
2006/07
2007/08
2008/09
2009/10
2010/11
2011/12
2012/13
19
14
13
12
13
10
10
12
14
13
17
12
11
10
12
9
9
10
12
11
21
16
15
14
15
12
12
14
16
16
180
130
120
120
130
100
100
120
140
140
160
120
100
100
110
80
90
100
120
110
200
150
140
140
150
120
120
140
170
170
Source: HBAI dataset, DWP.
6