Social Security Assurance
Tracking Process Error using
Sample Surveys
ITSEW 2009 Tallberg
Social Security Policy Branch – Random Sample Surveys
FaCSIA Fraud and Compliance Response
Cost of Compliance
Customer Attitude to Compliance
Compliance Strategy
Debt Recovery Strategy
Fraud Investigation/Prosecution
Action/Criminal Conviction
Reparation sought through the courts
Those customers whose
deliberate and organised noncompliance results in a
significant attack on the
integrity of the payment
Co
a
der
nsi
Those who deliberately not
comply
tion
for
or
err
Education Programmes e.g. Real Estate
and Overseas Pensions publicity
campaigns
tive
tra
is
min
Ad
.g.
re
ive
wa
bts
Targetted Prevention e.g.
Service Profiling
f De
Those who get it right
yo
Those who try to
comply but make
honest mistakes/do
not understand their
requirements
ion
r at
of
ide
r
ove
ost
ns
Co
Rec
com
plia
nce
ion
and
Un
it c
ut
sec
Pro
Those who do not
take reasonable
care to comply
g
sin
Rai
act
ion
for
Targetted Compliance Activities e.g. Data
Matching, Compliance Reviews
Small Debt Waiver: <$50
Adapted from Braithwaite Regulation Model
Social Security Policy Branch – Random Sample Surveys
Social Security Policy Branch – Random Sample Surveys
Random Sample Surveys Background
• Began in 98/99
• Continues as an ongoing project
• Methodology tested, improved and ratified by External Consulting
Actuaries
• Audited by the Australian National Audit Office (ANAO)
• Recommendations have been implemented
Social Security Policy Branch – Random Sample Surveys
Service level agreement
• FaHCSIA and Centrelink have an SLA in place to manage the RSS
• SLA is a set of agreed processes and procedures to be followed by both
parties
• Living Document – can be modified to meet changing political or
departmental requirements
Social Security Policy Branch – Random Sample Surveys
Sample Design
• Define Sample Population (Age Pension, Carer Payment)
• Make Exclusions where Necessary
• Define Strata – Population Groups defined by characteristics (eg Gender,
Rate, Marital Status)
• Consideration for Remote Customers (Clustering)
• Allocate Sample (operational requirements and strata)
Social Security Policy Branch – Random Sample Surveys
Exclusions and Substituons
• In an Ideal sample ALL customers have a chance of selection
• RSS Population is not a closed system
• Need to balance randomness of the sample against burden to the
customers
• Exclusion Rules ensure that:
• customers that have an outstanding or recently completed review;
• customers that are seriously ill or out of the country;
Are not selected
Social Security Policy Branch – Random Sample Surveys
Stratification
• The target population is divided into discrete, non-overlapping sub
populations
• This assists in ensuring a Representative Sample is selected
• Exploits the depth of administrative information held on populations
under review
• Sharpens the final estimates around variables associated with
compliance status
• Allows for unequal probability selection
Social Security Policy Branch – Random Sample Surveys
RSS SAMPLE SPECIFICATION 2008-09 - DISABILITY SUPPORT PENSION
STRATIFICATION
STRATUM ID POP Mar08
<50yrs
22,140
D01
Female 50+yrs
47,143
D02
<50yrs
33,402
D03
Male
Partnered
50+yrs
84,984
D04
<35yrs
35,302
D05
35-49yrs
54,720
D06
Female 50+yrs
90,896
D07
<35yrs
54,656
D08
35-49yrs
74,531
D09
Male
Max rate Unpartnered
50+yrs
93,920
D10
Partnered
52,644
D11
Unpartnered
42,384 Nonremote Reduced rate
D12
Partnered
46,629
D13
Unpartnered
Remote
46,736
D14
TOTAL
BY TRI
MESTER
43
62
53
84
54
67
87
67
79
88
66
59
55
55
SAMPLE SKIP
172
128
252
187
212
158
338
252
218
162
271
202
349
260
271
202
316
236
355
265
266
198
239
178
283
165
283
165
3000
2757
780,088
Note: sample is allocated using Caroll formula (that is proportional to the square root of the stratum population)
The allocation of sample between nonremote and remote areas is proportional to respective customer populations
Social Security Policy Branch – Random Sample Surveys
Clustering - Background
• Centrelink’s most remote Areas (North & Central QLD,
North Australia, Western Australia) have smaller
customer populations over wider geographic areas
• Need to balance the requirement to sample in these
locations with the cost of travel and distance
• Clustering concentrates the selections to a smaller
location within the remote Area
Social Security Policy Branch – Random Sample Surveys
Social Security Policy Branch – Random Sample Surveys
Social Security Policy Branch – Random Sample Surveys
Social Security Policy Branch – Random Sample Surveys
Selection
• Design Delivered to Centrelink
• Random Selection of Customers within Allocation Boundaries by
Centrelink
• FaHCSIA views selection
• Centrelink proceeds after approval
Social Security Policy Branch – Random Sample Surveys
Reviews
• Selected Customer attends an Interview in Centrelink Office (CSC or ASO)
• Full Re-Assessment of Entitlement conducted via Questionnaire (Income,
Assets, Residency, POI, Qualification, Accommodation)
• More Detailed information from Review filled out by Centrelink Officer after
review concluded and entered into RRRS
• Quality Assurance is performed by Centrelink on a sample of the results
Social Security Policy Branch – Random Sample Surveys
Review Results
Assessment of Current and Prior Entitlement can have the following results
• Cancellation of Payment
• Variation (Upward or Downward) of Current Fortnightly Rate
• Debt Raised for an overpayment in an earlier period
• Arrears Paid for an underpayment in an earlier period
These Results are not mutually exclusive. A Cancellation/Variation can
happen in conjunction with a Debt and/or Arrears.
Supplementary Components (Eg RA, PHA, TAL) are also reviewed.
Social Security Policy Branch – Random Sample Surveys
FaHCSIA Processing
• Centrelink Review Results Stored on ISIS/IRS/RRRS
• Data is Transmitted from Centrelink to FaHCSIA
• FaHCSIA tests the data integrity
• Validates a sample (10%) of the Reviews - (Centrelink QAs this sample)
• Tabulates and Analyses the Results
Social Security Policy Branch – Random Sample Surveys
Social Security Policy Branch – Random Sample Surveys
Results
• Results used by FaHCSIA to:
• Provide an estimate FaHCSIA Programme Outlay Accuracy
• Identify Emerging Risks
• Assess effectiveness of Compliance Budget measures
• Evidence Base for Compliance Policy
• Latent Debt Measure – Debt KPI
• Improve predictors for Service Profiling
Social Security Policy Branch – Random Sample Surveys
Outlay Accuracy
• Modelled Estimate of the Mispayed Amount / Outlayed Amount subtracted
from 100%
• Does not discriminate between customer and Centrelink errors
• Weighted by $ Value – a $450 cancellation has a greater effect than a
$0.05 upward variation
• Only counts current episodes of mispayment ie. is not affected by debts for
prior periods of mispayment
• Sourced from Dataset 1
Social Security Policy Branch – Random Sample Surveys
Payment Correctness
• Modelled Estimate of the Number of Customers with an Administrative
Error that affected Payment / Total Population subtracted from 100%
• Only counts customers where the error was made by Centrelink which
caused a dollar impact
• Does include dollar impact from debts (ie prior periods of mispayment)
• Does not count multiple errors – if any 1 of up to 4 Errors meets the criteria
the customer is counted once
• Not weighted by dollar value - Counts customers not amounts – a $35,000
debt contributes the same as a $0.05 variation
• Sourced from Dataset 4
Social Security Policy Branch – Random Sample Surveys
Latent Debt
• Measure of the level of undetected indebtedness in the benefit
population
• Key Prevention Indicator for Debts is the Latent Measure of
Indebtedness to Outlays
• Make the distinction between “Current Year” Indebtedness and New
Debts
Social Security Policy Branch – Random Sample Surveys
Figure 1.1 Current Year Indebtedness to Outlay (Latent measure)
by Benefit, 2003-2007
12 month moving aggregate, 1 Standard Error bar
AGE
DSP
New debts AGE
New debts DSP
CAR
New debts CAR
1.40%
1.00%
0.80%
0.60%
0.40%
0.20%
20
06
-0
7
Social Security Policy Branch – Random Sample Surveys
20
05
-0
6
20
04
-0
5
20
03
-0
4
0.00%
20
02
-0
3
Indebtedness:outlay (%)
1.20%
Addressing emerging risk
top 10 reasons for mispayment
• Random Sample Results are further subdivided by reason for
mispayment
• These indicate the Risks to Outlays to be addressed by Compliance
and Prevention measures
• Mispayments in debt and variation can be for the same or different
reasons
• Risks are ranked by number (incidence) and value (magnitude)
• Charts observe the Higher risks to outlays
Social Security Policy Branch – Random Sample Surveys
Age Pension - Top 10 Risks as at T1 08/09
Assets - business (including Farm)
Investment income from shares
Income from foreign pension
Reason for Variation
12 Months to T1 08/09
Self employed - income from business
precludes payment
12 Months to T3 07/08
Other
Employment income
Income from income stream
Assets - property/real estate
Assets - value of investments
Investment income from banks/building
societies etc
$0
$2,000
$4,000
Social Security Policy Branch – Random Sample Surveys
$6,000
$8,000
FN Variation Amt ($)
$10,000
$12,000
$14,000
Social Security Policy Branch – Random Sample Surveys
Random Sample Surveys - Actual Outlay Accuracy
Benefit Reviewed
Age Pension
No of Reviews
Fortnightly Amt
Variation Amt
Accuracy
No Debts
Debt Amt
Carer
No of Reviews
Fortnightly Amt
Payment
Variation Amt
Accuracy
No Debts
Debt Amt
Carer
No of Reviews
Fortnightly Amt
Allowance
Variation Amt
Accuracy
No Debts
Debt Amt
Disability
No of Reviews
Fortnightly Amt
Support
Variation Amt
Pension
Accuracy
No Debts
Debt Amt
Family Tax
No of Reviews
Fortnightly Amt
Benefit
Variation Amt
Accuracy
No Debts
Debt Amt
Special Benefit No of Reviews
Fortnightly Amt
Variation Amt
Accuracy
No Debts
Debt Amt
Survey Period
Trimester 1
1,666
689,241.64
12,076.42
98.25%
623
309,569.68
499
207,949.68
7,410.93
96.44%
133
82,086.46
84
8,337.32
98.50
98.82%
0
0.00
919
407,118.93
13,461.32
96.69%
190
283,469.87
0
0.00
0.00
#DIV/0!
0
0.00
0
0.00
0.00
#DIV/0!
0
0.00
Trimester 2
1,667
730,792.91
13,117.55
98.21%
626
333,658.37
498
214,720.42
5,444.86
97.46%
134
104,244.20
83
8,406.98
100.60
98.80%
0
0.00
919
420,268.90
10,819.27
97.43%
171
197,850.06
791
112,820.11
2,654.63
97.65%
17
13,484.20
498
178,507.46
6,916.30
96.13%
161
63,009.87
Social Security Policy Branch – Random Sample Surveys
Trimester 3
1,664
693,409.48
10,805.92
98.44%
537
325,313.97
499
205,873.27
5,590.83
97.28%
133
127,261.13
83
8,349.80
201.20
97.59%
1
359.12
920
430,148.63
5,823.34
98.65%
148
87,941.00
716
93,517.24
2,326.26
97.51%
8
2,902.52
0
0.00
0.00
#DIV/0!
0
0.00
Annual Total
4,997
2,113,444.03
35,999.89
98.30%
1,786
968,542.02
1,496
628,543.37
18,446.62
97.07%
400
313,591.79
250
25,094.10
400.30
98.40%
1
359.12
2,758
1,257,536.46
30,103.93
97.61%
509
569,260.93
1,507
206,337.35
4,980.89
97.59%
25
16,386.72
498
178,507.46
6,916.30
96.13%
161
63,009.87
Random Sample Surveys - Restored Outlay Accuracy
Survey Period
Benefit Reviewed
No of Reviews
Fortnightly Amt
Variation Amt
Accuracy
No Debts
Debt Amt
No of Reviews
Carer
Fortnightly Amt
Payment
Variation Amt
Accuracy
No Debts
Debt Amt
No of Reviews
Carer
Fortnightly Amt
Allowance
Variation Amt
Accuracy
No Debts
Debt Amt
No of Reviews
Disability
Fortnightly Amt
Support
Variation Amt
Pension
Accuracy
No Debts
Debt Amt
No of Reviews
Family Tax
Fortnightly Amt
Benefit
Variation Amt
Accuracy
No Debts
Debt Amt
Special Benefit No of Reviews
Fortnightly Amt
Variation Amt
Accuracy
No Debts
Debt Amt
Age Pension
Trimester 1
1,666
689,241.64
12,132.41
98.24%
616
309,131.03
499
207,949.68
7,410.93
96.44%
133
82,347.65
84
8,337.32
98.50
98.82%
0
0.00
919
407,118.93
13,927.48
96.58%
188
283,193.39
0
0.00
0.00
#DIV/0!
0
0.00
0
0.00
0.00
#DIV/0!
0
0.00
Trimester 2
1,667
730,792.91
13,073.02
98.21%
619
321,667.94
498
214,720.42
5,404.19
97.48%
130
98,154.77
83
8,406.98
100.60
98.80%
0
0.00
919
420,268.90
10,838.34
97.42%
169
195,016.42
791
112,820.11
2,486.65
97.80%
17
13,100.00
498
178,507.46
8,747.94
95.10%
152
60,948.71
Social Security Policy Branch – Random Sample Surveys
Trimester 3
1,664
693,409.48
11,186.50
98.39%
533
325,258.36
499
205,873.27
5,545.27
97.31%
131
126,986.99
83
8,349.80
201.20
97.59%
1
359.12
920
430,148.63
5,823.34
98.65%
146
87,890.70
716
93,517.24
2,263.36
97.58%
9
2,987.32
0
0.00
0.00
#DIV/0!
0
0.00
Annual Total
4,997
2,113,444.03
36,391.93
98.28%
1,768
956,057.33
1,496
628,543.37
18,360.39
97.08%
394
307,489.41
250
25,094.10
400.30
98.40%
1
359.12
2,758
1,257,536.46
30,589.16
97.57%
503
566,100.51
1,507
206,337.35
4,750.01
97.70%
26
16,087.32
498
178,507.46
8,747.94
95.10%
152
60,948.71
Random Sample Surveys - Restored Outlay Accuracy
Survey Period
Benefit Reviewed
Actual
Restored
Actual
Carer Payment
Restored
Carer Allowance Actual
Restored
Disability Support Actual
Restored
Pension
Actual
Family Tax
Restored
Benefit
Actual
Special Benefit
Restored
Age Pension
Trimester 1
98.25%
98.24%
96.44%
96.44%
98.82%
98.82%
96.69%
96.58%
#DIV/0!
#DIV/0!
#DIV/0!
#DIV/0!
Social Security Policy Branch – Random Sample Surveys
Trimester 2
98.21%
98.21%
97.46%
97.48%
98.80%
98.80%
97.43%
97.42%
97.65%
97.80%
96.13%
95.10%
Trimester 3
98.44%
98.39%
97.28%
97.31%
97.59%
97.59%
98.65%
98.65%
97.51%
97.58%
#DIV/0!
#DIV/0!
Annual Total
98.30%
98.28%
97.07%
97.08%
98.40%
98.40%
97.61%
97.57%
97.59%
97.70%
96.13%
95.10%
Random Sample Surveys - Replaced Case Result Comparison
Benefit
Reivewed
Actual Result
Restored Result
Downward No Review
Variation Conducted
Age Pension Downward Variation
1
Present Rate to Continue
2
Upward Variation
1
total
4
Carer
Downward Variation
1
Present Rate to Continue
0
Payment
Upward Variation
0
total
1
Carer
Present Rate to Continue
0
0
Allowance total
Disability
Present Rate to Continue
3
Upward Variation
0
Support
total
3
Pension
Family Tax Cancellation
0
Present Rate to Continue
0
Benefit
Upward Variation
0
total
0
Special
Cancellation
0
Downward Variation
1
Benefit
Present Rate to Continue
4
Upward Variation
0
total
5
Social Security Policy Branch – Random Sample Surveys
7
13
3
23
1
6
1
8
1
1
12
1
13
2
11
0
13
0
0
0
0
0
total
Present
Rate to
Continue
3
15
2
20
3
28
0
31
2
2
18
1
19
0
18
1
19
1
2
14
2
19
Upward
Variation
2
1
3
6
0
0
0
0
0
0
1
0
1
0
0
0
0
0
2
2
3
7
13
31
9
53
5
34
1
40
3
3
34
2
36
2
29
1
32
1
5
20
5
31
Benefit
Reivewed
Result Component
Age Pension No of Reviews
Fortnightly Amt
Actual Var Amt
Restored Var Amt
Actual No. Debts
Restored No. Debts
Actual Debt Amt
Restored Debt Amt
Carer
No of Reviews
Fortnightly Amt
Payment
Actual Var Amt
Restored Var Amt
Actual No. Debts
Restored No. Debts
Actual Debt Amt
Restored Debt Amt
Period
1
24
10,320.63
57.12
113.11
7
0
438.65
0
15
6,372.74
0
0
1
1
30.76
291.95
Social Security Policy Branch – Random Sample Surveys
2
15
6,962.89
93.42
48.89
8
1
12,011.77
21.34
13
5,642.59
41.27
0.6
5
1
6,092.87
3.44
Total
3
14
5,984.37
74.7
455.28
4
0
55.61
0
12
4,544.09
45.56
0
2
0
274.14
0
53
23,267.89
225.24
617.28
19
1
12,506.03
21.34
40
16,559.42
86.83
0.6
8
2
6,397.77
295.39
Carer
Allowance
Disability
Support
Pension
No of Reviews
Fortnightly Amt
Actual Var Amt
Restored Var Amt
Actual No. Debts
Restored No. Debts
Actual Debt Amt
Restored Debt Amt
No of Reviews
Fortnightly Amt
Actual Var Amt
Restored Var Amt
Actual No. Debts
Restored No. Debts
Actual Debt Amt
Restored Debt Amt
2
197.00
0
0
0
0
0
0
16
7,589.88
3.34
469.5
2
0
276.48
0
Social Security Policy Branch – Random Sample Surveys
0
0.00
0
0
0
0
0.00
0
10
4,010.15
0
19.07
3
1
2,834.16
0.52
1
100.60
0
0
0
0
0
0
10
4,923.61
0
0
2
0
50.3
0
3
297.60
0
0
0
0
0.00
0
36
16,523.64
3.34
488.57
7
1
3,160.94
0.52
Family Tax No of Reviews
Fortnightly Amt
Benefit
Actual Var Amt
Restored Var Amt
Actual No. Debts
Restored No. Debts
Actual Debt Amt
Restored Debt Amt
No of Reviews
Special
Fortnightly Amt
Benefit
Actual Var Amt
Restored Var Amt
Actual No. Debts
Restored No. Debts
Actual Debt Amt
Restored Debt Amt
0
0.00
0
0
0
0
0
0
0
0.00
0
0
0
0
0
0
Social Security Policy Branch – Random Sample Surveys
19
2,810.23
167.98
0
1
1
485.75
101.55
31
11,740.69
602.43
2434.07
10
1
2,061.21
0.05
13
1,245.20
62.9
0
0
1
0
84.8
0
0.00
0
0
0
0
0
0
32
4,055.43
230.88
0
1
2
485.75
186.35
31
11,740.69
602.43
2434.07
10
1
2,061.21
0.05
Social Security Policy Branch – Random Sample Surveys
Social Security Policy Branch – Random Sample Surveys
Age Pension Random Sample Cases 02/03 - 05/06
Overlap Between RSS Debt and Prior Review History
2002/03 2003/04 2004/05 2005/06
Age Pension
Sample Selected
No. Cust with Debts
No. Debts
No. Cust w Prior Reviews
No. Reviews
Cust with Debts and Reviews
Cust with Debts and No Reviews
Cust with Reviews and No Debts
Cust with no Debt or Review
No. Cust wit Overlap
No. Debts Overlap
Social Security Policy Branch – Random Sample Surveys
1,493
195
239
491
801
87
108
404
894
52
58
1,500
183
232
353
671
74
134
279
1,013
35
42
1,495
319
425
785
1,859
213
106
572
604
90
116
1,489
365
461
765
1,880
229
136
536
588
107
118