URBIS TEAM RESPONSIBLE FOR THIS REPORT:
Directors
Alison Wallace
Dr Linda Kurti
Associate Directors
Tomas Lopata
Duncan Rintoul
Senior Consultant
Zoe Cox
Sub Consultants
Murray Benton, INCA Consultancy
Acting Professor Margaret Kelaher, Centre for Health Policy, Programs and Economics,
Melbourne School of Population Health, University of Melbourne
Expert Advisors
Professor Ross Bailie and Dr Yin Paradies, Menzies School of Health Research
Group Support
Lynda Jones, Alison Rees, Jillian Yeomans
Job Code
KAJ44109
xdisclai mer x
Urbis Social Policy has received ISO 20252 certification, the new international
quality standard for Market and Social Research, for the provision of social
policy research and evaluation, social planning, community consultation,
market and communications research.
URBIS
Australia Asia Middle East
www.urbis.com.au
TABLE OF CONTENTS
Appendix A
ICDP Monitoring and Evaluation Reference Group Members............................... 3
Appendix B
List of Stakeholders Consulted ................................................................................ 4
Appendix C
DoHA Divisions Responsible for Managing ICDP Measures ................................ 5
Appendix D
Matrix Mapping Sources of Data (Package-wide and Measure-specific)
Reported Against ICDP Results as Specified in the Package-wide Program
Logic ........................................................................................................................... 7
Appendix E
Data Sources ............................................................................................................ 11
Appendix F
Data limitations ........................................................................................................ 19
Appendix G
Terminology ............................................................................................................. 26
Appendix H
Key Indicators with Data Sources .......................................................................... 29
ICDP Monitoring and Evaluation Framework Volume 3
APPENDICES
Appendix A
ICDP Monitoring and Evaluation Framework Volume 3
ICDP Monitoring and
Evaluation Reference Group
Members
Page 2
APPENDICES
ICDP Monitoring and Evaluation Reference Group Members
Name
Organisation
Ms Alison Killen
Assistant Secretary, Better Health Care Branch Office for Aboriginal and
Torres Strait Islander Health
Dr Ron Borland
Cancer Council Victoria
Dr Alex Brown
Baker IDI Heart and Diabetes Institute (Melbourne)
Professor Sandra Eades
Baker IDI Heart and Diabetes Institute (Melbourne)
Dr Lauren Cordwell
Royal Australian College of General Practitioners
Dr John Daniels
Redfern Aboriginal Medical Service
Dr Sophie Couzos
National Aboriginal Community Controlled Health Organisation (NACCHO)
Ms Dea Delaney Thiele
NACCHO
Mr Justin Mohamed
NACCHO
Dr Carmel Nelson
Kimberley Aboriginal Medical Services Council
Ms Indrani Peris-Caldwell
Australian Institute for Health and Welfare
Professor Lisa Jackson Pulver
Muru Marri Indigenous Health Unit, University NSW
Ms Leanne Wells
Australian General Practice Network
Professor Kerin O’Dea
Sansom Institute, University SA
Dr David Dumbrell
Department of Health and Ageing
Mr Jeff McKenzie
Department of Health and Ageing
ICDP Monitoring and Evaluation Framework Volume 3
Page 3
APPENDICES
Appendix B
List of Stakeholders
Consulted
Over sixty stakeholders provided feedback on various drafts of the Monitoring and Evaluation
Framework, drawn from the following:

The ICDP Monitoring and Evaluation Framework Reference Group

NACCHO

NACCHO Affiliates

AGPN

Australian Primary Care Collaboratives

State/Territory Departments of Health

DoHA personnel.
ICDP Monitoring and Evaluation Framework Volume 3
Page 4
APPENDICES
Appendix C
ICDP Monitoring and Evaluation Framework Volume 3
DoHA Divisions
Responsible for Managing
ICDP Measures
Page 5
APPENDICES
ICDP Measure Management Responsibility
Measure
DoHA Division Management Responsibility
A1 National Action to Reduce Indigenous Smoking Rates
 Business Group Division
 Mental Health and Chronic Disease Division
A2 Helping Indigenous Australians Reduce their Risk of
Chronic Disease
 Mental Health and Chronic Disease Division
A3 Local Indigenous Community Campaigns to Promote
Better Health
 Business Group Division
B1 Subsidising PBS Medicine Co-payments
 Pharmaceutical Benefits Division
B2 Higher Utilisation Costs for MBS and PBS
 Medical Benefits Division
B3 Supporting Primary Care Providers to Coordinate
Chronic Disease Management
 Primary and Ambulatory Care Division
B4 Improving Indigenous Participation in Health Care
through Chronic Disease Self Management
 Mental Health and Chronic Disease Division
B5 Increasing Access to Specialist and Multidisciplinary
Team Care
 Primary and Ambulatory Care Division
B6 Monitoring and Evaluation
 OATSIH
C1 Workforce Support, Education and Training
 Health Workforce Division
C2 Expanding the Outreach and Service Capacity of
Indigenous Health Organisations
 OATSIH
C3 Engaging Divisions of General Practice to Improve
Indigenous Access to Mainstream Primary Care
 Primary and Ambulatory Care Division
C4 Attracting More People to Work in Indigenous Health
 Business Group Division
C5 Clinical Practice and Decision Support Guidelines
 OATSIH
ICDP Monitoring and Evaluation Framework Volume 3
Page 6
APPENDICES
Appendix D
ICDP Monitoring and Evaluation Framework Volume 3
Matrix Mapping Sources of
Data (Package-wide and
Measure-specific) Reported
Against ICDP Results as
Specified in the Packagewide Program Logic
Page 7
APPENDICES
Appendix E
Matrix Mapping Sources of Data (Package-wide and
Measure-specific) Reported Against ICDP Results
as specified in the Package-wide Program Logic
MEDIUM TERM
RESULTS
Results
Chronic disease
risk factors
Chronic disease
manage-ment and
follow up care
ICDP
A1
A2
A3
There is a reduction in the incidence of preventable chronic disease risk factors
among Indigenous Australians.
X
X
X
X
Smoking rates among Indigenous Australians are reduced.
X
X
X
X
More Indigenous Australians with or at risk of chronic disease adopt healthy
lifestyle choices relating to smoking, nutrition and exercise.
X
X
X
X
ICDP-funded health care services deliver a comprehensive and coordinated
approach to chronic disease management, including increased and earlier
access to primary health care, specialist and allied health services, affordable
care and medicines.
X
More Indigenous Australians with or at risk of chronic disease actively
participate in their own health care.
X
Health outcomes are improved among Indigenous Australians with or at risk of
chronic disease who participate in ICDP measures.
X
ICDP Monitoring and Evaluation Framework Volume 3
X
X
X
X
Page 8
B1
B3
A
B3
B
B4
B5
A
B5
B
X
X
X
X
X
X
X
X
X
X
X
X
C1
C2
C3
C4
C5
APPENDICES
MEDIUM TERM
RESULTS
Results
Workforce
expansion and
support
A3
B3
A
B3
B
B4
B5
A
B5
B
C1
C2
C3
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
B1
B3
A
B3
B
B4
B5
A
B5
B
C1
C2
C3
C4
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
ICDP
A1
A2
B1
More health care providers are equipped to assist Indigenous Australians with
or at risk of chronic disease to make healthy lifestyle choices and to manage
their condition.
X
X
X
More health care providers are accessed by and provide quality care to
Indigenous Australians with or at risk of chronic disease.
X
X
X
There is an increase in the workforce providing primary health care and other
health services to Indigenous Australians.
X
X
X
ICDP
A1
A2
Resources for designing and delivering health promotion campaigns are
accessible, effective and evidence-based
X
X
Indigenous Australians who have had contact with the ICDP have a better
knowledge and understanding of the impact of preventable chronic disease risk
factors on their wellbeing.
X
X
X
X
Indigenous Australians who have had contact with the ICDP are more aware of
and utilise the expanded range of health services and supports available to
them to adopt healthy lifestyle choices and reduce smoking.
X
X
X
X
Indigenous Australians who have had contact with the ICDP make positive
decisions about their health and lifestyle.
X
X
X
X
ICDP-funded health system supports, incentives and subsidies are operating to
facilitate the provision of quality primary health care for Indigenous Australians
with chronic disease.
X
X
Financial and other barriers to accessing health care and medicines are
reduced.
X
X
Health services funded under the ICDP demonstrate cultural awareness and
commitment.
X
Care coordination within ICDP-funded services is improved for Indigenous
Australians with or at risk of chronic disease.
X
C4
C5
X
EARLY RESULTS
Results
Chronic disease
risk factors
Chronic disease
manage- ment and
follow-up care
Indigenous Australians with chronic disease or associated risk factors have
more services and supports available to help them manage their condition.
The number of Indigenous Australians with or at risk of chronic disease who
ICDP Monitoring and Evaluation Framework Volume 3
X
X
X
A3
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
Page 9
C5
APPENDICES
EARLY RESULTS
Results
A2
A3
B1
B3
A
X
X
B3
B
ICDP
A1
B4
Indigenous Australians in contact with ICDP measures value the enhanced
services.
X
X
The number of Indigenous Australians with or at risk of chronic disease who
access specialist and multi-disciplinary follow-up care is increased.
X
Health care providers demonstrate increased knowledge and improved
practice in relation to the prevention, early identification and management of
chronic disease for Indigenous Australians.
X
X
X
X
X
The ICDP workforce is retained and developed within funded services.
X
X
X
X
X
Marketing, training and recruitment strategies are successful in encouraging
more people to work in primary health care and other services available to
Indigenous Australians.
X
X
X
ICDP
A1
A2
A3
B1
B3
A
B3
B
The workforce required to implements the ICDP is recruited, oriented and
trained.
X
X
X
X
X
Package measures are efficiently implemented in accordance with agreed
guidelines and timelines
X
X
X
X
Monitoring and reporting requirements are met
X
X
X
Internal and external stakeholders, sentinel sites and service providers
consider the implementation of the Package to be addressing identified needs,
and enhancing the existing service system
X
X
X
B5
A
B5
B
X
X
X
X
C1
C2
C3
X
X
X
X
X
C4
C5
access primary health care services is increased.
Workforce
expansion and
support
X
X
X
X
X
X
X
X
X
X
X
X
X
B4
B5
A
B5
B
C1
C2
C3
C4
C5
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
OUTPUTS (YEAR
1 AND ONGOING)
Results
Workforce
expansion and
support
ICDP Monitoring and Evaluation Framework Volume 3
Page 10
APPENDICES
Appendix F
ICDP Monitoring and Evaluation Framework Volume 3
Data Sources
Page 11
APPENDICES
Defining data sources terminology in the Monitoring and Evaluation Framework
Terminology
Descriptor
Aboriginal and Torres Strait Islander
Health Performance Framework (HPF)
Published biennially by the AIHW incorporating a wide range of
chronic disease performance indicators collected through various
national and other databases/surveys (more information on HPF data
sources and technical notes are included in this Appendix)
AGPN
As a key stakeholder in a number of ICDP measures, the AGPN will
participate in a number of monitoring and evaluation activities
APCC
Program data from over 500 participating medical practices including
clinical and service enhancement measurements
BEACH
Collects information about clinical activities in general practice in
Australia. The BEACH database includes over 1,200,000 GP-patient
encounter records. Annually BEACH analyses data from 100
consecutive consultations from each of the randomly selected
sample of 1,000 GPs
CDSM register
Number of patients enrolled at the practice level in the Chronic
Disease Management Program
Clinical data systems and/or quality
improvement systems
Systematic collection and reporting of data relating to clinical
performance and clinical outcomes
Community members consultation
Face to face interviews or focus groups with members of local
Indigenous communities to assess knowledge, awareness and
impact of ICDP measures on local individuals, families, and
communities
Community-level survey
Quantitative survey tool used in A1 evaluation to assess impact of
health promotion and social marketing campaign activity, to be
determined when measure evaluations are developed
Consultation with …
Face to face or telephone interviews, focus groups, email or other
direct interaction with stakeholders (key stakeholders are identified in
each measure)
GPET
Data regarding postgraduate vocational training activity including the
number of services requesting a general practice registrar length of
time before these vacancies are filled and the number of placements
which are completed
Health workforce survey
One or more surveys of the ICDP workforce to be developed by the
evaluator to assess impact of ICDP measures on health care
professionals
Healthy for Life (H4L)
Program data set within OATSIH, including data from participating
Indigenous and other health services within a web-based reporting
system
MBS
Data held by Medicare Australia regarding uptake of clinical services
provided under the Medical Benefits Schedule
MSOAP
Program data held by DoHA including the number of patients per
location and in total, number of visits and the types of participating
organisations
National Aboriginal and Torres Strait
Islander Health Survey (NATSIHS)
A survey of a representative sample of Aboriginal and Torres Strait
Islander Australians, conducted at six-yearly intervals, which covers
a range of health and well-being issues
ICDP Monitoring and Evaluation Framework Volume 3
Page 12
APPENDICES
Terminology
Descriptor
NT AHKPI
Northern Territory has developed a set of core indicators based on a
range of clinical and social measurements in participating Indigenous
community health services
OATSIH Services Reporting (OSR)
OATSIH service reporting tool for annual reporting by ACCHSs
Organisational survey
One or more surveys to be developed by the evaluator for
Indigenous health services and general practices funded under the
ICDP to assess the impact of ICDP measures at a service level
PBS
Data held by DoHA regarding utilisation of pharmaceuticals
subsidised under the Pharmaceutical Benefits Scheme
PIP/PIPIHI
Data held by DoHA regarding the Practice Incentives Program,
including registrations and uptake of the ICDP PIP Indigenous Health
Incentive and process measures
Program documentation
The range of qualitative and quantitative information held by program
managers within DoHA including information regarding design and
implementation of measures, process data, and some outcome
measures
QAIHC
QAIHC has developed a set of core indicators based on a range of
clinical and social measurements in participating Indigenous
community health services
USOAP
Program reporting data held by DoHA including the number and type
of Program occasions of care, extent of Program participation by
specialists and allied health professionals, and reported success
factors in the implementation of the Program
ICDP Monitoring and Evaluation Framework Volume 3
Page 13
APPENDICES
Sources of data utilised in the ICDP Monitoring and Evaluation Frameworks (Package-wide and individual measures)
Data type/category
Data source
National Performance
Indicators as Reported in
the HPF Publication
Administrative/system
data sets
ICDP
A1
x
x
A2
A3
B1
x
B3a
B3b
x
x
x
PIP
x
x
x
PIP IHI
x
x
x
MBS
x
PBS
x
APCC
x
x
NT AH KPI
x
x
Healthy for Life
x
x
QAIHC
x
x
BEACH
x
OSR
x
B4
B5a
B5b
x
MSOAP
x
NATSIHS
x
C4
C5
x
x
x
x
x
x
x
ICDP Program
documentation
x
x
ICDP Program-specific
evaluations
x
x
ICDP health
workforce
x
x
Organisations
funded by the
x
x
ICDP Monitoring and Evaluation Framework Volume 3
C3
x
x
USOAP
C2
x
GPET
Surveys for ICDP
evaluation
C1
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
Page 14
APPENDICES
Data type/category
Data source
ICDP
A1
A2
A3
B1
B3a
B3b
B4
B5a
B5b
C1
C2
C3
C4
C5
ICDP
Website users
(C5)
x
Longitudinal
survey of health
promotion
program
participants
In-depth stakeholder
consultations conducted
for the ICDP evaluation
(one-on-one interviews
and focus groups;
stakeholders described in
each framework)
ICDP
stakeholders at
national,
State/Territory,
regional and
local level
x
x
Surveys and statistics
produced by associated
providers (e.g. RTOs,
Quitline)
x
Clinical data systems
and/or quality
improvement systems
Other
x
x
Communications
audit
ICDP Monitoring and Evaluation Framework Volume 3
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
Page 15
APPENDICES
Aboriginal and Torres Strait Islander Health Performance
Framework data sources
Following the advice of Department of Health and Ageing, Urbis understands that the Australian
Institute of Health and Welfare’s Aboriginal and Torres Strait Islander Health Performance Framework
(HPF) is the agreed standard for monitoring changes in the health of Indigenous Australians. The HPF
consists of 70 measures across three tiers - health status and outcomes, determinants of health and
health systems performance.
Of the HPF’s 70 performance indicators. Urbis has identified 24 which are of particular relevance to the
monitoring and evaluation of the ICDP. See Appendix G for Technical Notes on these data sources.
List of data sources utilised in HPF indicators as referenced in ICDP Monitoring and
Evaluation Framework
Number
Name
Data Sources
HPF #1.02
Top reasons for hospitalisation
 AIHW National Hospital Morbidity Database
HPF #1.05
Circulatory disease
 National Aboriginal and Torres Strait Islander
Health Survey (NATSIHS)
 Bettering the Evaluation and Care of Health
Survey (BEACH)
 AIHW National Hospital Morbidity database
HPF #1.07
High blood pressure
 National Aboriginal and Torres Strait Islander
Health Survey (NATSIHS)
 Bettering the Evaluation and Care of Health
Survey (BEACH)
 AIHW National Hospital Morbidity database
HPF #1.08
Diabetes
 National Aboriginal and Torres Strait Islander
Health Survey (NATSIHS)
 Bettering the Evaluation and Care of Health
Survey (BEACH)
 AIHW National Hospital Morbidity database
HPF #1.09
End stage renal disease
 Australia and New Zealand Dialysis and
Transplant Registry (ANZDATA)
 AIHW National Hospital Morbidity Database
 AIHW National Mortality Database
HPF #1.15
Perceived health status
 National Aboriginal and Torres Strait Islander
Health Survey (NATSIHS)
 National Aboriginal and Torres Strait Islander
Social Survey (NATSISS)
HPF #1.16
Social and emotional wellbeing
 National Aboriginal and Torres Strait Islander
Health Survey (NATSIHS)
 National Aboriginal and Torres Strait Islander
Social Survey (NATSISS)
 Western Australian Aboriginal Child Health
Survey
 Bettering the Evaluation and Care of Health
Survey (BEACH)
ICDP Monitoring and Evaluation Framework Volume 3
APPENDICES
Number
Name
Data Sources
 AIHW National Hospital Morbidity database
 AIHW National Mortality Database
 AIHW National Community Mental Health Care
Database
 AIHW National Residential Mental Health Care
Database
HPF #1.17
Life expectancy at birth
 2006 Census Population and Housing
 AIHW National Mortality Database
HPF #1.22
All causes age-standardised death
 AIHW National Mortality Database
HPF #1.23
Leading causes of mortality
 AIHW National Mortality Database
HPF #2.03
Environmental tobacco smoke
 National Aboriginal and Torres Strait Islander
Health Survey (NATSIHS)
HPF #2.18
Tobacco use
 2004-05 National Aboriginal and Torres Strait
Islander Health Survey (NATSIHS)
 2004 National Drug Strategy Household Survey
HPF #2.19
Tobacco smoking during pregnancy
 AIHW National Perinatal Data Collection
HPF #2.22
Level of physical activity
 National Aboriginal and Torres Strait Islander
Health Survey (NATSIHS)
HPF #2.23
Dietary behaviour
 National Aboriginal and Torres Strait Islander
Health Survey (NATSIHS)
HPF #2.26
Prevalence of overweight and obesity
 2004-05 National Aboriginal and Torres Strait
Islander Health Survey (NATSIHS)
HPF #3.03
Early detection and early treatment




HPF #3.04
Chronic disease management
 Service Activity Reporting (SAR)
 Healthy for Life data collections
 Divisions of General Practice National
Performance Indicators
HPF# 3.05
Differential access to key hospital
procedures
 AIHW National Hospital Morbidity Database
HPF #3.06
Ambulatory care sensitive hospital
admissions
 AIHW National Hospital Morbidity Database
HPF #3.10
Aboriginal and Torres Strait Islander people
in the health workforce
 2006 ABS Census of Population and Housing
HPF #3.12
Access to services by types of service
compared with need
 National Aboriginal and Torres Strait Islander
Health Survey (NATSIHS)
 National Aboriginal and Torres Strait Islander
Social Survey (NATSISS)
 Community Housing and Infrastructure Needs
ICDP Monitoring and Evaluation Framework Volume 3
Medicare Database
AIHW BreastScreen Australia Database
National Bowel Cancer Screening Register
2004-05 National Aboriginal and Torres Strait
Islander Health Survey (NATSIHS)
 AIHW National Mortality Database
 Service Activity Reporting (SAR)
APPENDICES
Number
Name
Data Sources









Survey (CHINS)
2006 ABS Census of Population and Housing
AIHW National Hospital Morbidity Database
General Practitioner data
Medicare Database
AIHW Labour Force surveys
Service Activity Report (SAR)
Drug and Alcohol Service Reporting (DASR)
Health expenditure data
Palliative care data
HPF #3.18
Aboriginal and Torres Strait Islander
peoples training for health-related
disciplines
 Australian Government Department of
Education, Science and Training’s Higher
Education Student Statistics Collection
 National Centre for Vocational education
Research Collection
HPF #3.20
Recruitment and retention of clinical
management staff (including GPs)
 Service Activity Reporting (SAR)
 Rural Workforce Agency National Minimum
Data
 GP Data
 AIHW Labour Force surveys
ICDP Monitoring and Evaluation Framework Volume 3
APPENDICES
Appendix G
ICDP Monitoring and Evaluation Framework Volume 3
Data limitations
APPENDICES
Data limitations and technical notes
Aboriginal and Torres Strait Islander Health Performance Framework
(HPF)
The following technical notes on HPF data sources and methods were prepared by the Department of
Health and Ageing.
Aboriginal and Torres Strait Islander peoples and non-Indigenous
‘Aboriginal and Torres Strait Islander peoples’ is the preferred term used throughout the HPF report.
‘People’ is an acceptable alternative to ‘peoples’ depending on context, but in general the term ‘peoples’
is used. The term Indigenous Australian is inclusive of all groups and is used where it is impractical to
use the preferred term ‘Aboriginal and Torres Strait Islander peoples’ (e.g. tables, charts, figures and
where the text requires the term to be repeated a number of times).
The term ‘non-Indigenous’ is used where the data collection allows not stated responses on the
Indigenous status question to be separately identified to non-Indigenous. Where the not stated
responses are included with non-Indigenous, the term ‘other Australians’ is used to refer to the
combined non-Indigenous and not stated data.
Age-standardisation
Age-standardisation controls for the effect of age, to allow comparisons of summary rates between two
populations that have different age structures. Age-standardisation is used throughout this report when
comparing Aboriginal and Torres Strait Islander Australians with non-Indigenous Australians for a range
of variables where age is a factor in relation to the variable being examined. The main disadvantages
with age-standardisation are that the resulting rates are estimates that are not the real rates for the
population, the age-standardised rates are not meaningful themselves, only as a means for comparison,
and they may hide important differences in distribution such as rates for particular ages. Lastly for agestandardisation to be effective the relationship between age and the variable needs to be approximately
consistent between the two populations.
Main Sources
The data the HPF are mainly drawn from national data collections and surveys. These include the
following:
National Aboriginal and Torres Strait Islander Health Survey (NATSIHS): The 2004-05 NATSIHS
included a sample of 10,400 Aboriginal and Torres Strait Islander Australians. This was considerably
larger than the supplementary Indigenous samples in the 1995 and 2001 National Health Surveys
(NHS) (3,681 in 2001). The NATSIHS sample was specifically designed to select a representative
sample of Aboriginal and Torres Strait Islander Australians and thus overcome the problem inherent in
most national surveys with small and unrepresentative Indigenous samples. The NATSIHS also uses
the standard Indigenous status question. Information recorded in this survey is essentially ‘as reported’
by respondents. Responses may be affected by imperfect recall or individual interpretation of survey
questions. Any data that are self-reported are likely to underestimate circumstances of which the
respondent is unaware e.g. certain health conditions, or where they may feel reluctant to tell the
interviewer e.g. weight, drug use. Selected non-Indigenous comparisons are available through the
National Health Survey (NHS). The NHS was conducted in major cities, regional and remote areas, but
very remote areas were excluded from the sample. In remote communities, there were some
modifications to the NATSIHS content in order to address language and cultural appropriateness in
traditional communities, as well as to assist respondents in understanding the concepts. Some
questions were excluded and some reworded. Also paper forms were used in remote areas and
ICDP Monitoring and Evaluation Framework Volume 3
APPENDICES
computer assisted interview (CAI) instruments were used in non-remote areas. Further information on
NATSIHS data quality issues can be found in the national publication ABS Cat. No. 4715.0 ABS 2006b.
The National Aboriginal and Torres Strait Islander Social Survey (NATSISS): The 2008 NATSISS
was conducted between August 2008 and April 2009. Information was collected by personal interview
from approximately 13,300 Aboriginal and Torres Strait Islander Australians aged 15 years and over
throughout Australia, including those living in remote areas. The NATSISS sample was specifically
designed to select a representative sample of Aboriginal and Torres Strait Islander Australians and thus
overcome the problems inherent in most national surveys with small and unrepresentative Indigenous
samples. The NATSISS uses the standard Indigenous status question. Information recorded in this
survey is essentially ‘as reported’ by respondents, and hence may differ from that which might be
obtained from other sources or by using other collection methodologies. Responses may be affected by
imperfect recall or individual interpretation of survey questions. Selected non-Indigenous comparisons
are available through the General Social Survey and a range of other surveys. Further details can be
obtained from ABS 2009. Time-series comparisons are available through the 1994 National Aboriginal
and Torres Strait Islander Survey.
Census
The Census uses the standard Indigenous status question and it is asked for each household member.
Measures that are drawn from Census data are subject to some data concerns relating to the
unexplained growth in the Aboriginal and Torres Strait Islander population since the 1991 Census, and
the limitations of Indigenous self-identification.
There are 4 principal sources of error in Census data: respondent error, processing error, partial response
and undercount. Quality management of the Census program aims to reduce error as much as possible,
and to provide a measure of the remaining error to data users, to allow them to use the data in an
informed way.
In 2006, the ABS enhanced the sample for the Post-Enumeration Survey to include remote areas. This
resulted in a doubling of the measured undercount for Indigenous Australians (from 5.7% to 11.5%).
The Census form may be completed by one household member on behalf of others. Incorrect answers
can be introduced to the Census form if the respondent does not understand the question or does not
know the correct information about other household members. Many of these errors remain in the final
data.
The processing of information from Census forms is now mostly automated. Quality assurance
procedures are used during Census processing to ensure processing errors are kept at an acceptable
level. Sample checking is undertaken during coding operations, and corrections are made where
necessary.
When completing their Census form, some people do not answer all the questions which apply to them. In
these instances, a ‘not stated’ code is allocated during processing, with the exception of non-response to
age, sex, marital status and place of usual residence. These variables are needed for population
estimates, so they are imputed using other information on the Census form, as well as information from
the previous Census.
Other Census data issues relate to the accuracy of the Census count itself, e.g. whether people are
counted more than once, or are undercounted.
Problems with the accuracy of Indigenous population estimates impacts on the accuracy of life
expectancy estimates and rate calculations eg mortality rates.
The 2006 Census based estimates were compared to previously published estimates using the 2001
Census based estimates. The analysis produced contradictory results, particularly with respect to “the
gap” between Indigenous and non-Indigenous mortality over time. A number of investigations were
undertaken to seek to understand the results and recommend options for future reporting of trends.
ICDP Monitoring and Evaluation Framework Volume 3
APPENDICES
Trend analysis using Indigenous population figures begin from 2001 and are based on 2006 Census
estimates (back cast and projections) Series B. This solution recognises the difficulty of undertaking long
trend analysis when there have been major shifts in Indigenous identification over time. Indigenous
identification has changed considerably with each Census but this is less so between the most recent
Censuses (i.e. between 2001 and 2006). A similar approach will again be required following the release of
estimates based on the 2011 Census.
In 2011 the ABS will implement improvements to Census Indigenous enumeration procedures and will
expand the scope of the Census Post Enumeration Survey to include very remote areas and discrete
Indigenous communities.
Community Housing and Infrastructure Needs Survey (CHINS): The CHINS collects data on discrete
Indigenous communities, including approximately 92,960 Aboriginal and Torres Strait Islanders in 2006 or
18% of the total estimated Indigenous population. The survey was also conducted in 2001 and 1999. The
data are collected from key personnel in Indigenous communities and housing organisations
knowledgeable about housing and infrastructure issues. This collection is not a population survey.
National Perinatal Data Collection: Birth notification forms are completed for all births of 20 weeks or
more gestation, or a birthweight of 400 grams or more. The Perinatal National Minimum Data Set includes
all births in Australia in hospitals, birth centres and the community. The state/territory of birth is provided
for all births in each state/territory. State-level data are based on place where birth occurred, not place of
usual residence. Complicated pregnancies from surrounding New South Wales may be referred to the
Australian Capital Territory and hence there may be poorer outcomes attributed to Australian Capital
Territory births. Because of this and the small numbers involved, care should be taken in interpreting data
from the Australian Capital Territory (Laws & Sullivan 2004a). A standard data item for Indigenous status
is specified in the Perinatal National Minimum Data Set. However, at this stage not all states and
territories use this standard wording for the Indigenous status question on their forms. This impacts on the
quality and comparability of the data collected (ABS & AIHW 2005). All jurisdictions collect Indigenous
status of the mother for each baby. However, this statistic does not necessarily indicate the Indigenous
status of the baby. Work is underway to include the Indigenous status of the baby in the Perinatal National
Minimum Data Set. Studies linking perinatal data with birth registration data and hospital admissions
show that Indigenous data are under-reported (Human Resources Development Canada & Health
Canada 2003). In 2007, the AIHW completed an assessment of the quality of Indigenous status
information in perinatal data in each state and territory. This involved a survey which was sent to the
midwifery managers across Australia to determine how many hospitals in each jurisdiction obtain
Indigenous status information of mothers giving birth from admission records and how many collect this
information independently. The assessment also involved analysis of the variability in the number and
proportion of mothers recorded as Indigenous in the perinatal data collection over time and across
jurisdictions for the period 1991–2004. The outcomes of this assessment showed that Indigenous status
data from New South Wales, Victoria, Queensland, Western Australia, South Australia and the Northern
Territory are suitable for trends analysis and national reporting. Perinatal data from Tasmania, although
improving, were deemed to be of insufficient quality. Although the most recent data in the Australian
Capital Territory were of publishable quality, the data were not yet of sufficient stability to support trends
analysis (AIHW: Leeds KL et al. 2007). All jurisdictions are working towards improving the quality of the
Indigenous status data. States and territories have agreed to improve Indigenous data collection
procedures in key data collections including implementation of the Best Practice Guidelines for the
collection of Indigenous status in health data collections. Data on Indigenous status are not reported for
Tasmania prior to 2005 as the not stated category for Indigenous status was included with the nonIndigenous category. The not stated category for Indigenous status is high for Tasmania (around 75%)
and low for the other jurisdictions, not more than 0.5% (Laws & Sullivan 2004b). The ‘not stated’ category
for birthweight was found to be small nationally in the evaluation of the Perinatal National Minimum Data
Set. Therefore, the exclusion of not stated for birthweight will not have a significant impact on these data.
AIHW will work with the states and territories in the development of an enhanced Perinatal National
Minimum Data Set to be operational by 2013.
National Hospital Morbidity Data: This is a data collection including all completed admitted patient
episodes in public and private hospitals across Australia. The number and pattern of hospitalisations
can be affected by differing admission practices among the jurisdictions and from year to year, and
ICDP Monitoring and Evaluation Framework Volume 3
APPENDICES
differing levels and patterns of service delivery. Some jurisdictions have slightly different approaches to
the collection and storage of the standard Indigenous status question and categories in their hospital
collections. The not stated category is missing from several collections. The incompleteness of
Indigenous identification means the number of hospital separations recorded as Indigenous is an
underestimate of hospitalisations involving Aboriginal and Torres Strait Islander people. For several
years, Queensland, South Australia, Western Australia and the Northern Territory reported that
Indigenous status in their hospital separations data was of acceptable quality (AIHW 2007). However,
the AIHW has completed an assessment of the level of Indigenous under-identification in hospital data
in all states and territories. Results from this assessment indicate that New South Wales, Victoria,
Queensland, Western Australia, South Australia and the Northern Territory have adequate Indigenous
identification (20% or less overall under-identification of Indigenous patients) in their hospital
separations data. It has therefore been recommended that reporting of Indigenous hospital separations
data be limited to aggregated information from New South Wales, Victoria, Queensland, Western
Australia, South Australia and the Northern Territory. Tasmania and ACT data are presented at the
state/territory level and should be used with caution, but they are not aggregated with the other 6
jurisdictions. AIHW will commence another audit process with audits in all states and territories and
invite full participation of jurisdictions in the process to assess improvements in the data quality. The
proportion of the Indigenous population covered by the 6 jurisdictions is 96% (ABS & AIHW 2005).
Interpretation of results should take into account the relative quality of the data from the jurisdictions.
Hospitalisation data for these 6 jurisdictions should not be assumed to represent the hospitalisation
experience in the Australian Capital Territory and Tasmania.
From the AIHW study it was possible to produce correction factors for the level of Indigenous underidentification in hospital data at the national level.
National Mortality Database: The mortality rate for Indigenous Australians can be influenced by
identification of Indigenous deaths, late registration of deaths, as a result of coronial inquiry, and
changes to death forms and/or processing systems. Due to the small size of the Indigenous population
these factors can significantly impact on trends over time and between jurisdictions.
All jurisdictions comply with the standard wording for the Indigenous status question and categories for
their death registration forms. However, Victoria, South Australia, the Northern Territory and the
Australian Capital Territory all have slightly different wording to the national standard for the instruction
on those with both Aboriginal and Torres Strait Islander origin (ABS & AIHW 2005). While the wording is
only slightly different, states and territories have agreed to the implementation of the Best Practice
Guidelines for the collection of Indigenous status in health data collections.
Almost all deaths in Australia are registered. However, the Indigenous status of the deceased is not
always recorded/recorded correctly. The incompleteness of Indigenous identification means the number
of deaths registered as Indigenous is an underestimate of deaths occurring in the Aboriginal and Torres
Strait Islander population (ABS 1997). As a result, the observed differences between Indigenous and
non-Indigenous mortality are underestimates of the true differences.
While the identification of Indigenous deaths is incomplete in all state and territory registration systems,
5 jurisdictions (New South Wales. Queensland, Western Australia, South Australia and the Northern
Territory) have been assessed by the ABS and the AIHW as having adequate identification. Longer term
mortality trend data are limited to 3 jurisdictions (Western Australia, South Australia and the Northern
Territory) with 10 years of adequate identification of Indigenous deaths in their recording systems. The
quality of the time-series data is also influenced by the late inclusion of a not stated category for
Indigenous status in 1998. Prior to this time, the not stated responses were probably included with the
non-Indigenous. The ABS calculated the identification rate of Indigenous deaths for the period 2006–
2007 through data linkage between the Census and death registration records to examine differences in
reporting of Indigenous status across the two datasets (ABS 2009b).
Table 1: Indigenous deaths identification rate, state/territory and Australia, 2006-2007
State/territory
Identification Rate
NSW
0.87
ICDP Monitoring and Evaluation Framework Volume 3
APPENDICES
State/territory
Identification Rate
Qld
0.94
WA
1.11
NT
1.09
Vic, SA, Tas, ACT, Overseas territories
0.65
Australia
0.92
Source: ABS 2009b
As part of the data development work funded under the National Indigenous Reform Agreement (NIRA)
the ABS will link Census records with death registration records to assess the level of identification
again in 2011.
BEACH: The Bettering the Evaluation And Care of Health (BEACH) survey collects information about
consultations with GPs, including GP and patient characteristics, patient reasons for the visit, problems
managed and treatments provided. The survey has been conducted annually since April 1998.
Information is collected from a random sample of approximately 1,000 GPs from across Australia each
year. Each GP provides details of 100 consecutive consultations.
Although the questionnaire contains an Indigenous identifier, it is unknown whether all GPs ask their
patients this question. In a sub-study Supplementary Analysis of Nominated Data of approximately 9,000
patients, it was found that if the question on Indigenous status was asked within the context of a series of
questions about origin and cultural background, 2.2% identified as Indigenous. This is twice the rate
routinely recorded in BEACH, indicating that BEACH may under-estimate the number of Indigenous
consultations.
Under the NIRA, state and territory governments have agreed to the implementation of the Best Practice
Guidelines for the collection of Indigenous status in health data collection, which include recommended
strategies for local service providers such as GPs and practice nurses to improve Indigenous identification
data.
Life expectancy at birth calculation
The gap in life expectancy between Indigenous and non-Indigenous Australians is currently estimated
to be between 9.7 years (females) and 11.5 years (males). Life expectancy estimates are based on
population and deaths data. Unfortunately there are significant problems with the quality of Indigenous
data in these data sets. This has led to a great deal of variability and uncertainty in the estimates of life
expectancy for Indigenous Australians. A number of 'indirect' methods for estimating Indigenous life
expectancy have been used previously. This method estimates Indigenous deaths through looking at
changes in the population and how many deaths would be expected between the Census periods.
However, the 'indirect' method was found to be highly sensitive to small variations in the population
estimates. Estimates of the gap between Indigenous and non-Indigenous Australians for the period
1996-2001 varied depending on the indirect method used eg 20 years, 17 years and 13 years. The ABS
recently decided to use 'direct' methods to calculate 2005-07 life expectancy for Indigenous Australians.
Indigenous deaths are estimated through data linkage of Census and deaths data. While the standard
direct method appears the most robust as it is less affected by variations in the underlying data, it will
still be affected by the assumptions made in estimating the Indigenous population and deaths
MBS/PBS Voluntary Indigenous Indicator
In November 2001, a Voluntary Indigenous Identifier (VII) question was included on all Medicare
enrolment forms. Until recently, the VII data was only used to analyse broad service trends and there
were questions about its reliability for more detailed reporting. However, recent assessments have
indicated VII data quality has improved substantially over recent years, and that the latest available VII
data are capable of providing statistically reliable estimates of Indigenous health service useage at the
ICDP Monitoring and Evaluation Framework Volume 3
APPENDICES
national and state/territory level (Presentation by Dr Darren Benham, OATSIH 2008). The AIHW used
information from Medicare VII data for estimating expenditure on Medicare and PBS for the report,
Expenditures for Health on Aboriginal and Torres Strait Islander People 2006-07.
The Australian Primary Care Collaboratives (APCC)
The data managers of APCC have indicated their willingness to discuss with DoHA how it might be
involved in the evaluation of the ICDP. The APCC data system contains many clinical outcome data
that would be useful for the evaluation. However, there are some restrictions on the type of data that
can be accessed. Moreover, the APCC operates with a selective group of general practices which are
unlikely to be representative of Australian general practices. There is a lack of rigorous published data
available from the APCC and it is acknowledged that the Aboriginal and Torres Strait Islander patient
status identifier is currently poorly recorded at a practice level. Strategies for improving the Indigenous
patient identifier would be an important precursor to extracting data on Aboriginal and Torres Strait
Islander patients for the purposes of the ICDP evaluation – with timing of this critical in determining
whether any changes in recorded service provision is a result of improved identification or improved
service delivery.
ICDP program documentation
As indicated previously (see Vol.1, section 3), the nature of data to be collected by ICDP program
management is still being determined in some areas. It will take a year or so to assess the quality and
comparability of data collected via ICDP program reporting. An assessment will need to be made at a
later date to assess the quality, utility and accessibility of the data for evaluation purposes in particular
of the ICDP evaluator.
References
Australian Bureau of Statistics 1997, Occasional paper: Mortality of Aboriginal and Torres Strait Islander
Australians. ABS cat. no. 3315.0, ABS, Canberra.
Australian Bureau of Statistics 2007b, Deaths Australia 2006. ABS cat. no. 3302.0., ABS cat. no.
3302.0, Canberra.
Australian Bureau of Statistics 2009, National Aboriginal and Torres Strait Islander Social Survey 2008,
ABS cat. no. 4714.0, Canberra
Australian Bureau of Statistics 2009b, Experimental Life Tables for Aboriginal and Torres Strait Islander
Australians 2005-2007, Australia, ABS cat. no. 3302.0.55.003, Canberra
Australian Bureau of Statistics & Australian Institute of Health and Welfare 2005, The health and welfare
of Australia’s Aboriginal and Torres Strait Islander peoples 2005, ABS cat. no. 4704.0; AIHW cat. no.
IHW 14, Canberra
Australian Institute of Health and Welfare 2007, Australian hospital statistics 2005–06, AIHW cat. no.
HSE 50, Health services series no. 30, AIHW, Canberra
Australian Institute of Health and Welfare, Leeds KL, Gourley M, Laws PJ, Zhang J, Al-Yaman F & EA,
S 2007, Indigenous mothers and their babies, Australia 2001–2004. AIHW cat. no. PER 38. Perinatal
statistics series no. 19., AIHW, Canberra.
Human Resources Development Canada & Health Canada 2003, The well-being of Canada’s young
children, Government of Canada, Ottowa.
Laws, PJ & Sullivan, EA 2004a, Australia’s mothers and babies 2002, Perinatal Statistics Series No. 15,
AIHW National Perinatal Statistics Unit, AIHW cat. no. PER 28, Sydney.
ICDP Monitoring and Evaluation Framework Volume 3
APPENDICES
Appendix H
ICDP Monitoring and Evaluation Framework Volume 3
Terminology
APPENDICES
Terminology
The following definitions refer to terminology used throughout the Monitoring and Evaluation
Framework.

Indigenous Australians – The term Indigenous Australians has been used throughout this report
to encompass Aboriginal and Torres Strait Islander peoples.

Aboriginal Community Controlled Health Service – An Aboriginal Community Controlled Health
Service is (as defined by the National Aboriginal Community Controlled Health Organisation):

an incorporated Aboriginal organisation

initiated by a local Aboriginal community

based in a local Aboriginal community

governed by an Aboriginal body which is elected by the local Aboriginal community

delivering a holistic and culturally appropriate health service to the community which controls it.
By definition, organisations controlled by government to any extent are excluded.

Indigenous health services – An Indigenous health service is a health service funded principally
to provide services to Indigenous Australians. Indigenous health organisations are not necessarily
community-controlled. Indigenous health services can be run by state or territory governments.

Indigenous health workforce – Depending on the context, the term Indigenous health workforce
has been used in the Framework to refer to:


Indigenous Australians who are employed in the health workforce

Non-Indigenous Australians who are employed in areas of the health workforce that deliver
specific services to Indigenous Australians.
Chronic disease – In using the term chronic disease, the definitions outlined in the 2005 National
Chronic Disease Strategy and in the Indigenous Chronic Disease Package Factsheet recently
released by DoHA have been used. Chronic diseases:

have complex and multiple causes

usually have a gradual onset, although they can have sudden onset and acute stages

occur across the life cycle, although they become more prevalent with older age

can compromise quality of life through physical limitations and disability

are long term and persistent, leading to gradual deterioration of health

while usually not immediately life threatening, they are the most common and leading cause of
premature mortality.
The Indigenous Chronic Disease Package Factsheet makes specific reference to the following
chronic diseases:

circulatory disease

diabetes

cancer

chronic respiratory disease.
The focus on these diseases is based on evidence that chronic diseases and associated risk
factors are responsible for about two-thirds of the health gap between Indigenous and nonIndigenous Australians. Aboriginal and Torres Strait Islander people are more likely to die from
these conditions than a non-Indigenous person with the same condition, 1.5 times more likely to die
from cancer and 12 times more likely to die from diabetes. Tobacco smoking alone is responsible
for some 20% of all deaths of Aboriginal and Torres Strait Islander people. The Indigenous Chronic
Disease Package will focus on preventing and better treating and managing those chronic diseases
identified as responsible for the majority of mortality in the Indigenous population, and their risk
factors.
ICDP Monitoring and Evaluation Framework Volume 3
APPENDICES

Primary health care service – Primary health care is the first level of contact of individuals,
families and community with the health care system. Within the Australian context there are a
variety of types of services that are referred to as primary health care services. These are:

general practices where the general practitioner is the principal service provider with a varying
number of other services provided through the practice

community health services

Indigenous health services

allied health services.
ICDP Monitoring and Evaluation Framework Volume 3
APPENDICES
Appendix I
ICDP Monitoring and Evaluation Framework Volume 3
Key Indicators with Data
Sources
APPENDICES
ICDP Monitoring and Evaluation Framework Volume 3
APPENDICES
Sydney
Level 21, 321 Kent Street
Sydney, NSW 2000
Tel: +612 8233 9900
Fax: +612 8233 9966
Brisbane
Level 12, 120 Edward Street
Brisbane, QLD 4000
Tel: +617 3007 3800
Fax: +617 3007 3811
Dubai
Level 4, Attareen Building,
Saaha Offices, Old Town Island
Downtown Burj Dubai, UAE
Tel: +971 4 4200212
Fax: +971 4 4200209
Melbourne
Level 12, 120 Collins Street
Melbourne, VIC 3000
Tel: +613 8663 4888
Fax: +613 8663 4999
Perth
Ground Floor, 53 Ord Street
West Perth, WA 6005
Tel: +618 9346 0500
Fax: +618 9321 7790
Australia • Asia • Middle East
www.urbis.com.au
info@urbis.com.au
ICDP Monitoring and Evaluation Framework Volume 3
Page 31