Approach of the Office of the C&AG to
Examining Value for Money
Seamus McCarthy
Deputy Director of Audit
Office of the Comptroller and Auditor General
Irish Evaluation Network
Friday 14 December 2007
Objectives
Place VFM examination work by OCAG in the
context of accountability and evaluation
structures in Ireland
Outline how evaluation challenges are dealt with
in VFM examination work
Briefly survey some possible future
developments for OCAG assessment of VFM
C&AG’s VFM reporting mandate
C&AG may examine and report on
– the economy and efficiency achieved by audited
bodies in the acquisition, use and disposal of
resources
– the adequacy of the systems, procedures and
practices used by audited bodies to evaluate the
effectiveness of their operations
C&AG may make such comparisons as he considers
approporate (including comparisons of systems,
procedures and practices)
Evolution of the mandate
C&AG role is part of the Westminster model of
government
Original focus on financial audit
– Appropriation accounts system
– State agencies’ accounts
Progressively more reports on cases of loss, waste or
extravagance e.g.
– Project cost overruns
– Fishery harbours
– Prison planning
Project audits: 1990-1993
Comptroller and Auditor General (Amendment) Act,
1993
The accountability cycle
Dáil Éireann
Report
Minute
Committee of Public
Accounts
Minister for Finance
Information
Public hearing
Departments/
agencies
Information
Report
Comptroller and
Auditor General
Public auditing principles
Independence
Objectivity/freedom from bias
Wide scope of mandate
Freedom to report publicly
Authority
Certification audit vs Reporting
Certification audit
Statutory/obligatory
Output is a certificate/pro forma
Accounts submitted for audit
Numbers based
Standards set
Set audit procedures
Answers are clear
Ongoing relationship with
auditees’ staff
Reporting
Discretionary
Output is a report
Nothing formal to ‘audit’
Narrative/analysis based
Standards must be developed
No fixed procedures
No ‘right’ answers
New relationships with auditees’
staff
Organisation of OCAG
C&AG
Director of Audit
Director of Audit
Secretary and
Director of Audit
Corporate
Services
Vote
audits
Certification
audit
Semi State
audits
Financial
management
reporting
VFM
exams
Value for money
reporting
Strategic reporting issues
Reporting governed by ethical/professional standards
Quality assurance
– Expert input
– Internal review/audit of working papers
– Challenge/clearance process
– ‘Cold’ review
Resources
– Internal staffing
• Developing capacity
– Consultants
• Ensuring independence/objectivity
• Understanding of requirements for VFM reporting
Stages in the examination process
Selection of
examinations
Planning
Investigation
Reporting
Follow-up
Selection of examinations
C&AG exercises discretion
Strategic themes/priorities e.g.
– Public sector management reforms
– PPPs
– Health sector
Matters arising (from audits, controversies, etc)
Consultation with PAC
Review of potential reporting issues based on
– Risks to value for money in the body under review
(performance risks)
– Potential added value of a C&AG report
– Feasibility
Some recent reports
Schools PPP
Waste management in hospitals
Rent supplement
The Financial Regulator
Follow up review of examinations carried out 19972000
eGovernment
Drug addiction treatment and rehabilitation
Examination planning
Scope setting - bound by statutory mandate
– Issues to be examined – may include financial
management issues
– Time frame
– Financial scope
Analytic framework development
Liaison with auditee(s)
Tests of data availability
Time and budget planning
Formal review of plan by C&AG
Public sector business model
Identify social needs
Set objectives
Budget
Financial outturn
Financial results
Capacity
management
Planned inputs
Inputs (resources)
Measures of
resource use
Activity
management
Planned activities
Activities/processes
Activity measures
Planned outputs
Outputs
Output measures
Planned results
Results
Planned impacts
Impacts
Budgeting
Business
targets
Socio/economic
statistics
Analytic framework
Core VFM concepts (economy, efficiency,
effectiveness) are very high level
Risk-based prioritisation of key performance areas
For each performance area, identify
– Performance standard in terms of 3 Es (+ equity)
– Performance measure(s)
– Target(s)
Focus on performance of relevant bodies – things
they control or influence, rather than context
indicators
Framework set at outset, but may evolve in light of
information
Engagement with auditees
Objectives
– Brief management about plans, and get their
views
– Identify data sources/data holders
– Agree working procedures
– Resolve difficulties
– Inform management about progress
– Share findings (no surprises principle)
– Get the management perspective
Investigation
Statutory access rights
Data sources
– Official documents
– Literature search
– Administrative data e.g. databases
– Semi-structured interviews (with managers, staff,
clients)
– Questionnaires/surveys
– Sampling of cases (e.g. for process review)
Data handling procedures
Analysis
Data checking/validation incl. system review
Forms of analysis used
– Exploratory data analysis
– Descriptive statistics
– Trend analysis
– Regression
– DCF-based financial models
– Activity-based costing
Review/audit of results
Analysis of professional judgements results
Reporting
Target audiences: members of the Dáil; the public;
public service managers
Style: non-specialist; concise but comprehensive;
clear
Conclude on examination questions
Include
– recommendations for improvement
– examples of transferrable good practice
Formal clearance of drafts
– two (or more) rounds of clearance
Follow-up
Publication of report
Briefing/update for C&AG for appearance at PAC
meeting
PAC hearing
Follow-up review or re-audit
Potential impacts (during or post examination)
– Expenditure savings
– Greater efficiency
– Improved service
– Better systems for evaluating effectiveness
Possible future developments
Review of C&AG’s legislative mandate
Reporting of transferable good practice
Ad hoc reference groups for specific evaluations
Development of performance reporting (e.g. Output
Statements, VFM reports)
Changing structure of public sector – new agencies,
inspectors, regulators, PPPs, community/voluntary sector,
etc.
– Revised performance models/accountability regimes
– New issues e.g. gaps in service provision; boundary
disconnection; contract/service level management
Discussion