National Statistics Quality Review of DASA’s Finance and Economic Statistics
Project Initiation Document
Introduction
All National Statistics outputs are reviewed at least every five years. In DASA we
undertook a review of personnel statistics in 2000/01, and last year reviewed both
medical statistics and the UK Defence Statistics (UKDS) publication. This year it is
the turn of financial and economic statistics. The review is one of the Agency’s Key
Targets for 2002/03: “To conduct a National Statistics quality review of DASA’s
financial and economic statistics, and recommend improvements in their quality
and scope”. The review has also been announced in the National Statistics work
programme.
Background
DASA’s finance and economic statistics include statistics on Defence expenditure,
disaggregated in a number of different ways, research and development (R&D)
statistics, Defence trade (and balance of payment) statistics, employment statistics,
and commercial statistics. These statistics are all published in Chapter 1 of UKDS,
though many form part of and are produced for other statistical publications.
This review will take account of, and build on last year’s review of UKDS, the
reviews of defence employment estimates and of DEFFORM 57 – a form that collects
the information used in the production of many financial statistics, and work on the
effects of RAB on our statistics. It will need to consider how changes in the financial,
economic and commercial environments and to the national statistical infrastructure
will impact on DASA’s statistics, and to consider the impact of changes in IT,
administrative and accountancy practices, within MoD, such as RAB, on our data
collection processes.
Objective
The objectives of the project are:
 to review current practice to produce a description of methods and processes for
the production of financial and economic statistics using the National Statistics
Analysis of Current Practice template;
 to review all aspects of the methods and processes for the production and
publication of finance and economic statistics, using the completed template as a
basis, utilising in the review the draft NS Code of Practice, other protocols (when
available), and any other appropriate professional guidance which can assist in
applying National Statistics standards, and where appropriate advice from external
sources;
 to understand how changes in the financial, economic and commercial
environments and to the national statistical infrastructure will impact on DASA’s
statistics;
 to consider the impact of changes in IT, administrative and accountancy practices
within MoD on our data collection processes;
 to report on this review, including recommendations on actions for improvement,
to the Project Board by 31 March 2003.
1
PID sent to ONS 8/7/2002
Scope
This review covers the Defence financial and economic statistics. These statistics are
all published in Chapter 1 of UKDS, though many form part of and are produced for
other statistical publications. Given the wide range covered by these statistics, we
have split the statistics up into a number of areas and look at these different areas in
turn and their major issues below.
Price indices, related price index forecasting and other economic/econometric advice
are all considered outside of the scope of this review. Externally sourced data, which
we use to compare our statistics with those for other Government departments, or
other countries, will not be covered in detail but we shall look at the comparability of
these statistics and see if we need to add any “ health warnings”.
Financial statistics on the breakdown of Defence expenditure and budget
These statistics are found in Chapter 1 of UKDS and include breakdowns of the
Defence budget, and breakdowns of total expenditure and of categories of
expenditure, such as equipment procurement.
We are already redesigning the financial statistics tables in UK Defence Statistics to
reflect resource consumed rather than cash spent. Most of the existing series will end
with final cash figures for 2000/01 which will be published in the 2002 edition in
September. Most tables will be based on resource consumed from 2001/02 onwards.
Changes in the accounting systems have necessitated this change - but we are yet to
experience customer reaction. We anticipate some customer disappointment when
external customers fully realise what will no longer be available.
The focus of the review of these statistics should be on gauging the customer reaction
to the changes, and using this to help plan and prioritise how we further develop these
statistics for UKDS 2003. This will probably be done in conjunction with our new
Publications branch, who are taking forward recommendations from the review of
UKDS done last year.
Trade statistics (imports and exports), and balance of payments statistics
UKDS and other publications include tables showing exports and imports of defence
equipment. There is also a table showing payments for defence services.
We have already done much work with HMC&E, DTI and others through the Defence
Trade Statistics Working Group chaired by DASA. This is currently focussing on
sources, data collection methods and definitions. To date this has concentrated on
export statistics but the Group has now expanded its remit to look at imports as well.
We plan to involve members of this Group (as stakeholders) in the review of trade
statistics.
2
PID sent to ONS 8/7/2002
With this background we feel the NS review should be strategic - the environment
(simplification of export controls etc), the needs of customers, how to respond to
changes etc. We are intending to split the review of these statistics into two parts:
exports and imports, and balance of payments, to reflect the distinct data sources and
customers.
Research and Development (R&D) Statistics
These statistics break down MoD expenditure on defence R&D in various ways.
Historically, this has been a difficult area. The introduction of RAB, the IPT structure
and the changes to DERA have greatly affected data collection. We need to consider
the resources involved in producing these statistics, and the compliance costs to our
suppliers. It is not clear that other Departments go to the same lengths as we do, but
MoD is a major player (defence R&D is 40% of government R&D) and there is
considerable interest from the House of Commons Defence Committee. We need to
focus here on the requirements of ONS and the ultimate customers, the approaches
taken in other Government Departments, the processes for capturing the data within
MoD and coherency with RAB definitions.
Commercial Statistics
These statistics provide information on the contracts placed by MoD. Our main
concerns are with quality and coverage. We draw on data from the Defence Bills
Agency but we know there are significant quality problems and have just launched a
short review on a sample of records. We also know that an increasing proportion of
bills are not paid by the DBA, but we have no data on these. There is interest in PFI
and PPP contracts, and there are a number of new initiatives, such as P2P (Payments
to Purchasers) that may affect data collection. We propose that the NS review should
focus on data sources and their quality and coverage.
Industrial statistics
At present these statistics cover little beyond a table in UKDS which shows MoD
payments to prime contractors by size of total payments. Last year’s UKDS review
revealed a demand for more statistics about the defence industrial and technological
base. We are hampered by the fact that there is no specific section or sub-sections of
the standard industrial classification (SIC) for the defence industry and no simple or
agreed definition of the industry itself.
The focus of this NS review should be to refine customer needs and consider how to
meet them. This will feed into the Key Target on producing a longer term strategy for
UK defence statistics.
Employment statistics
These statistics cover the estimates of the UK employment dependent on Defence
expenditure and defence equipment exports. A thorough review of the methodology,
involving independent external expertise, is already underway. The outcome of this
will feed into the NS review in summer 2002. We do not feel that any new work is
3
PID sent to ONS 8/7/2002
needed, but the NS review will provide us with the opportunity to set the findings of
this study in the context of the wider quality review, and its recommendations for the
future.
Format of review
The overall review does not fall neatly into any of the four categories described by the
ONS (strategic, fundamental, standard or targeted). There are elements of the
strategic, standard and targeted for different areas of the business. Indeed, the
different areas have different issues that require different approaches, and different
time constraints. For this reason we propose splitting the review into a series of
modules so that the individual modules can be addressed separately, and at different
times. This has advantages also in taking account of existing work that has been done,
or is underway and of spreading the workload on the Project Team.
Although these reviews will be modular they will feed into the overall review and
there will be one final report. By working this way we hope to be able to look at the
individual detail of the separate modules, but also to draw out the common themes of
these statistics.
Process
We intend following the same approach as we have done in other reviews but adapted
to take account of the modular nature of this review. Each review module will
complete the Analysis of Current Practice template. We will then use this template as
a starting point for discussions among the project team, with customers and others to
identify areas, issues and gaps that we will wish to look at further in the review itself.
Each module will have its own review, but we hope that by having an overall Project
Manager and Team members participating in the review of several modules that we
will be able to see and draw out the common themes.
We anticipate that the final report will include both general recommendations and
those specific to individual modules or area of statistics.
Timing
We plan to spread the review modules over the year. This will help us even out the
workload, take account of relevant work that is currently underway and of milestone
events such as the publication of UKDS 2002.
We intend to finish the review and produce a report to the Project Board by the end of
the financial year. This will provide the Project Board with an opportunity to assess
whether the Key Target has been achieved.
Although we intend to finish the review in 2002/03, it is not intended that all the
outcomes of the review will be addressed. In the other reviews undertaken by DASA
to date, the implementation of recommendations – and the development of the
implementation plan - has taken place after the review. This is the approach we expect
to adopt here as this review covers a very wide range of statistics.
4
PID sent to ONS 8/7/2002
In addition, the NS framework for a quality review requires that a report of the review
is sent to the National Statistician and then made publicly available on the NS
website, and sent to the Statistics Commission. We intend to do this in 2003/04.
Risks
We have identified and will need to manage a number of risks against successful
completion of the project, including:



Availability of resources,
Timing of completion of work intended to feed into review
Participation of customers and providers of other sources of information
Organisation
We will project manage this review in the normal way. Glen Watson, the Director
with responsibility for all financial and economic statistics, will chair the Project
Board. Brenda Mandeville, Principal Economist within DASA (Projects) will be
project manager/co-ordinator. Unlike other DASA NS reviews, she will act as a coordinator and be responsible for the review as a whole, but she will not necessarily
take the lead in all areas. Jason Bradbury, Tony Turner and Nick Bennett of DASA
(Procurement) will take the lead for certain modules of the review as appropriate. The
MoD National Statistics Steering Group will also monitor progress on this review.
Project Board:
The initial members of the Project Board are:
Glen Watson, D DASA IS&L - chair
Neil Davies, Director Economic Advice Division (representing DGFM’s interests)
Glenn Everett, Director of Statistics, DTI
Prof. Keith Hartley, Centre for Defence Economics, University of York.
Brenda Mandeville, DASA (Projects) - Project Manager
Jason Bradbury, Head of DASA (Proc)
Tony Turner, Principal consultant DASA (Proc)
Claire Stokes, DASA (Projects) – secretary
David Dick, the Corporate Financial Controller will attend later meetings. Other
members may be co-opted as necessary.
The terms of reference for the Project Board are at Annex A.
Project Team:
We will undertake this review using a core group of staff who will lead on the
modules and provide continuity, consisting of the Project Manager and Jason
Bradbury, Tony Turner, and Nick Bennett from DASA (Procurement). They will
draw on a wider project team - the above, plus all of the Finance Statistics Team,
Claire Stokes and others as required - to undertake the review of individual modules
and provide staff resources for the review.
The terms of reference for the Project Team are also at Annex A.
5
PID sent to ONS 8/7/2002
ANNEX A: Terms of Reference
Terms of reference for the Project Board
The Project Board will:
 Approve the project plan
 Advise, and facilitate the Project Team on resources required and actions to be
taken
 Review progress and agree any actions to keep the project to timetable
 Agree changes to project plans
 Agree and facilitate inputs to the review from external sources
 Determine whether the Key Target has been satisfactorily met
Terms of reference for the Project Team
The project team will undertake a National Statistics Quality Review of DASA’s
financial and economic statistics to meet the objectives in the PID, and in particular
to:








understand the quality of existing outputs, assess their fitness for purpose and, if
necessary, propose corrective actions
assess the level of risk in our existing outputs and ensure that the MoD is not
embarrassed by poor quality statistics
understand more about customer needs for Defence related financial and
economic statistics now and in the future
understand how changes in the financial, economic and commercial
environments and in the national statistical infrastructure will impact on DASA’s
statistics
consider the impact of changes in IT, administrative and accountancy practices
within MoD on our data collection processes
ensure coherency (or at least understand differences) between published
financial and economic statistics from other sources (Departmental Resource
Accounts, Departmental Performance Report, HMT Public Expenditure
Statistical Analysis, Major Projects Report, National Accounts etc)
provide input to the UKDS implementation Key Target, in particular the longer
term strategy
produce a clear set of recommendations for the future development of financial
and economic statistics.
6
PID sent to ONS 8/7/2002