National Statistics Quality Review of DASA’s Finance and Economic Statistics Project Initiation Document Introduction All National Statistics outputs are reviewed at least every five years. In DASA we undertook a review of personnel statistics in 2000/01, and last year reviewed both medical statistics and the UK Defence Statistics (UKDS) publication. This year it is the turn of financial and economic statistics. The review is one of the Agency’s Key Targets for 2002/03: “To conduct a National Statistics quality review of DASA’s financial and economic statistics, and recommend improvements in their quality and scope”. The review has also been announced in the National Statistics work programme. Background DASA’s finance and economic statistics include statistics on Defence expenditure, disaggregated in a number of different ways, research and development (R&D) statistics, Defence trade (and balance of payment) statistics, employment statistics, and commercial statistics. These statistics are all published in Chapter 1 of UKDS, though many form part of and are produced for other statistical publications. This review will take account of, and build on last year’s review of UKDS, the reviews of defence employment estimates and of DEFFORM 57 – a form that collects the information used in the production of many financial statistics, and work on the effects of RAB on our statistics. It will need to consider how changes in the financial, economic and commercial environments and to the national statistical infrastructure will impact on DASA’s statistics, and to consider the impact of changes in IT, administrative and accountancy practices, within MoD, such as RAB, on our data collection processes. Objective The objectives of the project are: to review current practice to produce a description of methods and processes for the production of financial and economic statistics using the National Statistics Analysis of Current Practice template; to review all aspects of the methods and processes for the production and publication of finance and economic statistics, using the completed template as a basis, utilising in the review the draft NS Code of Practice, other protocols (when available), and any other appropriate professional guidance which can assist in applying National Statistics standards, and where appropriate advice from external sources; to understand how changes in the financial, economic and commercial environments and to the national statistical infrastructure will impact on DASA’s statistics; to consider the impact of changes in IT, administrative and accountancy practices within MoD on our data collection processes; to report on this review, including recommendations on actions for improvement, to the Project Board by 31 March 2003. 1 PID sent to ONS 8/7/2002 Scope This review covers the Defence financial and economic statistics. These statistics are all published in Chapter 1 of UKDS, though many form part of and are produced for other statistical publications. Given the wide range covered by these statistics, we have split the statistics up into a number of areas and look at these different areas in turn and their major issues below. Price indices, related price index forecasting and other economic/econometric advice are all considered outside of the scope of this review. Externally sourced data, which we use to compare our statistics with those for other Government departments, or other countries, will not be covered in detail but we shall look at the comparability of these statistics and see if we need to add any “ health warnings”. Financial statistics on the breakdown of Defence expenditure and budget These statistics are found in Chapter 1 of UKDS and include breakdowns of the Defence budget, and breakdowns of total expenditure and of categories of expenditure, such as equipment procurement. We are already redesigning the financial statistics tables in UK Defence Statistics to reflect resource consumed rather than cash spent. Most of the existing series will end with final cash figures for 2000/01 which will be published in the 2002 edition in September. Most tables will be based on resource consumed from 2001/02 onwards. Changes in the accounting systems have necessitated this change - but we are yet to experience customer reaction. We anticipate some customer disappointment when external customers fully realise what will no longer be available. The focus of the review of these statistics should be on gauging the customer reaction to the changes, and using this to help plan and prioritise how we further develop these statistics for UKDS 2003. This will probably be done in conjunction with our new Publications branch, who are taking forward recommendations from the review of UKDS done last year. Trade statistics (imports and exports), and balance of payments statistics UKDS and other publications include tables showing exports and imports of defence equipment. There is also a table showing payments for defence services. We have already done much work with HMC&E, DTI and others through the Defence Trade Statistics Working Group chaired by DASA. This is currently focussing on sources, data collection methods and definitions. To date this has concentrated on export statistics but the Group has now expanded its remit to look at imports as well. We plan to involve members of this Group (as stakeholders) in the review of trade statistics. 2 PID sent to ONS 8/7/2002 With this background we feel the NS review should be strategic - the environment (simplification of export controls etc), the needs of customers, how to respond to changes etc. We are intending to split the review of these statistics into two parts: exports and imports, and balance of payments, to reflect the distinct data sources and customers. Research and Development (R&D) Statistics These statistics break down MoD expenditure on defence R&D in various ways. Historically, this has been a difficult area. The introduction of RAB, the IPT structure and the changes to DERA have greatly affected data collection. We need to consider the resources involved in producing these statistics, and the compliance costs to our suppliers. It is not clear that other Departments go to the same lengths as we do, but MoD is a major player (defence R&D is 40% of government R&D) and there is considerable interest from the House of Commons Defence Committee. We need to focus here on the requirements of ONS and the ultimate customers, the approaches taken in other Government Departments, the processes for capturing the data within MoD and coherency with RAB definitions. Commercial Statistics These statistics provide information on the contracts placed by MoD. Our main concerns are with quality and coverage. We draw on data from the Defence Bills Agency but we know there are significant quality problems and have just launched a short review on a sample of records. We also know that an increasing proportion of bills are not paid by the DBA, but we have no data on these. There is interest in PFI and PPP contracts, and there are a number of new initiatives, such as P2P (Payments to Purchasers) that may affect data collection. We propose that the NS review should focus on data sources and their quality and coverage. Industrial statistics At present these statistics cover little beyond a table in UKDS which shows MoD payments to prime contractors by size of total payments. Last year’s UKDS review revealed a demand for more statistics about the defence industrial and technological base. We are hampered by the fact that there is no specific section or sub-sections of the standard industrial classification (SIC) for the defence industry and no simple or agreed definition of the industry itself. The focus of this NS review should be to refine customer needs and consider how to meet them. This will feed into the Key Target on producing a longer term strategy for UK defence statistics. Employment statistics These statistics cover the estimates of the UK employment dependent on Defence expenditure and defence equipment exports. A thorough review of the methodology, involving independent external expertise, is already underway. The outcome of this will feed into the NS review in summer 2002. We do not feel that any new work is 3 PID sent to ONS 8/7/2002 needed, but the NS review will provide us with the opportunity to set the findings of this study in the context of the wider quality review, and its recommendations for the future. Format of review The overall review does not fall neatly into any of the four categories described by the ONS (strategic, fundamental, standard or targeted). There are elements of the strategic, standard and targeted for different areas of the business. Indeed, the different areas have different issues that require different approaches, and different time constraints. For this reason we propose splitting the review into a series of modules so that the individual modules can be addressed separately, and at different times. This has advantages also in taking account of existing work that has been done, or is underway and of spreading the workload on the Project Team. Although these reviews will be modular they will feed into the overall review and there will be one final report. By working this way we hope to be able to look at the individual detail of the separate modules, but also to draw out the common themes of these statistics. Process We intend following the same approach as we have done in other reviews but adapted to take account of the modular nature of this review. Each review module will complete the Analysis of Current Practice template. We will then use this template as a starting point for discussions among the project team, with customers and others to identify areas, issues and gaps that we will wish to look at further in the review itself. Each module will have its own review, but we hope that by having an overall Project Manager and Team members participating in the review of several modules that we will be able to see and draw out the common themes. We anticipate that the final report will include both general recommendations and those specific to individual modules or area of statistics. Timing We plan to spread the review modules over the year. This will help us even out the workload, take account of relevant work that is currently underway and of milestone events such as the publication of UKDS 2002. We intend to finish the review and produce a report to the Project Board by the end of the financial year. This will provide the Project Board with an opportunity to assess whether the Key Target has been achieved. Although we intend to finish the review in 2002/03, it is not intended that all the outcomes of the review will be addressed. In the other reviews undertaken by DASA to date, the implementation of recommendations – and the development of the implementation plan - has taken place after the review. This is the approach we expect to adopt here as this review covers a very wide range of statistics. 4 PID sent to ONS 8/7/2002 In addition, the NS framework for a quality review requires that a report of the review is sent to the National Statistician and then made publicly available on the NS website, and sent to the Statistics Commission. We intend to do this in 2003/04. Risks We have identified and will need to manage a number of risks against successful completion of the project, including: Availability of resources, Timing of completion of work intended to feed into review Participation of customers and providers of other sources of information Organisation We will project manage this review in the normal way. Glen Watson, the Director with responsibility for all financial and economic statistics, will chair the Project Board. Brenda Mandeville, Principal Economist within DASA (Projects) will be project manager/co-ordinator. Unlike other DASA NS reviews, she will act as a coordinator and be responsible for the review as a whole, but she will not necessarily take the lead in all areas. Jason Bradbury, Tony Turner and Nick Bennett of DASA (Procurement) will take the lead for certain modules of the review as appropriate. The MoD National Statistics Steering Group will also monitor progress on this review. Project Board: The initial members of the Project Board are: Glen Watson, D DASA IS&L - chair Neil Davies, Director Economic Advice Division (representing DGFM’s interests) Glenn Everett, Director of Statistics, DTI Prof. Keith Hartley, Centre for Defence Economics, University of York. Brenda Mandeville, DASA (Projects) - Project Manager Jason Bradbury, Head of DASA (Proc) Tony Turner, Principal consultant DASA (Proc) Claire Stokes, DASA (Projects) – secretary David Dick, the Corporate Financial Controller will attend later meetings. Other members may be co-opted as necessary. The terms of reference for the Project Board are at Annex A. Project Team: We will undertake this review using a core group of staff who will lead on the modules and provide continuity, consisting of the Project Manager and Jason Bradbury, Tony Turner, and Nick Bennett from DASA (Procurement). They will draw on a wider project team - the above, plus all of the Finance Statistics Team, Claire Stokes and others as required - to undertake the review of individual modules and provide staff resources for the review. The terms of reference for the Project Team are also at Annex A. 5 PID sent to ONS 8/7/2002 ANNEX A: Terms of Reference Terms of reference for the Project Board The Project Board will: Approve the project plan Advise, and facilitate the Project Team on resources required and actions to be taken Review progress and agree any actions to keep the project to timetable Agree changes to project plans Agree and facilitate inputs to the review from external sources Determine whether the Key Target has been satisfactorily met Terms of reference for the Project Team The project team will undertake a National Statistics Quality Review of DASA’s financial and economic statistics to meet the objectives in the PID, and in particular to: understand the quality of existing outputs, assess their fitness for purpose and, if necessary, propose corrective actions assess the level of risk in our existing outputs and ensure that the MoD is not embarrassed by poor quality statistics understand more about customer needs for Defence related financial and economic statistics now and in the future understand how changes in the financial, economic and commercial environments and in the national statistical infrastructure will impact on DASA’s statistics consider the impact of changes in IT, administrative and accountancy practices within MoD on our data collection processes ensure coherency (or at least understand differences) between published financial and economic statistics from other sources (Departmental Resource Accounts, Departmental Performance Report, HMT Public Expenditure Statistical Analysis, Major Projects Report, National Accounts etc) provide input to the UKDS implementation Key Target, in particular the longer term strategy produce a clear set of recommendations for the future development of financial and economic statistics. 6 PID sent to ONS 8/7/2002