Census Data Processing In BPS – Statistics Indonesia 1 About BPS – Statistics Indonesia Vision: BPS is the provider of quality statistics. A non-departmental government institution. Directly responsible to the President of the Republic of Indonesia. Headed by a director general. - Provide data to the government, and the public. - Assist statistics divisions of government ministries and other institutions, in developing the statistical system they need for setting up work programs and periodic reporting. - Develop and promote standards to be incorporated implementation of statistical techniques and methods. in the - Provide necessary services in the field of education and training in statistics. - Establish cooperation with international institutions and other countries for the benefit of Indonesia’s statistical development. Organizational Structure Director General Principal Secretary Institute of Statistics Main Inspectorate Inspectorate Area I Inspectorate Area II Inspectorate Area III Bureau of Program Management Education and Training Center Deputy DG for Production Statistics Deputy DG for Methodo logy and Statistical Information Directorate of Food Crops, Horticultural, and Estate Crops Statistics Directorate for Dev of Census & Survey Methodology Directorate of Animal Husbandry, Fishery, And Forestry Statistics Directorate of Indutrial Statistics Directorate of Statistical Dissemination Directorate of Statistical Information System Deputy DG for Social Statistics Bureau of Finance Bureau of Personnel & Legal Affairsl Deputy DG for Distribution and Services Statistics Bureau of General Affairs Deputy DG for National Account and Statistical Analysis Directorate of Population Statistics Directorate of Distribution Statistics Directorate of Production Account Directorate of Social Welfare Statistics Directorate of Price Statistics Directorate of Consumption Account Directorate of Social Resilience Statistics Directorate of Financial, IT, and Tourism Statistics Directorate of Statistical Analysis BPS Regional Statistics Offices Provincial and Regional Offices There are 33 BPS provincial offices 344 BPS regency offices and 96 BPS city offices. they are the vertical representative offices of BPS. At sub-district level, BPS employs a statistical staff (subdistrict level Statistical Coordinator). Technically and administratively, BPS regional offices are directly under the control of BPS-Statistics Indonesia. The number of BPS staffs is 13,465 (as July 2008) 1,466 persons in BPS Head Quarter (11%) 11,999 persons in BPS-Regional Offices (89 %). CHIEF OF BPS PROVINCIAL OFFICE ADMINISTRATION DIVISION Provincial Level Organization PROGRAM MANAGEMENT SUB DIVISION INTERNAL AFFAIRS SUB DIVISION PERSONNEL AND LEGAL AFFAIRS SUB DIVISION SOCIAL STATISTICS DIVISION PRODUCTION STATISTICS DIVISION DISTRIBUTION STATISTICS DIVISION REGIONAL ACCOUNT AND STATISTICAL ANALYSIS DIVISION DATA PROCESSING AND DISSEMINATION DIVISION DEMOGRAPHIC AND HOUSEHOLD STATISTICS SUB DIVISION AGRICULTURAL STATISTICS SUB DIVISION CONSUMER AND WHOLESALE PRICE STATISTICS SUB DIVICION PRODUCTION ACCOUNTS SUB DIVISION NETWORKING AND STATISTICAL REFERENCE SUB DIVISION MAN POWER STATISTICS SUB DIVISION INDUSTRIAL STATISTICS SUB DIVISION FINANCIAL AND PRODUCER PRICE STATISTICS SUB DIVICION CONSUMPTION ACCOUNTS SUB DIVISION DATA PROCESSING SOCIAL WELFARE STATISTICS SUB DIVISION MINING, ENERGY & CONSTRUCTION STATISTICS SUB DIVISION COMMERCE & SERVICES STATISTICS SUB DIVICION STATISTICAL ANALYSIS AND DEVELOPMENT SUB DIVISION DISSEMINATION & STATISTICAL SERVICE SUB DIVISION INTEGRATION SUB DIVISION ASSETS FINANCE SUB DIVISION MANAGEMENT SUB DIVISION Census in Indonesia Population census: in the years ending with 0. previous population census: in 1961, 1971, 1980, 1990, 2000. The next Population Census would be in 2010 Agricultural census: in the years ending with 3. previous agricultural census: in 1983, 1993, 2003. Economic census: in the years ending with 6. previous economic census: in 1986, 1996, 2006. Data Processing Support for Population Census 1980, 1990 • Using Mainframe: • (NEC ACOS 500 in 1980) • (NEC ACOS 1520 in 1990) • Batch Processing (not interactive) Data Processing Policy (after elimination of Mainframe) 1. Decentralized & Distributed Data Processing: • Encourage End-users Computing: subject matter units are entitled to process their own survey under coordination of data processing division. • Develop application in-house by HQ & decentralize operation to Regional offices. • Implement Distributed / Client-Server Data Processing Model. 2. IT Department (Directorate of SIS) focuses on IT infrastructures & complex DP Systems such as Censuses. 8 Data Processing Support for Population Census 2000 • PC based Client-Server Data Processing • Using Scanner: Kodak DS3500 • Speed: 3500 forms / hour • Use of OMR, ICR: Nestor Reader • ICR Ascent-Capture: started in 2003 (Census of Agriculture). PC 2000 Data Capture Stages Scan documents (produce image file) Recognize image (produce ascii characters) Verification & Correction (operators decide characters ambiguous to recognize engine) Completion completeness check Release save into database Experience from PC 2000 • Unexpected low quality of enumerator’s handwriting to recognize by ICR • Unable to recognize characters correctly – Rejected too many values (too high confidence level) – Accepted incorrect values (too low confidence level) realized from tabulation output – Should check-digit help to increase accuracy ? • Ineffective learning engine • In a LAN environment using MS-SQl Server, 4 concurrent scanning processes slow down the whole DP activities consisting of 3 recognize stations, 10 correction stations,and 25 computer editing stations. Experience from PC 2000 • Extra work to erase & rewrite Census Form • Powder from rubber eraser stick on Census forms & affected Scanner performance • Costly maintenance effort of Scanners which spread over all provinces • Relatively high cost (investment) for Data Capture H/W& S/W • However, Data Capture using scanner was able to shorten Data Processing Completion Time. • It was not easy to find out well motivated enumerators & Data Capture operators. PC2010 Data Processing Plan Main Documents: • Short Form :L for Listing & basis for 10% sample, using data entry (DE), :C using scanner, complete enumeration (100%) of households & members • Long Form :S using data entry / scanner, 10% sample. Estimation: • • • • • • • No of Districts / Regency = 456 No of Sub-districts = 5,901 No of villages = 72,084 No of Census Blocks = 658,308 (in year 2000) No of smallest Administrative Areas (SLS) = 900,000 No of households = 64 million No of population = 236 million Document: Batch Control Document C PC2010 Data Processing Plan (cont) Issues on existing Scanners: Kodak DS3500 procured for PC 2000: 79 units About 60 % of them need spare-parts replacement Spare-parts are discontinued. (Dicom Singapore eager to reserve all available spare-parts to BPS) Running out of Ascent Capture Counter (dongle) limit Operable uncertainty for PC 2010 PC 2010 Data Processing Strategy 1. Utilize scanners with character recognition technology (OMR, OCR, ICR), focused on OMR. 2. Expect ICR with better Learning Engine capability. 3. Procurement of Scanning H/W and S/W in 2009. 4. Formation of 8 Data Processing Centers. 5. Utilize SMS facility for transmitting and accessing data, as well as monitoring data collection activities. 6. Utilize web-based application for monitoring data processing activities. 7. In 2008, all of 33 province level offices and some (34) district / city level offices are connected using Virtual Private Network (VPN). 17 PC2010 Data Processing Plan (cont) Formation of 8 Processing Centers Medan 11 12 13 14 21 Palembang 15 16 17 18 19 DKI Jakarta 31 36 61 Bandung 32 Semarang 33 34 NAD Surabaya Sumatera Utara Sumatera Barat Riau Kepulauan Riau Banjarmasin Jambi Sumatera Selatan Bengkulu Makasar Lampung Kep. Bangka Belitung DKI Jakarta Banten Kalimantan Barat Jawa Barat Jawa Tengah DI Yogyakarta 35 51 52 53 62 63 64 71 72 73 74 75 76 81 82 91 92 Jawa Timur Bali NTB NTT Kalimantan Tengah Kalimantan Selatan Kalimantan Timur Sulawesi Utara Sulawesi Tengah Sulawesi Selatan Sulawesi Tenggara Gorontalo Sulawesi Barat Maluku Maluku Utara Irian Jaya Barat Papua 8 Data Processing Centers SMS / Web Based Progress Reporting & Monitoring Regional Offices Progress Reports Monitoring Headquarters Cellular Operator SMS Application Server Database Server GSM modem Regional Offices Progress Reports Internet / VPN Monitoring Headquarters Thank you for your attention September 2008