THEME:
EXPLOITING NEW TECHNOLOGIES AND NEW DATA SOURCES
TITLE:
TIMELY STATISTICS THROUGH INTEGRATED STATISTICAL SYSTEM
1)
2)
3)
4)
Ms. Faiza Rusrianti Tajul Arus
Ms. Che Norma Murad
Ms. Siti Haslinda Mohd Din
Datuk Dr. Abdul Rahman Hasan
Department of Statistics Malaysia
Block C6, Complex C,
62514 Putrajaya
MALAYSIA
Tel:
603-8885 7703 / 603-8885 7343 / 603-8885 7492
Fax: 603-8888 9209
E-mail: faiza@stats.gov.my / norma_murad@stats.gov.my /sitihaslinda@stats.gov.my /
abrahmanh@stats.gov.my /
1
ABSTRACT
Department of Statistics Malaysia (DOSM) is the leading official statistics producer. Being
the producer of national statistics, the department is responsive to the various challenges
related to the Malaysia National Development Policies and the National Vision, to become a
develop nation by 2020.
DOSM has modernised the way in producing statistics and embark on a journey to transform
the business process model from stovepipe to integrated approach. Through the integrated
proposed solution, it provided the main business functionalities of DOSM on pre collection,
collection, processing, analysis and dissemination. The web browser shall be the front end
interface to this integrated system and it provides wider access capabilities anyway and
anytime through web based application. It allows live-update statistical register and
automated sample selection. Centralised and online surveys approached enable easy and
faster access by clients. Online monitoring and validation will enhance the quality of data
produced. Dissemination of statistics to external users by means of publications and data
request are served through an enterprise portal.
The solution has improved the rising cost issues in producing statistics. It also enhances the
quality, integrity and the reliability of the official statistics as it is crucial in fulfilling the
demand for decision making and policy formulation.
2
1. INTRODUCTION
1.1
Background of the Department of Statistics, Malaysia
The Department of Statistics, Malaysia (DOSM) was established in 1949 under the
Statistics Ordinance 1949 and then known as the Bureau of Statistics. The Statistics
Ordinance 1945 was replaced by the Statistics Act 1965. Under this Act, the Bureau
of Statistics was known as Department of Statistics, headed by the Chief Statistician.
The Statistics Act 1965 was revised in 1989. The functions of DOSM are explicitly
defined under Section 2(2):
“The function of the Department shall be to collect and interpret statistics for the
purpose of furnishing information required in the formation or carrying out the
Government policy in any field or otherwise required for Government purposes or for
meeting the needs of trade, commerce, industry or agriculture (including forestry,
fishing and hunting)”.
1.2
Statistical System in Malaysia
A centralised statistical system is being practice in Malaysia. DOSM is the main
agency that is responsible in generating official statistics for the country. The
Statistics was compiled from censuses/surveys as well as data from various
agencies. The principal statistics produced by DOSM comprise economic indicator of
agriculture, mining and quarrying, manufacturing, construction, services, national
accounts, balance of payments, economic indicators and external trade1. Meanwhile,
social/demographic statistics comprises the statistics for population, labour force,
1
Department of Statistics Malaysia (2009a),p13
3
household income and expenditure, prices, environment, health, education and
housing.
2. INNOVATION IN NEW TECHNOLOGIES
Timely and reliable statistics are essential to ensure the implementation of Policies
and National Vision is at par with the Malaysia Vision 2020. This is also in line with
DOSM vision, to become a leading statistical organisation internationally by 2020. In
tandem with this, DOSM had determined the Strategic Plan 2010-2014 as framework
and guidance. Meanwhile new strategic plan for 2015-2019 is currently being
developed. One of the identified strategic plans is the enhancement of data
communication.
In 1950s, DOSM used hand-pick system whereby data were collected and captured
manually. This practice was insufficient to capture and process the data that might
lead to momentous non-sampling error. In year 1954 and 1957, ICT tabulators and
Key Punch Machines were introduced to speed up the process of producing
statistics, and in 1967, the workflow had migrated from mechanical to electronic data
processing following the installation of the main frame system. Meanwhile in 1980’s,
few generations of mainframe systems were installed to cope with the increasing
usage and users’ demand for timely data. DOSM started to use the Intelligent
Character Recognition (ICR) for data processing in 2005. ICR has definitely reduced
the time consumed in data processing and human error in data capturing.
Recently, DOSM has taken new initiatives to integrate different system applications
and all statistical workflows through a computerised system known as National
Enterprise-Wide Statistical System (NEWSS). The workflow includes designing,
processing, collecting, analysing, interpreting and disseminating activities in the
4
Generic Statistical Business Process Model (GSBPM). The NEWSS project kicked
off in 2008 and the first phase of development was completed and fully implemented
in 2010. NEWSS as an integrated system and serve the following purposes:
i.
To standardise, consolidate and improve the existing system/ application to
support the strategic requirement and the operation of DOSM;
ii.
To simplify, improve and expedite the process of statistical data dissemination;
iii.
To develop an integrated business process management that adheres to
international statistics standard; and
iv. To build up a central repository to facilitate data sharing between DOSM and
other government agencies.
3. INTEGRATED STATISTICAL SYSTEM
An efficient and effective delivery system is essential in disseminating information
and services to customers and users. The implementation of NEWSS based on the
DOSM ICT Architecture will definitely move DOSM forward leveraging on the use of
emerging technology. Currently DOSM has approximately 95 application systems
which are operational and maintained.
However in NEWSS, only six major
applications are revamped. These applications are:
(a)
Sistem Daftar Pusat (SIDAP) which is used to maintained the Enterprise
and Establishment Frame;
(b)
Sistem Pangkalan Data Rangka Banci (SPRDB) which is used to
maintain Household Frame;
(c)
Sistem Penyiasatan Tenaga Buruh (SPTB) which covers the surveys
and analysis in the areas of labour force, migration, salaries and wages;
(d)
Sistem Soal Selidik Ekonomi (SSE) which is used to process 35 types
of economics surveys;
5
(e)
Sistem Penyiasatan Pembuatan Bulanan (SPPB) which is used to
support the monthly manufacturing surveys; and
(f)
Sistem Pengkodan Bantuan Komputer which maintains the codes used
by DOSM.
This system is continuously enhanced in line with the development of the information
and communication technology through2:

generation of information that adhere to the customers and users
requirements;

coordination between data producers and users;

formation on institutional and legal framework;

adoption on international standards, methodology, concepts and definitions
and classifications;

enhancement of human capital; and

provision of the latest and appropriate information and communication
technology (ICT) infrastructure.
DOSM has transformed from common system with silo environment, obsolete
technology, not integrated environment and non-uniform platform and application
design to centralised repository, new technology, information support system and
integrated statistical system framework. The Integrated Statistical System Framework
(ISSF) was formulated to provide a structured and standardised framework that
would support the business functionalities for DOSM pre-collection, collection,
processing, analysis and dissemination at Head Quarters and State Offices
respectively. The system then was known as NEWSS (National Enterprise-Wide
Statistical System) the one stop center of statistical information.
2
Department of Statistics Malaysia (2010), p7.
6
3.1
NEWSS
NEWSS would be DOSM’s first implementation in compliance to the ISSF and
would lay the foundation of the enterprise architecture for all future DOSM
systems. The Architecture includes a centralised depository of statistical
information including standard codes and classifications that would be used by
DOSM for all its processing needs.
NEWSS would have alerts, reminders, escalation and other mechanism for
DOSM to efficiently monitor, manage and control the surveys and census
carried out by the department. Validation rules would also be built into the
system to improve the accuracy of the statistics information. With the
monitoring and validation functionalities provided by NEWSS, DOSM would be
able to improve the timeliness and reliability of the statistics information.
NEWSS would also be the conduit to increase dissemination of statistics
information to DOSM stakeholders. Overview of system flow is as in
appendix.
3.2
Integrated Statistical Workflows
Integrated Statistical workflows comprising five business functionalities for
DOSM, namely:

Pre-Collection
Consist of activities pertaining to preparation for pre-collection i.e.
sample selection from frame, extraction and loading contact information
on respondent and creation of survey form.

Collection
Involve activities such as data collection (primary and secondary), data
validation and verification, data capture and estimation.

Processing
7
Include the provision of clean data activities as statistical edit and
imputation,
calendarisation,
weighting
and
estimation
and
also
production-oriented analysis.

Analysis
This function includes activities on tabulation, review and quality
assurance, data extraction and report generation.

Dissemination
Consist of activities pertaining to import and archive micro and macro
data, produce, catalogue, market and disseminate statistical products
and services and process into DOSM warehouse.
3.3
Information Support Systems (ISS)
The systems within ISS are as follows:

Enterprise Portal (EP) which provides a single point of access to the
contents and records of the various modules in NEWSS;

Performance Management System (PMS) which allows DOSM to
measure performance via key performance indicator;

Geographic Information System (GIS) which supports the analysis
process using spatial data and information integration system in
obtaining related information from the establishment and household
frames;

Business Intelligence (BI) which is used for statistical analysis on
structured data.
BI system provides historical, current, and predictive
views of business operations, using data that has been gathered into a
data warehouse; and

Knowledge Management System (KMS) which promotes collaboration
and sharing of information within DOSM.
8
4. IMPACT OF INNOVATION
The impact of NEWSS on DOSM can be categorized into the following areas3:
4.1
(a)
Stakeholders;
(b)
Subject Matter Divisions;
(c)
Frame;
(d)
Central repository;
(e)
Dissemination;
(f)
System development; and
(g)
Hardware and software.
Stakeholders
The direct beneficiaries of NEWSS are the stakeholders. All requirements of
the stakeholders can be fulfilled in a shorter period with a better frame for
sampling. In this context, NEWSS will provide the interface to accept data
from selected organizations to update the enterprise and establishment
frames.
In this phase, organisations such as Companies Commission of
Malaysia (CCM), Construction Industry Development Board (CIDB) and
Employee Provident Fund (EPF) have indicated their willingness to share
data with DOSM. This approach will enable DOSM to update its frame for
better sampling.
Besides this, NEWSS will facilitate the data tapping from other government
agencies (OGAs). Through mutual understanding, these OGAs will provide
their softcopy data to DOSM. However, direct integration with their systems
through web-based application or other means, will be planned later on.
3
Mazlan Sulong (2011)
9
Computing facilities provided by NEWSS enable DOSM to customize the
request from stakeholders.
Dynamic analysis tools such as Business
Intelligence (BI) and Extensible Business Reporting Language (XBRL) allow
DOSM to analyse data easily and prepare various types of reports with the
look and feel of the stakeholders.
4.2
Subject Matter Divisions (SMDs)
Efficient and effective application systems designed using Java and workflow,
allow SMDs to do weighted or exported sampling.
Besides that, preparation of the sampling will be well organised and verified
using the business rules set up in the applications.
This will ensure the
accuracy and consistency of the sampling.
SMDs can easily view the gradual changes in tabulation based on the
generated sets of data. This feature facilitates the SMDs to trace and rectify
errors that surfaced during tabulation.
Performance Management System enables top management to monitor the
progress of all activities pertaining to censuses and surveys. The status can
be updated by the SMDs and retrieved by top management or supervisors for
monitoring purposes. With such facilities more responses can be gathered
and that ensures the timing and the achievement of designated milestones.
4.3
Frame and SIDAP
The direct impact of NEWSS on frame and SIDAP is the improvement in
sampling method.
Prerequisite data obtained from OGAs such as CCM,
CIDB and EPF, as well as from GIS, provide DOSM with a consolidated view
10
of all frames and statistics of each census and survey area. These updated
and timeliness data help DOSM to formulate the right and accurate sampling.
The use of workflow in the system enables check and balance of data that are
updated. The outcome is that data are better quality and more accurate.
Sampling methods such as Dalenius Hodges Method can be used to
determine the optimum point of stratification. This essentially reduces the
manual process and human judgement. Besides that, other methods such as
Neyman Method, Equal Method, Proportional Method and Optimum Method
can be used for stratified sampling.
4.4
Central Repository
The Central Repository Database facilities will elevate the level of data
management in providing services to all customers. It will off load SMDs from
serving customers and as a result SMDs can concentrate fully on surveys.
Eventually the Central Repository will become the “National Statistics
Repository” to serve the country as the leading statistics organisation and
business intelligence users.
4.5
Dissemination of Statistics
Dissemination of statistics to external users by means of publications and
data request are served through an enterprise portal.
The enterprise portal is a framework for integrating information, people and
process across organisational boundaries.
It provides a secure unified
access point in the form of a web-based user interface. The portlet is a user
interface components that are managed and displayed in the web portal.
11
Customer management will become easy and service-oriented with the
provision of on-line request and on-line payment, embedded in the enterprise
portal.
4.6
System Development Skill Sets
The adoption of Fujitsu Framework-software components enables the system
developers to leverage on the new emerging enterprise-wide Java
technology.
The
Fujitsu
Framework-software
components
are
the
presentation tier for the http servers; business logic tier for portal framework;
service tier for application framework and ISS; interface tier for Java EE
platform; and resource tier for central repository and metadata.
The Fujitsu Framework can run on different operating system platforms and is
upward scalability. It has a lot of readily available library for the developers to
use and share.
4.7
ICT Infrastructure and Software
The ICT infrastructure and software are designed to sustain reliability,
availability and serviceability (RAS) of the equipment twenty-four by seven. It
has the capabilities of vertical upgrade and horizontal upgrade in the form of
clustering and consolidation.
5. RELEVANT
With NEWSS, the Department can meet responsibilities by providing data with
greater efficiency, quality, reliable, relevant and accessible to reevaluate the
effectiveness of government policy. Monthly Manufacturing(MM) data has been
12
assisting the government in the National Key Economic Areas (NKEA) which focuses
on the manufacture of products such as petroleum; electrical and electronics; palm
oil; and ICT. The Department has provided relevant data for The Performance
Management & Delivery Unit, Malaysia (PEMANDU) to oversee implementation and
assess progress of the Economic Transformation Programme and the Government
Transformation Programme to realise the goal of achieving a high-income nation
status by 2020.
In Malaysia, the manufacturing sector is the second most important industry after the
service sector and contributed 27.5% to the Malaysia’s Gross Domestic Product
(GDP) 4 . Thus, the availability of timely and accurate data is essential in the
formulation of government policy.
MM data are used as short-term indicators in the compilation of GDP. In addition, two
publications issued from MM data namely Industrial Production Index (IPI) and the
Monthly Manufacturing Statistics (PPB). IPP includes an index of manufacturing,
mining and electricity while PPB includes the sales figures, number of employees
and salaries and wages.
MM data are meant and useful to businesses by helping access to the business
market, analysis company performance to the current growth of manufacturing
industries, identifying the manufacturing sub-sector competitiveness (whether
domestic or export-oriented industries) and identify potential investment.
The researchers, administrators and the general public will also benefit from MM data
for research, analysis and generate manufacturing-related writing.
4
Department of Statistics Malaysia (2011) p5.
13
MM data are used for comparison between countries. Among international
organisations that are using economic data compilation are the United Nations
Statistics Division (UNSD), the Organisation of the United Nations, World Bank,
ASEAN Secretariat, the International Monetary Fund (IMF) and the United Nations
Economic and Social Commission for Asia and the Pacific (ESCAP).
NEWSS MM will help the Department to realise its responsibilities in meeting the
needs of the Government; companies and business associations; research agencies;
public and international statistical organisations. NEWSS MM is also used in the
dissemination of publications through the facility to download free of charge at the
website of both IPP and PPB publications since April 2010.
6. SIGNIFICANT
Among the significant highlights in NEWSS MM is the collaboration of members of
the system users in the State Office, Operations Centre, business companies and
headquarters. Integration process of investigation work includes updating the list and
profiles of businesses, the selection of the survey sample, address label printing
businesses, updating information visit outside (field work), survey data capture,
quality checks, generating table analysis (data mining) and publishing. MM final data
will be stored in a centralised repository for time series analysis.
The status of the investigation progress can be monitored by all the officers involved
in the Head Office and State Office including the top management of the Department.
This ensures compliance with the implementation schedule of the investigation and
publication can be removed in a timely manner. Before NEWSS MM was
implemented, all processes are done separately without going through the system,
except for the data capture process. The time taken to capture and data processing
14
can also be reduced. In addition, the operating costs of data collection in the field can
be reduced.
Mechanisms, such as workflow, functions checks, verification and approval functions
are provided in every work process flow in NEWSS MM. For example, the State
Office prior to the validation process (revised), verification (authentication) and
approved by the approving officer before sending the clean data to HQ.
The system will generate analysis reports and tables from clean and quality data to
enable officials to check and compare data. The Government then can use the
current and accurate reports and publications to make policies regarding
manufacturing.
The reports and publications can assist businesses to evaluate current economic
conditions and future prospects of their businesses. This information will assist the
businesses in increasing their productivity, competitiveness and retain ability. This
will indirectly generate economic activities and provide jobs to the people.
The integrated statistical system known as NEWSS MM is widely used throughout
the country including the Head Office, State Offices and the SMDs. Data was
collected from the field, processed through the validation and verification by State
Offices and the relevant divisions and submitted to HQ on online basis. Based on this
data, response rate calculation can be executed automatically and accurately.
7. EFFECTIVENESS
Effectiveness of the integrated statistical system (NEWSS) can be identified by:
15

Profiles and company information (of businesses/enterprises) updates were
performing centrally in the NEWSS database. The 2.8 million profiles and
company
information
(of
businesses/enterprises)
are
the
list
of
establishments registered under legal entity operating in Malaysia.

Use of GIS in NEWSS through its integrated system has helped the
Department to monitor all areas covered by the census and surveys. Its helps
to increase the accuracy of the sampling method used in tandem with the
definition of the studied population and as a result the outcome of the study
will be of better quality and accuracy. To date, a total of 74,600 enumeration
blocks have been updated in the GIS.

Profile of the survey, industry coverage and sample survey can be accessed
through a single system. Uniformity in the use of codes and classifications in
NEWSS has facilitates the system to operate smoothly and orderly according
to the Department standards.

Sample cases were distributed evenly among the field officers involved in
every State Offices. This allows efficient distribution of workloads and
optimum usage of resources. Currently, there is an average of 5,000
establishments under the responsibility of 550 field officers in the Department.

NEWSS MM provides offline and online data capture in order to expedite the
processes as well as the function of uploading secondary data capture.

Reports and tables are generated by using BI and Extensible Business
Reporting Language (XBRL) for the purpose of analysis. BI allows the
generation of reports in various formats and criteria as required by
stakeholders.
16

Online dissemination of quality data analyses to stakeholders are met on
every 28th of the month according to the format and time required.
Publications can be done in line with international statistical publications.

Enterprise portal is a facility for the dissemination and data request where it
provides an interface for an organisation or the public to obtain statistical
data. Customer services in the Department will become easy and will be an
online customer oriented services. All payment transactions for data
requested can be made online through this interface.
8. ACHIEVEMENT
The system was developed using the approach of Life Cycle System Development
methodology. Implementation was made through the process of getting User
Requirements (URS), System Function Design (FDS) and System Design
Specification (SDS). It also has undergone three stages of testing to ensure the
quality assurance system meets the required standards. The three (3) stages of
testing are System Integration Testing (SIT), Provisional Acceptance Test (PAT) and
Final Acceptance Test (FAT).
Since NEWSS was implemented, the department has received recognition from
agencies within and outside the country. For example the Central Bank of Malaysia
has benchmark this system to guide the development of their statistical system.
In addition, the Department has received visits from foreign delegations such as
Thailand and Bangladesh for a pilot study of an integrated statistical system. This
proves that NEWSS are reliable in terms of planning, implementation and outcomes.
17
The Department has also been given the opportunity to share experiences relating
NEWSS internationally. Among other meetings and forums attended were the
Government Information Forum 2009 in Singapore on 4th-5th March 2009, meeting
on the Management of Statistical Information Systems (MSIS 2010) in Korea on 26th29th April 2010 and the Expert Group Meeting in Thailand on 20th-22nd June 2011.
NEWSS is an inspiration for the countries to improve their statistical system.
To ensure high quality standards are maintained at all times, the Department, has
formed several Committees to monitor the project:
i.
ICT Steering Committee (SC NEWSS) is chaired by the Chief Statistician and
its members are the directors in the Department. The meeting is held four (4)
times per year.
ii.
Group projects (PIT) is chaired by ICT Manager and members are the
directors of respective divisions in the Department. The meeting is held on a
monthly basis
iii.
NEWSS monitoring committee is chaired by the Senior Director of the
Information Management Division and its members are the owners of the
modules and ICT Officers. The meeting is held on a monthly basis.
iv.
NEWSS Technical Committee is chaired by the Senior Director of the
Information Management Division and its members are NEWSS ICT Officers
and suppliers. The meeting is held on a monthly basis.
v.
Team Committee is chaired by NEWSS Senior Director of Information
Management and its members are the NEWSS ICT Officers. The frequency is
once every 2 weeks.
18
9. WAY FORWARD
The Malaysian Prime Minister, Datuk Seri Najib Tun Razak has highlighted the
importance of public services to shift to a new paradigm that features a combination
of productivity, creativity and innovation. Therefore, the Department responded to
empowering and expanding existing NEWSS to several implementation phases in
order to strengthen the impact of the Department’s productivity through NEWSS
Phase II and Phase III. This initiative indirectly supports the Department's main
innovation to integrate the entire existing system in the Department in one system to
support NEWSS as a one stop center of statistical information.
As a producer of national statistics, the Department is ready and willing to do
collaboration and data sharing across agencies (horizontal) to produce high-impact
productivity statistics and support the national development policy. NEWSS as a
catalyst and detonator in developing statistical data production will be a leading edge
of the collaboration initiative. This direction supports G2G (Government to
Government) and G2C (Government to Citizen) thereby increasing economic growth.
NEWSS is also successful in meeting the aspirations of the government to increase
paperless works which directly enhance the appreciation and practice of widely ICT
green technology in government services.
NEWSS as a leading integrated statistical system has been designed to support a
centralised repository to enable users to access data and time series data. This
centralised repository development uses tools that enable users to perform data
mining directly for the purpose of statistical analysis.
The Department also supports the transformation of government services and online
payment services by 2015. Online payment services is a module for the purchasing
19
or subscription of publications online for the end user, such as researchers, analysts
and economic planners, is ready and awaiting approval from the relevant ministry in
Malaysia. This module will ensure operational cost savings and manual printing of
publications.
Development and operation of NEWSS are based on GSBPM adopted by
international standards. However, the evaluation component is yet to be use in
NEWSS. At present, the Department is taking action to develop the components of
this assessment in all its operating modules. This implementation of full GSBPM will
ensure that data produced by the Department are relevant to internal and external
users, and accepted with full confidence and recognised internationally.
10. CONCLUSION
NEWSS is the new ICT initiative which facilitates the automated processes that
interact with the services within DOSM as well as external services.
The
implementation of NEWSS definitely has the impact on the stakeholders; SMDs;
frame and the use of sampling methods; central repository with its intelligence;
dissemination of statistics via enterprise portal; human capital expertise in system
development; and ICT infrastructure and software. Its impact provides the baseline
for DOSM to move forward. Beyond this, NEWSS will continue to expand in realising
the vision and mission of DOSM.
11. ACKNOWLEDGEMENTS
The innovations proposals opportunity with ESCAP Statistics Division was a great
chance for learning and professional development for myself. Hence, I consider
myself as a very lucky young statistician as I was given the opportunity to be part of
ESCAP Innovation Showcase 2014.
20
Without the great support from Dr. Haji Abdul Rahman Hasan, Malaysia’s Chief
Statistician, it will not possible for me to write this innovation proposal. Madam Siti
Haslinda Mohd Din and Madam Che Norma Murad, Director and Principle Assistant
Director of Methodology and Research Division, DOSM has always been very
responsive in providing necessary information and guidance.
My thanks and appreciation to my family and colleagues for their assistance in
preparing and finalising this proposal within the limited time frame.
12. REFECENCES
Department of Statistics Malaysia. (2009). Department of Statistics Malaysia: 60
Years Statistics.
Department of Statistics Malaysia. (2010). Department of Statistics Malaysia:
Strategic Plan 2010-2014.
Mazlan bin Sulong. (2011). National Enterprise-wide Statistical System: Impact to
Department of Statistics Malaysia.
Retrieved 28 October 2012 from
www.unescap.org/stat/MSIS/egm-Jun2011/session1-Malaysia.pdf
Department of Statistics Malaysia. (2011). Department of Statistics Malaysia:
Executive Summary, AIICT Report 2011.
21
13. APPENDIX
22