Data Management &
Data lifecycle
•
•
•
•
Survey Conception
Data System Architecture
Data collection management
Data Analysis & Dissemination
Introduction
Type of info per usage
Global
reports
Project
report
Outreach
material
Indicators
(Focus)
Registration
X
X
X
X
IDP profiling
Protection Incident
monitoring
Protection situation
monitoring
Population movement
monitoring
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
Sectoral assessment
Partners activities
monitoring
Absorbtion capacity
evaluation
Data Management & Data Lifecycle
X
X
X
X
X
From Data.. to Information
Introduction
Operation Data Manager should be involved in all
the steps of a “Data Lifecycle”.
Any break of this cycle ends with the
failure of the system :
Survey
Conception
Data Analysis
&
Dissemination
Data
System
Architecture
Data
Collection
management
Data Management & Data Lifecycle
•
A data collection form that is illdesigned either because it does not
satisfy operational information
requirements or is flawed from a
technical standpoint
•
A well designed survey with a poorly
designed and therefore poorly
maintained database
•
A structurally well designed database
with no data, as data collection cycles
have not been integrated/respected
•
A well populated database without
implemented reports and queries and
therefore no output
Before the Form…
Survey Conception
• Avoid reinventing the wheel – check what has
been designed and piloted before
• Consultation with all stakeholders – avoid
duplication of efforts and assessment fatigue of
beneficiaries
• Layers of data collection
• Collect Simple base reference data first
• Embark on detailed info based on samples defined from
the base reference
• Data collection frequency should vary according to how
frequently the phenomena being tracked or measured
changes
Data Management & Data Lifecycle
Good practices for Data Collection Forms
Survey Conception
1. Questionnaires used in survey research
should be clear and well presented.
2. Think about the form of the questions,
3. Keep the survey as short as possible.
4. Make definitions of data elements consistent
with standard definitions and analytic
conventions
5. Plan clearly how answers will be analyzed.
6. Test the survey for “understandability” and
respondent effort through focus groups
Data Management & Data Lifecycle
Data System Architecture
Data model
•
•
Data models are the key for
interoperability (i.e easy data
exchange with partners)
Implementing partners
should not have to draft and
decide on a core data model;
it should be the same
everywhere and just adapted
locally where necessary;
support (guidelines) need to
be there
Site / community
Beneficiary
registration
Assessment
Demographics
Bio Data
Vulnerability
Needs
Base
indicators
Delivered
Assistance
Site
Organization
Importance of a common
referential
Data Management & Data Lifecycle
Infrastructure
Inventory
Who’s doing
what where?
Project activities description
Performance Indicators
Activity monitoring
•
Multi sectoral
assessment:
-Health
-Education
-Water
System architecture
Data System Architecture
• Building an Interface for data collection:
• Mobile
• Offline desktop
• Web/Server based
• OCR* ready form (can be scanned)
• Integration of external data source (ETL**)
• Offering analysis capacity (OLAP*** and Stats)
* Mechanical or electronic translation of scanned images of handwritten, typewritten or printed
text into machine-encoded text
** Extract, transform, and load (ETL) is a process in database usage that involves Extracting
data from outside sources, Transforming it to fit operational needs (which can include quality
levels), Loading it into the end target (database or data warehouse)
*** An OLAP (Online analytical processing) cube is a data structure that allows fast analysis of
data.
Data Management & Data Lifecycle
Reports are part of the data system
Data System Architecture
Queries and tools to extract data from the
databases need to be designed along with
the database
Must give abilities for reporting officers to
- Set up queries and reports without high
level IT knowledge
- To be clear on the standard indicators
these queries should be based on
Data Management & Data Lifecycle
Data collection strategies
Data collection management
• Direct coordination with partners
•ex : Somali protection cluster
• Establishment of a « data collection project »
•ex : UNOPS Goma
• Specific Contract with a dedicated partner
•Ex: CartONG in Uganda
Data Management & Data Lifecycle
Data collection management
Implementation matrix
Dedicated
Implementing
UNHCR
Project partner
partner
Direct Government
Registration
IDP profiling
Protection Incident
monitoring
Protection situation
monitoring
Population movement
monitoring
Sectoral assessment
Partners activities
monitoring
Absorbtion capacity
evaluation
X
X
X
X
X
X
X
X
X
X
Avoid conflict of interest
Data Management & Data Lifecycle
X
X
X
X
X
X
X
X
PDF reports and maps
• Targets mostly local
partners and decision
makers
• Can be disseminated
through
• mailing list (cf
Somali protection)
• Google group (cf
Goma Update)
• Website (cf
ReliefWeb)
Data Management & Data Lifecycle
Data Dissemination
GeoPortal and Open Data API
Data Dissemination
GeoPortal:
• is a tool to ensure institutional memory and
“Master Data” management
• Can be a tool for desk officers to visualize a
situation and use map extracts in their reporting
Data API:
• Can be used for global dissemination: cf
Worldbank Data API or Google public data
• Offers material for data journalism (e.g. computer
assisted reporting on data through journalists)
Data Management & Data Lifecycle
Data, Law & License
For all data sets that do not
fall under the “Guidelines
for the Regulation of
Computerized Personal
Data Files” (for instance
protection data) ….
http://www.unhcr.org/refworld/pdfid/3ddcafaac.pdf
…. The “Open database
license” (ODBL) can give
a legal frame to all our
data collection
activities
http://www.opendatacommons.org/licenses/odbl/1.0
Data Management & Data Lifecycle
Data Dissemination
Providing support for the 4 phases of the
process
Conclusion
4 specific types of expertise that are difficult to combine in one profile:
•Statistician/Analyst: Creating a questionnaire and compiling
analyzing the resulting statistics
• IS Architect: Building the information system
• Manager: Managing the stakeholder consultation process during
the design phase, the collection in the field and dissemination of
results
• Data journalist: Developing sound and sexy reports
Need to find where are the gap among the “Operation Data
Management” officers network
Need to define the training & support need for each of those specific
domains
Data Management & Data Lifecycle