An Evaluation of Open
Source GIS Routing Tools in
Direct Vaccine Delivery in
Kano State, Northern Nigeria
KehindeA. Adewara, Snr. GISCoordinator
Presentation to FOSS4G Seoul 2015 Conference
18 September 2015
ACKNOWLEDGMENT AND CONTRIBUTIONS
Before I begin permit me to start my presentation with acknowledgment and contribution. It is
absolutely necessary that I thank you all for finding time to come, to every one who had
contributed in every little ways to the success of the FOSS4G Seoul 2015 conference, for the
travel grant award I have received as well as selfless support from the Management and staff
of eHealth Africa (eHA), Nigeria.
Outline
1) Introduction
2) Justification for the Research
3) Conceptualization
4) Approach/Methodology
1.Drive Test Survey
2.Desktop Routing Estimation
3.Multi Criteria Ranking
5) Research Outcomes
1.Desktop Routing Outputs
2.Comparative Outcomes of Error Margin
3.Scoring/Ranking
6) Research Findings, Conclusion and Recommendations
Open Source GIS Routing Tools
Introduction
• The enormous challenges associated with vaccine delivery to support routine immunization
in northern Nigeria have been tackled by government and development partners. Such
immense efforts need to be complemented with a measure that is cost effective and
efficient in view of dwindling resources.
• Hence this article took advantage of the enormous benefits of open source resources to
addressing some of these challenges. The comparative capability of selected number of
open source GIS routing tools were investigated in this regards.
-
- A pragmatic support for routine immunization and health care delivery in Northern Nigeria.
1
Open Source GIS Routing Tools
Justification for the Research
• This research was influenced by the
1. The need to eliminate software license cost and optimize vaccine delivery activities using
reliable, credible and cost effective open source routing tools
2. Need to take advantage of emerging open source GIS routing tools,
3. Need to build confidence in the use of open source GIS routing tools as a result of
proliferation
4. Need to embrace open source GIS routing tools from a cautious, pragmatic and objective
perspective (Graser, Straub, & Dragaschnig, 2015).
5. Need to determine the most appropriate routing tools in view of its consequence on
decision making outputs.
-
- A pragmatic support for routine immunization and health care delivery in Northern Nigeria.
2
Open Source GIS Routing Tools
Conceptualization
•
Drive test and desktop routing techniques were
based on shortest path concept or graph theory
which considered travel time as constrains or
impediment.
•
It is evident that some applications use Euclidean
Fig. 1 - Euclidean Route
route calculation as the basis for routing (fig. 1).
The output from this calculation may not be
reflective of ground reality. Hence spatial route
calculation helps to address this limitation (fig. 2).
Fig. 2 - Spatial Route
-
- A pragmatic support for routine immunization and health care delivery in Northern Nigeria.
4
Open Source GIS Routing Tools
Approach/Methodology
• In determining the most appropriate open source GIS routing tools, the following
techniques were adopted
1. Drive Test Survey,
2. Desktop Routing Estimation and
3. Multi Criteria Ranking
-
- A pragmatic support for routine immunization and health care delivery in Northern Nigeria.
3
Open Source GIS Routing Tools
Drive Test Survey
• Drive test survey has been used in different studies as a control measure depending on
study objective and scope. It is widely used in telecommunication studies (Sanders, Linder,
Pratt, Dickherber, Floyd, & Pickering, 2004; Boxberger, Lawver, & Smithey, 2010). However
it was used in this case as a technique for acquiring baseline information which serves as a
benchmark for determining accuracy among different routing tools.
• It may be practically impossible to conduct drive test survey for all the facilities in the state
due to logistic challenge, hence a representative sample size of 326km was considered
with minimum error of margin using an online sample calculator. The sample size was
determined at 95% confidence level and narrow confidence interval (CI) of 5 (Myles,
Douglas, & Eric, 2013).
-
- A pragmatic support for routine immunization and health care delivery in Northern Nigeria.
5
Open Source GIS Routing Tools
Drive Test Survey – Cont’d
Table 1 - Sample Distribution
Zone
Farthest
Distance
(km)
Sample
Size
(km)
Number
of
facilities
served
Distant
Covere
d (km)
Rano
196
130
2
115.06
Bichi
90
73
3
130.4
151
123
5
166.28
437
326
10
411.74
• The sample population (437km) was determined as
proximity/distance between the state cold store and the
farthest facility in each zone (table 1). Hence,
approximately a buffer distance 85.74km above the
sample benchmark (326km) was covered in the survey.
The actual distant covered was 411.74 km for 10
facilities.
Nass
araw
a
/Wud
il
Total
-
- A pragmatic support for routine immunization and health care delivery in Northern Nigeria.
6
Open Source GIS Routing Tools
Drive Test Survey – Cont’d
• The travel distance and time during the drive test
survey was determined using mobile mapping GPS
enabled solution called OsmAnd tablet device (fig. 3).
The device is made up of a detailed base map,
navigation tool and a plugin called trip recording. The
plugin is the app used for recording distances covered
during the drive test survey.
Fig. 3 – OsmAnd Interface
-
- A pragmatic support for routine immunization and health care delivery in Northern Nigeria.
7
Open Source GIS Routing Tools
Drive Test Survey – Cont’d
Table 2 – Kano State Primary Health Facilities targeted
for Routing Destinations
S/N
Primary Name
Settlement
Local Government
Name
1
Gwarzo General Hospital
Tudun Burtu
Gwarzo
2
Shanono Comprehensive
Health Center
Unguwar
Hakimi
Shanono
the state cold store and the 10
3
Tsanyawa Comprehensive
Health Center
Yan Amar
Tsanyawa
selected health facilities (table 2).
4
Gezawa General Hospital
Kuka
Gezawa
5
Kunya Primary Health Center
Panda Model Primary Health
Center
Kunya
Minjibir
Albasu
8
Hungu Primary Health Center
Kachako Primary Health
Center
Wudilawa
Gidan Tudu
Yamma
Unguwar
Tsamiya
9
Falgore Health Post
Tudun Ningi
Doguwa
10
Dadin Kowa Health Post
Tasha
Doguwa
• Drive test and routing estimation
were conducted to determine actual
travel distant measurement between
6
7
-
- A pragmatic support for routine immunization and health care delivery in Northern Nigeria.
Albasu
Takai
8
Open Source GIS Routing Tools
Table 4 - GIS Routing Tools
Desktop Routing Estimation
S/N
GIS Routing Tool
Platform Type
• The desktop routing estimation was used
1
QGIS Road Graph plugin (QRG)
Desktop
2
Open Street Routing Machine (OS RM)
Online
3
Google Maps Engine (GME)
Online
(5) open source GIS routing tools (Haklay,
4
GraphHopper (GH)
Online
2010). It was conducted for all the
5
OsmAnd
Mobile
to compare routing outputs from the five
delivery routes connecting the 10 facilities
during the drive test survey. These tools
were primarily classified as online, mobile
and desktop routing tools (see table 4).
-
- A pragmatic support for routine immunization and health care delivery in Northern Nigeria.
9
Open Source GIS Routing Tools
Desktop Routing Estimation (cont’d)
• The delivery route is an optimum route (with shortest possible distance) connecting
the state cold store to the 10 health facilities across the state.
• Certain assumptions were made and certain limitations were unresolved. It is
assumed that all routing tools consider travel by vehicle and fastest (not shortest)
path option.
-
- A pragmatic support for routine immunization and health care delivery in Northern Nigeria.
10
Open Source GIS Routing Tools
Desktop Routing Estimation (cont’d)
• Limitations
include
inability
to
handle
traffic
conditions as well as inability to consider same
routes among all the routing tools due to insufficient
base map.
• These limitations and other factors (geo positioning
accuracy,
model)
limitations
were
the
of
inbuilt
reasons
for
routing/network
the
observed
discrepancies in the routing outputs using same
base map (see figure 4).
Figure 4 - Different Outputs from Routing
Tools using same base map
-
- A pragmatic support for routine immunization and health care delivery in Northern Nigeria.
11
Open Source GIS Routing Tools
• Multiplicative weighing score was able to
Multi Criteria Ranking
• In view of the limitation associated with using
error margin condition between drive test and
routing estimation outputs as a sole factor for
determining best routing tools, it is imperative to
consider other ‘win and loose’ advantages
associated with these tools.
• There are several methods of conducting such
overcome certain limitations with additive
weigh. It is commonly used in the
decision
making
hierarchy
of
world
leading organizations such as UNDP in
developing annual HDI, an instrument
used for ranking nations based on human
per capita income, life expectancy and
education (UNDP, 2010).
ranking well documented in the literature, most
common is the additive weighing factor approach
(Tofallis, 2014).
Figure 5 – Multi Criteria Ranking Support
-
- A pragmatic support for routine immunization and health care delivery in Northern Nigeria.
12
Open Source GIS Routing Tools
Defining Ranking Criteria & Its Measures
• Those criteria considered in this research –
i. Routing Output/Drive Test Error margin
Table 4 - Definition of Ranking Criteria
Criteria
1. Routing Output/Drive
Test Error margin
Measurement
Cumulative margin of error
between drive test and routing
outputs.
Analysis of % coverage of vector
base map compared with imagery,
considering average of five 4x4
sqkm clusters each in urban and
rural areas (Haklay, 2010).
Unit
meters
3. Capacity For Multiple
Routing
The capacity to support multiple
routing was determine by the
options provided by the app
designed feature.
Yes or no
4. Support For Traffic
Input
Determine by onboard designed
feature that support traffic
modeling
Designed platform for
implementation
Yes or no
Provision for choice of alternatives
Yes or no
2. Base map
content/completeness
i. Capacity For Multiple Routing
ii. Base-Map content/completeness
iii. Support For Traffic Input
iv. Routing Platform
v. Alternative Route Option
• The measure of these criteria (table 4) were
based
on
established
techniques
(Haklay
and
referenced
method).
5. Routing Platform
Expert
judgement was used to determine % coverage
6. Alternative Route
Option
%
Mobile, desktop,
online/web
in each cluster.
-
- A pragmatic support for routine immunization and health care delivery in Northern Nigeria.
13
Open Source GIS Routing Tools
Threshold Definition and Weighing
• The thresholds for each of the criteria
were
defined
and
ordinal
weights
• Hence the average (60%) of the two was used for
assigning weight of 1 for greater than 60% and 0
for less than.
Table 5 - Normalized Thresholds
assigned (table 5).
• Threshold for base map completeness
was derived from average consensus
option expressed in percentage for both
five sample clusters in rural and urban.
The threshold values for rural and urban
clusters were determined as 50% and
70% respectively by consensus.
-
Criteria
Value Range
1. Routing
Output/Drive Test
Cumulative Error
margin
2. Base map
content/completenes
s
3. Capacity For
Multiple Routing
4. Support For Traffic
Input
46.17 – 67.52 km
5. Routing
Platform
6. Alternative Route
Option
0 - 100%
Yes/No
Threshold
condition
> 56.62
< 56.62
≤ 60
≥ 60
Assigned
Weight s
0
1
0
1
No
Yes
No
Yes
0
1
0
1
Mobile, desktop,
online/web
Mobile/Online
Desktop
0
1
Yes/No
No
Yes
0
1
Yes/No
- A pragmatic support for routine immunization and health care delivery in Northern Nigeria.
14
Open Source GIS Routing Tools
Table 6 - Desktop Routing Outputs
Research Outputs: Desktop Routing
• The outcomes derived was contrary to
expectation that routing tools using same
base map would have same outputs. QRG,
GH, OSRM and OsmAnd use same base
map yet the routing outputs varied between
467.26 km (min) for QRG to 488.62 km
(max) for OSRM (table 6). This represents
about ± 21.36km discrepancy which is
about 4% of the entire distance coverage.
• This may be understandable for GME
because different base map was used.
-
Routes
State cold storeGwarzo General
Hospital
Gwarzo Gen HospShanono
Comprehensive
Health Center
Shanono CHCTsanyawa
Comprehensive
Health Center
State cold storeKunya Primary
Health Center
Kunya PHC-Gezawa
General Hospital
State cold storeFalgore Health Post
Falgore HP-Dadin
Kowa Health Post
State cold storeHungu Primary
Health Center
Hungu PHC-Panda
Model Primary
Health Center
Panda MPHCKachako Primary
Health Center
Total
Drop
Order
QRG
GME
Distance
(km)
Distance
(km)
Distance
(km)
Distance (km)
Distance
(km)
1 Bichi
69.71
71.00
71.19
71.20
73.70
2 Bichi
14.97
15.40
15.63
15.60
15.60
3 Bichi
30.41
32.00
46.96
30.60
23.90
1 Nassarawa
35.90
36.00
36.08
36.10
36.00
2 Nassarawa
29.51
29.40
29.56
29.60
29.50
1 Rano
132.51
135.00
135.00
143.00
143.00
2 Rano
52.77
58.80
52.62
52.60
53.00
1 Wudil
70.54
70.60
70.65
70.70
70.30
2 Wudil
5.55
5.40
5.54
5.54
5.53
3 Wudil
25.39
467.26
25.20
478.80
25.39
488.62
25.40
480.34
25.30
475.83
Zone
- A pragmatic support for routine immunization and health care delivery in Northern Nigeria.
GH
OS RM
OsmAnd
15
Open Source GIS Routing Tools
Research Outputs: Desktop Routing (cont’d)
• The choice of different routes used by the routing
tools during the desktop routing exercise (figure 6)
has been responsible for the discrepancy noted in
the desktop routing outputs (table 6). This choice
was largely influenced by
1. Geo-positioning accuracy of the routing tools,
2. Base
map
quality
in
terms
of
content
and
completeness,
3. In-built routing algorithms
Figure 6 – Non uniform Choice of delivery routes during
Desktop Routing Estimation
-
- A pragmatic support for routine immunization and health care delivery in Northern Nigeria.
16
Open Source GIS Routing Tools
Research
Outputs:
Comparative
Error
Table 7 – Comparative Discrepancy Outcomes
Margin
• There were discrepancies in the error
margin reported between the drive test
average and routing outputs (table 7).
• Factors responsible for these errors are not
limited to the use of different routing
algorithms (a case for future investigation)
but also the base map quality in terms of
content (completeness) as well as geopositioning accuracy of the routing tools.
-
Routes (Origin-Destination)
State cold store-Gwarzo
General Hospital
Gwarzo Gen Hosp-Shanono
Comprehensive Health Center
Shanono CHC-Tsanyawa
Comprehensive Health Center
State cold store-Kunya
Primary Health Center
Kunya PHC-Gezawa General
Hospital
State cold store-Falgore
Health Post
Falgore HP-Dadin Kowa
Health Post
State cold store-Hungu
Primary Health Center
Hungu PHC-Panda Model
Primary Health Center
Panda MPHC-Kachako
Primary Health Center
Sum Total
Drop
Order
Zone
QRG
Error
Margin
(km)
GME
Error
Margin
(km)
GH
Error
Margin
(km)
OS RM
Error
Margin
(km)
OsmAnd
Error
Margin
(km)
1 Bichi
4.85
3.56
3.37
3.36
0.86
2 Bichi
0.52
0.09
-0.14
-0.11
-0.11
3 Bichi
31.92
30.33
15.37
31.73
38.43
1 Nassarawa
11.33
11.23
11.15
11.13
11.23
2 Nassarawa
1.98
2.09
1.93
1.89
1.99
1 Rano
9.98
7.49
7.49
-0.51
-0.51
2 Rano
1.26
-4.77
1.41
1.43
1.03
1 Wudil
5.33
5.28
5.22
5.18
5.58
2 Wudil
0.00
0.15
0.01
0.01
0.02
3 Wudil
0.36
67.52
0.55
55.99
0.36
46.17
0.35
54.45
0.45
58.96
- A pragmatic support for routine immunization and health care delivery in Northern Nigeria.
17
Open Source GIS Routing Tools
Table 8 - Normalization Outcome
Scoring/Ranking Outcome
• In view of this outcome (table 8) with
reference to normalized threshold table
(5), scoring was derived based on
frequency of 1 occurrence while the
highest frequency of 4 was ranked 1st
and least score of 1 was ranked 4th.
• Based on the overall criteria, QRG
Ranking Criteria
1.
Routing
Output/Drive Test Error
margin
2.
Capacity For
Multiple Routing
3.
Base map
content/completeness
4.
Support For Traffic
Input
5.
Routing Platform
6.
Alternative Route
Option
Score
QRG
GME
OS
RM
GH
OsmAnd
0
1
1
1
0
1
1
1
0
0
1
0
1
1
1
1
1
0
0
0
0
0
0
0
0
0
4
1
3
0
3
0
2
0
1
emerged 1st ranked because it’s the only
Table 9- Ranking Outcome
one
that
platform
supported
and
traffic
desktop
modelling
routing
while
OsmAnd was last ranked. Both GME and
GH were ranked 2nd (table 9).
-
Routing Tools
QRG
GME
GH
OS RM
OsmAnd
Score
4
3
3
2
1
Rank
1
2
2
3
4
- A pragmatic support for routine immunization and health care delivery in Northern Nigeria.
18
Open Source GIS Routing Tools
RESEARCH
FINDINGS,
CONCLUSION
AND
RECOMMENDATIONS
• It is not surprising to find significant variation in the routing outputs
• Scheduling delivery and routing within Kano metropolis
of tools using different base maps (OSM and google) because
was largely constrained by lack of traffic details, hence the
different choice of routes are chosen. But to discover a significant
routing outputs which considered speed limit as an input
variation in routing outputs of routing tools using same OSM base
for travel time estimation was not consistent with travel
map is worrisome. It is an indication that there is an inconsistency
time output derived from drive test survey. Thus it is
in the routing algorithm used. The outcome of drive test clearly
anticipated
shown that GH tool has a better routing algorithm with lowest
investigating metropolitan traffic. Speed limit consideration
cumulative error margin. Future research is hereby encouraged to
is still valid for interstate and rural areas.
investigate how road graph plugin of QRG would integrate GH
routing algorithm for better performance.
that
future
research
would
consider
• QRG is at the moment constrained by inability to handle
batch routing. All the routing tools considered are equally
• It is important to emphasize that the GME better drive test
unable to do batch routing. Hence it is expected that
performance over QRG was just because the selected delivery
future research would focus of developing batch routing
routes were largely within urban area where GME base map
component and to integrate other features such as
content is relatively good. It is thus recommended that outstanding
alternative route options.
GME features such as alternative routing option should be
considered in QRG integration.
-
- A pragmatic support for routine immunization and health care delivery in Northern Nigeria.
19
THANKS FOR
COMING
ANY
QUESTIONS?
An Evaluation of Open Source GIS Routing Tools in Direct Vaccine Delivery in Kano State, Northern Nigeria /
18 September 2015