Networked Systems in Developing Regions
Challenges and Opportunities
Lakshminarayanan Subramanian
Courant Institute of Mathematical Sciences
New York University
Joint work with many fantastic collaborators!
(Courant, NYU Med, NYU Wagner, UC Berkeley, Intel,
Univ of Washington, Univ of Colorado, Amrita University,
Aravind Eye Hospitals)
Disclaimers of a realist…

“I am not a philanthropist”

“NGOs are great, but may not scale or be sustainable”

“Information and Communication Technologies (ICT) can
only be an aid but cannot alleviate poverty”

“I believe in cell-phones, not yet in $100 laptops”
2
Causes of poverty

Lack of awareness/opportunity

A negative earnings cycle

Serious healthcare problems

Prolonged debt

Inheritance from family/society
3
Strategies to eliminate poverty

Jeffrey Sachs: “Provide aid to the underprivileged”

William Easterly: “2.3 trillion dollars of aid, nothing
much to show. Promote homegrown solutions”

C.K. Prahlad: “Tap the fortune at the bottom of the
pyramid”

Amartya Sen: “Promote development at the rural level”
4
The Bottom: A Brief Description





3-4 billion people with per-capita equivalent
purchasing power (PPP) less that US$2,000 per year
Could swell to 6-8 billion over the next 25 years
Most live in rural villages or urban slums and shanty
towns—movement towards urbanization
Education levels are low or no-existent (especially
for women)
Markets are hard to reach, disorganized, and very
local in nature
5
The cost of being Poor
Bombay area:
Credit (APR)
Dharavi
Warden Road
(shantytown)
600-1000%
12-18%
Ratio
60-75x
Water (100 gal)
$0.43
$0.011
37x
Phone
(cents/min)
Diarrhea Meds
4-5
2.5
2x
$20
$2
10x
$0.28
$0.24
1.2x
Rice ($/kg)
6
Even the Very Poor Spend

Dharavi, one of the poorest villages in India:





Even the poorest of the poor in Bangladesh:


85% have a TV
50% have a pressure cooker
21% have a telephone
… but can’t afford a house
devote 7 percent of income to communications services
(GrameenPhone)
These are valid markets…
7
ICT: A Big Missing Piece

Technology can impact everyone


“Bottom of the Pyramid”
Not just Internet access:


Enable profitable businesses

Must be scalable and sustainable





Health, education, government, commerce
Poor are a viable market
Focus on income creation, supply chain efficiency
Not charity, not financial aid
Promotes stability, entrepreneurism and social mobility
First World technology is a bad fit

New research agenda
8
How can ICT help?

Communications


Awareness, access to external world, phone calls
Healthcare

“Where there is No Doctor?”: Rural healthcare system




Finance


Telemedicine/consultation
Continuing Medical Education for Health-workers
Low-cost diagnostic tools
Microfinance audit, insurance schemes
Education

Educational modules, distance learning
9
Other important areas
Commerce
 Supply Chain
 Agriculture
 E-Governance

10
Where has ICT helped so far?
Very few ICT based efforts worldwide have led
to a large-scale success
 ICT adoption successes



Cell-phone, Radio, TV
Why?




Figure out what they need
Never thrust a technology
Economics not in favor
Make change a gradual phenomenon!
11
A five-step approach





1.
2.
3.
4.
5.
Identify basic real-world problem
Where can ICT help?
Research challenges?
What is the appropriate solution?
Deployment?
12
ICT research
challenges
Network connectivity is key!
Traditional wire-line connectivity solutions are
not economically viable!
 Potential options



Develop new low-cost connectivity solution!
Leverage existing low-bandwidth wireless solutions


Cellular, Satellite, CDMA450, WiMax
Intermittent links are a fact of life




Budget constrained links
SMS
Power outages
Physical transportation links
14
Research Challenges

Low-cost high-performance network connectivity



Intermittent distributed systems



Leveraging the next-generation cell-phone
Redesigning applications to tolerate intermittency
“Interactive applications” in low bandwidth environments


WiLDNet: WiFi-based Long Distance Networks
Wireless Mesh Networks
What can you do using 30 Kbps bandwidth?
Application specific research challenges

Security, User-interface, NLP, Image processing etc.
15
Research Challenges

Low-cost high-performance network connectivity



Intermittent distributed systems



Leveraging the next-generation cell-phone
Redesigning applications to tolerate intermittency
“Interactive applications” in low bandwidth environments


WiLDNet: WiFi-based Long Distance Networks
Wireless Mesh Networks
What can you do using 30 Kbps bandwidth?
Application specific research challenges

Security, User-interface, NLP, Image processing etc.
16
Rural Connectivity in Developing Regions
Optical
Fiber
Village
Kiosk
Rural
Clinic
3 Com
Town
Current solutions:

Wired

Village
Kiosk


Rural
Clinic
School


Dial-up/DSL
Optical fiber
Cellular
VSAT
WiMax
17
WiFi-based Long Distance Networks


WiLD links use standard 802.11
radios
Longer range up to 150km



Directional antennas (24dBi)
Line of Sight (LOS)
Why choose WiFi:

Low cost of $500/node




Volume manufacturing
No spectrum costs
Customizable using open-source
drivers
Good datarates

11Mbps (11b), 54Mbps (11g)
18
WiLD Deployments
Bay Area:
• 7 links up to 45km
• Experimental testbed
India at Aravind Hosp:
• 12 links up to 15km
• 9 Rural Clinics
• 25,000 patients/yr
• Scaling to 50 clinics
Ghana:
• 5 links up to 100km
• Electronic Library
Aravind Network, Theni, India
Other Projects:
Digital Gangetic Plains
• 30 links
• upto 39 km
• 802.11 APs
Akshaya Wireless
• 400 nodes
• 2-25 km
• commercial APs
CRCNet,New Zealand
• 17 links
• upto 13 km
• Soekris SBCs
19
Akshaya: A Case study
Joint project between the State of Kerala in
India and Tulip IT
 Wireless IP network set up in the district of
Malappuram – 630 eCenters in all
 Backhaul and last mile links are based on
proprietary technologies
 1 center for every 2000 families

20
Total Costs for network
deployment
WiFi/WiMax is the most economically viable solution
Fiber/WiMax is the least economically viable
21
Experience with WiLD Networks


In the field, point-to-point performance is bad
On a 60km link in Ghana


We get 0.6 Mbps TCP vs 6 Mbps UDP
On a relay (single channel)

We get only 2 Mbps TCP
22
WiLDNet Design Overview

Fix 802.11 protocol problems

Replace CSMA -> TDMA

Enforce synchronization of multiple links

Variable channel loss


Adaptive loss recovery
Combine retransmissions and FEC
23
Design Constraints

No hardware changes

Modify WiLD routers, not endpoints

Routers are inexpensive machines



low processing power
low energy budget (solar)
We want to be spectrum efficient
24
Problem with 802.11: ACKs

Low utilization



Large propagation delays
Stop & wait inefficient
RTS/CTS makes it worse

ACK timeouts


ACK doesn’t arrive in time
Retransmissions until retry limit
reached
25
Problem: Propagation Delay

Large propagation delay  high collision probability
A
B
26
Design Choices for WiLDNet

Use Sliding Window flow control




802.11 MAC ACKs disabled
Packet batches sent every slot
Slot allocation determined by demand
Replace CSMA with TDMA on every link

Alternate send and receive slots
27
Inter-Link Interference
Simultaneous Send
Simultaneous Receive
Send & Receive
B
B
B
1
1
1
A
A
A
2
2
2
C
C

Disable CCA

C
12dB isolation
28
Implicit Synchronization for TDMA



Every packet is time-stamped in TX slot
Slots are offset because of propagation delay
We don’t use explicit marker packets to signify end of TX slot*
∆
TX slot
RX slot
Sender
Receiver
* 2P MAC protocol (Raman et al. Mobicom ’05)
29
Channel Loss: From external traffic

Strong correlation between loss and external traffic

Source (A) and interferer (I) do not hear each other
A
I
B
30
Loss Recovery: Bulk ACKs + FEC

Bulk ACKs:




Adaptive FEC:



Aggregate ACKs (bit-vectors) sent with every packet
Use retransmissions for loss recovery
Retry limit can be per-packet
Sender performs encoding of packets proactively
Packet level FEC
Tradeoff of BW and Delay

Bandwidth efficient: use Bulk ACKs


TCP, bulk traffic
Delay efficient: use Adaptive FEC

Voice, Video
31
WiLDNet Design Recap
Replace CSMA with TDMA
 Loose time synchronization




To eliminate inter-link interference
Overcome variable channel loss
Adaptive loss recovery layer at link layer


Using Bulk ACKs: BW efficient
Using FEC: Delay efficient
32
Evaluation: Multiple Hops outdoors

2 hop network




19km, 1.5km
WiLDNet: Similar
throughput

Channels
Same channel OR
Diff. channels
More spectrum
efficient
Through-put
for TCP
(Mbps)
802.11
Same
2.11
802.11
Diff.
4.50
WiLDNet
Same
4.86
WiLDNet
Diff.
4.90
33
Deployment
34
Wireless Mesh Networks

Rapidly deployable high performance wireless
mesh networks


What we are investigating




Current mesh networks have poor performance in
multi-hop settings
Multi-radio, Multi-channel
Nodes with directional antennas
Understanding multi-AP interactions
Visit the 12-node testbed in 715 Broadway!
35
Research Challenges

Low-cost high-performance network connectivity



Intermittent distributed systems



Leveraging the next-generation cell-phone
Redesigning applications to tolerate intermittency
“Interactive applications” in low bandwidth environments


WiLDNet: WiFi-based Long Distance Networks
Wireless Mesh Networks
What can you do using 30 Kbps bandwidth?
Application specific research challenges

Security, User-interface, NLP, Image processing etc.
36
Intermittent Distributed Systems

How do we build conventional distributed
systems in intermittent environments?





Connectivity is intermittent and unpredictable
Net bandwidth is limited
Potentially high delays
Potentially lossy environments
Examples


A distributed system of cell-phones using GPRS links
and SMS messages to communicate
Web search from a rural cafe
37
Intermittent Web Search
A typical search today involves 4-8 queries!
 Can we do web search in one round?
 What we have done


Change the query interface



Specify all that you know about what you are searching for
Intermittent proxy issues multiple queries,
prefetches and bundles response pages
Local proxy enables search within retrieved bundle
38
Intermittent ATM
How do you operate ATMs over intermittent
environments?
 Our solution




Enable offline authentication
Use “approximate consistency” results to split a
bank balance into smaller entities
Provide a “risk model” to enable extra cash
availability at ATMs
39
Research Challenges

Low-cost high-performance network connectivity



Intermittent distributed systems



Leveraging the next-generation cell-phone
Redesigning applications to tolerate intermittency
“Interactive applications” in low bandwidth environments


WiLDNet: WiFi-based Long Distance Networks
Wireless Mesh Networks
What can you do using 30 Kbps bandwidth?
Application specific research challenges

Security, User-interface, NLP, Image processing etc.
40
Low-bandwidth video streaming

Multi-hop satellite network for distance
learning /telemedicine


Very high delays, low bandwidth
Questions

Enhancing QoS on low bandwidth environments



OverQoS: Overlay based QoS
On the fly transcoding
Can we develop an appropriate video codec that is
easily “adaptable” to different rates?
41
Intermittent proxy

Imagine a distributed system over a multi-hop
intermittent/low-bandwidth network


Content distribution, distributed databases, video
streaming, client/server transactions, web search
Intermittent proxy



A generic optimization engine that performs
resource allocation across flows on a lowbandwidth/intermittent link
Maintains “soft application state” to enhance
performance
Performs local hop-hop recovery
42
Research Challenges

Low-cost high-performance network connectivity



Intermittent distributed systems



Leveraging the next-generation cell-phone
Redesigning applications to tolerate intermittency
“Interactive applications” in low bandwidth environments


WiLDNet: WiFi-based Long Distance Networks
Wireless Mesh Networks
What can you do using 30 Kbps bandwidth?
Application specific research challenges

Security, User-interface, NLP, Image processing etc.
43
Cell-phone based applications

Cell-phone based Micro-finance




Cell-phone based Health record system



Use SIM as a cheap identity
Use programmable smart-phones to provide low-cost
authentication
Benefits: Reduce transaction costs and corruption
Health-workers in the field use cell-phones to enter health
records
Need a distributed database syncronization/search mechanism
which works over SMS-links
Cell-phone based cheap Inventory management

Why need RF-ID based systems?
44
Low cost paper-watermarking


Every piece of paper has an
inbuilt random speckle pattern
Can we extract this speckle
pattern to watermark any
paper?


Use USB 60x microscope
Applications


Verification of paper based
records
Cheap inventory management
45
Watermarking initial results
Nearly 90% match
Less than 20% match
46
Automated Diabetic Retinopathy
47
Medical Education Modules

Can we create medical education modules to
teach a healthcare worker to become a malaria
expert?


WiSE-MD modules for surgical education in NYU Med
Tailor modules for two focus areas:


Malaria and Infectious diseases
Obstetrics and Gynecology
48
Other interesting problem areas
Traffic Signaling Networks
 User interfaces




Text-free interfaces for semi-literate or illterates
Speech interfaces
Natural language processing


People interested in content in local language
Language translation
49
Questions?
Thank you!
Synchronization in Bipartite
Graphs
X%
A
D

2 global slots

B

send
:X% of time
receive :(1-X)% of time
E
C
1-X%
51
Implicit Synchronization for TDMA



Every packet is time-stamped in TX slot
Slots are offset because of propagation delay
We don’t use explicit markers based synchronization
58
Rural Connectivity in Developing Regions
Optical
Fiber
Village
Kiosk
Rural
Clinic
3Com
Town
Current
 VSATsolutions:
Village
Kiosk

Wired

 Dial-up
WiMAX


Rural
Clinic
School


Optical fiber
DSL
Point-to-Point
Cellular
59
CapEx for network deployment
WiFi/CDMA450 has the lowest deployment
Largest cost for fiber is installation
60
OpEx for network deployment
WiFi/CDMA450 suffers from recurring spectrum lease cost
Termination costs for a large portion of the OpEx
61
Conclusions for Akshaya






WiFi with directional antennas + WiMax has most
attractive economics.
WiFi/CDMA450 has lowest cost of deployment.
Largest cost for Fiber is installation.
Wireless backhaul (both WiFi and VIP) technologies have
at most 1/8 the backhaul CapEx VS Fiber
Largest component of the capital investment for
providing connectivity is the cost of the end-user
devices.
Cost of backhaul/access radio equipment on towers is
miniscule
62
Technologies considered
Backhaul Technologies
Access Technologies
WipLL
WiMax
WiFi
X
Fiber
X
(directional)
VIP
WiFi
CDMA450
(Omni)
X
X
63
Implementation

Driver modifications (Atheros madwifi)



Click Modular router framework



Disable ACKs
Disable Carrier Sense
Creates virtual network interfaces
Intercept packets and modify
FEC encoding/decoding


Original packets sent first, redundant next
Decoding performed only on loss
64
Evaluation Overview

How well does WiLDNet mitigate 802.11 protocol
induced losses?

Can multiple link synchronization in WiLDNet achieve
spectrum efficiency?

How does WiLDNet combine Bulk ACKs and FEC to
achieve desired delay and bandwidth?
65
Evaluation: Single Hop


TCP: Measured on indoor channel emulator
Without channel loss: 2.5x improvement at 80km
WiLDNet
constant
at 6 Mbps
66
Evaluation: Single Hop


TCP: Measured on indoor channel emulator
With 10% channel loss: 2x improvement at 80km
WiLDNet
constant
at 5 Mbps
67
Evaluation: Single Hop outdoors
Link
Distance
(km)
802.11 CSMA
TCP (Mbps)
WiLDNet
TCP (Mbps)
One
direction
Both
directions
One
direction
Both
directions
B-R
8
5.03
4.95
3.65
5.86
P-S
45
3.62
3.52
3.10
4.91
Ghana
65
2.80
0.68
2.98
5.51


WiLDNet’s improvement increases with distance
TDMA sends only 50% of time in one direction
68
Tradeoffs of Bulk ACKs and FEC
Increasing
retries


Delay increases with number of retries
Overhead increases with increasing FEC
69
Summary and Conclusions

Single hop:



Multiple hops:


Up to 8x better at 60km
5.5 Mbps vs 0.7 Mbps for CSMA
Can achieve optimal performance even with same
channel operation
Can combine FEC and Bulk ACKs to achieve
required delay and BW
70
Future Work

Global scheduling

Future deployments


Aravind Eye Hospitals (India)
Ghana, Guinea Bissau

Remote management

Network planning
71
Problem: Propagation Delay

On Channel emulator with bi-directional UDP
72
Problem with 802.11: ACKs

On Channel emulator with unidirectional UDP
Decreasing utilization
with distance
Not waiting for 802.11 ACKs
Sharp drop at 110km
73