CENTAUR: Realizing the Full Potential of Centralized WLANs Through a Hybrid Data Path Vivek Shrivastava* , Shravan Rayanchu, Suman Banerjee University of Wisconsin-Madison Nabeel Ahmed, Srinivasan Keshav University of Waterloo, Ontario Konstantina Papagiannaki Intel Labs, Pittsburgh Arunesh Mishra Google Inc. * viveks@cs.wisc.edu Centralization of Enterprise WLANs Clients Internet Access Point Wireless controller Vivek Shrivastava 2 2 Centralization of Enterprise WLANs Common control plane functions Power control Channel assignment Vivek Shrivastava 3 3 Centralization of Enterprise WLANs What about data plane functions? Power control Channel assignment Data scheduling ? Vivek Shrivastava 4 4 Can centralized scheduling help? Vivek Shrivastava 5 5 Can centralized scheduling help? Hidden terminals Vivek Shrivastava 6 6 Can centralized scheduling help? Hidden terminals Vivek Shrivastava 7 7 Can centralized scheduling help? Hidden terminals Vivek Shrivastava 8 8 Can centralized scheduling help? Hidden terminals 1. Carrier sense Vivek Shrivastava 9 9 Can centralized scheduling help? Hidden terminals 2. Channel free, transmit Vivek Shrivastava 10 10 Can centralized scheduling help? • Collision! • Backoffs • Low throughputs 3. Collision ! Vivek Shrivastava 11 11 How bad is it ? Experiments on production building-wide WLANs •W1: 5 floors 9 APs, 45 clients • W2: 1 floor 21 APs, 51 clients 10% links suffer more than 70% throughput reduction Vivek Shrivastava 12 12 A lost opportunity ? Suppose infrastructure can gather conflict data Vivek Shrivastava 13 13 A lost opportunity ? And when packets arrive … Vivek Shrivastava 14 14 A lost opportunity ? … realize interference will happen … Vivek Shrivastava 15 15 A lost opportunity ? 1. Transmit first packet Vivek Shrivastava 16 16 A lost opportunity ? 1. Transmit first packet 2. Transmit second packet with delay Vivek Shrivastava 17 17 Use an in-band scheduler Simple FIFO schedule with interference avoidance Scheduling functionality Vivek Shrivastava 18 18 What about exposed terminals ? • In our experiments, about 41%of link pairs suffer from exposed terminal interference • Disabling carrier sense to solve the problem can be dangerous for uplink, non-enterprise traffic • We will show how centralization can help even here Vivek Shrivastava 19 19 Challenge • Centralization has obvious overheads • How to make this feasible and useful under • Real applications • Common large-scale wireless environments • Presence of uplink and non-enterprise traffic • Requirements of no client modifications Vivek Shrivastava 20 20 Contributions 1.Design CENTAUR, a hybrid (partly centralized, partly distributed) scheduling approach • Resolves hidden and exposed terminals • Requires no client modifications • Requires no carrier sense disabling 2.Evaluate CENTAUR on two WLAN testbeds with real-world traffic traces • 1.48x greater throughput for bulk data traffic • 1.38x reduction in web-transaction times Vivek Shrivastava 21 21 Outline • A naïve attempt at centralized scheduling (DET) • Our hybrid centralized scheduler (CENTAUR) • Evaluation • Related Work • Summary Vivek Shrivastava 22 22 Outline • A naïve attempt at centralized scheduling (DET) • Our hybrid centralized scheduler (CENTAUR) • Evaluation • Related Work • Summary Vivek Shrivastava 23 23 DET: A Simple Deterministic Scheduler Vivek Shrivastava 24 24 DET: A Simple Deterministic Scheduler Input: Conflict graph Vivek Shrivastava 25 25 DET: A Simple Deterministic Scheduler Goal: Schedule each incoming downlink packet Input: Conflict graph Vivek Shrivastava 26 26 DET: A Simple Deterministic Scheduler Goal: Schedule each incoming downlink packet Input: Conflict graph Transmission 1 2 3slots 4 Vivek Shrivastava 27 27 DET: A Simple Deterministic Scheduler 1. New packet arrives Transmission 1 2 3slots 4 Vivek Shrivastava 28 28 DET: A Simple Deterministic Scheduler 1. New packet arrives 2. Find the earliest conflict free slot Transmission 1 2 3slots 4 Vivek Shrivastava 29 29 DET: A Simple Deterministic Scheduler 1. New packet arrives 2. Find the earliest conflict free slot Transmission 1 2 3slots 4 Vivek Shrivastava 30 30 DET: A Simple Deterministic Scheduler 1. New packet arrives 2. Find the earliest conflict free slot 3. Schedule the packet in that slot Transmission 1 2 3slots 4 5 Vivek Shrivastava 31 31 Performance of DET No gains for exposed terminals; Non4xperform worse under load conflicting links Vivek Shrivastava 32 Outline • A naïve attempt at centralized scheduling (DET) • Our hybrid centralized scheduler (CENTAUR) • Evaluation • Related Work • Summary Vivek Shrivastava 33 33 Overview of CENTAUR • Incorporate basic DET scheduler • Tackle DET’s shortcomings: • Amortize scheduling overhead • Improve performance for exposed links • Avoid degrading normal links • Coexist with non-enterprise and uplink traffic Vivek Shrivastava 34 34 (1) Avoid Scheduling Overheads • Problem: Per-packet scheduling performs poorly under high network loads • Solution: • Vivek Shrivastava Schedule packets in batches (or epochs) 35 35 (1) Avoid Scheduling Overheads • Problem: Per-packet scheduling performs poorly under high network loads • Solution: • Vivek Shrivastava Schedule packets in batches (or epochs) 36 36 (1) Avoid Scheduling Overheads • Problem: Per-packet scheduling performs poorly under high network loads • Solution: • Schedule packets in batches (or epochs) Wired ack Vivek Shrivastava 37 37 (1) Avoid Scheduling Overheads • Problem: Per-packet scheduling performs poorly under high network loads • Solution: • Vivek Shrivastava Schedule packets in batches (or epochs) 38 38 (1) Avoid Scheduling Overheads • Problem: Per-packet scheduling performs poorly under high network loads • Solution: • Vivek Shrivastava Schedule packets in batches (or epochs) 39 39 (1) Avoid Scheduling Overheads • Problem: Per-packet scheduling performs poorly under high network loads • Solution: • Vivek Shrivastava Schedule packets in batches (or epochs) 40 40 (1) Avoid Scheduling Overheads • Problem: Per-packet scheduling performs poorly under high network loads • Solution: • Vivek Shrivastava Schedule packets in batches (or epochs) 41 41 (1) Avoid Scheduling Overheads • Problem: Per-packet scheduling performs poorly under high network loads • Solution: • Schedule packets in batches (or epochs) Wired ack Vivek Shrivastava 42 42 (1) Avoid Scheduling Overheads • Problem: Per-packet scheduling performs poorly under high network loads • Solution: • Vivek Shrivastava Schedule packets in batches (or epochs) 43 43 (2) Improve Exposed Terminals • Problem: Exposed links can operate in parallel but don’t due to carrier-sensing • Solution: • Schedule batch of packets, fix backoff periods Vivek Shrivastava 44 44 (2) Improve Exposed Terminals • Problem: Exposed links can operate in parallel but don’t due to carrier-sensing • Solution: • Vivek Shrivastava Schedule batch of packets, fix backoff periods 45 45 (2) Improve Exposed Terminals • Problem: Exposed links can operate in parallel but don’t due to carrier-sensing • Solution: • Schedule batch of packets, fix backoff periods Variable wired delay Vivek Shrivastava 46 46 (2) Improve Exposed Terminals • Problem: Exposed links can operate in parallel but don’t due to carrier-sensing • Solution: • Vivek Shrivastava Schedule batch of packets, fix backoff periods 47 47 (2) Improve Exposed Terminals • Problem: Exposed links can operate in parallel but don’t due to carrier-sensing • Solution: • Schedule batch of packets, fix backoff periods Carrier sense, deferral Vivek Shrivastava 48 48 (2) Improve Exposed Terminals • Problem: Exposed links can operate in parallel but don’t due to carrier-sensing • Solution: • Vivek Shrivastava Schedule batch of packets, fix backoff periods 49 49 (2) Improve Exposed Terminals • Problem: Exposed links can operate in parallel but don’t due to carrier-sensing • Solution: • Vivek Shrivastava Schedule batch of packets, fix backoff periods 50 50 (2) Improve Exposed Terminals • Problem: Exposed links can operate in parallel but don’t due to carrier-sensing • Solution: • Schedule batch of packets, fix backoff periods Schedule packets in batches Vivek Shrivastava 51 51 (2) Improve Exposed Terminals • Problem: Exposed links can operate in parallel but don’t due to carrier-sensing • Solution: • Vivek Shrivastava Schedule batch of packets, fix backoff periods 52 52 (2) Improve Exposed Terminals • Problem: Exposed links can operate in parallel but don’t due to carrier-sensing • Solution: • Vivek Shrivastava Schedule batch of packets, fix backoff periods 53 53 (2) Improve Exposed Terminals • Problem: Exposed links can operate in parallel but don’t due to carrier-sensing • Solution: • Schedule batch of packets, fix backoff periods First packets can be out of sync Vivek Shrivastava 54 54 (2) Improve Exposed Terminals • Problem: Exposed links can operate in parallel but don’t due to carrier-sensing • Solution: • Schedule batch of packets, fix backoff periods Waiting packets synchronized by carrier sense ! Vivek Shrivastava 55 55 (2) Improve Exposed Terminals • Problem: Exposed links can operate in parallel but don’t due to carrier-sensing • Solution: • Vivek Shrivastava Schedule batch of packets, fix backoff periods 56 56 (2) Improve Exposed Terminals • Problem: Exposed links can operate in parallel but don’t due to carrier-sensing • Solution: • Schedule batch of packets, fix backoff periods After first packet, both APs transmit simultaneously. Vivek Shrivastava 57 57 (3) Avoid Degrading Normal Links Hidden terminals A B C D Vivek Shrivastava Nonhidden/Nonexposed 58 58 (3) Avoid Degrading Normal Links A B C D Scheduler Vivek Shrivastava 59 59 (3) Avoid Degrading Normal Links A B C D Scheduler Vivek Shrivastava 60 60 (3) Avoid Degrading Normal Links A B C D Scheduler Vivek Shrivastava 61 61 (3) Avoid Degrading Normal Links A B Hybrid scheduling C D Scheduler Vivek Shrivastava 62 62 Outline • A naïve attempt at centralized scheduling (DET) • Our hybrid centralized scheduler (CENTAUR) • Evaluation • Related Work • Summary Vivek Shrivastava 63 63 Large-Scale Experiments • Platform: • Two WLAN testbeds in separate buildings • Topology • Representative: 7 APs, 12 clients • Traffic and metrics • UDP, TCP, VoIP, HTTP (real traces) • Throughput, delay, MOS, web transaction delay Vivek Shrivastava 64 64 Result 1: UDP/TCP Performance Vivek Shrivastava 65 65 Result 1: UDP/TCP Performance DCF Vivek Shrivastava 66 66 Result 1: UDP/TCP Performance DCF Vivek Shrivastava 67 67 Result 1: UDP/TCP Performance DCF Vivek Shrivastava Per 68 68 Result 1: UDP/TCP Performance DCF Vivek Shrivastava Per 69 69 Result 1: UDP/TCP Performance DCF Vivek Shrivastava Per 70 Epoch Based 70 Result 1: UDP/TCP Performance Hidden terminal starves some clients DCF Vivek Shrivastava Per 71 Epoch Based 71 Result 1: UDP/TCP Performance Better fairness than DCF DCF Vivek Shrivastava Per 72 Epoch Based 72 Result 1: UDP/TCP Performance Exploits exposed terminals, higher system throughput DCF Vivek Shrivastava Per 73 Epoch Based 73 Result 1: UDP/TCP Performance Avg. delay is smallest for epoch based scheduling DCF Vivek Shrivastava Per 74 Epoch Based 74 Result 1: UDP/TCP Performance 90th percentile delays are higher for epoch based scheduling DCF Vivek Shrivastava Per 75 Epoch Based 75 Result 1: UDP/TCP Performance CENTAUR yields up to 60% higher total throughput and 50% lower per-packet delay compared to DCF DCF Vivek Shrivastava Per 76 Epoch Based 76 More Results in Paper • Centaur micro-benchmarks: performance for exposed and hidden terminals under centaur • Uplink traffic: coexistence and persistent gains with different fractions of uplink traffic • Data rate: robust to changes in data rate and ARF • Realistic HTTP traces: significant reduction in web transaction delay • VoIP traffic: better performance (MOS) for voice traffic with small epoch duration Vivek Shrivastava 77 77 Related Work • Commercial WLAN offerings (Aruba, Meru) • Theoretical formulations (Vaidya ‘00, Kanodia ‘01) • Epoch based scheduling (Kompella ‘05, 802.11n/e) • Interference mitigation (CMAP, SIC, Shuffle) Vivek Shrivastava 78 78 Summary • • • • Interference a growing problem in enterprises Careful design of a centralized data plane provides substantial performance gains CENTAUR implements a hybrid data path to improve aggregate performance without client modifications CENTAUR does not disable carrier sense and co-exists with non-enterprise and uplink traffic Vivek Shrivastava 79 79 Future work • • Even more efficient conflict graph generation What if we were allowed client modifications ? Questions ? Vivek Shrivastava 80 Characterizing System Latencies System delays are high and variable, leading to inaccuracies for per-packet scheduling 81 Evaluation of Micro-Probing Topologies 20 node 30 node Bandwidth Tests 16.2 mins 1hr 11 mins MicroProbing ~4 secs ~11 secs Can be computed in stages with each instance taking ~ 2.5ms Vivek Shrivastava 82 82 Result 2: Impact of Uplink Traffic • Vary proportion of downlink/uplink traffic • 6 different configurations • 80/20 ➔ 40/60 (downlink/uplink) • Results: • Downlink: 1.6x ➔ 6.8x gain in throughput • Uplink: 1x ➔ 1.18x gain in throughput CENTAUR provides persistent gains for different proportions of uplink and downlink traffic load Vivek Shrivastava 83 83 Result 3: Impact of Topology • Three topologies • Hidden Heavy topology ➔ 10 links • Exposed Heavy topology ➔ 6 links • Mixed Topology ➔ 12 links • Results: Up to 50% gain in overall system throughput • Up to 6x gain for HT; Up to 1.7x gain for ET Improvements from using CENTAUR can be seen across many different network topologies Vivek Shrivastava 84 84 Result 2: Impact of Uplink Traffic CENTAUR provides persistent gains for different proportions of uplink and downlink traffic load Vivek Shrivastava 85 85 Result 3: Impact of Topology Improvements from using CENTAUR can be seen across many different network topologies Vivek Shrivastava 86 86 CENTAUR Micro-Benchmarks (1) Exposed Terminals Vivek Shrivastava 87 87 CENTAUR Micro-Benchmarks (II) Hidden Terminals Vivek Shrivastava 88 88 Result 1: UDP/TCP Performance Client Index (1-12) CENTAUR yields up to 60% higher total throughput and 50% lower per-packet delay compared to DCF Client Index (1-12) Vivek Shrivastava 89 89 Other approaches to Hidden/Exposed Terminals Mechanis m CMAP [NSDI ‘07] ZigZag [Sigcomm ’08] SIC [Mobicom ’08] Adaptive RTS/CTS [VTC ’03] Centau r Target Problem Approach Expose d Hidden Disable CS Signal manipulation Extra Hidden Signaling Expose dHidden Centralized Scheduling Client Changes Evaluation Yes 802.11 Yes GNU Radio Yes 802.11 No 802.11 Other approaches to Hidden/Exposed Terminals Mechanis m CMAP [NSDI ‘07] ZigZag [Sigcomm ’08] SIC [Mobicom ’08] Adaptive RTS/CTS [VTC ’03] Centau r Target Problem Approach Expose d Hidden Disable CS Signal manipulation Extra Hidden Signaling Expose dHidden Centralized Scheduling Client Changes Evaluation Yes 802.11 Yes GNU Radio Yes 802.11 No Solve both hidden/exposed 802.11 Other approaches to Hidden/Exposed Terminals Mechanis m CMAP [NSDI ‘07] ZigZag [Sigcomm ’08] SIC [Mobicom ’08] Adaptive RTS/CTS [VTC ’03] Centau r Target Problem Approach Expose d Hidden Disable CS Signal manipulation Extra Hidden Signaling Expose dHidden Centralized Scheduling No client side changes for Centaur Client Changes Evaluation Yes 802.11 Yes GNU Radio Yes 802.11 No 802.11 CENTAUR: Realizing the Full Potential of Centralized WLANs Through a Hybrid Data Path • • Vivek Shrivastava* , Shravan Rayanchu, Suman Banerjee University of Wisconsin-Madison Nabeel Ahmed, Srinivasan Keshav University of Waterloo, Ontario Konstantina Papagiannaki Arunesh Mishra Intel Labs, Pittsburgh Google Inc. Growth of Interference in Enterprise Wireless LANs WLANs HP Labs Seoul National University Exposed Terminals 39% 9% 50% Hidden Terminals 43% 70% 39% Our Testbed Interference an increasing problem according to leading enterprise WLAN vendor Vivek Shrivastava 94 94 Centralization of Enterprise WLANs Centralized control for better network security and manageability Vivek Shrivastava 95 95 Can Centralized Data Plane Scheduling Help? Vivek Shrivastava 96 96 (2) Improve Exposed Terminals • Problem: Exposed links can operate in parallel but don’t due to carrier-sensing • • • Solution: Schedule batch of packets to each exposed AP Fix back-off periods and use carrier-sensing to align transmissions -- double throughput! Vivek Shrivastava 97 97 Related Work • Commercial WLAN offerings (Aruba, Meru) • Research proposals (MiFi, DenseAP, Smarta) • TXOP in 802.11e/802.11n packet aggregation • Interference Mitigation (CMAPs, ZigZag, SIC) Vivek Shrivastava 98 98 Types of Interference Hidden Terminals Exposed Terminals Focus on downlink conflicts Vivek Shrivastava 99 11 Can Centralized Data Plane Scheduling Help? X Vivek Shrivastava Y 100 100 Can Centralized Data Plane Scheduling Help? X Vivek Shrivastava Y 101 101 Can Centralized Data Plane Scheduling Help? X Vivek Shrivastava Y 102 102 Can Centralized Data Plane Scheduling Help? Hidden terminals Vivek Shrivastava 103 103 Quantifying Downlink Hidden Terminals 10% links suffer severe hidden terminal interference Vivek Shrivastava 104 104 Quantifying Downlink Exposed Terminals 41% links can obtain double the throughput with CS disabled, indicating exposed terminal interference Vivek Shrivastava 105 105 Result 1: UDP/TCP Performance Vivek Shrivastava 106 106 Result 1: UDP/TCP Performance Vivek Shrivastava 107 107 Result 1: UDP/TCP Performance Vivek Shrivastava 108 108 Centralization of Enterprise WLANs Vivek Shrivastava 109 109 (2) Improve Exposed Terminals • Problem: Exposed links can operate in parallel but don’t due to carrier-sensing • Solution: • Schedule batch of packets to each exposed AP • Fix back-off periods and use carrier sensing to align transmissions -- double throughput! Vivek Shrivastava 110 110 (2) Improve Exposed Terminals • Problem: Exposed links can operate in parallel but don’t due to carrier-sensing • Solution: • Schedule batch of packets to each exposed AP • Fix back-off periods and use carrier sensing to align transmissions -- double throughput! Vivek Shrivastava 111 111 (3) Avoid Degrading Normal Links Vivek Shrivastava 112 112 Can Centralized Data Plane Help? 1. Can it solve any interference problems prevalent in enterprise WLANs • Hidden and exposed terminals • 88% of links suffer some losses due to cochannel interference (Jigsaw, Sigcomm 2007) 2. If so, how can we implement it efficiently for practical WLAN deployments • No client modifications, support legacy clients • Vivek Shrivastava Coexistence with non-enterprise, uplink traffic 113 113 Quantifying Downlink Interference • Prior Work: Jigsaw [Sigcomm2006] analysis reveals 56% of all interference traffic is downlink in nature. • Our Work: Two production WLANs • W1: 5 floors, 9 APs, 45 clients • W2: 1 floor, 21 APs, 51 clients • Download ‘bulk’ traffic from the Internet Vivek Shrivastava 114 114 Growth of Interference in Enterprise Wireless LANs WLANs UW-Madison UW-Ontario Exposed Terminals 39% 9% Hidden Terminals 43% 70% Interference an increasing problem according to leading enterprise WLAN vendor Vivek Shrivastava 115 115 Centralization of Enterprise WLANs Can centralized data plane be useful for improving performance in WLANs ? • About 70-80% of enterprise traffic is downlink in nature Vivek Shrivastava 116 116 (3) Avoid Degrading Normal Links A B C D Scheduler Vivek Shrivastava 117 117 (3) Avoid Degrading Normal Links A B C D Scheduler Vivek Shrivastava 118 118 (3) Avoid Degrading Normal Links A B C D Scheduler Vivek Shrivastava 119 119 (3) Avoid Degrading Normal Links A B Hybrid scheduling C D Scheduler Vivek Shrivastava 120 120 Outline • DET - A deterministic scheduler • CENTAUR - A hybrid centralized scheduler • Evaluation • Related Work • Summary Vivek Shrivastava 121 121 Outline • DET - A deterministic scheduler • CENTAUR - A hybrid centralized scheduler • Evaluation • Related Work • Summary Vivek Shrivastava 122 122 Performance of DET No gains for exposed terminals; Nonconflicting links4x perform worse under load Vivek Shrivastava 123 Outline • DET - A deterministic scheduler • CENTAUR - A hybrid centralized scheduler • Evaluation • Related Work • Summary Vivek Shrivastava 124 124 DET: A Simple Deterministic Scheduler • • Key Idea: Perform per-packet scheduling Given • • • Scheduled packets {P1, P2, ..., Pr}; Unscheduled Pr+1 Objective • • • Conflict graph G = (L,E) Minimize t(Pr+1) Constraint: No two packets on interfering links are scheduled together Schedule downlink packets only Vivek Shrivastava 125 125 Can centralized scheduling help? 1. What are the problems it can solve ? • Hidden and exposed terminals 2. Can we implement it efficiently ? • No client modifications • Coexistence with non-enterprise & uplink traffic • Don’t disable carrier sensing Vivek Shrivastava 126 126 Can centralized scheduling help? Vivek Shrivastava 127 127 DET: A Simple Deterministic Scheduler • Schedule one packet at a time • Consider packets in order of arrival Transmission 1 2 3slots 4 Vivek Shrivastava 128 Packets arriving 128 What about exposed terminals ? 41% links can obtain double the throughput with CS disabled, indicating exposed terminal interference Vivek Shrivastava 129 129 Use an in-band scheduler Simple FIFO schedule with interference avoidance scheduler Vivek Shrivastava 130 130