Yuan Feng, Zimu Liu, Baochun Li
Department of Electrical and Computer Engineering, University of Toronto
Presented by Zuo Yinbo ( 左银波 )
June 8, Wed., 2011

GestureFlow: Streaming Gestures to an Audience, Yuan Feng et al., (IEEE INFOCOM2011)
Background
Use multi-touch gestures or body motion gestures to interact with computers
2

GestureFlow: Streaming Gestures to an Audience, Yuan Feng et al., (IEEE INFOCOM2011)
Background, cont.
• Collaborations among users in real time need streaming gestures in a broadcast fashion from one user to all participating users.
• Implement a gesture broadcast protocol to accomplish this aim.
3

GestureFlow: Streaming Gestures to an Audience, Yuan Feng et al., (IEEE INFOCOM2011)
Contents
•
•
– Coding gesture broadcast sessions
– Coding across multiple broadcast sessions
•
– Overall performance
– Network coding performance
•
•
4

GestureFlow: Streaming Gestures to an Audience, Yuan Feng et al., (IEEE INFOCOM2011)
Introduction & Related work
• In this paper, the author presents the design of GuestFlow, a gesture broadcast
protocol designed for concurrent gestureintensive streams in multiple broadcast sessions.
• It achieves effectiveness and practicality by using inter-session network coding, and solve the challenges introduced by linear dependence.
5

GestureFlow: Streaming Gestures to an Audience, Yuan Feng et al., (IEEE INFOCOM2011)
Introduction & Related work, cont.
• Random network coding
– maximize information flow rates in multicast sessions
– reduce download time of P2P systems
– achieve high playback rate and system throughput
• In this work, however, network coding is used for ensuring reliable delivery with low delays in low bit-rate traffic from gesture events.
6

GestureFlow: Streaming Gestures to an Audience, Yuan Feng et al., (IEEE INFOCOM2011)
Introduction & Related work, cont.
• Network coding(NC) in transport protocols
– improve unicast throughput (by intra-session NC)
– achieve low multicast latency in streaming
– provide better support to a unicast TCP session with sliding window
• In this paper, however, the transport protocol (GestureFlow) is designed specifically for multiple broadcast sessions, and it specifies how the coding window advances itself.
7

GestureFlow: Streaming Gestures to an Audience, Yuan Feng et al., (IEEE INFOCOM2011)
Design & Implement
• Gesture streams’ transporting features
– low bit rate
– bursty traffic
– sensitive to packets loss and delay
– continuous playing back of a gesture stream
• TCP protocol is not suitable
– varying available bandwidth over time
– high delay jitters with congested links
8

GestureFlow: Streaming Gestures to an Audience, Yuan Feng et al., (IEEE INFOCOM2011)
Design & Implement, cont.
• The transport solution to gestures streams
– a protocol called GestureFlow
– “all-to-all” broadcast ( each node broadcast to all other nodes and multiple broadcast sessions exist concurrently)
– low-delay streaming with guaranteed reliability
• random network coding
• stream coded packets using UDP flows rather than TCP
• either a direct UDP link between source and the receiver or a two-hop path which uses a single relay node
• allow relay nodes to recode packets
9

GestureFlow: Streaming Gestures to an Audience, Yuan Feng et al., (IEEE INFOCOM2011)
Design & Implement, cont.
Fig. 1. Streaming coded blocks along multiple paths in GestureFlow.
Transmit Data packets from Node 1 to Node 4 in coded form, either with a direct link ,or relayed by Node 2 and Node 3 after being recoded with their own packets.
10

GestureFlow: Streaming Gestures to an Audience, Yuan Feng et al., (IEEE INFOCOM2011)
Design & Implement, cont.
• Coding gesture broadcast sessions coding coefficients: choose from k

Receiver decodes as soon as it has received k linearly independent coded block. Loss of particular blocks will not cause failure of decoding and thus guarantee reliable delivery.
11

GestureFlow: Streaming Gestures to an Audience, Yuan Feng et al., (IEEE INFOCOM2011)
Design & Implement, cont.
• Problem with coding of fixed k data blocks
– all the previous k-1 blocks will be waiting for coding until the k th block arrives (delay)
• The solution in GestureFlow
– use a sliding coding window with a maximum size
– sequence number of the earliest original block in the window is embedded within the UDP packet
– after receiving ACKs of the earliest original block from all the receivers in the broadcast session, the coding window advances itself by removing the earliest original block
12

GestureFlow: Streaming Gestures to an Audience, Yuan Feng et al., (IEEE INFOCOM2011)
Design & Implement, cont.
The coding window advances and adapts itself over time according to the traffic.
13

GestureFlow: Streaming Gestures to an Audience, Yuan Feng et al., (IEEE INFOCOM2011)
Design & Implement, cont.
• How to acknowledge the sender?
– do not acknowledge a newly decoded data block immediately for there will be additional overhead and complexity
– use cumulative acknowledgements of decoded data blocks
– receiver sends an acknowledgement for a block only if all earlier blocks with smaller sequence numbers have been decoded
– the earliest sequence number is embedded in each packet (block)
14

GestureFlow: Streaming Gestures to an Audience, Yuan Feng et al., (IEEE INFOCOM2011)
Design & Implement, cont.
Receiver sends a cumulative acknowledgement (for b3) only when all earlier blocks (b1, b2, b3) have been decoded
15

GestureFlow: Streaming Gestures to an Audience, Yuan Feng et al., (IEEE INFOCOM2011)
Design & Implement, cont.
• Use single-hop and two-hop path together to reduce delay jitters
– relay nodes are receivers themselves, no additional bandwidth is consumed
– data block will arrive via a path with the lowest delay
– three-hop will unlikely offer a lower delay
• Use inter-session coding
– relay node receive blocks belonging to different sessions
– all incoming coded blocks are mixed together for recoding and then transmitted to other nodes
16

GestureFlow: Streaming Gestures to an Audience, Yuan Feng et al., (IEEE INFOCOM2011)
Design & Implement, cont.
Relay node (Node 2) mixes all incoming coded blocks ( S1, S3, S4 ) together in its recoding process, and sends coded inter-session blocks out.
17

GestureFlow: Streaming Gestures to an Audience, Yuan Feng et al., (IEEE INFOCOM2011)
Design & Implement, cont.
• How do relay nodes produce coded blocks ?
– relay node also maintains a sliding recoding window and remove the decoded original blocks
– after the source of a session advances its coding window by removing its earliest original block, all relay nodes will easily detect it as the sequence number of the earliest original block is embedded within a coded block
– a relay node removes original data blocks whose sequence number is smaller than the starting sequence number in a newly received coded block from the source
18

GestureFlow: Streaming Gestures to an Audience, Yuan Feng et al., (IEEE INFOCOM2011)
Design & Implement, cont.
removed from the recoding window newly received coded block a relay node removes blocks from its recoding window
19

GestureFlow: Streaming Gestures to an Audience, Yuan Feng et al., (IEEE INFOCOM2011)
Experiences with GestureFlow
• MusicScore: a collaborative music composition application on iPad
– users interact with it to compose music using only multitouch gestures
– it uses GestureFlow to stream gesture events among multiple participating users
• Experiments scenarios
– four nodes in “all-to-all” broadcast sessions
– two iPads connecting to the
Internet through Wi-Fi, and two iPhone 3GS devices connecting to the Internet through 3G and
EDGE(2.75G), respectively.
20

GestureFlow: Streaming Gestures to an Audience, Yuan Feng et al., (IEEE INFOCOM2011)
Experiences with GestureFlow, cont.
21

GestureFlow: Streaming Gestures to an Audience, Yuan Feng et al., (IEEE INFOCOM2011)
Experiences with GestureFlow, cont.
Percentage of recoded blocks from relay nodes over all received blocks
Streaming delays with different numbers of participating nodes
22

GestureFlow: Streaming Gestures to an Audience, Yuan Feng et al., (IEEE INFOCOM2011)
Experiences with GestureFlow, cont.
Bandwidth overhead per user
The CPU load and memory usage of network coding in an iPhone 3GS device
23

GestureFlow: Streaming Gestures to an Audience, Yuan Feng et al., (IEEE INFOCOM2011)
Experiences with GestureFlow, cont.
Average delays along with
95% confidence intervals in different experiment settings
Streaming delay get minimum value when coding window size is 8
24

GestureFlow: Streaming Gestures to an Audience, Yuan Feng et al., (IEEE INFOCOM2011)
Experiences with GestureFlow, cont.
90% of the coding windows have a size of no more than 5 blocks, which indicates coding and decoding process add little delay, as blocks do not have to wait too long to be transmitted or relayed.
90% of the recoding windows have a size of no more than 11 blocks, with an average around 4 blocks.
this indicates that there is only one block or two required to be recoded at the relay node.
25

GestureFlow: Streaming Gestures to an Audience, Yuan Feng et al., (IEEE INFOCOM2011)
Experiences with GestureFlow, cont.
CDF of linear dependence in different settings of maximum coding window size (W).
90% of the coded blocks contain
Waste of other.
Received coded blocks are considered useful only when they are linear independent with each other, or else they are considered as redundant blocks.
The linear dependence ratio is higher when the coding window size is getting smaller, 18% when W = 8.
26

GestureFlow: Streaming Gestures to an Audience, Yuan Feng et al., (IEEE INFOCOM2011)
Mitigating linear dependence
• Mitigate the bandwidth overhead due to linearly dependent blocks
– apply Reed-Solomon codec based on the
Vandermonde matrix on the source node in a
GestureFlow broadcast session
– a k × k submatrix of a Vandermonde matrix has a nonzero determinant and is nonsingular
– use k × k submatrix to code original blocks to guarantee linear independence among all coded blocks
– relay nodes still use random linear networking coding to recode received blocks
27

GestureFlow: Streaming Gestures to an Audience, Yuan Feng et al., (IEEE INFOCOM2011)
Mitigating linear dependence, cont.
A Vandermonde matrix with n-k rows and k columns
Any k × k submatrix of G has a nonzero determinant and is nonsingular, and as a result every subset of k rows of G is guaranteed to be linearly independent. As such, linear independence among all original and coded blocks transmitted from the source guaranteed with the use of Vandermonde matrix to generate coded blocks.
28

GestureFlow: Streaming Gestures to an Audience, Yuan Feng et al., (IEEE INFOCOM2011)
Mitigating linear dependence, cont.
90 th percentile of linear dependence is significantly reduced by using the new design, from 18% to 6%.
The remaining dependence is caused by the recoding process in relay nodes
29

GestureFlow: Streaming Gestures to an Audience, Yuan Feng et al., (IEEE INFOCOM2011)
Mitigating linear dependence, cont.
Streaming delays and their standard deviations from Wi-Fi 1 User to other three users by using the new and original GestureFlow.
By using RS codec at the source, average streaming delays through different kinds of connections have been evidently reduced. Since the linear dependence is mitigated, a received block can be used to decode with a higher probability, reducing the decoding delay.
30

GestureFlow: Streaming Gestures to an Audience, Yuan Feng et al., (IEEE INFOCOM2011)
Conclusion
• Propose the new paradigm for users to interact with computers--Gesture
• Design and implement a transportation protocol
(GestureFlow) for streaming gesture events with low but bursty bit rates
• Use random linear network coding and RS codec to solve the challenges brought by low streaming delays and multiple concurrent broadcast sessions
• Explain thoroughly the design and improvement of the streaming protocol
• Evaluate the proposed protocol with an iPad application Music Score
31

32