Scalable, Secure and Efficient
Content Distribution and Services
(Brief overview of some of our recent papers)
Niklas Carlsson
Linköping University, Sweden
@ Students, October 15, 2014
Background: Research overview
• Design, modeling, and performance evaluation of
distributed systems and networks
• Current topics include




Scalable content delivery
Energy-efficient and sustainable ICT
Measurement analysis and modeling
Security and emerging services
Background: Research overview
• Design, modeling, and performance evaluation of
distributed systems and networks
• Current topics include




Scalable content delivery
Energy-efficient and sustainable ICT
Measurement analysis and modeling
Security and emerging services
Background: Research overview
• Design, modeling, and performance evaluation of
distributed systems and networks
• Current topics include




Scalable content delivery
Energy-efficient and sustainable ICT
Measurement analysis and modeling
Security and emerging services
5590%
of Internet traffic
Background: Research overview
• Design, modeling, and performance evaluation of
distributed systems and networks
• Current topics include




Scalable content delivery
Energy-efficient and sustainable ICT
Measurement analysis and modeling
Security and emerging services
Background: Research overview
• Design, modeling, and performance evaluation of
distributed systems and networks
• Current topics include




Scalable content delivery
Energy-efficient and sustainable ICT
Measurement analysis and modeling
Security and emerging services
Background: Research overview
• Design, modeling, and performance evaluation of
distributed systems and networks
• Current topics include




Scalable content delivery
Energy-efficient and sustainable ICT
Measurement analysis and modeling
Security and emerging services
Background: Research overview
• Design, modeling, and performance evaluation of
distributed systems and networks
• Current topics include




Scalable content delivery
Energy-efficient and sustainable ICT
Measurement analysis and modeling
Security and emerging services
Background: Research overview
• Design, modeling, and performance evaluation of
distributed systems and networks
• Current topics include




Scalable content delivery
Energy-efficient and sustainable ICT
Measurement analysis and modeling
Security and emerging services
Background: Research overview
• Design, modeling, and performance evaluation of
distributed systems and networks
• Current topics include




Scalable content delivery
Energy-efficient and sustainable ICT
Measurement analysis and modeling
Security and emerging services
Background: Research overview
• Design, modeling, and performance evaluation of
distributed systems and networks
• Current topics include




Scalable content delivery
Energy-efficient and sustainable ICT
Measurement analysis and modeling
Security and emerging services
Background: Research overview
• Design, modeling, and performance evaluation of
distributed systems and networks
• Current topics include




Scalable content delivery
Energy-efficient and sustainable ICT
Measurement analysis and modeling
Security and emerging services
Background: Research overview
• Design, modeling, and performance evaluation of
distributed systems and networks
• Current topics include




Scalable content delivery
Energy-efficient and sustainable ICT
Measurement analysis and modeling
Security and emerging services
Background: Research overview
• Design, modeling, and performance evaluation of
distributed systems and networks
• Current topics include




Scalable content delivery
Energy-efficient and sustainable ICT
Measurement analysis and modeling
Security and emerging services
Background: Research overview
• Design, modeling, and performance evaluation of
distributed systems and networks
• Current topics include




Scalable content delivery
Energy-efficient and sustainable ICT
Measurement analysis and modeling
Security and emerging services
Background: Research overview
• Design, modeling, and performance evaluation of
distributed systems and networks
• Current topics include




Scalable content delivery
Energy-efficient and sustainable ICT
Measurement analysis and modeling
Security and emerging services
Background: Research overview
• Design, modeling, and performance evaluation of
distributed systems and networks
• Current topics include




Scalable content delivery
Energy-efficient and sustainable ICT
Measurement analysis and modeling
Security and emerging services
Background: Research overview
• Design, modeling, and performance evaluation of
distributed systems and networks
• Current topics include




Scalable content delivery
Energy-efficient and sustainable ICT
Measurement analysis and modeling
Security and emerging services
Background: Research overview
• Design, modeling, and performance evaluation of
distributed systems and networks
• Current topics include




Scalable content delivery
Energy-efficient and sustainable ICT
Measurement analysis and modeling
Security and emerging services
Background: Research overview
• Design, modeling, and performance evaluation of
distributed systems and networks
• Current topics include




Scalable content delivery
Energy-efficient and sustainable ICT
Measurement analysis and modeling
Security and emerging services
Proc. ACM/SPEC ICPE 2013
Background: Research overview
• Design, modeling, and performance evaluation of
distributed systems and networks
• Current topics include




Scalable content delivery
Energy-efficient and sustainable ICT
Measurement analysis and modeling
Security and emerging services
Proc. ACM/SPEC ICPE 2013
Background: Research overview
• Design, modeling, and performance evaluation of
distributed systems and networks
• Current topics include




Scalable content delivery
Energy-efficient and sustainable ICT
Measurement analysis and modeling
Security and emerging services
Proc. ACM/SPEC ICPE 2011
Background: Research overview
• Design, modeling, and performance evaluation of
distributed systems and networks
• Current topics include




Scalable content delivery
Energy-efficient and sustainable ICT
Measurement analysis and modeling
Security and emerging services
Proc. ACM/SPEC ICPE 2011
Background: Research overview
• Design, modeling, and performance evaluation of
distributed systems and networks
• Current topics include




Scalable content delivery
Energy-efficient and sustainable ICT
Measurement analysis and modeling
Security and emerging services
Proc. ACM/SPEC ICPE 2011
Background: Research overview
• Design, modeling, and performance evaluation of
distributed systems and networks
• Current topics include




Scalable content delivery
Energy-efficient and sustainable ICT
Measurement analysis and modeling
Security and emerging services
Proc. ACM/SPEC ICPE 2011
Background: Research overview
• Design, modeling, and performance evaluation of
distributed systems and networks
• Current topics include




Scalable content delivery
Energy-efficient and sustainable ICT
Measurement analysis and modeling
Security and emerging services
Proc. ACM/SPEC ICPE 2011
Background: Research overview
• Design, modeling, and performance evaluation of
distributed systems and networks
• Current topics include




Scalable content delivery
Energy-efficient and sustainable ICT
Measurement analysis and modeling
Security and emerging services
Proc. ACM/SPEC ICPE 2011
Background: Research overview
• Design, modeling, and performance evaluation of
distributed systems and networks
• Current topics include




Scalable content delivery
Energy-efficient and sustainable ICT
Measurement analysis and modeling
Security and emerging services
Proc. ACM/SPEC ICPE 2011
Background: Research overview
• Design, modeling, and performance evaluation of
distributed systems and networks
• Current topics include




Scalable content delivery
Energy-efficient and sustainable ICT
Measurement analysis and modeling
Security and emerging services
Proc. ACM/SPEC ICPE 2011
Background: Research overview
• Design, modeling, and performance evaluation of
distributed systems and networks
• Current topics include




Scalable content delivery
Energy-efficient and sustainable ICT
Measurement analysis and modeling
Security and emerging services
Proc. ACM/SPEC ICPE 2011
Background: Research overview
• Design, modeling, and performance evaluation of
distributed systems and networks
• Current topics include




Scalable content delivery
Energy-efficient and sustainable ICT
Measurement analysis and modeling
Security and emerging services
• Design, modeling, and performance evaluation of
distributed systems and networks
• Current topics include




Scalable content delivery
Energy-efficient and sustainable ICT
Measurement analysis and modeling
Security and emerging services
ACM TWEB 2011
Proc. IFIP Performance
Background: Research overview
• Design, modeling, and performance evaluation of
distributed systems and networks
• Current topics include



Scalable content delivery
Energy-efficient and sustainable ICT
Measurement analysis and modeling
Security and emerging services
Views (v)

Rank (r)
Proc. IMC 2009
Background: Research overview
• Design, modeling, and performance evaluation of
distributed systems and networks
• Current topics include




Scalable content delivery
Energy-efficient and sustainable ICT
Measurement analysis and modeling
Security and emerging services
C D
B
A
Background: Research overview
• Design, modeling, and performance evaluation of
distributed systems and networks
• Current topics include




Scalable content delivery
Energy-efficient and sustainable ICT
Measurement analysis and modeling
Security and emerging services
ACM TWEB 2011
Background: Research overview
• Design, modeling, and performance evaluation of
distributed systems and networks
• Current topics include




Scalable content delivery
Energy-efficient and sustainable ICT
Measurement analysis and modeling
Security and emerging services
ACM TOIT 2011
Background: Research overview
• Design, modeling, and performance evaluation of
distributed systems and networks
• Current topics include




Scalable content delivery
Energy-efficient and sustainable ICT
Measurement analysis and modeling
Security and emerging services
Background: Research overview
• Design, modeling, and performance evaluation of
distributed systems and networks
• Current topics include




Scalable content delivery
Energy-efficient and sustainable ICT
Measurement analysis and modeling
Security and emerging services
Proc. PAM 2013
Background: Research overview
• Design, modeling, and performance evaluation of
distributed systems and networks
• Current topics include




Scalable content delivery
Energy-efficient and sustainable ICT
Measurement analysis and modeling
Security and emerging services
I own
66.174.161.0/20
Verizon
Internet
UK ISP
China
Telecom
Proc. PAM 2013
Background: Research overview
• Design, modeling, and performance evaluation of
distributed systems and networks
• Current topics include




Scalable content delivery
Energy-efficient and sustainable ICT
Measurement analysis and modeling
Security and emerging services
I own
66.174.161.0/20
I own
66.174.161.0/20
Verizon
Internet
UK ISP
China
Telecom
Proc. PAM 2013
Background: Research overview
• Design, modeling, and performance evaluation of
distributed systems and networks
• Current topics include




Scalable content delivery
Energy-efficient and sustainable ICT
Measurement analysis and modeling
Security and emerging services
I own
66.174.161.0/20
I own
66.174.161.0/20
Verizon
Internet
UK ISP
China
Telecom
Proc. PAM 2013
Background: Research overview
• Design, modeling, and performance evaluation of
distributed systems and networks
• Current topics include




Scalable content delivery
Energy-efficient and sustainable ICT
Measurement analysis and modeling
Security and emerging services
I own
24
I own
66.174.161.0/20
Verizon
66.174.161.0/
Internet
UK ISP
China
Telecom
Proc. PAM 2013
Background: Research overview
• Design, modeling, and performance evaluation of
distributed systems and networks
• Current topics include




Scalable content delivery
Energy-efficient and sustainable ICT
Measurement analysis and modeling
Security and emerging services
I own
24
I own
66.174.161.0/20
Verizon
66.174.161.0/
Internet
UK ISP
China
Telecom
Proc. PAM 2013
Background: Research overview
• Design, modeling, and performance evaluation of
distributed systems and networks
• Current topics include




Scalable content delivery
Energy-efficient and sustainable ICT
Measurement analysis and modeling
Security and emerging services
I own
24
I own
66.174.161.0/20
Verizon
66.174.161.0/
Internet
UK ISP
China
Telecom
Background: Research overview
• Design, modeling, and performance evaluation of
distributed systems and networks
• Current topics include




Scalable content delivery
Energy-efficient and sustainable ICT
Measurement analysis and modeling
Security and emerging services
Background: Research overview
• Design, modeling, and performance evaluation of
distributed systems and networks
• Current topics include




Scalable content delivery
Energy-efficient and sustainable ICT
Measurement analysis and modeling
Security and emerging services
• Active in the “performance” community
Methodology overview
• Measurements
• Data mining/analytics/statistics
• Modeling
• System and policy design
• Analytic modeling
• Optimization
• Implementation
• Evaluation
• Analytic, simulations, and instrumentation
A final note …
• I am looking for ambitious students
• Projects, BSc/MSc thesis, and PhD positions
• Please do not hesitate to contact me if you are
interested in one (or more) topic(s) related to
• Computer networks, distributed systems, security, ...
• Data mining/analytics/statistics, measurements,
analytical modeling, optimization, simulation,
implementation or system design
• There are always lots interesting problems!!!
• You can also find out more about my research
here: www.ida.liu.se/~nikca/
48
Thank you!
Niklas Carlsson (niklas.carlsson@liu.se)
Research overview and pubs: www.ida.liu.se/~nikca/
Dynamic Content Allocation for Cloud-assisted Service
of Periodic Workloads
György Dán and Niklas Carlsson
Proc. IEEE INFOCOM, 2014
Internet Content Delivery
• Large amounts of data with varying popularity
• Multi-billion market ($8B to $20B, 2012-2015)
• Goal: Minimize content delivery costs
• Migration to cloud data centers
Internet Content Delivery
Young videos
Old videos
E.g., Borghol et al., “Characterizing and Modeling Popularity
of User-generated Videos”, Proc. IFIP Performance, Oct. 2011.
• Large amounts of data with varying popularity
• Multi-billion market ($8B to $20B, 2012-2015)
• Goal: Minimize content delivery costs
• Migration to cloud data centers
Internet Content Delivery
• Large amounts of data with varying popularity
• Multi-billion market ($8B to $20B, 2012-2015)
• Goal: Minimize content delivery costs
• Migration to cloud data centers
Internet Content Delivery
• Large amounts of data with varying popularity
• Multi-billion market ($8B to $20B, 2012-2015)
• Goal: Minimize content delivery costs
• Migration to cloud data centers
Motivation
• Goal: Minimize content delivery costs
• Capped servers: fixed bandwidth (and storage) cap
• Elastic cloud bandwidth: flexible, but pays premium
• Dynamic content allocation: Want to utilize capped
bandwidth (and storage) as much as possible
cloud
servers
55
Motivation
• Goal: Minimize content delivery costs
• Capped servers: fixed bandwidth (and storage) cap
• Elastic cloud bandwidth: flexible, but pays premium
• Dynamic content allocation: Want to utilize capped
bandwidth (and storage) as much as possible
cloud
servers
56
peers
Cost minimization formulation
Cost minimization formulation
Total demand
Cost minimization formulation
Demand of files in
capped BW storage
Cost minimization formulation
Capped BW limit (U)
Cost minimization formulation
Cost minimization formulation
Served from capped
BW storage
Cost minimization formulation
Served using elastic
cloud resources
Cost minimization formulation
Traffic due to allocation
Cost minimization formulation
Cost minimization formulation
• Traffic of files only in cloud
• Spillover traffic
• Traffic due to allocation
• Total expected cost
• Optimal policy
Cost minimization formulation
• Traffic of files only in cloud
• Spillover traffic
• Traffic due to allocation
• Total expected cost
• Optimal policy
Cost minimization formulation
• Traffic of files only in cloud
• Spillover traffic
• Traffic due to allocation
• Total expected cost
• Optimal policy
Cost minimization formulation
• Traffic of files only in cloud
• Spillover traffic
• Traffic due to allocation
• Total expected cost
• Optimal policy
Cost minimization formulation
• Traffic of files only in cloud
• Spillover traffic
• Traffic due to allocation
• Total expected cost
• Optimal policy
Cost minimization formulation
• Traffic of files only in cloud
• Spillover traffic
• Traffic due to allocation
• Total expected cost
• Optimal policy
Cost minimization formulation
• Traffic of files only in cloud
• Spillover traffic
• Traffic due to allocation
• Total expected cost
• Optimal policy
Cost minimization formulation
• Traffic of files only in cloud
• Spillover traffic
• Traffic due to allocation
• Total expected cost
• Optimal policy
Utilization maximization
Cost minimization formulation
• Traffic of files only in cloud
• Spillover traffic
• Traffic due to allocation
• Equivalent formulation
• Total expected cost
• Optimal policy
Utilization maximization
Cost minimization formulation
• Traffic of files only in cloud
• Spillover traffic
• Traffic due to allocation
• Equivalent formulation
• Total expected cost
• Optimal policy
Utilization maximization
Cost minimization formulation
• Equivalent formulation
Utilization maximization
Cost minimization formulation
Two file example
• Equivalent formulation
Utilization maximization
Cost minimization formulation
Two file example
• Equivalent formulation
Utilization maximization
Cost minimization formulation
Two file example
• Equivalent formulation
Utilization maximization
Cost minimization formulation
Two file example
• Equivalent formulation
Utilization maximization
Cost minimization formulation
Two file example
• Equivalent formulation
Dynamic content allocation problem
• Formulate as a finite horizon dynamic
decision process problem
• Show discrete time decision process
is good approximation
• Exact solution as MILP
• Provide computationally feasible
approximations (and prove properties
about approximation ratios)
• Validate model and algorithms using
traces from Spotify
82
Caching and Optimized Request Routing in Cloud-based
Content Delivery Systems
Niklas Carlsson, Derek Eager, Ajay Gopinathan, and Zongpeng Li,
Proc. IFIP PERFORMANCE, 2014.
Internet Content Delivery
• Large amounts of data with varying popularity
• Multi-billion market ($8B to $20B, 2012-2015)
• Goal: Minimize content delivery costs
• Migration to geographically distributed cloud data centers
Motivation
• Geographically distributed cloud
• Elastic cloud bandwidth and storage
• When sufficiently expensive storage costs, not all contents
should be cached at all locations
85
Motivation
• Geographically distributed cloud
• Elastic cloud bandwidth and storage
• When sufficiently expensive storage costs, not all contents
should be cached at all locations
• Two policy questions arise
• What content should be cached where?
• How should requests be routed?
86
Motivation
• Geographically distributed cloud
• Elastic cloud bandwidth and storage
• When sufficiently expensive storage costs, not all contents
should be cached at all locations
• Two policy questions arise
• What content should be cached where?
• How should requests be routed?
Motivation
• Geographically distributed cloud
• Elastic cloud bandwidth and storage
• When sufficiently expensive storage costs, not all contents
should be cached at all locations
• Two policy questions arise
• What content should be cached where?
• How should requests be routed?
Motivation
• Geographically distributed cloud
• Elastic cloud bandwidth and storage
• When sufficiently expensive storage costs, not all contents
should be cached at all locations
• Two policy questions arise
• What content should be cached where?
• How should requests be routed?
Motivation
• Geographically distributed cloud
• Elastic cloud bandwidth and storage
• When sufficiently expensive storage costs, not all contents
should be cached at all locations
• Two policy questions arise
• What content should be cached where?
• How should requests be routed?
90
Request routing optimization
• Minimize content delivery costs
• Cache miss cost
• Cache storage cost
• Remote routing cost
91
Request routing optimization
Aggregate request
rate at server
location i
• Minimize content delivery costs
• Cache miss cost
• Cache storage cost
• Remote routing cost
92
Request routing optimization
Remote routing cost
Cache storage cost
Cache miss cost
• Minimize content delivery costs
• Cache miss cost
• Cache storage cost
• Remote routing cost
93
Request routing optimization
Cache miss cost
• Minimize content delivery costs
• Cache miss cost
• Cache storage cost
• Remote routing cost
94
Request routing optimization
Cache storage cost
• Minimize content delivery costs
• Cache miss cost
• Cache storage cost
• Remote routing cost
95
Request routing optimization
Remote routing cost
• Minimize content delivery costs
• Cache miss cost
• Cache storage cost
• Remote routing cost
96
Request routing optimization
Remote routing cost
Cache storage cost
Cache miss cost
• Minimize content delivery costs
• Cache miss cost
• Cache storage cost
• Remote routing cost
97
Aggregate request
rate at server
location i
Request routing optimization
Conservation constraints
• Minimize content delivery costs
• Cache miss cost
• Cache storage cost
• Remote routing cost
98
Request routing optimization
• Minimize content delivery costs
• Cache miss cost
• Cache storage cost
• Remote routing cost
99
Request rate at location
Properties of optimal request routing
Either all request served locally or all request served remotely
[Theorem 4]
For Theorem 5 [sets and properties], first …
Order server location based on request rate
Request rate rank of location
Request rate at location
Properties of optimal request routing
Four (4) potentially empty sets of server locations
S4
S3
S2
Request rate rank of location
S1
Request rate at location
Properties of optimal request routing
S4
S3
S2
Request rate rank of location
S1
Properties of optimal request routing
Request rate at location
Servers in set S2 and S4 serves only local request
Servers in set S3 serve both local and remote
Servers in set S3 serve the same request rates
Servers in set S1 inactive
S4
S3
S2
Request rate rank of location
S1
Properties of optimal request routing
Request rate at location
Servers in set S2 and S4 serves only local request
Servers in set S3 serve both local and remote
Servers in set S3 serve the same request rates
Servers in set S1 inactive
S4
S3
S2
Request rate rank of location
S1
Properties of optimal request routing
Request rate at location
Servers in set S2 and S4 serves only local request
Servers in set S3 serve both local and remote
Servers in set S3 serve the same request rates
Servers in set S1 inactive
S4
S3
S2
Request rate rank of location
S1
Properties of optimal request routing
Request rate at location
Servers in set S2 and S4 serves only local request
Servers in set S3 serve both local and remote
Servers in set S3 serve the same request rates
Servers in set S1 inactive
S4
S3
S2
Request rate rank of location
S1
Properties of optimal request routing
Request rate at location
Servers in set S2 and S4 serves only local request
Servers in set S3 serve both local and remote
Servers in set S3 serve the same request rates
Servers in set S1 inactive
S4
S3
S2
Request rate rank of location
S1
Request rate at location
Finding the optimal request routing
O(N2) candidate solution to consider;
each at a computational cost O(N)
S4
S3
S2
Request rate rank of location
S1
Request rate at location
Finding the optimal request routing
O(N2) candidate solution to consider;
each at a computational cost O(N)
Note: Size of S1 and S2 decides the rest
S4
S3
S2
Request rate rank of location
S1
Contributions
• Propose new delivery approach using distributed clouds
• Request routing periodically updated
• Cache content updated dynamically
• Formulate optimization problem
• Non-convex, so standard techniques not directly applicable
• Identify and prove properties of optimal solution
• Leverage properties to find optimal solution
• Comparison with optimal static placement and routing, as
well as with baseline policies
• Present a lower-cost approximation solution that achieve
within 2.5% of optimum
110
Methodology overview
• Measurements
• Data mining/analytics/statistics
• Modeling
• System and policy design
• Analytic modeling
• Optimization
• Implementation
• Evaluation
• Analytic, simulations, and instrumentation
A final note …
• I am looking for ambitious students
• Projects, BSc/MSc thesis, and PhD positions
• Please do not hesitate to contact me if you are
interested in one (or more) topic(s) related to
• Computer networks, distributed systems, security, ...
• Data mining/analytics/statistics, measurements,
analytical modeling, optimization, simulation,
implementation or system design
• There are always lots interesting problems!!!
• You can also find out more about my research
here: www.ida.liu.se/~nikca/
112
Thank you!
Niklas Carlsson (niklas.carlsson@liu.se)
Research overview and pubs: www.ida.liu.se/~nikca/
Helping Hand or Hidden Hurdle: Proxy-assisted HTTPbased Adaptive Streaming Performance
Vengatanathan Krishnamoorthi, Niklas Carlsson, Derek Eager, Anirban
Mahanti, and Nahid Shahmehri,
Proc. IEEE MASCOTS, 2013.
HTTP-based adaptive Streaming (HAS)
1,4
2,4
3,4
4,4
5,4
6,4
7,4
1,4
2,4
3,4
4,4
5,4
6,4
7,4
1,3
2,3
3,3
4,3
5,3
6,3
7,3
1,3
2,3
3,3
4,3
5,3
6,3
7,3
1,2
2,2
3,2
4,2
5,2
6,2
7,2
1,2
2,2
3,2
4,2
5,2
6,2
7,2
1,1
2,1
3,1
4,1
5,1
6,1
7,1
1,1
2,1
3,1
4,1
5,1
6,1
7,1
Client
Server
115
HTTP-based adaptive Streaming (HAS)
1,4
2,4
3,4
4,4
5,4
6,4
7,4
1,4
2,4
3,4
4,4
5,4
6,4
7,4
1,3
2,3
3,3
4,3
5,3
6,3
7,3
1,3
2,3
3,3
4,3
5,3
6,3
7,3
1,2
2,2
3,2
4,2
5,2
6,2
7,2
1,2
2,2
3,2
4,2
5,2
6,2
7,2
1,1
2,1
3,1
4,1
5,1
6,1
7,1
1,1
2,1
3,1
4,1
5,1
6,1
7,1
Client
Server
116
HTTP-based adaptive Streaming (HAS)
1,4
2,4
3,4
4,4
5,4
6,4
7,4
1,4
2,4
3,4
4,4
5,4
6,4
7,4
1,3
2,3
3,3
4,3
5,3
6,3
7,3
1,3
2,3
3,3
4,3
5,3
6,3
7,3
1,2
2,2
3,2
4,2
5,2
6,2
7,2
1,2
2,2
3,2
4,2
5,2
6,2
7,2
1,1
2,1
3,1
4,1
5,1
6,1
7,1
1,1
2,1
3,1
4,1
5,1
6,1
7,1
Client
Server
117
HTTP-based adaptive Streaming (HAS)
1,4
2,4
3,4
4,4
5,4
6,4
7,4
1,4
2,4
3,4
4,4
5,4
6,4
7,4
1,3
2,3
3,3
4,3
5,3
6,3
7,3
1,3
2,3
3,3
4,3
5,3
6,3
7,3
1,2
2,2
3,2
4,2
5,2
6,2
7,2
1,2
2,2
3,2
4,2
5,2
6,2
7,2
1,1
2,1
3,1
4,1
5,1
6,1
7,1
1,1
2,1
3,1
4,1
5,1
6,1
7,1
Client
Server
118
HTTP-based adaptive Streaming (HAS)
1,4
2,4
3,4
4,4
5,4
6,4
7,4
1,4
2,4
3,4
4,4
5,4
6,4
7,4
1,3
2,3
3,3
4,3
5,3
6,3
7,3
1,3
2,3
3,3
4,3
5,3
6,3
7,3
1,2
2,2
3,2
4,2
5,2
6,2
7,2
1,2
2,2
3,2
4,2
5,2
6,2
7,2
1,1
2,1
3,1
4,1
5,1
6,1
7,1
1,1
2,1
3,1
4,1
5,1
6,1
7,1
Client
Server
119
HTTP-based adaptive Streaming (HAS)
1,4
2,4
3,4
4,4
5,4
6,4
7,4
1,4
2,4
3,4
4,4
5,4
6,4
7,4
1,3
2,3
3,3
4,3
5,3
6,3
7,3
1,3
2,3
3,3
4,3
5,3
6,3
7,3
1,2
2,2
3,2
4,2
5,2
6,2
7,2
1,2
2,2
3,2
4,2
5,2
6,2
7,2
1,1
2,1
3,1
4,1
5,1
6,1
7,1
1,1
2,1
3,1
4,1
5,1
6,1
7,1
Client
Server
120
HTTP-based adaptive Streaming (HAS)
1,4
2,4
3,4
4,4
5,4
6,4
7,4
1,4
2,4
3,4
4,4
5,4
6,4
7,4
1,3
2,3
3,3
4,3
5,3
6,3
7,3
1,3
2,3
3,3
4,3
5,3
6,3
7,3
1,2
2,2
3,2
4,2
5,2
6,2
7,2
1,2
2,2
3,2
4,2
5,2
6,2
7,2
1,1
2,1
3,1
4,1
5,1
6,1
7,1
1,1
2,1
3,1
4,1
5,1
6,1
7,1
Client
Server
121
HTTP-based adaptive Streaming (HAS)
1,4
2,4
3,4
4,4
5,4
6,4
7,4
1,4
2,4
3,4
4,4
5,4
6,4
7,4
1,3
2,3
3,3
4,3
5,3
6,3
7,3
1,3
2,3
3,3
4,3
5,3
6,3
7,3
1,2
2,2
3,2
4,2
5,2
6,2
7,2
1,2
2,2
3,2
4,2
5,2
6,2
7,2
1,1
2,1
3,1
4,1
5,1
6,1
7,1
1,1
2,1
3,1
4,1
5,1
6,1
7,1
Client
Server
122
Problem: Proxy-assisted HAS
In this paper …
• Evaluation of proxy-assisted HAS policies
123
Example: Default “best-effort”
124
Example: Default “best-effort”
1,4
2,4
3,4
4,4
5,4
6,4
7,4
1,4
2,4
3,4
4,4
5,4
6,4
7,4
1,3
2,3
3,3
4,3
5,3
6,3
7,3
1,3
2,3
3,3
4,3
5,3
6,3
7,3
1,2
2,2
3,2
4,2
5,2
6,2
7,2
1,2
2,2
3,2
4,2
5,2
6,2
7,2
1,1
2,1
3,1
4,1
5,1
6,1
7,1
1,1
2,1
3,1
4,1
5,1
6,1
7,1
Client
125
Proxy
Example: Default “best-effort”
1,4
2,4
3,4
4,4
5,4
6,4
7,4
1,3
2,3
3,3
4,3
5,3
6,3
7,3
1,2
2,2
3,2
4,2
5,2
6,2
7,2
1,1
2,1
3,1
4,1
5,1
6,1
7,1
Client 1
126
1,4
2,4
3,4
4,4
5,4
6,4
7,4
1,3
2,3
3,3
4,3
5,3
6,3
7,3
1,2
2,2
3,2
4,2
5,2
6,2
7,2
1,1
2,1
3,1
4,1
5,1
6,1
7,1
Proxy before
1,4
2,4
3,4
4,4
5,4
6,4
7,4
1,3
2,3
3,3
4,3
5,3
6,3
7,3
1,2
2,2
3,2
4,2
5,2
6,2
7,2
1,1
2,1
3,1
4,1
5,1
6,1
7,1
Proxy after
Example: Default “best-effort”
1,4
2,4
3,4
4,4
5,4
6,4
7,4
1,3
2,3
3,3
4,3
5,3
6,3
7,3
1,2
2,2
3,2
4,2
5,2
6,2
7,2
1,1
2,1
3,1
4,1
5,1
6,1
7,1
Client 1
127
1,4
2,4
3,4
4,4
5,4
6,4
7,4
1,3
2,3
3,3
4,3
5,3
6,3
7,3
1,2
2,2
3,2
4,2
5,2
6,2
7,2
1,1
2,1
3,1
4,1
5,1
6,1
7,1
Proxy before
1,4
2,4
3,4
4,4
5,4
6,4
7,4
1,3
2,3
3,3
4,3
5,3
6,3
7,3
1,2
2,2
3,2
4,2
5,2
6,2
7,2
1,1
2,1
3,1
4,1
5,1
6,1
7,1
Proxy after
Example: Default “best-effort”
1,4
2,4
3,4
4,4
5,4
6,4
7,4
1,3
2,3
3,3
4,3
5,3
6,3
7,3
1,2
2,2
3,2
4,2
5,2
6,2
7,2
1,1
2,1
3,1
4,1
5,1
6,1
7,1
Client 2
128
1,4
2,4
3,4
4,4
5,4
6,4
7,4
1,3
2,3
3,3
4,3
5,3
6,3
7,3
1,2
2,2
3,2
4,2
5,2
6,2
7,2
1,1
2,1
3,1
4,1
5,1
6,1
7,1
Proxy before
1,4
2,4
3,4
4,4
5,4
6,4
7,4
1,3
2,3
3,3
4,3
5,3
6,3
7,3
1,2
2,2
3,2
4,2
5,2
6,2
7,2
1,1
2,1
3,1
4,1
5,1
6,1
7,1
Proxy after
Example: Default “best-effort”
1,4
2,4
3,4
4,4
5,4
6,4
7,4
1,3
2,3
3,3
4,3
5,3
6,3
7,3
1,2
2,2
3,2
4,2
5,2
6,2
7,2
1,1
2,1
3,1
4,1
5,1
6,1
7,1
Client 2
129
1,4
2,4
3,4
4,4
5,4
6,4
7,4
1,3
2,3
3,3
4,3
5,3
6,3
7,3
1,2
2,2
3,2
4,2
5,2
6,2
7,2
1,1
2,1
3,1
4,1
5,1
6,1
7,1
Proxy before
1,4
2,4
3,4
4,4
5,4
6,4
7,4
1,3
2,3
3,3
4,3
5,3
6,3
7,3
1,2
2,2
3,2
4,2
5,2
6,2
7,2
1,1
2,1
3,1
4,1
5,1
6,1
7,1
Proxy after
Example: Default “best-effort”
1,4
2,4
3,4
4,4
5,4
6,4
7,4
1,3
2,3
3,3
4,3
5,3
6,3
7,3
1,2
2,2
3,2
4,2
5,2
6,2
7,2
1,1
2,1
3,1
4,1
5,1
6,1
7,1
Client 3
130
1,4
2,4
3,4
4,4
5,4
6,4
7,4
1,3
2,3
3,3
4,3
5,3
6,3
7,3
1,2
2,2
3,2
4,2
5,2
6,2
7,2
1,1
2,1
3,1
4,1
5,1
6,1
7,1
Proxy before
1,4
2,4
3,4
4,4
5,4
6,4
7,4
1,3
2,3
3,3
4,3
5,3
6,3
7,3
1,2
2,2
3,2
4,2
5,2
6,2
7,2
1,1
2,1
3,1
4,1
5,1
6,1
7,1
Proxy after
Example: Default “best-effort”
1,4
2,4
3,4
4,4
5,4
6,4
7,4
1,3
2,3
3,3
4,3
5,3
6,3
7,3
1,2
2,2
3,2
4,2
5,2
6,2
7,2
1,1
2,1
3,1
4,1
5,1
6,1
7,1
Client 3
131
1,4
2,4
3,4
4,4
5,4
6,4
7,4
1,3
2,3
3,3
4,3
5,3
6,3
7,3
1,2
2,2
3,2
4,2
5,2
6,2
7,2
1,1
2,1
3,1
4,1
5,1
6,1
7,1
Proxy before
1,4
2,4
3,4
4,4
5,4
6,4
7,4
1,3
2,3
3,3
4,3
5,3
6,3
7,3
1,2
2,2
3,2
4,2
5,2
6,2
7,2
1,1
2,1
3,1
4,1
5,1
6,1
7,1
Proxy after
Policies and policy classes
I have these
fragments
I have this buffer
occupancy
Proxy works ahead using prefetching
Client picks from these qualities
(when possible)
Client-proxy cooperation policies
• Buffer oblivious (priority-based prefetching)
• Buffer aware (conservative quality during low buffer conditions)
132
Thank you!
Niklas Carlsson (niklas.carlsson@liu.se)
Research overview and pubs: www.ida.liu.se/~nikca/
Quality-adaptive Prefetching for Interactive Branched
Video using HTTP-based Adaptive Streaming
V. Krishnamoorthi, N. Carlsson, D. Eager, A. Mahanti, and N. Shahmehri
Proc. ACM Multimedia, 2014.
Empowering the Creative User: Personalized HTTPbased Adaptive Streaming of Multi-path Nonlinear Video
V. Krishnamoorthi, P. Bergström, N. Carlsson, D. Eager, A. Mahanti, and N. Shahmehri
Proc. ACM SIGCOMM Workshop on Future Human-Centric Multimedia
Networking (FhMN), 2013 (Also in special best-paper issue of ACM CCR).
Best paper award
Most of us have seen Tom & Jerry
movies …
135
Most of us have seen Tom & Jerry
movies …
What if they could create their own
movie versions??
136
For a minute...think that you are Tom
Dislikes:
Likes:
137
Now, lets look at his rival Jerry
Dislikes:
Likes:
138
Contributions (example paper: FhMN ‘13)
• Framework that allows the creator to easily create
customized playback experiences for the viewer
• Combines ideas of personalized multi-path video and
HTTP-based adaptive streaming (HAS)
139
Contributions (example paper: FhMN ‘13)
• Framework that allows the creator to easily create
customized playback experiences for the viewer
• Combines ideas of personalized multi-path video and
HTTP-based adaptive streaming (HAS)
140
Contributions (example paper: FhMN ‘13)
• Framework that allows the creator to easily create
customized playback experiences for the viewer
• Combines ideas of personalized multi-path video and
HTTP-based adaptive streaming (HAS)
141
Contributions (example paper: FhMN ‘13)
• Framework that allows the creator to easily create
customized playback experiences for the viewer
• Combines ideas of personalized multi-path video and
HTTP-based adaptive streaming (HAS)
142
Contributions (example paper: FhMN ‘13)
• Framework that allows the creator to easily create
customized playback experiences for the viewer
• Combines ideas of personalized multi-path video and
HTTP-based adaptive streaming (HAS)
• Viewer traverse through the video by interacting with the
player and choose among multiple path options
143
Contributions (example paper: FhMN ‘13)
• Framework that allows the creator to easily create
customized playback experiences for the viewer
• Combines ideas of personalized multi-path video and
HTTP-based adaptive streaming (HAS)
• Viewer traverse through the video by interacting with the
player and choose among multiple path options
144
Contributions (example paper: FhMN ‘13)
• Framework that allows the creator to easily create
customized playback experiences for the viewer
• Combines ideas of personalized multi-path video and
HTTP-based adaptive streaming (HAS)
• Viewer traverse through the video by interacting with the
player and choose among multiple path options
145
Contributions (example paper: FhMN ‘13)
• Framework that allows the creator to easily create
customized playback experiences for the viewer
• Combines ideas of personalized multi-path video and
HTTP-based adaptive streaming (HAS)
• Viewer traverse through the video by interacting with the
player and choose among multiple path options
• Seamless video playback
146
Contributions (example paper: FhMN ‘13)
• Framework that allows the creator to easily create
customized playback experiences for the viewer
• Combines ideas of personalized multi-path video and
HTTP-based adaptive streaming (HAS)
• Viewer traverse through the video by interacting with the
player and choose among multiple path options
• Seamless video playback
•
Rate-adaptive prefetching and buffer management based on
current network conditions
147
Contributions (example paper: FhMN ‘13)
• Framework that allows the creator to easily create
customized playback experiences for the viewer
• Combines ideas of personalized multi-path video and
HTTP-based adaptive streaming (HAS)
• Viewer traverse through the video by interacting with the
player and choose among multiple path options
• Seamless video playback
•
Rate-adaptive prefetching and buffer management based on
current network conditions
148
Contributions (example paper: FhMN ‘13)
• Framework that allows the creator to easily create
customized playback experiences for the viewer
• Combines ideas of personalized multi-path video and
HTTP-based adaptive streaming (HAS)
• Viewer traverse through the video by interacting with the
player and choose among multiple path options
• Seamless video playback
•
Rate-adaptive prefetching and buffer management based on
current network conditions
149
Characterizing Large-scale Routing Anomalies: A Case
Study of the China Telecom Incident
Rahul Hiran, Niklas Carlsson, and Phillipa Gill
Proc. PAM, 2013
China Telecom incident
151
China Telecom incident
• The incident occurred on 8th April 2010
• The congress report, 2010 in USA mentions the incident
• Questions about what was done with the data: attack or accident
• We characterize this incident using only publicly available data such
as Routeviews, iPlane
152
China Telecom incident
• The incident occurred on 8th April 2010
• The congress report, 2010 in USA mentions the incident
• Questions about what was done with the data: attack or accident
• We characterize this incident using only publicly available data such
as Routeviews, iPlane
I own
66.174.161.0/20
Verizon
Internet
UK ISP
153
China
Telecom
China Telecom incident
• The incident occurred on 8th April 2010
• The congress report, 2010 in USA mentions the incident
• Questions about what was done with the data: attack or accident
• We characterize this incident using only publicly available data such
as Routeviews, iPlane
I own
66.174.161.0/20
I own
66.174.161.0/20
Verizon
Internet
UK ISP
154
China
Telecom
China Telecom incident
• The incident occurred on 8th April 2010
• The congress report, 2010 in USA mentions the incident
• Questions about what was done with the data: attack or accident
• We characterize this incident using only publicly available data such
as Routeviews, iPlane
I own
66.174.161.0/20
I own
66.174.161.0/20
Verizon
Internet
UK ISP
155
China
Telecom
China Telecom incident
• The incident occurred on 8th April 2010
• The congress report, 2010 in USA mentions the incident
• Questions about what was done with the data: attack or accident
• We characterize this incident using only publicly available data such
as Routeviews, iPlane
I own
24
I own
66.174.161.0/20
Verizon
66.174.161.0/
Internet
UK ISP
156
China
Telecom
China Telecom incident
• The incident occurred on 8th April 2010
• The congress report, 2010 in USA mentions the incident
• Questions about what was done with the data: attack or accident
• We characterize this incident using only publicly available data such
as Routeviews, iPlane
I own
24
I own
66.174.161.0/20
Verizon
66.174.161.0/
Internet
UK ISP
157
China
Telecom
China Telecom incident
• The incident occurred on 8th April 2010
• The congress report, 2010 in USA mentions the incident
• Questions about what was done with the data: attack or accident
• We characterize this incident using only publicly available data such
as Routeviews, iPlane
I own
24
I own
66.174.161.0/20
Verizon
66.174.161.0/
Internet
UK ISP
158
China
Telecom
China Telecom incident
• The incident occurred on 8th April 2010
• The congress report, 2010 in USA mentions the incident
• Questions about what was done with the data: attack or accident
• We characterize this incident using only publicly available data such
as Routeviews, iPlane
I own
24
I own
66.174.161.0/20
Verizon
66.174.161.0/
Internet
UK ISP
159
China
Telecom
Third-party Identity Management Usage on the Web
Anna Vapen, Niklas Carlsson, Anirban Mahanti, and Nahid Shahmehri
Proc. PAM, 2014
Third-party Web Authentication
Web Authentication
• Registration with each website
• Many passwords to remember
Third-party authentication
• Use an existing IDP (identity provider) account
to access an RP (relying party)
• Log in less often; Stronger authentication
• Increase personalization opportunities
• Share information between websites
161
Methodology
• Popularity-based logarithmic sampling
• 80,000 points uniformly on a logarithmic range
• Capturing data from different popularity segments
1 million
most
popular
websites
162
Sampled
websites
Methodology (2)
• Selenium-based crawling and relationship identification
• Able to process Web 2.0 sites with interactive elements
• Low number of false positives
• Validation with semi-manual classification and text-matching
1 million
most
popular
websites
163
Sampled
websites
IDP Selection
• Popular sites as IDPs, instead of specialized IDPs
Popular sites with
• Lots of existing users
• Personal information
Specialized IDPs with stronger
authentication methods
164
Comparison with Content Services
• Content: scripts, images and other third-party objects
• IDPs much more popular sites than content providers
165
Cultural and Geographical Analysis
• North American and Chinese RPs use local IDPs to a large extent
• Content delivery usage less biased to local providers
North America
Europe
China
Russia
Asia (all)
Identity
management
Content
delivery
166
North America
China
Asia (rest)
Europe
Russia
Other
Thank you!
Niklas Carlsson (niklas.carlsson@liu.se)
Research overview and pubs: www.ida.liu.se/~nikca/