ITU Kaleidoscope 2015
Trust in the Information Society
A unified framework for
Internet Speed Measurements
Eduardo Saiz
Faculty of Engineering of Bilbao
University of the Basque Country (UPV/EHU), Spain
eduardo.saiz@ehu.eus
Barcelona, Spain
9-11 December 2015
Outline
•
•
•
•
•
Introduction
Background
Framework
Test procedure
Conclusions
Barcelona, Spain, 9-11 December 2015
ITU Kaleidoscope 2015 - Trust in the Information Society
1
Introduction
•
•
•
•
•
Introduction
Background
Framework
Test procedure
Conclusions
Barcelona, Spain, 9-11 December 2015
ITU Kaleidoscope 2015 - Trust in the Information Society
Introduction
• The challenge:
– Users demand QoS
– ICT players verify SLAs
– Providers: ensure users trust
USERS
trust.
OPERATORS
& PROVIDERS
REGULATORS
Barcelona, Spain, 9-11 December 2015
ITU Kaleidoscope 2015 - Trust in the Information Society
2
Introduction
• The challenge:
– Users demand QoS
– ICT players verify SLAs
– Providers: ensure users trust
SPEEDTEST
USERS
trust.
OPERATORS
& PROVIDERS
REGULATORS
RTR-NetTest
Barcelona, Spain, 9-11 December 2015
ITU Kaleidoscope 2015 - Trust in the Information Society
2
Introduction
Sample Case: Orange Spain – FTTH 30Mbps/5Mbps
BILBAO (ES)
MADRID (ES)
AMSTERDAM (NL)
VIRGINIA (US)
20.412 Mbps
10.105 Mbps
6.788 Mbps
4.047 Mbps
3.871 Mbps
2.802 Mbps
Barcelona, Spain, 9-11 December 2015
ITU Kaleidoscope 2015 - Trust in the Information Society
3
Background
•
•
•
•
•
Introduction
Background
Framework
Test procedure
Conclusions
Barcelona, Spain, 9-11 December 2015
ITU Kaleidoscope 2015 - Trust in the Information Society
Background
• Lack of standardization on
Internet speed measurement
• ITU-T SG11:
Internet Speed
Measurement
by End-Users
ITU-T Draft
Q. FW_Sp_Test
Framework &
Test Procedure
ITU-T Draft
Q. TM_Sp_Test
Testing
Methodologies
ITU-T Draft
Q. Int_Speed_Test
Barcelona, Spain, 9-11 December 2015
ITU Kaleidoscope 2015 - Trust in the Information Society
4
The Framework
•
•
•
•
•
Introduction
Background
Framework
Test procedure
Conclusions
Barcelona, Spain, 9-11 December 2015
ITU Kaleidoscope 2015 - Trust in the Information Society
The Framework
Scope
•
•
•
•
Measurement scenarios
Measurement parameters
Measurement procedures
Requirements for the measurement algorithm
The key goal of the framework is to provide
transparent, trust-based measurement results
that can be accepted by all ICT players.
ITU-T Q.FW_Sp_Test Draft
Barcelona, Spain, 9-11 December 2015
ITU Kaleidoscope 2015 - Trust in the Information Society
5
The Framework
Global scenario
Measurement Agent
MA
Exchange Point
EP
Telecom
Operator
Internet Resource
IR
Internet
Network Internet speed test path
A
B
Internet resource speed test path
Barcelona, Spain, 9-11 December 2015
ITU Kaleidoscope 2015 - Trust in the Information Society
6
The Framework
Test Facilities
CONTROLLER
DATA
ACQUISITION
COLLECTOR
DB
CONTROLLING
& MEASUREMENT
RESULTS
MA
MEASUREMENT
AGENT
RESULT
DATA
PARAMETERS
MEASUREMENT
Barcelona, Spain, 9-11 December 2015
ITU Kaleidoscope 2015 - Trust in the Information Society
RESULTS
RESULT
DATA
MEASUREMENT
PEERS
MP
7
The Framework
System Architecture
LEGEND
MA
MP
EP
IR
Measurement Agent
Measurement Peer
Exchange Point
Internet Resource
Internet Speed
Measurement System
CONTROLLER
COLLECTOR
At regional/national level
MP
MA
Telecom Operator
(fixed / mobile)
CUSTOMER
MP
Internet
EP
A
B
IR
Network Internet speed test path
Internet resource speed test path
Barcelona, Spain, 9-11 December 2015
ITU Kaleidoscope 2015 - Trust in the Information Society
8
The Framework
A. Network Internet speed test
MA
MP
End-to-end testing
Telecom
Operator
(fixed / mobile)
Internet
External
Interface of EP
Network Internet speed test
LEGEND
Barcelona, Spain, 9-11 December 2015
ITU Kaleidoscope 2015 - Trust in the Information Society
MA
MP
EP
IR
Measurement Agent
Measurement Peer
Exchange Point
Internet Resource
9
The Framework
B. Internet resource speed test
MP
Close
location
MA
Telecom
Operator
(fixed / mobile)
Internet
IR
EP
Internet resource speed test
LEGEND
Barcelona, Spain, 9-11 December 2015
ITU Kaleidoscope 2015 - Trust in the Information Society
MA
MP
EP
IR
Measurement Agent
Measurement Peer
Exchange Point
Internet Resource
10
Test procedure
•
•
•
•
•
Introduction
Background
The Framework
Test procedure
Conclusions
Barcelona, Spain, 9-11 December 2015
ITU Kaleidoscope 2015 - Trust in the Information Society
Test procedure
Test Phases
EXCHANGE INTERNET
POINT RESOURCE
MEASUREMENT
AGENT
TEST
INITIALIZATION
ITU-T Draft
Q. TM_Sp_Test
TEST
EXECUTION
TEST
FINALIZATION
MEASUREMENT
PEER
DATA
ACQUISITION
COLLECTOR
WEB
SITE
CONTROLLER
1
2
3
2.1
2.2
2.3
2.4
2.5
DOWNLINK PRE-TEST
LATENCY
DOWNLINK TEST
UPLINK PRE-TEST
UPLINK TEST
MEASUREMENT
RESULTS
Barcelona, Spain, 9-11 December 2015
ITU Kaleidoscope 2015 - Trust in the Information Society
MEASUREMENT
RESULTS
13
Test procedure
Test Parameters
• Download data transmission speed
MEASUREMENT
AGENT
DOWNLINK
MEASUREMENT
PEER
• Upload data transmission speed
MEASUREMENT
AGENT
UPLINK
MEASUREMENT
PEER
• Two-way delay (RTT)
– Twice “the time required for a packet to traverse the
network or a segment of the network” (ITU-T G.1050)
Barcelona, Spain, 9-11 December 2015
ITU Kaleidoscope 2015 - Trust in the Information Society
14
Test procedure
Issues under study
ITU-T Draft
Q. TM_Sp_Test
Limited data
Max timeout
Adaptable data
Max timeout
Barcelona, Spain, 9-11 December 2015
ITU Kaleidoscope 2015 - Trust in the Information Society
Testing
Methodologies
Unlimited data
Limited time
Unlimited data
Until stabilized
15
Test procedure
Methodology concerns
Limited data
Max timeout
ITU-T Draft
Q. TM_Sp_Test
Testing
Methodologies
Under study (2016)
Adaptable data
Max timeout
Unlimited data
Limited time
Unlimited data
Until stabilized
• Protocols to be used
• Additional/optional parameters to be measured
• Impact of different OS & browsers…
Barcelona, Spain, 9-11 December 2015
ITU Kaleidoscope 2015 - Trust in the Information Society
15
Conclusions
•
•
•
•
•
Introduction
Background
The Framework
Test procedure
Conclusions
Barcelona, Spain, 9-11 December 2015
ITU Kaleidoscope 2015 - Trust in the Information Society
Conclusions
• Framework:
– Neutral and trust-based approach
to unify Internet Speed estimation
• Operators: Reliable tool for SLA verification
• Users: Valuable information, comparative results
– Global QoS observatory
• Strengths and weaknesses of operator’s network
• Extendable to other Internet based services
Barcelona, Spain, 9-11 December 2015
ITU Kaleidoscope 2015 - Trust in the Information Society
16
THANK YOU
A unified framework for
Internet Speed Measurements
Eduardo Saiz, Eva Ibarrola, Eneko Atxutegi, Fidel Liberal
{eduardo.saiz, eva.ibarrola, eneko.atxutegi, fidel.liberal}@ehu.eus
University of the Basque Country (UPV/EHU), Spain
ITU Kaleidoscope 2015
Trust in the Information Society
Barcelona, Spain
9-11 December 2015
QoXphere
(ITU-T Kaleidoscope 2013)
Barcelona, Spain, 9-11 December 2015
ITU Kaleidoscope 2015 - Trust in the Information Society
A
Introduction
Trust: User fidelity
QoXphere Framework
E. Ibarrola, Kaleidoscope 2013
QoS BUSINESS
CUSTOMER
FIDELITY
ASSESSED
QoS
INTRINSIC
QoS
PERCEIVED
QoS
Barcelona, Spain, 9-11 December 2015
ITU Kaleidoscope 2015 - Trust in the Information Society
B
Basic methodology
Test Execution: Steps
MA
MP
Request Pre-DL
ACK
Get Time
OK
Step 2.1
Downlink Pre-test
Downlink pretest
duration time Tp
Request Pre-DL
ACK
Get Time
OK
STEP 1. DOWNLINK PRE-TEST
Barcelona, Spain, 9-11 December 2015
ITU Kaleidoscope 2015 - Trust in the Information Society
C
Basic methodology
Test Execution: Steps
MA
MP
PING#1
1
ACK#
OK
Get Time
Step 2.2
Latency
Subtest
PING#1
0
10
ACK#
OK
Get Time
LTS
RESU
STEP 2. LACENCY SUBTEST
Barcelona, Spain, 9-11 December 2015
ITU Kaleidoscope 2015 - Trust in the Information Society
D
Basic methodology
Test Execution: Steps
MP
MA
Request to send data to
Server
Data Stream
Step 2.3
Downlink
test
Downlink subtest
duration time Td
Get Time
OK
LTS
RESU
STEP 3. DOWNLINK TEST
Barcelona, Spain, 9-11 December 2015
ITU Kaleidoscope 2015 - Trust in the Information Society
E
Basic methodology
Test Execution: Steps
MA
MP
Request to send data to
server
OK
Data Streams
Accept
Step 2.4
Uplink Pre-test
Uplink pretest
duration time Tp
Request to send data to
server
OK
Data Streams
Accept
STEP 4. UPLINK PRE-TEST
Barcelona, Spain, 9-11 December 2015
ITU Kaleidoscope 2015 - Trust in the Information Society
F
Basic methodology
Test Execution: Steps
MA
MP
Request to send data to
server
OK
Data Stream
Step 2.5
Uplink pretest
duration time tu
Uplink Test
Time; Bytes
Data Stream
Time
STEP 5. UPLINK TEST
Barcelona, Spain, 9-11 December 2015
ITU Kaleidoscope 2015 - Trust in the Information Society
G