Communication solution design for field trial setup
D3.3.2
Communication solution design for
field trial setup
Deliverable report
SUNSEED, Grant agreement No. 619437
Page 1 of 56
Communication solution design for field trial setup
NOTICE
The research leading to the results presented in the document has received funding from the
European Community's 7th Framework Programme under the Grant Agreement number 619437.
The content of this document reflects only the authors’ views. The European Commission is not liable
for any use that may be made of the information contained herein.
The contents of this document are the copyright of the SUNSEED consortium.
SUNSEED, Grant agreement No. 619437
Page 2 of 56
Communication solution design for field trial setup
Document Information
Call identifier
FP7-ICT-2013-11
Project acronym
SUNSEED
Project full title
Sustainable and robust networking for smart electricity distribution
Grant agreement number
619437
Deliverable number
D3.3.2
WP / Task
WP3 / T3.3
Type (distribution level)1
PU
Due date of deliverable
31.1.2016 (Month 24)
Date of delivery
30.1.2016 V1.0, updated 31.3.2016
Status, Version
V2.0
Number of pages
56 pages
Responsible person, Affiliation
Jurij Jurse, EP
Authors
Ljupco Jorguseski, Manolis Chrysallos, Haibin Zhang, Onno Mantel, Casper
van den Broek TNO
Jimmy J. Nielsen (AAU)
Herve Ganem GTOSA
Tomaz Lovrencic, Blaž Petrnel, Robert Tošnjak, Bojan Dovč TS
Žiga Hribar, Bojan Miličić ES
Jurij Jurše EP
Reviewers
Mihael Mohorčič, JSI
Zhong Fan (TREL)
1
PU
RP
RE
CO
Public
Restricted to other programme participants (including the Commission Services)
Restricted to a group specified by the consortium (including the Commission Services)
Confidential, only for members of the consortium (including the Commission Services)
SUNSEED, Grant agreement No. 619437
Page 3 of 56
Communication solution design for field trial setup
Revision history
Version
0.1
0.2
Date
20.11.2015
14.12.2015
1.0
30.2.2016
1.1
15.3.2016
1.2
29.3.2016
1.3
31.3.2016
Author(s)
L. Jorguseski
J.
J.
Nielsen,
L.
Jorguseski, T. Lovrencic
J.
J.
Nielsen,
L.
Jorguseski, T. Lovrencic,
B., Miličič, J. Jurse, H.
Ganem
B. Petrnel, R. Tošnjak, B.
Dovč
J.
J.
Nielsen,
L.
Jorguseski, J. Jurse
J.Jurše
SUNSEED, Grant agreement No. 619437
Notes
ToC
Chapter 4, Chapter 5
Status
Draft
Draft
Chapter 1, Chapter 2, Chapter 3
Submission
Version
Update of Chapter 2 and 3
Completion
Merging and final editing
Completion
Final
Page 4 of 56
Communication solution design for field trial setup
Table of Contents
Document Information ........................................................................................................................... 3
List of Figures........................................................................................................................................... 7
List of Tables ............................................................................................................................................ 8
Abbreviations and acronyms ................................................................................................................... 9
SUNSEED project ................................................................................................................................... 10
Executive Summary ............................................................................................................................... 11
1
Introduction................................................................................................................................... 13
2
Communication solutions for the SUNSEED field trial .................................................................. 15
2.1
Considered communication solutions for WAMS ................................................................. 15
2.1.1
Scenario A: WAMS SPM/PMC – mobile or fixed modem/router .................................. 15
2.1.2
Scenario B: WAMS SPM – satellite modem/router ....................................................... 20
2.2
Considered communication solutions for smart meters....................................................... 21
2.2.1
Scenario C: mobile or fixed modem/router – PLC data concentrator – smart meter... 21
2.2.2
Scenario D: mobile or fixed modem/router – Ethernet or RS485 to Ethernet converter
- smart meter................................................................................................................................. 22
2.3
3
4
Communication solution in individual functional location (transformer station) ................ 23
Security solution for the SUNSEED field trial ................................................................................ 25
3.1
Communication security and architecture for field trial ....................................................... 25
3.2
Security architecture for credential management ................................................................ 29
3.3
Credential provisioning to the WAMS node.......................................................................... 31
Communication measurement procedures for the SUNSEED field trial ....................................... 33
4.1
Smart grid traffic characterization measurements ............................................................... 33
4.1.1
PCAP tracing locations ................................................................................................... 33
4.1.2
Available information from PCAP traces ....................................................................... 34
4.1.3
KPIs and statistics of interest from PCAP traces ........................................................... 35
4.1.4
Traffic Characterization Measurement procedure........................................................ 36
4.2
WAMS-SPM end-to-end delay measurement ....................................................................... 37
4.2.1
4.3
Measurement approach ................................................................................................ 38
End-to-end throughput measurement .................................................................................. 39
4.3.1 Throughput measurement setup in controlled laboratory environment............................ 40
4.4
5
Radio access related measurements ..................................................................................... 44
4.4.1
Relevant measurements derived from statistical counters/KPIs at the eNB ................ 45
4.4.2
Gemalto LTE modem ..................................................................................................... 48
4.4.3
External communication equipment ............................................................................. 49
Field trial communication measurements and relation to SUNSEED use cases ........................... 51
5.1
Measurements related to state estimation use case ............................................................ 51
SUNSEED, Grant agreement No. 619437
Page 5 of 56
Communication solution design for field trial setup
5.2
Measurements related to demand-response use case ......................................................... 52
5.3
Measurements related to SG outage localization use case .................................................. 53
6
Conclusions.................................................................................................................................... 54
7
References ..................................................................................................................................... 56
SUNSEED, Grant agreement No. 619437
Page 6 of 56
Communication solution design for field trial setup
List of Figures
Figure 1 High-level overview of the SUNSEED trial network (incl. different locations, communication
systems)................................................................................................................................................. 14
Figure 2: Communication scenario A.1: WAMS – LTE mobile modem/router...................................... 15
Figure 3: Communication scenario A.2: WAMS – UMTS mobile modem/router. ................................ 16
Figure 4: Communication scenario A.3: WAMS – xDSL or FTTH modem/router. ................................. 16
Figure 5: WAMS-SPM typical wiring diagram scheme in communication scenarios A.1 and A.2......... 17
Figure 6: WAMS-PMC typical wiring diagram scheme in communication scenarios A.1 and A.2. ....... 18
Figure 7: An example of inventory material with technical specification for scenario A ..................... 19
Figure 8: Communication scenario B.1: WAMS – satellite modem/router in combination with Wi-Fi
(options B2). .......................................................................................................................................... 20
Figure 9: Communication scenario B: WAMS – satellite modem/router (options B1) in combination
with Wi-Fi (options B2). ......................................................................................................................... 20
Figure 10: Communication scenario C.1: mobile modem/router – PLC data concentrator (DC) – smart
meter. .................................................................................................................................................... 21
Figure 11: Communication scenario C.2: fixed modem/router – PLC data concentrator (DC) – smart
meter. .................................................................................................................................................... 22
Figure 12: Communication scenario D.1: mobile modem/router – Ethernet or RS485 to Ethernet
converter - smart meter. ....................................................................................................................... 22
Figure 13: Communication scenario D.2: fixed modem/router – Ethernet or RS485 to Ethernet
converter - smart meter. ....................................................................................................................... 23
Figure 14: Selected communication scenarios for WAMS SPM devices and smart meters on individual
transformer station. .............................................................................................................................. 24
Figure 15: Field trial security architecture ............................................................................................ 25
Figure 16: SUNSEED general back-end infrastructure........................................................................... 26
Figure 17: SUNSEED APNs related back-end infrastructure. ................................................................. 27
Figure 18: SUNSEED xDSL related infrastructure. ................................................................................. 28
Figure 19: SUNSEED VPN related back-end infrastructure. .................................................................. 29
Figure 20 : Communication and security architecture .......................................................................... 30
Figure 21: Practical implementation of security architecture............................................................... 31
Figure 22: security and credential provisioning workflow .................................................................... 32
Figure 23: PCAP location for packet sniffing within Telekom Slovenija network used in the trial ....... 34
Figure 24: Example of PCAP trace, showing implementation of a SIP protocol from [2] ..................... 35
Figure 25 Schematic view on the end-to-end delay measurement procedure for WAMS-SPM nodes 38
Figure 26: Measurement collection from WAMS-SPM. Upon completion of a measurement burst, a
packet is assembled and sent to the TS core, where it is detected and a timestamp is made at the
APN, and finally at the DB server, another timestamp is made when the data is stored in the
database. ............................................................................................................................................... 39
Figure 27: End-to-end scheme .............................................................................................................. 41
Figure 28: Lab environment - test bed for end-to-end telecommunication compatibilities ................ 41
Figure 29: time of reading 10 registers from 1 smart meter is less than 2 seconds) ............................ 42
Figure 30: Laboratory test LV power grid configuration ....................................................................... 43
Figure 31: SNMP community ................................................................................................................. 50
SUNSEED, Grant agreement No. 619437
Page 7 of 56
Communication solution design for field trial setup
List of Tables
Table 1 ................................................................................................................................................... 45
SUNSEED, Grant agreement No. 619437
Page 8 of 56
Communication solution design for field trial setup
Abbreviations and acronyms
APN
EP
DB
DC
DPSK
FTTH
GGSN
HSPDA
IP
IT
KPI
LTE
LV
M2M
MQTT
MPLS
OFDM
PCAP
PCB
P-GW
PLC
SG
SGSN
S-GW
SM
S-FSK
SNMP
SPM
TAC
TCP
TS
UDP
UMTS
WAMS
VPN
WAMS – PMC
WAMS – SPM
WAN
XMPP
Access Point Name
Elektro Primorska
Database
Data Concentrator
Differential Phase Shift Keying
Fiber-to-the-Home
Gateway GPRS Support Node
High Speed Downlink Packet Access
Internet Protocol
Information Technology
Key Performance Indicator
Long Term Evolution
Low Voltage
Machine to Machine
MQ Telemetry Transport
Multiprotocol Label Switching
Orthogonal Frequency Division Multiplexing
Packet Capture
Printed Circuit Board
Packet Gateway
Power Line Carrier
Smart Grid
Serving GPRS Support Node
Serving Gateway
Smart Meter
Spread Frequency-shift Keying
Simple Network Management Protocol
Synchro Phasor Measurement
Tracking Area Code
Transmission Control Protocol
Telekom Slovenija
User Datagram Protocol
Universal Mobile Telecommunications Service
Wide Area Measurement and Supervision
Virtual Private Network
WAMS Power Measurement and Control
WAMS Synchro-Phasor Measurement
Wide Area Networks
Extensible Messaging and Presence Protocol
SUNSEED, Grant agreement No. 619437
Page 9 of 56
Communication solution design for field trial setup
SUNSEED project
SUNSEED proposes an evolutionary approach to utilization of already present communication networks from
both energy and telecom operators. These can be suitably connected to form a converged communication
infrastructure for future smart energy grids offering open services. Life cycle of such communication network
solutions consists of six steps: overlap, interconnect, interoperate, manage, plan and open. Joint
communication networking operations steps start with analysis of regional overlap of energy and
telecommunications operator infrastructures. Geographical overlap of energy and communications
infrastructures identifies vital DSO energy and support grid locations (e.g. distributed energy generators,
transformer substations, cabling, ducts) that are covered by both energy and telecom communication
networks. Coverage can be realised with known wireline (e.g. copper, fiber) or wireless and mobile (e.g. WiFi,
4G) technologies. Interconnection assures end-2-end secure communication on the physical layer between
energy and telecom, whereas interoperation provides network visibility and reach of smart grid nodes from
both operator (utility) sides. Monitoring, control and management gathers measurement data from wide area
of sensors and smart meters and assures stable distributed energy grid operation by using novel intelligent real
time analytical knowledge discovery methods. For full utilization of future network planning, we will integrate
various public databases (e.g. municipality GIS, weather). Applications build on open standards (W3C) with
exposed application programming interfaces (API) to 3rd parties enable creation of new businesses related to
energy and communication sectors (e.g. virtual power plant operators, energy services providers for optimizing
home energy use) or enable public wireless access points (e.g. WiFi nodes at distributed energy generator
locations). SUNSEED life cycle steps promise much lower investments and total cost of ownership for future
smart energy grids with dense distributed energy generation and prosumer involvement.
Project Partners
1.
2.
3.
4.
5.
6.
7.
8.
9.
TELEKOM SLOVENIJE D.D.; TS; Slovenia
AALBORG UNIVERSITET; AAU; Denmark
ELEKTRO PRIMORSKA, PODJETJE ZA DISTRIBUCIJO ELEKTRICNE ENERGIJE D.D.; EP; Slovenia
ELEKTROSERVISI, ENERGETIKA, MERILNI LABORATORIJ IN NEPREMICNINE D.D.; ES; Slovenia
INSTITUT JOZEF STEFAN; JSI; Slovenia
GEMALTO SA; GTOSA; France
GEMALTO M2M GMBH; GTOM2M; Germany
NEDERLANDSE ORGANISATIE VOOR TOEGEPAST NATUURWETENSCHAPPELIJK ONDERZOEK - TNO; TNO;
The Netherlands
TOSHIBA RESEARCH EUROPE LIMITED; TREL; United Kingdom
Project webpage
http://www.sunseed-fp7.eu/
SUNSEED, Grant agreement No. 619437
Page 10 of 56
Communication solution design for field trial setup
Executive Summary
The deliverable outlines the specific scenarios for communication solutions on separate field trial
locations which are geographically quite distinguished. Regarding the proposed scenarios the general
communication SUNSEED back-end infrastructure with security architecture and provisioning the
security credentials to the end nodes is defined.
In particular, we focus on considered solutions for WAMS (SPM and PMC) and for smart meters,
respectively. Of course, due to costs reduction and where possible, WAMS and smart meters can use
the same gateway for a considered communication solution. For WAMS scenarios the typical wiring
diagram scheme and an example of inventory material with technical specification was made. On the
basis of coverage analyses from deliverable D5.1.2 the most appropriate communication scenario per
each transformer station as the main functional location was assigned.
In-depth analyses of the field trial locations and of the Telekom Slovenije infrastructure reviled that
mobile and satellite communications are the most appropriate to connect majority of WAMS devices
into the SUNSEED cloud. Therefore, we distinguish two communication scenarios, where WAMS
(SPM/PMC) are using mobile or fixed modem/router (Scenario A) and WAMS SPM are using satellite
modem/router (Scenario B). With respect to the analysis of fixed network access points presence and
mobile network coverage from deliverables published in WP5 scenario A is further divided into three
sub-scenarios, where gateway (modem/router) could connects the LTE network (scenario A.1), UMTS
network (scenario A.2) or xDSL/FTTH network (scenario A.3). Regarding the position of a satellite
antenna scenario B is also divided into two sub-scenarios where WAMS could be located at utility
functional location and it is connects directly to satellite modem/router, while the location can have
also Wi-Fi bridge to subordinate functional location (Scenario B.1). Other possibility is WAMS
positions at subordinate functional location and connects to the satellite modem/router via Wi-Fi
bridge (Scenario B.2).
Similar as WAMS in scenario A, smart meters use mobile or fixed modems/routers as gateways for
connecting into the SUNSEED cloud via mobile or fixed network. However, with regard to the exact
smart meter model, they are connected to such gateways in different manner. We distinct two
scenarios, where smart meters are connected to PLC data concentrator (DC), which is connected to
mobile or fixed modem/router (Scenario C) or they are connected to mobile or fixed modem/router
via RS485 to Ethernet converter (Scenario D).
Sunseed IT selected security will insure the proper filtering of data transmission at the IP level . It
consist of four main parts in general to support WAMS connectivity with the data centre via by using
mobile connections with private APNs, WAMS connectivity with the data centre via xDSL or FTTH
connections and WAMS connectivity with the data centre via VPN by using satellite links. The
datacentre is behind the firewall with load balancing functionality for physical separation from the
access network part. For end WAMS nodes access the authorization delegation model is used. . In
such model, “Resource servers” (entities controlling access to some “resource” delegate the
administration of the access control to some or all of their resources to an external authorization
server.
During the trial measurements it is important to characterize the ongoing traffic in terms of e.g.
packet sizes, inter-packet arrival times, aggregated amount of traffic, etc. This realistic traffic
SUNSEED, Grant agreement No. 619437
Page 11 of 56
Communication solution design for field trial setup
characterization based on the trial communication sessions is useful for modifying the existing traffic
models used for performance analysis of communication networks for smart grids. In this deliverable
a detailed procedure is explained regarding capturing packets from the smart grid applications during
the trial via PCAP traces as well as definitions of the most important traffic KPIs (e.g. traffic volume,
packet size, time series of packet generation, etc.). Next to the traffic characterization it is important
to measure the communication network performance parameters. This deliverable describes end-toend delay and throughput measurement procedures that will be used in the trial. For the end-to-end
delay it is important to realize sufficiently accurate time synchronization between the two
communication end-points. Finally, during the communication sessions it is important to measure
the most important radio access communication network conditions (e.g. signal level, interference
level, radio access load, etc.) in order to understand the end-to-end delay and throughput
performance. This deliverable defines two approaches for measuring the radio access network
conditions: the approach of using radio access related performance counters as provided by the
vendor of radio access network, and the approach of using AT commands and/or SNMP commands
to extract the necessary radio access performance information from the communication modems.
During the SUNSEED trial the measurements of the electricity grid and the communication network
will be used to assess the feasibility and the performance requirements for the three defined
SUNSEED use cases, namely the real-time state estimation, demand response, and outage
localization use case. For the electricity grid state estimation use case a procedure is defined
regarding the testing of the accuracy and up to date information of the state-estimation function. As
the reporting interval of the WAMS-PMC and WAMS-SPM nodes is varied from e.g. 15 min down to 1
s the impact on the communication network and end-to-end delays will be investigated as well as the
ability of the state-estimation algorithm to predict a correct value and on time. For the demand
response use case next to the measurements needed for the state estimation additional
measurements and control information messages will be collected. As the delay in demand response
cycles is typically around 15 min, for this use case it is important to check the reliability of these
additional measurements and control messages (i.e. to check the stability and reliability of the
control process) and not on the end-to-end delay performance for these messages. Finally, by
instructing the WAMS-SPM and WAMS-PMC nodes to initiate ‘fake outage’ messages enriched with
location information (e.g. GPS coordinates for WAMS-SPM) the feasibility of automatic grid outage
localization will be assessed as the final SUNSEED use case.
SUNSEED, Grant agreement No. 619437
Page 12 of 56
Communication solution design for field trial setup
1 Introduction
At the moment of writing of this deliverable most of the locations and solutions regarding the
communication part of the SUNSEED field trial are determined. The goal of this deliverable is to
present in detail the actually used communication network for the SUNSEED trial. Throughout the
deliverable a reference is given to D5.1.2 where the preliminary/possible communication options are
also presented.
The end nodes in the SUNSEED trial (e.g. SM, WAMS-SPM, and WAMS-PMC) are communicating via
the LTE and UMTS wireless telecommunication network of Telekom Slovenija, with exception at
location Kneža, where also satellite links are utilized. The communication network set-up is
illustrated in Figure 1 for the four SUNSEED field trial areas:
1) Area Kromberk, where:
a. LTE base stations (or eNBs) are used for covering 10 WAMS-SPM nodes, 1 WAMSPMC nodes, 6 PLC concentrator nodes and 116 SM nodes.
2) Area Bonifika, where:
a. LTE base stations (or eNBs) are used for covering 7 WAMS-SPM nodes, 2 WAMS-PMC
nodes, 5 PLC concentrator nodes and 535 SM nodes.
3) Area Razdrto, where:
a. UMTS base stations (or eNBs) are used for covering 17 WAMS-SPM nodes, 2 WAMSPMC nodes, 7 PLC concentrator nodes and 10 SM nodes.
4) Area Kneža, where:
a. LTE base stations (or eNBs) are used for covering 3 WAMS-SPM nodes, 0 WAMS-PMC
nodes, 2 PLC concentrator nodes and 0 SM nodes.
b. Satellite links covering 5 WAMS-SPM nodes, 0 WAMS-PMC nodes, 2 PLC
concentrator nodes and 2 SM nodes. This is not illustrated in Figure 1 due to clarity
reasons.
The data packets in the domain of the wireless communication network of Telekom Slovenija are
flowing through the dedicated Access Point Name (APN) sunseed.ts.si. Outside the mobile
communication network domain the measurements are forwarded to the dedicated SUNSEED field
trial database, also illustrated in Figure 1.
SUNSEED, Grant agreement No. 619437
Page 13 of 56
Communication solution design for field trial setup
Figure 1 High-level overview of the SUNSEED trial network (incl. different locations,
communication systems)
The structure of the deliverable is as follows. Chapter 2 presents the different access network
possibilities for the wide (or neighbourhood) area networks (WAN) for the WAMS-SPM,
WAMS-CPM, PLC concentrators and SM nodes installed in the SUNSEED field trial. Chapter 3
explains the security solution for protecting the packet data streams and for provisioning the
security credentials to the end nodes. In Chapter 4 we present the different communication
measurement procedures that will be used during the SUNSEED trial execution. The relation
between the measurements in the trial and the defined SUNSEED use cases is presented in
Chapter 5. The deliverable is finalized with the conclusions in Chapter 6.
SUNSEED, Grant agreement No. 619437
Page 14 of 56
Communication solution design for field trial setup
2 Communication solutions for the SUNSEED field trial
This chapter focuses on communication solutions for the SUNSEED field trial. In particular, chapter
2.1 focuses on considered solutions for WAMS (SPM and PMC) and chapter 2.2 for smart meters,
respectively. Of course, due to costs reduction and where possible, WAMS and smart meters can use
the same gateway for a considered communication solution.
2.1
Considered communication solutions for WAMS
The preliminary plan from D5.1.2 proposes and describes possible communication solutions for the
WAMS with respect to gateway type (internal or external). However, in-depth analyses of the field
trial locations and of the Telekom Slovenije infrastructure reviled that mobile and satellite
communications are the most appropriate to connect majority of WAMS devices into the SUNSEED
cloud. Therefore, we distinguish two communication scenarios, where:


WAMS (SPM/PMC) are using mobile or fixed modem/router (Scenario A),
WAMS SPM are using satellite modem/router (Scenario B).
2.1.1 Scenario A: WAMS SPM/PMC – mobile or fixed modem/router
In scenario A WAMS devices use mobile or fixed modems/routers as gateways for connecting into
the SUNSEED cloud via mobile or fixed network, as depicted in Figure 2. With respect to the analysis
of fixed network access points presence and mobile network coverage (see deliverables D5.1.1 and
D5.1.2.) scenario A is further divided into three sub-scenarios, where:
-
gateway (modem/router) connects to the LTE network (scenario A.1),
gateway (modem/router) connects to the UMTS network (scenario A.2),
gateway (modem/router) connects to the xDSL or FTTH network (scenario A.3).
With regard to each deployment location specifics, scenarios A.1 and A.2 are implemented so that
communication antenna of the gateway is mounted either inside or outside the electrical box or the
object where WAMS is mounted to overcome the Faraday cage/shield effect.
Communication network
WAN
Utility functional location
(Transformer station)
SURGE ARRESTER
ETHERNET
Cellular network
LTE
MODEM/ROUTER
WAMS
(SPM/PMC)
Data
concentrator
Figure 2: Communication scenario A.1: WAMS – LTE mobile modem/router.
SUNSEED, Grant agreement No. 619437
Page 15 of 56
Communication solution design for field trial setup
Utility functional location
(Transformer station)
Communication network
WAN
SURGE ARRESTER
ETHERNET
Cellular network
UMTS
MODEM/ROUTER
WAMS
(SPM/PMC)
Data
concentrator
Figure 3: Communication scenario A.2: WAMS – UMTS mobile modem/router.
Communication network
WAN
Utility functional location
(Transformer station)
Network
ETHERNET
xDSL
FTTH
MODEM/ROUTER
WAMS
(SPM/PMC)
Data
concentrator
Figure 4: Communication scenario A.3: WAMS – xDSL or FTTH modem/router.
Next, Figure 5 and Figure 6 depict typical wiring of WAMS-SPM and WAMS-PMC in most often
considered communication scenarios A.1 and A.2. The one should note that deployment locations’
wiring diagrams in practice can differ from the depicted examples due to locations’ specifics.
Considering such diagrams, we prepared also detailed inventory of power and communication
material and technical specification for each deployment location. An example of such
documentation is depicted in Figure 7.
SUNSEED, Grant agreement No. 619437
Page 16 of 56
Communication solution design for field trial setup
Figure 5: WAMS-SPM typical wiring diagram scheme in communication scenarios A.1 and A.2.
SUNSEED, Grant agreement No. 619437
Page 17 of 56
Communication solution design for field trial setup
Figure 6: WAMS-PMC typical wiring diagram scheme in communication scenarios A.1 and A.2.
SUNSEED, Grant agreement No. 619437
Page 18 of 56
Communication solution design for field trial setup
COMMUNICATION SCENARIO A WAMS – cellular modem/router
Description
Unit
Quantity
piece
1
piece
3
piece
3
piece
3
m
m
m
m
m
3
1
1
3
1
piece
1
piece
1
piece
1
piece
1
piece
1
piece
1
m
m
m
m
3
1
1
1
POWER EQUIPEMENT
Power protection elements
Miniature circuit breaker 230 V AC,
SPECIFICATIONS:
- Rated current I N =6A
- Tripping characteristic: B
- Other: DIN rail, 3-pole, 3 module
Surge arrester (for ovehead transformer station, 3 pieces per one WAMS)
SPECIFICATIONS:
- Max. continuous operating voltage U c = 275 - 320 V AC
- Protection lavel U p at I n (8/20) <= 3,2 kV
- Max. discharge current (8/20) I max >= 80 kA
- Impulse current I imp >= 25 kA
- Other: DIN rail, 1-pole, 1,5 module, Terminal cross section: 25mm2 (stranded), IEC: I,II
Surge arrester (for cable transformer station, 1 pieces per one WAMS)
SPECIFICATIONS:
- Max. continuous operating voltage U c = 275 - 320 V AC
- Protection lavel U p at I n (8/20) <= 1,5 kV
- Max. discharge current (8/20) I max >= 40 kA
- Impulse current I imp >= 12,5 kA
- Other: DIN rail, 3-pole, 3 module, Terminal cross section: 25mm2 (stranded), IEC: I,II
Fusse (low voltage NH knife-blade type, 1 pieces per one location)
SPECIFICATIONS:
- Rated current I N =100A
- inserted into 160 A fusse base
Power supply wires (quantities per one location)
PVC insulated wire H07V-K 1,5 mm2, black (fine stranded), for electonics devices supply
PVC insulated wire H07V-K 1,5 mm2, blue (fine stranded), for electonics devices supply
PVC insulated wire H07V-K 1,5 mm2, yellow/green (fine stranded), for electonics devices supply
PVC insulated wire H07V-K 10 mm2, black (fine stranded), for phase to surge arrester conncetion
PVC insulated wire H07V-K 16 mm2, yellow/green (fine stranded), for surge arrester to ground conncetion
Other power material (din rail, connectors, busbars…)
COMMUNICATION EQUIPEMENT
Modem/router
SPECIFICATIONS:
- 4G LTE Router, WAN, 3xLAN, WiFi 802.11.b/g/n
- dual SIM, VPN
Power supply 24 V, 40 W, 2 module
SPECIFICATIONS:
- AC240V/DC 24V
- In 1,66A , 40W
Protection for modem/router ( 2A)
SPECIFICATIONS:
-20kA lighting surge protection, one ground position
- Two passsive 10/100/1000 Ethernets ports
Protection for power supply Line-up terminal with fuse holder VSV4
SPECIFICATIONS:
'- DC 2A
Antenna for LTE
SPECIFICATIONS:
- 2G/3G/4G Terminal Antenna for Cellular Gateways and Routers, LTE/GSM
Antenna for GPS
SPECIFICATIONS:
- gain 30Db, output impedance 50 ohm
Communication wires (quantities per one location)
PVC insulated wire H07V-K 1,5 mm2, - za ožičenje WAMSA
Antene cable LTE 50 ohm, female, male SMA connectors
Antene cable GPS 50 ohm, female, male SMA connectors
STP cable cat 5e -data cable + 2x RJ45STP
Figure 7: An example of inventory material with technical specification for scenario A
SUNSEED, Grant agreement No. 619437
Page 19 of 56
Communication solution design for field trial setup
2.1.2 Scenario B: WAMS SPM – satellite modem/router
Scenario B enables connection of WAMS devices in regions with bad mobile network coverage, no
mobile network coverage and no fixed broadband access points in the vicinity. In the SUNSEED field
trial, an example of such region is the area of Kneža where we overcome natural challenging
properties with the satellite communication links.
With respect to the position of a satellite antenna we distinct two sub-scenarios, where:
-
-
WAMS is located at utility functional location and connected directly to satellite
modem/router, while the location can have also Wi-Fi bridge to subordinate functional
location (Scenario B.1),
WAMS is located at subordinate functional location connected to the satellite modem/router
via Wi-Fi bridge (Scenario B.2).
Scenario B.1 is depicted in Figure 8 and Scenario B.2 is depicted in Figure 9. However, the wiring
diagram and inventory of material for Scenario B is unfortunately unavailable at the time of writing
of this deliverable.
Utility functional location
(Transformer station)
Communication network
WAN
SURGE ARRESTER
Satellite
communication
ETHERNET
MODEM/ROUTER
WAMS
(SPM)
Figure 8: Communication scenario B.1: WAMS – satellite modem/router in combination with
Wi-Fi (options B2).
Utility functional location
(Transformer station)
Communication network
WAN
SURGE ARRESTER
Satellite
communication
ETHERNET
MODEM/ROUTER
Wi-Fi bridge
Wi-Fi bridge
(to subordinate functional location)
ETHERNET
SURGE ARRESTER
WAMS
(SPM)
Utility subordinate functional location
(Transformer station)
Wireless
Comunication
(Wi-Fi)
Communication network
NAN
Figure 9: Communication scenario B: WAMS – satellite modem/router (options B1) in
combination with Wi-Fi (options B2).
SUNSEED, Grant agreement No. 619437
Page 20 of 56
Communication solution design for field trial setup
2.2
Considered communication solutions for smart meters
Though the solutions for connecting smart meters were discussed into details in D5.1.2, we had to
identify most feasible options for their connection in the field trial. Similar as WAMS in scenario A,
smart meters use mobile or fixed modems/routers as gateways for connecting into the SUNSEED
cloud via mobile or fixed network. However, with regard to the exact smart meter model, they are
connected to such gateways in different manner. We distinct two scenarios, where:


smart meters are connected to PLC data concentrator (DC), which is connected to mobile or
fixed modem/router (Scenario C),
smart meters are connected to mobile or fixed modem/router via RS485 to Ethernet
converter (Scenario D).
2.2.1 Scenario C: mobile or fixed modem/router – PLC data concentrator – smart
meter
One or three phase residential smart meters in the field trial use Power Line Carrier (PLC)
communications to bind at least four customers (consumers, producers or prosumers) to data
concentrator, which is typically connected to WAN gateway in a transformer station. As such,
scenario C is further divided into two sub-scenarios, where such:
gateway (modem/router) connects to the LTE network (scenario C.1),
gateway (modem/router) connects to the UMTS network (scenario C.2).
For visual presentation Figure 10 depicts Scenario C.1 and Figure 11 scenario C.2.
The advantage of PLC is in overall coverage alongside the low voltage power grid, which eliminates
the problems with extra communication infrastructure in the last mile. The most common PLC
technologies use a single-carrier system with S-FSK (Spread Frequency-shift Keying) modulation,
which lacks flexibility in selecting the carrier frequency, and hence, results in a low throughput, but
reliability ([E. Hossain, Z. Han, H.V. Poor, 2013]). It is defined in standard IEC 61334 and often called
PLAN defined. However, in Razdrto area common PLC technology will be upgraded to PLC G3, which
is a narrow band technology offering a great balance between robustness, quality of service, data
rate and cost. It uses Orthogonal Frequency Division Multiplexing (OFDM) and Differential Phase Shift
Keying (DPSK).
Communication network
WAN
Utility functional location
(Transformer station)
PLC communication
NAN
ETHERNET
SURGE ARRESTER
Cellular network
LTE
UMTS
WAMS
(SPM/PMC)
MODEM/ROUTER
LV Power grid
0,4 kV
Data
concentrator
Figure 10: Communication scenario C.1: mobile modem/router – PLC data concentrator (DC)
– smart meter.
SUNSEED, Grant agreement No. 619437
Page 21 of 56
Communication solution design for field trial setup
Communication network
WAN
Utility functional location
(Transformer station)
PLC communication
NAN
ETHERNET
xDSL, FTTH
WAMS
(SPM/PMC)
LV Power grid
MODEM/ROUTER
0,4 kV
Data
concentrator
Figure 11: Communication scenario C.2: fixed modem/router – PLC data concentrator (DC) –
smart meter.
2.2.2 Scenario D: mobile or fixed modem/router – Ethernet or RS485 to Ethernet
converter - smart meter
This scenario enables connection of individual smart meters or a group of smart meters on the same
functional location. In the field trial it is considered for industrial smart meters which do not support
PLC connection and in case of LV power grid with very low number of smart meters (three or less) or
very long distance among them.
Scenario D is further divided into sub-scenarios, where:
gateway (modem/router) connects to the UMTS or LTE network (scenario D.1),
gateway (modem/router) connects to the xDSL or FTTH network (scenario D.2).
The scenario D.1 is conceptually presented in Figure 12 and scenario D.2 in Figure 13. In both
scenarios there are two possibilities of connecting smart meters to modem/router. First is a parallel
connection of individual devices via Ethernet. Second is a serial connection through RS485 protocol.
Utility functional location
(customer connection box)
Communication network
WAN
ETHERNET
SURGE ARRESTER
Cellular network
LTE
UMTS
MODEM/ROUTER
RS485Ethernet
converter
RS485
RS485
Figure 12: Communication scenario D.1: mobile modem/router – Ethernet or RS485 to
Ethernet converter - smart meter.
SUNSEED, Grant agreement No. 619437
Page 22 of 56
Communication solution design for field trial setup
Utility functional location
(customer connection box)
Communication network
WAN
ETHERNET
xDSL, FTTH
MODEM/ROUTER
RS485Ethernet
converter
RS485
RS485
Figure 13: Communication scenario D.2: fixed modem/router – Ethernet or RS485 to Ethernet
converter - smart meter.
2.3
Communication solution in individual functional location
(transformer station)
The aim of this chapter is assigning the most appropriate communication solution per each
transformer station being considered as the main functional location. The assignment of
communication solution is based on comprehensive communication network coverage analyses from
D5.1.1 and their recent updates. The table in Figure 14 depicts planned communication solutions to
connect WAMS devices and smart meters in all transformer stations in the field trial.
SUNSEED, Grant agreement No. 619437
Page 23 of 56
Communication solution design for field trial setup
Field trial
location
Utility functional location - Communication
Nm. of WAMS Nm. Of WAMS
Transformer substation scenario for WAMS
SPM
PMC
(TS)
SPM connection
Notes
Communication
Communication scenario
Number of PLC Number of non
Number of
Number of Data
scenario for
for residental smart
residental smart PLC residental
industrial smart
concetrators
industrial smart
meters
meters
smart meters
meters
meters
KROMBERK (NOVA GORICA)
1
Modem/router
conform with IEC
61850-3
Meblo
A1
1
C
1
11
Primarna
A1
1
Kotlarna
A1
2
Površinska
A1
1
A+A
A1
1
Pikolud
A1
1
Jogi
A1
1
MFE Ilmest
A1
1
not presented
SE Meblo Jogi
A1
1
not presented
Bonifika 2
A1
1
Agraria
A1
1
Bonifika 1
A1
1
Bonifika 3
A1
1
Špar
A1
1
Črpališče Kanal grande
A1
1
D
D
2
C
2
24
D
12
C
1
23
D
6
D
1
not presented
Modem/router
conform with IEC
61850-3
Modem/router
conform with IEC
61850-3
Modem/router
conform with IEC
61850-3
Modem/router
conform with IEC
61850-3
Modem/router
conform with IEC
61850-3
not presented
2
C
1
19
D
3
C
1
8
D
3
D
1
D
1
D
11
D
1
BONIFIKA (KOPER)
Modem/router
conform with IEC
61850-3
2
1
RP Razdrto
A2
1
Profiles peletirnica
A2
1
1
37
C
1
81
D
8
C
1
13
D
6
D
2
C
1
4
D
3
C
1
360
D
8
not presented
Obrtni center
A1
Modem/router
conform with IEC
61850-3
Modem/router
conform with IEC
61850-3
Modem/router
conform with IEC
61850-3
without WAMS
installation
not in operation
Koper 3 (DVTP)
Markovec vzhod
C
not presented
D
not presented
D
1
RAZDRTO (SEŽANA)
Modem/router
conform with IEC
61850-3
C
1
9
Modem/router
conform with IEC
61850-3
D
Razdrto vas
A2
1
C
1
52
KZ Razdrto
A2
1
C
1
10
Kovinoplastika
A2
1
D
Razdrto
A2
1
Razdrto Kamnolom
A2
1
Asfaltna baza Laže
undefined
1
Laže
not presented
not presented
1
Modem/router
conform with IEC
61850-3
Modem/router
conform with IEC
61850-3
1
not presented
not presented
1
D
1
D
C
1
6
1
not presented
D
1
not presented
D
1
undefined
1
Asfaltna baza Senožeče
A2
1
D
1
D
Ravni
A2
1
D
3
not presented
Senožeče 1
A2
1
C
Nanos
A2
1
not presented
Tri hiše
A2
1
C
Hruševje mlin
A2
1
D
3
not presented
C.P. Nanos
A2
1
D
2
not presented
1
1
not presented
D
1
not presented
D
MVE Razdrto
Laže Cestarska hiša
1
C
1
32
not presented
1
1
20
not presented
1
11
not presented
D
1
not in operation
KNEŽA (TOLMIN)
Modem/router
conform with IEC
61850-3
Zuza
A1
1
Klavže
A1
1
C
1
37
D
Borovnica
undefined
1
C
1
5
not presented
Sela nad Podmelcem
undefined
1
C
1
22
Logar
undefined
1
D
Loje
undefined
1
C
Mohor
B2
1
D
Knežke Ravne
B1
1
C
1
4
not presented
B1,B2
1
C
1
5
D
1
A1
1
not presented
D
1
undefined
1
not presented
D
1
mHE Knežke Ravne 2
B1,B2
1
not presented
D
1
mHE Strmec Hvala
B1,B2
1
not presented
D
1
Knežke ravne vas
HE Podmelec
MHE Kneža
1
not presented
1
1
1
5
not presented
not presented
2
not presented
Figure 14: Selected communication scenarios for WAMS SPM devices and smart meters on
individual transformer station.
SUNSEED, Grant agreement No. 619437
Page 24 of 56
Communication solution design for field trial setup
3 Security solution for the SUNSEED field trial
3.1
Communication security and architecture for field trial
Figure 15 provide an overview of the IT level security in place for the trial. Those security measure
will insure the proper filtering of data transmission at the IP level.
Figure 15: Field trial security architecture
The SUNSEED back-end infrastructure is depicted in Figure 16. It consist of four main parts in general
supporting:
- WAMS connectivity with the data centre via by using mobile connections with private APNs
- WAMS connectivity with the data centre via xDSL or FTTH connections
- WAMS connectivity with the data centre via VPN by using satellite links.
SUNSEED, Grant agreement No. 619437
Page 25 of 56
Communication solution design for field trial setup
Figure 16: SUNSEED general back-end infrastructure.
The datacentre is behind the firewall with load balancing functionality for physical separation from
the access network part. It consist of several virtual servers hosting XMPP and MQTT brokers for
receiving WAMS measurements as well as management servers for controlling WAMS nodes.
Measurements are then sent from XMPP or MQTT server to the RABBITMQ broker, which distributes
the data into several queues, each for a different measurement type (according to the description in
deliverable D4.1.1). The data from each queue is then stored into raw database (MongoDB replica
Set). There is also virtual server hosting GUI and API, for accessing and writing the data into the
database. Access to the API has several layers of security:
 API access secured with personal tokens,
 HTTPS instead of unencrypted HTTP for transferring the data,
 Implemented rate limiters for preventing too frequent requests or even DDOS attacks.
The ASR900 is a demarcation aggregation services router between Telekom Slovenije’s core network,
other services delivery networks and the SUNSEED data centre. It is connected through Telekom
IP/MPLS core to the JUNIPER MX480 MPLS core router. Juniper MX480 also has a connection to
GGSN/EPG where WAMS traffic from different APNs is aggregated. On the other hand JUNIPER M320
router is used as a laboratory demarcation router to provide connection between IP/MPLS network
and laboratory test environment. This separates the test environment from the IP/MPLS network
and is used to for injecting test systems or services into the Telekom Slovenije’s core network. This
router also serves as an interconnecting point to the Sunseed network for WAMS traffic coming via
VPNs. Next, there is a test laboratory firewall (PF SENSE) used for firewalling purposes and the
termination of OpenVPN sessions used for transporting WAMS traffic over the public internet via
SUNSEED, Grant agreement No. 619437
Page 26 of 56
Communication solution design for field trial setup
satellite links. It enables also admin access for external users and connecting to test laboratory
servers where we host open source monitoring platform Nagios. This is used for monitoring all the
SUNSEED field trial devices via SNMP.
On the mobile network part we use private APNs with no access to public internet and RADIUS server
for authentication of the on-field devices. For the purpose of routing on the LTE router we use
routing behind mobile station. In this case we send the LAN IP address information, (e.g., WAMS IP
address) from the LDAP server to the GGSN as a radius profile server attribute called “framed-route”.
This way we notify GGSN of a routing prefix for the specific location or mobile device, so that the
network has the information about the LAN IP addresses of devices using such type of connection.
Similar solution is used in case of devices connected through xDSL.
Satellite access uses open VPN connections established between the on-field routers/modems and
the test laboratory PF SENSE firewall. In all three cases the on-field devices receive IPs from the
private address space. The traffic through the Telekom Slovenije IP MPLS core is transported using
layer 3 MPLS VPN service, via several firewalls with or without load balancing functionality. Firewalls
with such functionality enable additional layer of security by limiting the traffic from/to allowed
hosts within the Sunseed cloud, filtering the source/destination ports and providing the security also
on the application layer.
The part related to mobile network’s APNs is more into details depicted in Figure 17.
Aggregation
Services Router
JUNIPER
M320 LAB
Multiservice Edge
Router
JUNIPER
MX480
WAMS
and/or
data
concentrator
WAMS
and/or
data
concentrator
WAMS
and/or
data
concentrator
WAMS
and/or
data
concentrator
WAMS
and/or
data
concentrator
MODEM
MODEM
APN
sunseed.ng.ts.si
RUT950
RUT950
or
or
IR915P
IR915P
MODEM
APN
sunseed.to.ts.si
TS MPLS
RUT950
or
IR915P
MODEM
APN
sunseed.ra.ts.si
GGSN
RUT950
or
IR915P
MODEM
Aggregation
Services Router
ASR 9000
APN
sunseed.kp.ts.si
RUT950
or
IR915P
MODEM
DATACENTER
FW
BALANCER
APN
sunseed.test.ts.si
RUT950
or
IR915P
DATACENTER
SERVERS
Figure 17: SUNSEED APNs related back-end infrastructure.
WAMS and/or PLC data concentrators are connected to the mobile modems/routers. These connect
to private APNs, where:
- “sunseed.ng.ts.si” is used for the area of Kromberk,
- “sunseed.to.ts.si” is used for the area of Kneža,
- “sunseed.ra.ts.si” is used for the area of Razdrto,
- “sunseed.kp.si” is used for the area of Bonfika, and
- “sunseed.test.test.si” is used for testing the services.
SUNSEED, Grant agreement No. 619437
Page 27 of 56
Communication solution design for field trial setup
In case of APN connections, the GGSN is used as the PDP contexts’ termination point. It is also
connected into the SUNSEED VRF (virtual route forwarding) instance of the MPLS core via JUNIPER
MX480 to access test laboratory servers and the SUNSEED datacentre.
The part related to xDSL is depicted in more detail in Figure 18.
Aggregation
Services Router
JUNIPER
M320 LAB
BRAS
SE600
TS MPLS
WAMS
and/or
data
concentrator
xDSL
CPE
Aggregation
Services Router
ASR 9000
DATACENTER
FW
BALANCER
DATACENTER
SERVERS
Figure 18: SUNSEED xDSL related infrastructure.
When WAMS devices and/or data concentrators are using xDSL access points they connect to
customer premises equipment (CPE), which are typically xDSL modems with DHCP functionality. CPE
establishes the PPPoE connection to the BRAS SE600 which enables connection with the SUNSEED
VRF instance within the IP MPLS network.
The part related to VPN is depicted mode detail in Figure 19.
SUNSEED, Grant agreement No. 619437
Page 28 of 56
Communication solution design for field trial setup
VPN
SATELITE
WAMS
Option 1
SAT
MODEM
PF SENSE
FW BALANCER
Public internet
INHAND
915
Aggregation
Services Router
JUNIPER
M320 LAB
WI-FI
BRIDGE
TS MPLS
WI-FI
BRIDGE
Aggregation
Services Router
ASR 9000
Option 2
WAMS
DATACENTER
FW
BALANCER
DATACENTER
SERVERS
Figure 19: SUNSEED VPN related back-end infrastructure.
Here, WAMS devices connect to modem/router directly or over Wi-Fi bridges. Next, the satellite
modem connects to the satellite network, connected to public Internet. Therefore, we use OpenVPN
connection between the modems/routers and the PF Sense firewall balancer in the Telekom
Slovenije’s test laboratory, to make the connection private. As already discussed above, the traffic is
then securely routed to datacentre via JUNIPER M320 and ASR 9000 routers.
3.2
Security architecture for credential management
The security architecture implemented in SUNSEED is described in Figure 20. This architecture is
based on an authorization delegation model. In such model, “Resource servers” (entities controlling
access to some “resource” delegate the administration of the access control to some or all of their
resources to an external authorization server. Figure 20 summarizes this security architecture
enabling smart devices (WAMS or smart meters) to communicate securely with remote applications
located in the cloud.
On the left hand side of the figure the data collecting devices (smart meters or WAMS) need to
obtain credentials in order to send their data to some »data dissemination' service«. Such service
will be in the case of SUNSEED based upon the use of a publish/subscribe protocol such as MQTT or
XMPP.
The devices negotiate and obtain the credentials required during a security bootstrap phase involving
the authenticated interaction with an authorization server centralizing the definition of all access
rights policies for the SUNSEED communication ecosystem.
SUNSEED, Grant agreement No. 619437
Page 29 of 56
Communication solution design for field trial setup
Figure 20 : Communication and security architecture
For WAMS devices, the credentials obtained after the security bootstrap phase are securely stored in
an embedded secure element soldered on the WAMS Printed Circuit Board.
On the right hand of Figure 20, the data transmitted by the WAMS devices is handled by a number
of cloud applications which need to »subscribe« to the information streams originating for the
SUNSEED devices. Credentials are be also required by those applications to enable them to subscribe
to the desired information. Those credentials are also obtained in a security bootstrap phase via a
negotiation with the authorization server.
Figure 21 shows a specific implementation of the architecture described in Figure 20 corresponding
to the actual workflow which will be validated in during the SUNSEED pilot.
SUNSEED, Grant agreement No. 619437
Page 30 of 56
Communication solution design for field trial setup
Figure 21: Practical implementation of security architecture
This pilot involves the deployment of WAMS device in the field. Such devices may be either
connected directly to the Wide area network using for example a global LTE connection. They may
also be located in a Local Area Network and transmit their data via a router or possibly a data
concentrator.
MQTT is the protocol used by the WAMS to transmit their data. The transmitted MQTT messages
transit via an MQTT server located in the cloud.
In order to publish data via the MQTT server, the WAMS devices need to obtain credentials from the
Authorization server using the protocols described [1].
In the pilot, the data originating from the WAMS will be received by a database application located
in the cloud will index and archive the data. At a later stage, the data will be delivered to requesting
data processing and analysis applications.
The database application will subscribe to the information topics used used by the WAMS devices to
publish. Credentials will be required to be able to subscribe; they will also be obtained via a
negotiation with the authorization server.
3.3
Credential provisioning to the WAMS node
Figure 22: describes the manufacturing and provisioning workflow involved to produce an
operational WAMS node. This workflow involves the following steps:
1. The embedded secure element is loaded at manufacturing time with a software image and with
initial secrets enabling at a later stage to remotely provision diversified new secrets inside the
secure element.
2. The initially configured secure element is soldered on the WAMS PCB during the WAMS
hardware manufacturing phase.
SUNSEED, Grant agreement No. 619437
Page 31 of 56
Communication solution design for field trial setup
3. A predefined software image including the operating system and all the application software is
loaded in the WAMS flash memory. At this point the WAMS may be deployed in the field and
connected to either to a Wide area or a private area network.
4. An initial security bootstrap phase is taking place, including the following steps:
a. The initial secrets provisioned in all secure elements (embedded in all manufactured
WAMS) at manufacturing time are mass loaded during a one time operation in the
secure element management platform to make possible remote management by this
platform of the credentials contained in the WAMS secure elements.
b. The communication services to secure (in the SUNSEED case, MQTT publishing) are
configured in the authorization server as well as access control policies, granting
publishing rights to the suitable MQTT topics to all the WAMS deployed in the fields
c. The definition of access control policies in the authorization server triggers the
authorization server to initiate the remote provisioning of the authentication credentials
as well as the access control lists in the MQTT server via a resource server proxy as
described in [1].
d. The authorization server initiates a request to the secure element management platform
to provision the authentication credentials ( in the SUNSEED case PKI client certificates)
in the secure element. The flow of operations involved is also described in [1]. The
credentials are provisioned via the WAMS IP communication channel (PLC, wifi, 4G...)
5. The WAMS is operational and ready to securely publish MQTT messages using the credentials
remotely provisioned in the secure element.
Figure 22: security and credential provisioning workflow
SUNSEED, Grant agreement No. 619437
Page 32 of 56
Communication solution design for field trial setup
4 Communication measurement procedures for the SUNSEED field
trial
This chapter focuses on the measurement procedures envisaged to be used during the SUNSEED
trial. First, the measurement procedure for derivation of the traffic characteristics generated by the
different measurement end nodes is explained in Section 4.1. The procedure for deriving the most
important communication network key performance indicators (KPIs) such as end-to-end
communication delay and throughput are illustrated in Section 4.2 and Section 4.3, respectively. The
chapter is finalized with the measurement procedure for deriving the overall radio access KPIs in
Section 4.4.
4.1
Smart grid traffic characterization measurements
By measuring the network traffic of WAMS and SM nodes in the field trial, an empirical traffic model
that describes their communication patterns can be established. While the counter-based
measurements used for the traffic measurement campaign in D3.1 did reveal some details about the
WAMS-like devices under study (that were used since the WAMS nodes were not available at that
time), a lot of information could not be extracted from those measurements. For example, it was not
possible to characterize the distributions of packet sizes and inter-arrival times of packet flows, which
is usually considered key to a traffic model. Only the measurement of cumulated amount of bytes
within certain time intervals was possible.
Therefore, during the SUNSEED trial, we intend to obtain more fine grained measurements of the
communication network traffic in a smart grid network. The standard way to obtain such fine grained
measurements is to capture network traffic traces, for example by using Wireshark, TCPdump or
similar tools that output packet capture (PCAP) trace files.
4.1.1 PCAP tracing locations
PCAP traces can be captured on different types of devices provided that the operating system allows
the tracing application to monitor the relevant network interfaces. This is possible for most off-theshelf laptop and desktop computers running Windows, Linux and Mac OS operating systems. The
most important decision for the SUNSEED trial in regards to tracing is where in the trial network to
capture the trace files. For this, we have considered four options for trace locations, namely 1)
tracing on all source nodes, 2) placement of tracing bridge between WAMS/SM and communication
network, 3) tracing in TS core network, and 4) tracing at destination node. These options are
described in more detail in Appendix A.
Out of the considered options, the most suitable and realistic choice would be to use tracing in the TS
core network. Even though the traffic model that can be extracted from such measurements may be
slightly inaccurate as discussed in Appendix A, we consider the benefit of having a single
measurement location and thus the possibility of capturing all trial measurements easily to outweigh
this downside.
SUNSEED, Grant agreement No. 619437
Page 33 of 56
Communication solution design for field trial setup
Figure 23: PCAP location for packet sniffing within Telekom Slovenija network used in the trial
4.1.2 Available information from PCAP traces
The PCAP trace files will provide us with a timestamped list of events on the monitored network
interfaces. For each layer in the protocol stack, we will have access to the header and payload
information, allowing us to perform very detailed analysis. The information that is considered
important for our future analyses is the following:








Timestamp when the IP packet was generated
Source IP address and port
Destination IP address and port
Transport protocol used, e.g., UDP or TCP
Application type
Application payload size
Application type payload content
TCP sequence number
An example of a PCAP trace captured with a Wireshark is shown in Figure 24.
SUNSEED, Grant agreement No. 619437
Page 34 of 56
Communication solution design for field trial setup
Figure 24: Example of PCAP trace, showing implementation of a SIP protocol from [2]
4.1.3 KPIs and statistics of interest from PCAP traces
In the following we will consider the KPIs and statistics of interest in the traffic analysis. First and
foremost, KPIs should be measured that reveal the load that the SUNSEED trial imposes on TS’
cellular and core networks. In addition, the measurements should be used to create a statistical
traffic model, which can be used to create and evaluate the load and performance of artificial
scenarios, e.g. using network simulations. These statistical models should thus describe the traffic
generation behaviour of individual WAMS and SM nodes. The KPIs and statistics can be calculated for
different levels, namely:





for the entire SUNSEED trial (i.e., all data captured in PCAP trace)
for a specific trial location
for a particular device
per flow (TCP)
per message type (e.g., control data, measurement, …)
For the first three levels, it makes sense to consider KPIs for both uplink and downlink directions.
The specific features of interest that we intend to characterize are addressed in the following and are
inspired by similar work [3],[4]. Each of the features may be considered for the different levels
specified above.

Traffic volume/bandwidth: Both the uplink and downlink traffic volumes should be
considered, as well as the uplink/downlink ratio. From this, by taking the trial duration into
SUNSEED, Grant agreement No. 619437
Page 35 of 56
Communication solution design for field trial setup
account, the occupied bandwidth/throughput can also be calculated. Both mean values and
empirical probability distributions are of interest.

Packet size: The packet size is a key parameter to be determined for a traffic generation
model. An empirical probability distribution of the used packet sizes is of highest interest
here, since typically in M2M packets are either small control packet or large bulk payload
packets, therefore the mean is not very representative.

Time series of packet generation: The timely behaviour of packet generation can be
characterized using either the packet inter-arrival time (subtracting timestamps of
consecutive packet events) or by analysing the periodicity from the power spectral density
(periodogram) or autocorrelation function.

Session analysis: For TCP sessions, there are several aspects that would be interesting to
consider, namely the active time, average session length, session inter-arrival time, and
session total traffic volume. Both empirical probability distributions and mean values are of
interest.

Joint parameter determination: Fitting inter-arrival and packet size independently may not
result in the correct aggregated traffic volume. Therefore, a joint model parameter fitting
procedure should be used to find the combination of parameters that best describe the
observed data. For this, machine learning techniques can be used to model parameter
estimation.
4.1.4 Traffic Characterization Measurement procedure
The SUNSEED communication architecture uses a publish-subscribe principle, where the number of
data flows depends on the number of active traffic sources and the number of subscribers.
We need to know the following configuration information while measuring the traffic:
a) How many nodes are installed and what is the configuration for reporting data (e.g. the time
interval)?
b) How many nodes are activated (or allowed to publish part of the information)?
c) What information can be published (as specified in D2.2.2 [5]) and what has been subscribed
to by different applications?
It should be stressed here that the non-SUNSEED nodes will have an influence on the
delay/throughput of the generated packets and not on the generated traffic volume and inter-arrival
times from the SUNSEED nodes.
To be able to characterize how well the proposed communication architecture scales with an
increasing number of nodes, it may be necessary to gradually increase the number of active devices
in the field trial traffic characterization experiment.
That is, we should have different phases, where for each phase we note down the configuration and
then we capture PCAP traces for a period of time, e.g., 1 week. Hereafter, we repeat for the next
phase with a higher number of devices.
Before the measurement campaign is started for each of the deployed SM, WAMS-SPM, and WAMSPMC node the following information should be determined and saved:
a) The IP address of the node, which will be fixed and permanently bound to the IMSI of the LTE
modems for the whole duration of the SUNSEED trial.
SUNSEED, Grant agreement No. 619437
Page 36 of 56
Communication solution design for field trial setup
b) The serving cell of the node (and its access type UMTS or LTE, correspondingly TAC, etc.) i.e.
where in the TS cellular network this node is located and served. It is expected that the
serving cell is constant most of the time, since devices are static. However, handovers may
happen in case that the serving cell becomes unavailable or if the radio propagation
conditions change, e.g., due to adverse weather or new/moving obstacles affecting the
propagation channel.
During the measurement campaign, the measurement reporting period of the different nodes
(where applicable) should be known and saved. For example, as the WAMS-SPM measurement
reporting period is configurable during the measurement campaign it should be known (logged) what
reporting period was configured per WAMS-SPM node. Similarly, this measurement reporting
configuration should be known and saved also for the SM and WAMS-PMC nodes.
Additionally, during the measurement campaign the packets transmitted from the installed SM,
WAMS-SPM and WAMS-PMC nodes should be marked with regard to their payload in order to
distinguish between:
a) Power line measurement reports in uplink, commands towards the installed nodes, etc.
which are all relevant for the derivation of the SG traffic models.
b) Communication network related reports in uplink from e.g. WAMS-SPM or WAMS-PMC node
or from the Teltonika LTE modems (connected to PLC concentrators) as these packets should
be excluded from the derivation of the SG traffic models.
4.2
WAMS-SPM end-to-end delay measurement
This section details the measurement procedure for the end-to-end delay from WAMS-SPM nodes.
The following assumptions hold (see also Figure 25):
-
WAMS-SPM nodes send power line measurements with time-stamp based on GPS time
reference.
WAMS-SPM nodes, APN server in LTE/UMTS core network and database server where all
SUNSEED field trial data is stored are time-synchronized via GPS time reference.
The current proposal for deriving the end-to-end delay is to use IP packet trace as all WAMS-SPM
nodes in the trial will have fixed IP addresses. Consequently, the time-stamps of the data packets are
read at:
-
-
APN server in LTE/UMTS core network. Subtract the time-stamp in the IP packet from the
current GPS based time at APN server to calculate the WAMS-SPM to APN server delay
TAPN,LTE and TAPN,UMTS for the LTE and UMTS network, respectively.
SUNSEED database server. Subtract the time-stamp in the IP packet from the current GPS
based time at SUNSEED’s database server to calculate the end-to-end delay between WAMSSPM to SUNSEED APN delay.
Note here that in the SUNSEED trial the SM and WAMS-PMC nodes are not planned for GPS time
synchronization with the e.g. SUNSEED’s APN server or the database server. Therefore, the end-toend delay measurements for these nodes will not be possible.
The measurement set-up also assumes that the delay between the APN server at Telekom Slovenija
and the SUNSEED measurement database TAPN, Database is the same regardless if the packet has
travelled via the LTE or the UMTS access network.
SUNSEED, Grant agreement No. 619437
Page 37 of 56
Communication solution design for field trial setup
For the statistical processing of the measurement data the end-to-end delay measurements can be
considered as one set or, alternatively, the end-to-end delay measurements can be divided into
groups depending if the access network is LTE, UMTS, or satellite.
Figure 25 Schematic view on the end-to-end delay measurement procedure for WAMS-SPM
nodes
4.2.1 Measurement approach
The phasor measurements obtained in the WAMS-SPM nodes are collected in a proactive manner to
the SUNSEED database (DB) server, meaning that upon finishing a measurement burst, the WAMSSPM assembles an IP packet, which is immediately queued and sent towards the DB server. For
measuring the end-to-end delay of the time-critical WAMS-SPM measurements, we will exploit the
fact that the WAMS-SPM nodes are very accurately time synchronized. Figure 26 shows how the
latency measurements for the two measurement points (APN and DB) are obtained. In this figure, τ 1
represents the WAMS-SPM to LTE/UMTS APN delay, while τ2 represents the end-to-end WAMS-SPM
to SUNSEED APN delay.
SUNSEED, Grant agreement No. 619437
Page 38 of 56
Communication solution design for field trial setup
The delay introduced by the communication network depends largely on the traffic load in the
network, as well as the priority level of smart grid traffic in scheduling. During the SUNSEED trial
there will be no prioritization of the SUNSEED traffic, in other words the traffic will be handled in best
effort manner. Optionally, some prioritization policy of the SUNSEED traffic might be used in the core
network (BRAS level).
Thus, it would be interesting and relevant to measure the end-to-end delay performance at different
traffic load of the day, and with (if feasible) different scheduling priority levels.
Figure 26: Measurement collection from WAMS-SPM. Upon completion of a measurement
burst, a packet is assembled and sent to the TS core, where it is detected and a timestamp is
made at the APN, and finally at the DB server, another timestamp is made when the data is
stored in the database.
4.3
End-to-end throughput measurement
In order not to distort the end-to-end delay measurements in Section 4.2 and consequently also the
throughput measurements, as explained in this section, the reporting of the communication network
measurements should be non-overlapping with the actual power line measurements reporting. For
example, the reporting of the communication network measurements is preferably done at the end
of each power line measurement reporting session, which session can be for example at the end of
the measurement hour, day or week.
It should be stressed here that from the point of view of SG real-time control of the end-to-end
throughput is not as relevant as the end-to-end delay. Furthermore, due to the limitations in the
level of details in the communication network measurements it will not be feasible to determine the
throughput for all individual nodes (e.g. the installed SM and WAMPS-PMC nodes due to lack of GPS
time reference) and on all network segments (i.e. throughput in the radio access segment and/or PLC
segment, mobile core, TS transport network after the SUNSEED APN etc.). Therefore, it is expected
that the only end-to-end communication throughput measurement will be feasible for the WAMSSPM nodes defined as the packet size divided by the packet end-to-end delay as determined in
Section 4.2..
SUNSEED, Grant agreement No. 619437
Page 39 of 56
Communication solution design for field trial setup
Additionally, as already mentioned above, another option for the throughput measurements would
be to derive it from the (uplink and downlink) traffic volumes generated by the SM and WAMS-PMC
nodes. Then appropriate measures have to be taken in order to estimate the “useful content”. By
useful content we define the total data stripped out of packet headers and transmission artefacts
(retransmissions, redundant coding etc.). This can be done in two ways:


From the PCAP traces (used for traffic characterisation) we can filter out measurement
packets from a specific source-destination pair and calculate avg. throughput.
Count the amount of received data in database.
From this, by taking into account the aggregation of the downlink and uplink traffic volume, and the
trial measurement duration, the occupied bandwidth/throughput can also be calculated. Both mean
values and empirical probability distributions are of interest. Note that this approach for the end-toend measurements is less accurate than the approach based on the end-to-end delay of the WAMSSPM packets.
4.3.1 Throughput measurement setup in controlled laboratory
environment
We investigate the limits of PLC communication from smart meters to PLC concentrators under
varying number of smart meters, reporting intervals and line conditions. Real time smart meter
communication is bound at 1 min to 15 min reporting intervals. PLC concentrators are linked with LTE
router-modems via telecom operator network to central database within utility network
management environment. Gathered data are used for real time state estimation.
Background
It is preferred to use as small number of PMU units to achieve observation of distribution grids. Real
time smart meter readings are used to augment PMU measurements to achieve full observability.
Gathering measurements in real time from smart meters (e.g. 1 min reporting interval) may present
challenge to PLC communication channel as it differs greatly from classical AMI for billing purposes
smart meter data gathering, i.e. once per day.
Test setup
We set up a test measurement facility in laboratory-controlled environment that consists of:
 Different types of Smart meters (37) (Landis+Gyr, Iskraemeco)
 PLC concentrator (Landis+Gyr)
 LTE router-modem (Teltonika RUT550)
 PLC communication channel (power line), between smart meters and PLC concentrator
 Ethernet communication channel, between PLC concentrator and LTE mode
 Wireless communication channel, between LTE modem and database
 Power load
 Grid line measurement unit (Elspec G4500)
 Ethernet packet measurement unit (Wireshark)
 Remote management software for PLC concentrator and LTE modem
 Database
PLC channel is S-FSK 1, 3 phase smart meters of different types and producers are connected with
single PLC line to PLC concentrator, and their reporting interval is varying (1 – 15 min).
SUNSEED, Grant agreement No. 619437
Page 40 of 56
Communication solution design for field trial setup
Figure 27: End-to-end scheme
Figure 28: Lab environment - test bed for end-to-end telecommunication compatibilities
All measurements are gathered at PLC concentrator and transported to database via high bandwidth
LTE (4G) cellular network. Transport is via RabbitMQ message queue protocol over TCP directly to
SQL database. With using existing S-FSK technology, we transmit all needed billing data for DSO with
common reliability and we try to definite additional data volume transfer for other needs like our
applications in the project. We describe additional tasks in concentrator for periodic register reading
for each meter. To get real financial valuation we decide to calibrate WAMS devices with official
registered meters.
SUNSEED, Grant agreement No. 619437
Page 41 of 56
Communication solution design for field trial setup
To receive full information we decided that is sufficient to read 10 primary values from meter
registers (U1, U2, U3, I1, I2, I3, I0, PF1, PF2, PF3).
In practice, we managed to read all data from 34 meter in time less than 2 minutes. In long-term
test, we get all billing data (daily, 15min periods, and monthly data) and periodical reading of our 10
registers per meter each 5 minutes with 100% success.
Figure 29: time of reading 10 registers from 1 smart meter is less than 2 seconds)
Results and conclusion
Having dense network of PLC-connected smart meters communicating with short reporting intervals
over present S-FSK-specified PLC channels is a hard task. Error rates influence the state estimation
significantly. LTE cellular communication is a viable communication link for gathered smart meter
data.
Disturbances from different home devices could override signal and there could be blind windows for
receiving measuring data.
With incoming new technology PLC OFDM based on ERDF G3 protocol we are expecting more
reliable signal transfers and possibly more capacity for transmitted signals. Further work is required
to valuate new incoming communication technologies and to establish field trial test results under
operating distribution grid conditions that is planned within project itself.
Objective:
Imitation of real conditions and environment for operating measurement system. The goal is to
imitate some real topology segments of future field test. On this model we can measure and observe
behaviour of PLC communications depending on other influences (overvoltage’s, transients,
disturbances, noise of nonlinear loads at end users....). The physical model can help us to test PLC
communications at extreme and rear circumstances in the net and to fine tuning of computer model
of the net to get more accurate results closer to reality.
SUNSEED, Grant agreement No. 619437
Page 42 of 56
Communication solution design for field trial setup
Equipment:
• HES/MDC
• concentrator DC450
• Meters E450
• Basic Test bed with modular structure allowing different configurations
• Changeable modules to imitate length and characteristics of power lines
• Coupling and Filter circuits for injection of disturbance in to net
• RF Signal Generator to imitate disturbances
• Standard loads and loads from field coursing malfunction of PLC communication
Test configuration:
Figure 30: Laboratory test LV power grid configuration
Purpose of measurement:
The main purpose of this testing is to construct permanent modular testing bed to imitate real
circumstances on the real power net and have possibility to repeat some transients or other
SUNSEED, Grant agreement No. 619437
Page 43 of 56
Communication solution design for field trial setup
phenomena and repeated it many times and observe how they influence on correct work or what
caused malfunction or aging of elements of measurement system.
We like to build and define circuit with a common topology and constructed from most common
widget of today measurement system with common loads to get some kind of “Nominal model”,
where all parameter and values can be measured and results can be repeated. This measurement
should be calibrate and comparable to field test as much as possible. On this basis we can in future
compare new equipment (new products, different produces...), technologies (PLC G3) and evaluate
with measured quantities and numbers to define advantages, cost efficient, interoperability and
complementary with existing system. Some typical measured parameters are: Maximum length of
line to the next meter, life time, capacity and reliability of data transmission, noise rate, resistance on
different disturbances, and time between maintenance...
To have possibility to examine some single events from field trials and to fine turning of computer
simulations to get as much closer result to reality.
The model will help us to do case studies to find reason for multiple similar malfunction of
equipment by imitating the suspected influences in similar environments. Some typical problems on
the fields are: over current disconnection without known reasons, low success rate of
communication and too many relay connection on short distances, malfunction of power supply of
meters because of high frequency disturbances, crosstalk over MV transformer stations...
The method of measurement:
1) Definition and description of the network;
2) Calculating and preparing appropriate 4 pole wiring for imitation of topology of different
realistic field scenarios; Commissioning the system and get familiar with its calibrations,
possibilities and options; By changing the configuration of the circuit - determine the limits of
the system
3) 4. Verifying operation of the test bed with field tries and makes calibrations
4) 5. To reconcile all scenarios and test with boundary conditions to ensure the repeatability of
“normal model”. Task is to determinate the common values for today existing measurement
systems in DSO.
5) With the addition of equipment from different manufacturers - compliance of equipment
operation (interoperability)
6) Establishment of operation and testing of various types of communications between the
communicator and measurement centres (PLC, GPRS ...)
7) Comparison of existing technology FSK with G3, and on the comparative methods to identify
the differences and benefits of the new technologies
4.4
Radio access related measurements
The radio access related measurements are useful for obtaining information about the performance
of the LTE (and UMTS where applicable) radio interface in uplink and downlink at the LTE (or UMTS)
cells of interest during the SUNSEED trial.
The approach for collecting these radio access related measurements is at the eNode B side via
statistical counters as explained in Section 4.4.1. Additionally, relevant radio access information can
be gathered via Gemalto's or Teltonika's LTE (or UMTS) modem as presented in Section 4.4.2 and
4.4.3, respectively.
SUNSEED, Grant agreement No. 619437
Page 44 of 56
Communication solution design for field trial setup
4.4.1 Relevant measurements derived from statistical counters/KPIs at the eNB
Table 1
Parameter
RACH (%)
User download
throughput
(kbit/s)
User upload
throughput
(kbit/s)
Cell_DL_thr
Cell_UL_thr
IntP
I_Pucch
PrbDl% Avg
PrbUl%Avg
Equation
(EUtranCellFDD.pmRaSuccCbra +
EUtranCellFDD.pmRaSuccCfra) /
EUtranCellFDD.pmRaAttCbra +
EUtranCellFDD.pmRaAttCfra) * 100
(EUtranCellFDD.pmPdcpVolDlDrb +
EUtranCellFDD.pmPdcpVolDlDrbTransUm –
EUtranCellFDD.pmPdcpVolDlDrbLastTTI) /
EUtranCellFDD.pmUeThpTimeDl * 1000
EUtranCellFDD.pmUeThpVolUl /
EUtranCellFDD.pmUeThpTimeUl * 1000
EUtranCellFDD.pmPdcpVolDlDrb +
EUtranCellFDD.pmPdcpVolDlDrbTransUm –
EUtranCellFDD.pmPdcpVolDlDrbLastTTI) / ROP2
EUtranCellFDD.pmUeThpVolUl / ROP
AVERAGE(EUtranCellFDD.pmRadioRecInterferencePwr)
AVERAGE(EUtranCellFDD.pmRadioRecInterferencePwrPucch)
AVERAGE(EUtranCellFDD.pmPrbUtilDl)
AVERAGE(EUtranCellFDD.pmPrbUtilUl)
Table 2: LTE radio counters.
Counter
EUtranCellFDD.pmRaSuccCbra
EUtranCellFDD.pmRaSuccCfra
EUtranCellFDD.pmRaAttCbra
EUtranCellFDD.pmRaAttCfra
EUtranCellFDD.pmPdcpVolDlDrb
Description
The number of successfully
detected RA Message 3 for CBRA3.
The number of successfully
detected RA Message 3 for CFRA4.
The number of detected CBRA
preambles in the cell.
The number of detected CFRA
preambles in the cell.
The total volume (PDCP SDU) on
Data Radio Bearers that has been
transferred (UM and AM) in the
downlink direction.
Unit
PEG
PEG
PEG
PEG
1 kbit
When carrier aggregation is used,
2
ROP: result output period (in seconds), e.g. ROP = 900 in case of 15 min reports.
CBRA: contention-based random access preamble. Randomly selected preamble used for initial
network access, access following a radio link failure, handover between cells, downlink data transfer
requiring UE synchronization, uplink data transfer requiring UE synchronization.
4 CFRA: contention-free (explicitly signalled) random access preamble. Reserved preamble used for
handover between cells and downlink data transfer requiring UE synchronization.
3
SUNSEED, Grant agreement No. 619437
Page 45 of 56
Communication solution design for field trial setup
EUtranCellFDD.pmPdcpVolDlDrbTransUm
EUtranCellFDD.pmPdcpVolDlDrbLastTTI
EUtranCellFDD.pmUeThpTimeDl
EUtranCellFDD.pmUeThpVolUl
SUNSEED, Grant agreement No. 619437
a PDCP SDU can be sent over
multiple cells (PCell/SCell(s)). The
total volume (PDCP SDU) on Data
Radio Bearers that has been
transferred (UM and AM) in the
downlink direction is measured on
PCell.
The total volume (PDCP SDU) on
1 kbit
Data Radio Bearers for RLC UM
that has been transmitted in the
downlink direction in the PDCP
layer.
When carrier aggregation is used,
a PDCP SDU can be sent over
multiple cells (PCell/SCell(s)). The
total volume (PDCP SDU) on Data
Radio Bearers for RLC UM that has
been transmitted in the downlink
direction in the PDCP layer is
measured on PCell.
The total volume (PDCP SDU) on
1 kbit
Data Radio Bearers that has been
transferred (acknowledged by the
UE) in the downlink direction in
the last TTI when a buffer is
emptied.
The effective DL transport time
ms
comprises those periods from
when the first part of the PDCP
SDU of the DL buffer was
transmitted on Uu until the buffer
is emptied, excluding the TTI
emptying the buffer.
When carrier aggregation is used,
a PDCP SDU can be sent over
multiple cells (PCell/SCell(s)). The
effective DL transport time,
comprising those periods from
when the first part of the PDCP
SDU of the DL buffer was
transmitted on Uu until the buffer
is emptied, excluding the TTI
emptying the buffer, is registered
on PCell.
The UL DRB volume used for UL
UE Throughput. It comprises of
the MAC SDU volume received on
Uu, excluding the volume
kbit
Page 46 of 56
Communication solution design for field trial setup
EUtranCellFDD.pmUeThpTimeUl
EUtranCellFDD.pmRadioRecInterferencePwr
EUtranCellFDD.pmRadioRecInterferencePwrPucch
EUtranCellFDD.pmPrbUtilDl
received in the first 4 data
receptions of an UL buffer
transfer and the TTI emptying the
UL buffer.
The UL volume transfer time used
for UL UE Throughput. It
comprises of time periods from
when the 5th MAC SDU data
reception of an UL buffer transfer
on Uu until the buffer is emptied,
excluding the TTI emptying the
buffer.
The accumulated interference
power for one PRB.
The measured Noise and
Interference Power on PUCCH,
according to 36.214.
ms
mW * 2^(44)
dBm/PRB5
PDF ranges:
[0]: N+I <= -121
[1]: -121 < N+I <= -120
[2]: -120 < N+I <= -119
[3]: -119 < N+I <= -118
[4]: -118 < N+I <= -117
[5]: -117 < N+I <= -116
[6]: -116 < N+I <= -115
[7]: -115 < N+I <= -114
[8]: -114< N+I <= -113
[9]: -113 < N+I <= -112
[10]: -112 < N+I <= -108
[11]: -108 < N+I <= -104
[12]: -104 < N+I <= -100
[13]: -100 < N+I <= -96
[14]: -96 < N+I <= -92
[15]: -92 < N+I
A distribution that shows the
downlink Physical Resource Block
(PRB) pair utilization (total
number of used PRB pairs by
available PRB pairs) on the
Physical Downlink Shared Channel
(PDSCH).
PDF ranges:
[0]: 0 % <= Utilization < 10 %
[1]: 10 % <= Utilization < 20 %
5
PRB: physical Resource Block. PRB defines number of subcarriers for a predetermined amount of
time (time-frequency dimension) and it is handled by a scheduling function at the eNB.
SUNSEED, Grant agreement No. 619437
Page 47 of 56
Communication solution design for field trial setup
EUtranCellFDD.pmPrbUtilUl
[2]: 20 % <= Utilization < 30 %
[3]: 30 % <= Utilization < 40 %
[4]: 40 % <= Utilization < 50 %
[5]: 50 % <= Utilization < 60 %
[6]: 60 % <= Utilization < 70 %
[7]: 70 % <= Utilization < 80 %
[8]: 80 % <= Utilization < 90 %
[9]: 90 % <= Utilization
A distribution that shows the
uplink Physical Resource Block
(PRB) pair utilization (total
number of used PRB pairs by
available PRB pairs) on the
Physical Uplink Shared Channel
(PUSCH).
PDF ranges:
[0]: 0 % <= Utilization < 10 %
[1]: 10 % <= Utilization < 20 %
[2]: 20 % <= Utilization < 30 %
[3]: 30 % <= Utilization < 40 %
[4]: 40 % <= Utilization < 50 %
[5]: 50 % <= Utilization < 60 %
[6]: 60 % <= Utilization < 70 %
[7]: 70 % <= Utilization < 80 %
[8]: 80 % <= Utilization < 90 %
[9]: 90 % <= Utilization
4.4.2 Gemalto LTE modem
The relevant AT command for collecting the radio access parameters when the LTE modem is
camping on the cell is:
Syntax:^SMONI: ACT,EARFCN,Band,DL bandwidth,UL bandwidth,Mode,MCC,MNC,TAC,Global Cell ID,Phys-ical
Cell ID,Srxlev,RSRP,RSRQ,Conn_state
Example Answer: ^SMONI: 4G,6300,20,10,10,FDD,262,02,BF75,0345103,350,33,-94,-7,NOCONN
The relevant parameter to be measured when the LTE modem is camping on the cell are RSRP and
RSRQ. These two parameters can be used for deriving the coverage conditions (e.g. pathloss via RSRP
value and knowledge of the transmission power of the cell’s reference signal – RS) and downlink
interference/SINR conditions (e.g. by deriving the downlink SINR from the RSRQ value). Additionally,
the Global Cell ID should be also recorded in order to be known which cell is serving the WAMS node.
The relevant AT command when the LTE modem is in active state is:
^SMONI: ACT,EARFCN,Band,DL bandwidth,UL bandwidth,Mode,MCC,MNC,TAC,Global Cell ID,Phys-ical Cell
ID,TX_power,RSRP,RSRQ,Conn_state
Example Answer: ^SMONI: 4G,6300,20,10,10,FDD,262,02,BF75,0345103,350,90,-94,-7,CONN
SUNSEED, Grant agreement No. 619437
Page 48 of 56
Communication solution design for field trial setup
The difference between idle and active ^SMONI response is that instead of the parameter Srxlev (i.e.
signal strength threshold for base station selection as defined in TS 25.304) the Tx power (i.e. used
uplink power) is measured and that the connection state changes from NOCONN to CONN. The used
uplink power is useful to determine the following:
a) Maximum number of PRBs that can be used for the uplink transmission based on the
estimated path-loss (from RSRP measurements) and average uplink interference level from
the 15 min counter IntP or I_Pucch as illustrated in Section 4.4.1.
b) Possible uplink SINR for the uplink transmission, based on the average uplink interference
level from the 15 min counter IntP or I_Pucch as illustrated in Section 4.4.1.
4.4.3 External communication equipment
Monitoring of communication equipment should be done in organized and manageable way. SNMP,
or Simple Network Management Protocol, is a standard network management protocol widely used
in TCP networks and provides a method of managing the device – SNMP agent, e.g. LTE modem (or
other equipment) through the running the central computer of network management software, i.e.
SNMP server. In production there is 3 different versions (v1, v2 and v3), which mainly differs in
performance, security, confidentiality and manager-to-manager communications. For mobile clients
in Sunseed pilot, comm. security is handled when setting up the connection, i.e. on mobile gateway
(with private APNs), therefore there is no need to use controversial SNMP security model, but
Community-Based SNMP v2 (SNMP v2c) satisfy the network management requirements. At protocol
level SNMP defines following protocol data units (PDUs) namely:
-
GetRequest to retrieve the value of variable or list of variables from client,
SetRequest to request for changing the value of variable or list of variables,
GetNextRequest to discover available variables ant their values,
GetBulkRequest where server request client to respond with multiple bindings in the request,
Response return variable bindings and acknowledge server’s request,
Trap which is used to send asynchronous notification from the agent to the server and
InformRequest which acknowledges asynchronous notification (due to unreliable UDP nature
of SNMP protocol).
In Sunseed pilot monitoring the communication infrastructure is managed both by the asynchronous
(traps) and synchronous, i.e. period reporting (pooling) approach, such that first provides any sudden
warning or errors, like “mobile signal strength has fallen below defined threshold” and the latter for
monitoring statistics, like “number of transmitted upload or download bytes”. Request is defined by
OID (Object Identifier), which uniquely identify managed objects in a MIB (Management Information
Base) hierarchy. Example of pooling technique on Inhand LTE modem is reading the status of RSSI
signal:
Code when numeric-based OID is dislayed
Numeric OIDs and raw values can also be displayed in more meaningful texual names and sensible
formatted values using MIB files and cmd tool like snmptranslate. The example above identifies a
particular object, which can be displayed using list of MIB identifiers (provided by vendor of comm.
equipment, i.e. Inhand) to:
Code when text-based OID is dislayed
SUNSEED, Grant agreement No. 619437
Page 49 of 56
Communication solution design for field trial setup
SNMP community management also provide different access levels (“read-only”, “read-write”),
hence enabling and using specific community strings also provides operating the equipment on
remote (Figure 31).
Figure 31: SNMP community
Different equipment provides various OID, some of them are “general purpose”, while others remain
in vendor’s domain. Regarding the Sunseed pilot, OIDs related to RF signal (RSSI signal strength,
attached mobile base station), identification (imsi, equipment name, serial number) and network
layer (number of UL and DL bytes, mobile IP, connection uptime) is of great importance. SNMP data
from server is then being processed with monitoring tool (Nagios), where additional parameters can
be calculated and displayed in graphical form, e.g. graphs of outages. Furthermore, data is also
populated in dedicated GIS system, where one (SG operator) have overview and control over SNMP
events in geo-spatial domain.
SUNSEED, Grant agreement No. 619437
Page 50 of 56
Communication solution design for field trial setup
5 Field trial communication measurements and relation to
SUNSEED use cases
5.1
Measurements related to state estimation use case
The WAMS-SPM power line measurements, as well as additional SM and WAMS-PMC power line
measurements will be reported via TCP/IP protocol meaning that the power line measurements will
be eventually available at the SUNSEED database for post-processing. As a consequence, the issue is
if the reported power line measurements is available on time (to be post-processed by the state
estimation algorithm) and less of an issue if the power line measurements will be available at all.
However, it can happen that particular measurement is not available or not valid due to some other
reasons (e.g. transient phenomena in SPM as discussed in D4.1.1 [D4.1.1], communication outage
from some nodes, …).
The state estimation uses the following power line measurement:


SPM:
o
o
PMC:
o
o

measurements/data: node_id , v1, v2, v3, th1, th2 , th3, psp_v, psp_th, status, f1, f2,
f3, f4, df1, df2, df3, df4, , stamp1, stamp2
period: 1 second or more; format JSON see D2.2.2
measurements/data: node_id, v1, v2, v3, i1, i2, i3, i4, f1, p1, p2, p3, q1, q2, q3,
stamp,
period: 1 second or more; format JSON see D2.2.2
SM:
o
o
measurements/data: node_id, v1, v2, v3, i1, i2, i3, i4, f1, p1, p2, p3, q1, q2, q3, vv1,
vv2, vv3, stamp,
period: 15 minutes; format JSON see D2.2.2
The overall delay Dall at which the data is available for further processing in data storage can be
calculated as follow:
Dall = Dprocesing_node + Dcommunications+ Dprocesing_storage
While the processing delays at the node and on the server side are quite deterministic, the
communication’s delay depends on the availability/congestion and the type of the communication
path. Although the state estimation works also if some measurements are not available (resulting in
higher error), it is of paramount importance that it waits at least Dall before processing the data and
SE calculation. Thus, the knowledge about the communication delays is essential in particular as in
some cases where the satellite link is used (Kneža testbed) it can be more than 250ms, which is a few
times more than in LTE.
During the SUNSEED trial, the reporting power line measurement period for the WAMS-SPM nodes
will be configured from few minutes (e.g. 15, 10, 5, or 1) down to 1 second. Next to this, the
reporting period of power line measurements of WAMS-PMC might be also configured towards
lower values (e.g. lower than 15 min) in particular as it can provide more “up to date” injection
information, which improves the state estimation. Thus, it is advantageous that WAMS-PMC
SUNSEED, Grant agreement No. 619437
Page 51 of 56
Communication solution design for field trial setup
measurements are in line with WAMS-SPM. The SM measurements will be collected on 15 minutes
interval, which is the standard metering interval.
The trial measurements will aim at answering the following important questions:
a) Is it feasible to have real-time SG state estimation with the installed types of measurement
nodes (WAMS-SPM, WAMS-PMC and SM) and using the installed communication
technologies (e.g. LTE, UMTS, and PLC)? How real-time can we follow the SG state, is this on
seconds or few minutes level? What are changes in DS between two consecutive intervals?
b) What is the impact on the communication network at the telecom operator (e.g. LTE or
UMTS cellular network) in terms on resource/capacity use for different real-time SG state
estimation periods (e.g. from 1 second to 15 min)?
c) Do the higher SE update rates justify the higher use of communication resources (e.g. use of
communication resources vs. SE performance)?
d) Anything else…
5.2
Measurements related to demand-response use case
Demand-response use case can be considered as the subset of SE use case from the power line
measurement perspective. It means that if the SE power measurements are in place no additional
power line measurements need to be collected, as all necessary data are already available. However,
in the case that demand response use case is used without SE, then the following power line
measurements need to be collected:

PMC:
o
o

measurements/data: node_id, v1, v2, v3, i1, i2, i3, i4, f1, p1, p2, p3, q1, q2, q3,
stamp,
period: 15 minutes or less; format JSON see D2.2.2
SM:
o
o
measurements/data: node_id, v1, v2, v3, i1, i2, i3, i4, f1, p1, p2, p3, q1, q2, q3, vv1,
vv2, vv3, stamp,
period: 15 minutes; format JSON see D2.2.2
In addition to power line data there is additional data send from/to nodes that are connected to
controllable appliances. Using FPAI these nodes send data regarding the control space of their energy
consumption and receive data regarding allocation of their energy consumption:

Control space information
o Registration of control space
o Period On demand, typically less than once every 15 minutes; format XML see D2.2.2
and D4.4.1
o Update of control space
o Period: On demand, typically less than once every 15 minutes; format XML see
D2.2.2 and D4.4.1Allocation information

Allocation information
o Allocation
o Period: On demand, typically less than once every 15 minutes; format XML see
D2.2.2 and D4.4.1
o Allocation update
SUNSEED, Grant agreement No. 619437
Page 52 of 56
Communication solution design for field trial setup
o
Period: On demand, typically less than once every 15 minutes; format XML see
D2.2.2 and D4.4.1
As the general period for the demand-response use case is 15 minutes, the communication is not
that sensible to delay, but rather to the reliability in particular for the control data.
5.3
Measurements related to SG outage localization use case
During the SUNSEED trial the installed WAMS-SPM and WAMS-PMC nodes will be configured to
transmit ‘artificial power outage alarms/events’ while it is expected that the deployed SM nodes
from Landys Gyr cannot be configured to send these ‘artificial outage messages’. As part of this
outage alarms the GPS location of the WAMS-SPM or WAMS-PMC nodes will be reported. Using
these enriched power outage messages it will be shown during the trial how quickly a power outage
can be localized in the power grid.
SUNSEED, Grant agreement No. 619437
Page 53 of 56
Communication solution design for field trial setup
6 Conclusions
The deliverable outlines the specific scenarios for communication solutions on separate field trial
locations which are geographically quite distinguished. Regarding the proposed scenarios, the
general communication SUNSEED back-end infrastructure with security architecture and provisioning
the security credentials to the end nodes is defined.
In particular, we focus on considered solutions for WAMS (SPM and PMC) and for smart meters,
respectively. Of course, due to costs reduction and where possible, WAMS and smart meters can use
the same gateway for a considered communication solution. For WAMS scenarios the typical wiring
diagram scheme and an example of inventory material with technical specification was made. On the
basis of coverage analyses from deliverable D5.1.2 the most appropriate communication scenario per
each transformer station as the main functional location was assigned.
In-depth analyses of the field trial locations and of the Telekom Slovenije infrastructure reviled that
mobile and satellite communications are the most appropriate to connect majority of WAMS devices
into the SUNSEED cloud. Therefore, we distinguish two communication scenarios, where WAMS
(SPM/PMC) are using mobile or fixed modem/router (Scenario A) and WAMS SPM are using satellite
modem/router (Scenario B). With respect to the analysis of fixed network access points presence and
mobile network coverage from deliverables published in WP5 scenario A is further divided into three
sub-scenarios, where gateway (modem/router) could connects the LTE network (scenario A.1), UMTS
network (scenario A.2) or xDSL/FTTH network (scenario A.3). Regarding the position of a satellite
antenna scenario B is also divided into two sub-scenarios where WAMS could be located at utility
functional location and it is connected directly to satellite modem/router, while the location can
have also Wi-Fi bridge to subordinate functional location (Scenario B.1). Other possibility is WAMS
positions at subordinate functional location and connects to the satellite modem/router via Wi-Fi
bridge (Scenario B.2).
Similar as WAMS in scenario A, smart meters use mobile or fixed modems/routers as gateways for
connecting into the SUNSEED cloud via mobile or fixed network. However, with regard to the exact
smart meter model, they are connected to such gateways in different manner. We distinct two
scenarios, where smart meters are connected to PLC data concentrator (DC), which is connected to
mobile or fixed modem/router (Scenario C) or they are connected to mobile or fixed modem/router
via RS485 to Ethernet converter (Scenario D).
Sunseed IT selected security will insure the proper filtering of data transmission at the IP level. It
consists of four main parts in general to support WAMS connectivity with the data centre via by using
mobile connections with private APNs, WAMS connectivity with the data centre via xDSL or FTTH
connections and WAMS connectivity with the data centre via VPN by using satellite links. The
datacentre is behind the firewall with load balancing functionality for physical separation from the
access network part. For end WAMS nodes access the authorization delegation model is used. . In
such model, “Resource servers” (entities controlling access to some “resource” delegate the
administration of the access control to some or all of their resources to an external authorization
server.
During the trial measurements it is important to characterize the ongoing traffic in terms of e.g.
packet sizes, inter-packet arrival times, aggregated amount of traffic, etc. This realistic traffic
SUNSEED, Grant agreement No. 619437
Page 54 of 56
Communication solution design for field trial setup
characterization based on the trial communication sessions is useful for modifying the existing traffic
models used for performance analysis of communication networks for smart grids. In this deliverable
a detailed procedure is explained regarding capturing packets from the smart grid applications during
the trial via PCAP traces as well as definitions of the most important traffic KPIs (e.g. traffic volume,
packet size, time series of packet generation, etc.). Next to the traffic characterization it is important
to measure the communication network performance parameters. This deliverable describes end-toend delay and throughput measurement procedures that will be used in the trial. For the end-to-end
delay it is important to realize sufficiently accurate time synchronization between the two
communication end-points. Finally, during the communication sessions it is important to measure
the most important radio access communication network conditions (e.g. signal level, interference
level, radio access load, etc.) in order to understand the end-to-end delay and throughput
performance. This deliverable defines two approaches for measuring the radio access network
conditions: the approach of using radio access related performance counters as provided by the
vendor of radio access network, and the approach of using AT commands and/or SNMP commands
to extract the necessary radio access performance information from the communication modems.
During the SUNSEED trial the measurements of the electricity grid and the communication network
will be used to assess the feasibility and the performance requirements for the three defined
SUNSEED use cases, namely the real-time state estimation, demand response, and outage
localization use case. For the electricity grid state estimation use case a procedure is defined
regarding the testing of the accuracy and up to date information of the state-estimation function. As
the reporting interval of the WAMS-PMC and WAMS-SPM nodes is varied from e.g. 15 min down to 1
s the impact on the communication network and end-to-end delays will be investigated as well as the
ability of the state-estimation algorithm to predict a correct value and on time. For the demand
response use case next to the measurements needed for the state estimation additional
measurements and control information messages will be collected. As the delay in demand response
cycles is typically around 15 min, for this use case it is important to check the reliability of these
additional measurements and control messages (i.e. to check the stability and reliability of the
control process) and not on the end-to-end delay performance for these messages. Finally, by
instructing the WAMS-SPM and WAMS-PMC nodes to initiate ‘fake outage’ messages enriched with
location information (e.g. GPS coordinates for WAMS-SPM) the feasibility of automatic grid outage
localization will be assessed as the final SUNSEED use case.
SUNSEED, Grant agreement No. 619437
Page 55 of 56
Communication solution design for field trial setup
7 References
[1] FP7 SUNSEED Project, Deliverable D3.4.1, “Security architecture for smart grids“, SUNSEED
Consortium, 2015.
[2] Available from http://www.backtrack-linux.org/wiki/index.php/Pentesting_VOIP, retrieved
on March 2016.
[3] Shafiq, Muhammad Zubair, et al. "A first look at cellular machine-to-machine traffic: large
scale measurement and characterization." ACM SIGMETRICS Performance Evaluation
Review 40.1 (2012): 65-76.
[4] Shafiq, M. Zubair, et al. "Large-scale measurement and characterization of cellular machineto-machine traffic." IEEE/ACM Transactions on Networking (TON) 21.6 (2013): 1960-1973.
[5] FP7 SUNSEED Project, Deliverable D2.2.2, “Preliminary specification of SUNSEED field trial
requirements for communication, metering, and data collection“, SUNSEED Consortium,
2016.
SUNSEED, Grant agreement No. 619437
Page 56 of 56