Ether Testing by FPGA Implementation of Genetic
Algorithms
Olli Rauhala , Mathias Björk , Staffan Järn , Mika Ruohonen , Hayder Mohammed
Ali , and Jarmo Alander
University of Vaasa, Faculty of Technology, Vaasa, Finland
u96759@student.uva.fi , jal@uva.fi
Abstract. The goal of our Ether Tester project was to develop an FPGA platform implementing evolutionary computation based search method for testing
Intelligent Electronic Devices (IEDs) which communicate using Ethernet. Ether
Tester was implemented to test protection relays communicating using the IEC
61850 protocol. The results show that the Ether Tester was able to find bugs in
the software of the Device under Test (DUT) and thus proves the validity of using Ether Tester as a testing platform for IEDs.
Keywords: Genetic Algorithms· Ethernet· IEC 61850· FPGA
1
Introduction
Ether Tester is the pilot phase of our Teho-FPGA project that aims to promote and
enhance the knowledge about Field Programmable Logic Array (FPGA) technology
and its possible implementations in the fields of automation, energy and telecommunication. The project was carried out in our laboratory in collaboration with local
industries: Wärtsilä, ABB, Vamp, Vacon, and Wapice and Vaasa University of applied sciences. [5]
1.1
Related work
Li (2004) proposed a genetic algorithm for detecting and classifying network intrusion and blocking threatening ones [8]. Gong, Zulkernine & Abolmaesumi (2005)
implemented this detection scheme in a Java program [9]. These two researches can
be analyzed as complements to our research since our research presumes that the testing device is physically located inside the substation network with IEC 61850 compliant devices.
Kuffel, Ouellette & Forsyth (2010) have studied the impact of abnormal IEC 61850
GOOSE and Sampled Values (SV) protocol data on a DUT, which is very similar to
this thesis research. Kuffel et. al. on the other hand did manage to gather important
data from intelligent electronics devices (IEDs). Also they did not use genetic algorithms in their research [10].
Hor, Crossley & Millar (2007) managed to create a hybrid of a rough set theory and a
genetic algorithm (hybrid RS-GA), to obtain additional knowledge from operational
data from IEDs [11].
The above mentioned works does relate to our work partially, but in our research
more emphasis is put on disabling a DUT’s functionality using genetic algorithms.
1.2
Outline of this Paper
Section 2 explains in details the test setup, parameters and performance metric, it also
explains the reason behind implementing the testing platform using FPGA technology.
Sections 3, 4, and 5 illustrate the work and explains the results obtained from experiments.
Section 7 summarizes the obtained results.
Section 8 suggested improvements and further future work.
1.3
IEC61850
IEC 61850 protocol was introduced to simplify the configuration and set up effort
required to enable different Intelligent Electronic Devices (IEDs) from different manufacturers to communicate through Substation Automation Systems (SAS). It allows
high speed Ethernet switched communication suitable for time critical applications
such as protection relays. [3]
1.4
GSE Protocol
Generic Substation Events (GSE) is a control model defined for the IEC 61850 standard. It ensures the same event message is received by multiple physical devices using
multicast or broadcast services. The GSE control model is further subdivided into
GOOSE (Generic Object Oriented Substation Events) and GSSE (Generic Substation
State Events). [4]
1.5
Ethernet
Ethernet is a computer communication technology that allows fast serial data transmission in local area (LAN) and larger networks. Ethernet operates in the physical
and data link layers of the OSI model. It was commercially introduced in 1980 and
first standardized in 1983 as IEEE 802.3. Data rates and communication distance are
being updated continuously and the current technology offers a speed up to 100 gigabits per second compared to the original 10 megabits per second. [1]
Data streams are divided into frames. Each frame contains source and destination
addresses and a Frame Check sequence to allow foreword error correction. Figure 1
shows contents of an Ethernet frame.
Fig. 1. Structure Ethernet frame (Adapted from IEEE 802.3:2012: 53).
1.6
Genetic Algorithms
The basic principle of Genetic Algorithms is that if randomly generated solutions
provide good results, those solutions can be combined and used as building blocks to
generate better solutions. Solutions are evaluated by calculating a fitness function,
they are then modified using techniques inspired by natural evolution, such as inheritance, mutation, selection, and crossover. The new generated solutions are then revaluated and the procedure is repeated until the target of fitness optimization is
achieved or a predefined number of iterations is reached. [6]
2
Test Set-up
The goal of the Ether Tester is to investigate the effect of transmitting interfering
Ethernet frames to a DUT. Figure 2 shows the test set up used to validate the Ether
Tester functionality. The DUT is a protection relay exchanging GOOSE protocol
messages with a Raspberry Pi using Ethernet. The Raspberry Pi is programmed and
configured to communicate using IEC 61850 and to calculate the average message
exchange time and report it back to the Ether Tester to be used as a GA fitness function when creating new payload data for new Ethernet frames. Interfering frames are
generated and sent using the Ether Tester. Several parameters can be modified
through a system console running on a PC and communicating with the Ether Tester
through the JTAG interface. These parameters include the destination MAC address,
payload data and the transmission rate of the interfering frames which is possible to
adjust through modifying the Idle Cycles between Frames (ICBF).
Fig. 2. Test Set-up
2.1
The Use of Genetic Algorithms
Ethernet payload can extend up to 1500 bytes allowing 212000 unique possible frames.
Transmitting at 1 gigabit per second and a frame size of 1500 byte, the number of
transmitted frames per second can be calculated in equation 1.
109
= ~83333
8 ∗ 1500
(1)
The number of years required to test the effect of all possible payload data structures
therefore would be as in equation 2.
212000
83333∗365∗24∗60∗60
= ~211958
(2)
Genetic Algorithms (GA) uses the message exchange time provided via the Raspberry
Pi though the serial RS-232 interface as fitness function to produce new generations
of more harmful Ethernet frames and converge to optimum solution using minimum
possible number of iterations.
2.2
The use of FPGA
Network Interface Cards (NICs) installed on PCs have device drivers that are configured to filter and discard malformed Ethernet frames (those that contain invalid addresses or data lengths). Furthermore, transmission rate can be negotiated or fixed at
10/100/1000 megabits per second and the data will be always sent at maximum possible speed. Using custom made software and hardware on FPGA, it is possible to
transmit malfunctioning random frames and freely adjust the transmission rate
through varying the ICBF parameter. Furthermore, FPGA allows implementing several GA blocks in parallel to optimize the payload data in real time while sending at
high data rates.
3
Ether Tester
Altera’s DE4 development board was chosen for the implementation because it offers
4 gigabit Ethernet ports and Stratix IV with 250k Logic Elements to insure the possibility of implementing parallel genetic algorithms later. The design is based on Altera’s Triple Speed Ethernet reference design. However, Altera’s software was replaced with two Verilog HDL modules; the frame stormer and frame analyser. Frame
stormer generates Ethernet frames. It allows full controls over destination and source
addresses, payload, Ethernet type and size as well as the possibility of sending malformed packets. Data rates can be controlled via the ICBF parameter where the system main clock is 100 MHz. [7] [8]
The frame analyzer can calculates statistics and identify the number of received and
dropped frames, repeated frames, out of order frames and corrupted ones. [8]
Fig. 3. Ether Tester
4
Nios2 EthGA
A genetic algorithm was implemented on the Nios2 processor using C language in
order to inspect the effect of the payload data structure of the interfering frames on the
DUT’s performance. Experiments with four different levels of interference were performed.
4.1
Interference Level 1
In this test, EthGA sends Ethernet packet with MAC address other than the MAC
address of the DUT. The payload is a random data, which is the same in all of the
transmitted frames. When the data rate of the transmission speed exceeds 5 megabits
per second, the device reboots with a probability of 70% and sometimes goes into safe
mode. If the data rate of interference exceeds 10 megabits per second, the device will
certainly reboot itself. Figure 4 illustrates those results as a function of ICBF.
Fig. 4. Probability of DUT rebooting versus ICBF
4.2
Interference Level 2
In this test, the MAC address of the interfering frames is the same as the MAC address of the DUT. Some proportional relation between the interference rate and the
message exchange time between the DUT and the raspberry-pi was expected. The
results show an increase in the message exchange time; however, the obtained values
are oscillating as show in Figure 5.
Fig. 5. Message exchange time versus ICBF for interference level 2
4.3
Interference Level 3
In this test, the MAC address of interfering frames is the same as the mac address of
the DUT and the payload is optimized using GA. After analyzing 1000 samples as
shown in Figure 6, it was concluded that there is no relation between the structure of
the payload data and the message exchange time.
Fig. 6. Mean message exchange time for interference level 3.
5
GA Hardware
A genetic algorithm was implemented on FPGA’s hardware using Verilog HDL and
integrated into the Ether Tester. The results obtained agree with those obtained by
EthGA software; device failure and reboot was again observed when the destination
mac address of the interfering packets is other than the DUT’s address. This time,
however, it was found that the device reboots in safe mood when the destination address is 0x00:00:00:00:00:00 and with full transmission rate [2].
When interfering frames have a destination MAC address equal to the DUT’s MAC
address, oscillating values for the message exchange time similar to the results ob-
tained by EthGA were observed. However, a higher ICBF was found to induce higher
latency as shown in Figure 7.
Fig. 7. Mean Message Exchange time obtained by GA Hardware when interfering frames have
a destination address equal to the DUT’s destination address.
6
Conclusion
It was proven through field tests that the Ether Tester platform is able to detect bugs
and faults in IEDs communicating with IEC 61850 standard. The Ether Tester is a
handy tool offering several configurable parameters to experiment with different interference situations. It feature adjustable packet length, ICBF, source and destination
addresses. It bypasses device driver restrictions to allow sending malformed Ethernet
frames and it offers the possibility of optimizing payload data in real time using Genetic Algorithms.
Results showed that the tested protection relay suffers from the following faults:
 Interfering packets with destination address other than the device MAC address at
data rate between 5~10 Mbps causes the device to reboot with 60~70% probability.
Increasing the data rate of interference above 10 Mbps will cause certain reboot.
 Interfering packets with destination address 0x00:00:00:00:00:00 and with ICBF=0
(~0.86 Gbps) will cause the device to reboot and enter safe mode.
 Interfering packers with destination equal to the device’s mac address will cause
oscillation in message exchange time with lowest average delay recorded when
ICBF=2 million.
 It was concluded through the use of GA that the structure of payload data of the
Ethernet packet does not have any effect on the message exchange time for the
specific test case.
7
Future Work
Implementing message exchange time measurement in FPGA instead of using Raspberry Pi will eliminate the need for using slow RS-232 communications.
Implementing data logging functionality with pattern recognition can simplify testing
compared to the limited console view.
Performing tests on other devices to verify the validity of using Ether Tester as a platform for detecting bugs in IEDs. In addition, testing several different IEDs is needed
to reach a conclusion about the relation between the structure of the Ethernet frame
payload and the message exchange time.
References
1. Spurgeon, Charles E. & Joann Zimmerman (2014), Ethernet: The Definitive Guide, 2nd
Edition, Designing and managing Local Area Networks.
2. Järn, Staffan (2014), Implementation of Genetic Algorithms on a FPGA-Ethernet Tester,
Master’s Thesis. University of Vaasa.
3. Björk, Mathias (2014), Development and Testing of IEC61850 Network Interference
Equipment – A Case Study. Master’s thesis. University of Vaasa.
4. C. Kriger, S. Behardien & J. Retonda-Modiya (2013), A Detailed Analysis of the GOOSE
Message Structure in an IEC 61850 Standard-Based Substation Automation System, INT J
COMPUT COMMUN, ISSN 1841-9836.
5. Alander, Jarmo (2012). TehoFPGA project plan. Faculty of Technology, Electrical Engineering and Energy Technology, University of Vaasa.
6. Alander, Jarmo (1992). On optimal population size of genetic algorithms, Comp Euro ‘92.
'Computer Systems and Software Engineering’, Proceedings, Print ISBN: 0-8186-2760-3.
7. Rauhala, Olli (2013a). Personal communication
8. Li, Wei (2004). Using Genetic Algorithm for Network Intrusion Detection. Department of
Computer Science and Engineering, Mississippi State University.
9. Gong, Ren Hui, Mohammad Zulkernine & Purang Abolmaesumi (2005). A software implementation of a genetic algorithm based approach to network intrusion detection. In:
Software Engineering, Artificial Intelligence, Networking and Parallel/Distributed Computing, 2005 and First ACIS International Workshop on Self-Assembling Wireless Networks.
SNPD/SAWN 2005. Sixth International Conference.
10. Kuffel, Rick, Dean Ouellette & Paul Forsyth (2010). Real Time Simulation and Testing
Using IEC 61850. In: Modern Electric Power Systems (MEPS), 2010 Proceedings of the
International Symposium.
11. Hor, Chinglai, Peter A. Crossley & Dean L. Millar (2007). Application of Genetic Algorithm and Rough Set Theory for Knowledge Extraction. In: Power Tech, 2007 IEEE Lausanne.