University of Duisburg-Essen
Chair of Dynamics and Control
Univ.-Prof. Dr.-Ing. Dirk Söffker
————————————————————————————————
Safe Systems
Replay and Visualization of driving scenarios in
inland vessel-shipping
Mukesh Thenraj
Xu Fengkai
Advisor
Abderahman Bejaoui, M.Sc.
Univ.-Prof. Dr.-Ing. Dirk Söffker
April 2022
————————————————————————————————
Declaration
I declare that the work is up to the official supervision by the person responsible for the
task was written independently. The sources used and the resources used are given in
full. Verbatim parts of text and images and drawings are marked in each case.
Place, Date
signature
Copyright Agreement
The present work was created with the intensive cooperation of the named supervisors
People in the context of research work in the Chair of Control, Regulation and Control
System dynamics (SRS) of the University of Duisburg-Essen. The ones contained in this
work Solutions and ideas are therefore not only subject to the copyright of those to be
qualified Person, but jointly with the named group of people. The personal, private as
well as the use in the research context of the chair SRS is unaffected..
Place, Date
signature
Abstract[XU]
The aim of this project is to develop a complete set of code tools to visualize and replay inland vessel movement scenes based on ENC (Electronic Navigational Charts) and
AIS (Automatic Identification System) datasets in a highly automated manner within the
ROS (Robot Operating System) framework. During the development of this project, a
dataset processing tool in Python is firstly developed. This tool includes the function of
separating geographical layers independently from the ENC dataset and the function of
extracting and filtering vessel movement information from the big AIS dataset. Then,
static map visualization and dynamic visualization of vessel movement are implemented
in C++ within the ROS framework, and the problem of synchronous visualization of
multiple vessel nodes with time-series information is solved in the dynamic visualization
process. Finally, based on the previous step, the template code is written and the code
automatic generation tool is designed to automate the visualization process of a large
number of vessel nodes. It is validated at multiple time periods in multiple river zones
and the tool is able to accurately visualize and replay scenes in all situations. Through
the implementation of this project, it can analyze the vessel trajectory in both time dimension and space dimension more intuitively, evaluate the vessel driver’s behavior more
accurately and provide a technical basis for realizing remote control or automatic driving
of inland river vessels.
Keyword: Inland shipping, ROS, Visualization, Automation tools.
I
Contents
1 Introduction[Mukesh]
1.1 Motivation
1.2 Problem statement
1.3 Steps and structure of work
1.4 State of art
1.4.1 Electronic Navigational Chart - ENC
1.4.2 Automatic Identification System - AIS
1.4.3 Robot Operating System - ROS
1
1
2
3
3
3
4
7
2 Map visualization using ENC data[Mukesh]
2.1 Definition of ENC data
2.2 Converting ENC data to shapefile
2.3 Layers in Shapefile
2.3.1 Fairway Layer
2.3.2 Land Layer
2.3.3 Bridge Layer
2.3.4 Harbour Layer
2.3.5 Lake Layer
2.3.6 Canal Layer
2.4 Database consisting the infrastructure data
2.4.1 Comma Separated Values (CSV)
2.5 Setting Thresholds
2.6 Publish Information to Rviz
11
11
12
14
15
15
16
16
17
17
18
18
18
20
3 Dynamic visualization with AIS data[Xu]
3.1 Background of model motion
3.2 AIS data
3.3 Processing big data
3.4 Converting static to dynamic
3.4.1 Attitude and position
3.4.2 Historical trajectory
3.5 Multi-node visualization with time sequence synchronization
3.6 Auto code generator for multiple ROS nodes
23
23
25
25
28
28
30
31
35
4 Results[Mukesh and Xu]
4.1 Static map
4.2 Dynamic visualization
37
37
41
CONTENTS
II
5 Conclusion[Xu]
48
References
49
III
List of Figures
1.1
2.1
2.2
2.3
2.4
2.5
2.6
2.7
2.8
2.9
2.10
2.11
3.1
3.2
3.3
3.4
3.5
3.6
3.7
3.8
3.9
3.10
3.11
4.1
4.2
4.3
4.4
Structure of work.
3
Representation of Duisburg port (778 km - 781 km) using electronic navigational chart.
12
Converting ENC to Shapefile.
13
Visualisation of the polygon ”HRBBSN” of the considered area Duisburg.
14
Visualisation of the layer ”Fairway” of the considered area Duisburg.
15
Visualisation of the layer ”LNDARE” of the considered area Duisburg.
16
Visualisation of the layer ”BRIDGE” of the considered area Krefeld.
16
Visualisation of the layer ”HRBBSN” of the considered area Krefeld.
17
Visualisation of the layer Fairway in area Duisburg, left with open points
before setting threshold, right with closed points after setting threshold.
19
Visualization of the area Krefeld (760 km - 769 km), left from Atlas map.
20
Layers of Bridges, Harbours, and Fairway published by single noded ROS. 21
Static map of area Krefeld (760 km -769 km) image published by Rviz
using the implemented tool.
22
Example of motion attitude and position.
23
Example of historical motion path.
24
3-DOF simplified vessel kinematic model.
24
Data processing procedures.
26
Trajectories of all vessels in Neuss before data filtering.
27
Trajectories of all vessels in Neuss after data filtering.
28
The model of vessel in static map in ROS.
29
The algorithm of time sequence synchronization.
32
The algorithm of time sequence synchronization with multi-node.
33
Optimized time sequence synchronization in ROS.
34
Structure of the automatic code generator.
36
Static map of the Neuss area of the region 736 km to 745 km according to
the Atlas notation with the implemented Tool.
37
Static map of the of Haus-Knipp-Eisenbahnbrücke and Bundesautobahnbrücke
(HKEB) areas of the region 782 km to 788 km according to the Atlas notation with the implemented Tool.
38
Static map of the Krefeld area of the region 760 km to 769 km according
to the Atlas notation with the implemented Tool.
39
Static map of the Duisburg area of the region 778 km to 783 km according
to the Atlas notation with the implemented Tool.
40
LIST OF FIGURES
4.5
4.6
4.7
4.8
4.9
4.10
4.11
4.12
4.13
4.14
Static map of the Rheinvorland im Orsoyer Rheinbogen area of the region
796 km to 803 km according to the Atlas notation with the implemented
Tool.
The scene of the vessel before entering the port in Neuss area.
The scene of the vessel after entering the port in Neuss area.
The scene of the vessels before meeting in HKEB area.
The scene of the vessel after meeting in HKEB area.
The scene of the vessel before departing the port in Duisburg area.
The scene of the vessel after departing the port in Duisburg area.
The scene of the vessels decelerate at river-bend in area Rheinvorland im
Orsoyer Rheinbogen.
The scene of the vessels anchorage at krefeld area day 1.
The scene of the vessels anchorage at krefeld area day 2.
IV
41
42
42
43
43
44
45
46
46
47
V
List of Tables
2.1
3.1
Structure of CSV file.
Structure of AIS data in this project.
18
25
Chapter 1. Introduction[Mukesh]
1
1
Introduction[Mukesh]
The flow of trade between nations and regions is expanding all the time as the social
economy develops, and the aggregate number of ships in the social economy grows. Then
the volume of vessel traffic on numerous key rivers, straits, and ports increases, resulting in
an increase in all types of marine traffic incidents. Now is the time for everyone to rethink
how to handle crowded waterways and ports. Germany’s overall water transportation flow
has increased in recent years. These vital waterway transportation hubs, such as those
along the coast and on the Rhine, have made significant contributions to the economic
and social growth of our country [HY15].
However, because to a lack of data on human decision behavior, there are unsettling
inconsistencies between sophisticated marine technology equipment. To address these
issues, captains behavior for remote controlled vessels must be analyzed. Forecasting
captains behavior in different situations and in different places is a crucial part of Visualization of driving scenarios in inland vessel-shipping. It lays the theoretical groundwork
for increasing traffic efficiency and safety while also enhancing the channel’s navigational
capacity. There have been a many new approaches in recent years. Many approaches
for studying ship traffic flow forecasting models have emerged in recent years, including
linear regression analysis, particle swarm optimization, maximum likelihood estimation,
support vector machine, and neural network [WZC+ 17].
In this method Electronic navigational chart (ENC) is used instead of Google API to
visualize the static map of Rhine river. It’s because ENC has more advantage over Google
API’s. ENC is an open source once the data is downloaded it can be used without internet
and without GPS. But google map is not an open source and network connection or GPS
must be need to use the service. ENC offers separate layers for bridges, lands, fairways,
canals, lakes, harbours, etc. But in google maps these features are not available. Rviz
is used instead of Gazebo to show the visualization of static map and driving scenarios
because Rviz is specific tool designed for visualization.
1.1
Motivation
According to previous researches, the human decision-making process is highly non-linear,
therefore non-linear models would be more ideal for constructing advanced decisionmaking models than linear models. Even as technological advancements contribute more
and more to the creation of autonomous systems that assign the operators the exclusive
function of supervisors, the interaction between captain-vessel-interaction has become a
major problem in inland vessel-shipping and safety analysis. This tendency may be justified by the fact that the great majority of accidents are caused by a mix of circumstances,
Chapter 1. Introduction[Mukesh]
2
all of which invariably include human error and/or improper behavior. In many circumstances, such as when human erroneous behaviors are paired with inadequate or restricted
protection and safety measures, the environmental and casualty repercussions of catastrophes are staggering. On the other hand, working at the level of human contribution rather
than the machine component is the most effective strategy to increase the dependability
of the human-machine system [KYK08]. This issue has been approached from several
angles:
• Illustration of captains behavior for remote controlled vessels.
• Supervision of captain-vessel-interaction.
• Knowledge base including captains behavior in different situations and in different
places.
• Collecting of different possible captains behaviors/actions sequences.
• Visualization of driving scenarios based on complex area and action data.
• Visualization of vessel movement in different places.
1.2
Problem statement
The tool box is developed with the following function that it can process ENC data and
big AIS data and visualize different vessels in different places this way the plan in ROS
high automatically. With the tool, the analysis of ship motion and driver behavior will be
much more easier, direct and intuitive. Developing a tool box that could process the large
amount of AIS-data automatically and visualize different vessel’s behaviours in different
places for many days. Development of the kinematic model is helpful to understand the
motion of ship. The development of this tool allows the visualization of vessels behavior
so that the motions and behaviors can be analyzed understood. Finally analyze the result
of movement of vessels obtain from the tool.
Chapter 1. Introduction[Mukesh]
1.3
3
Steps and structure of work
Figure 1.1: Structure of work.
In Chapter 1 the state of art is discussed and ENC, ROS, AIS data used in different
projects are reviewed. In chapter 2 the implementation of the tool allowing the visualization of the static map based on ENC-Data is presented. In chapter 3 the implementation
of the dynamic visualisation using AIS-Data is explained. In chapter 4 the results of
dynamic and static visualization are shown and the considered areas are analyzed and
shown. The steps, methods, and results in this work are concluded in chapter 5.
1.4
1.4.1
State of art
Electronic Navigational Chart - ENC
Electronic nautical charts (ENC) have replaced printed navigational charts as the digital
standard. These formats open a world of options in terms of data management. With
rising data volumes and degrees of freedom, however, the problem of efficiently displaying
Chapter 1. Introduction[Mukesh]
4
and processing ENC data has arguably gotten more difficult, and solutions are primarily
devised on a case-by-case basis. Open application programming interfaces (APIs) for
public usage, in particular, are limited. Given the current lack of resources for polygonbased marine settings, it is clear that an open-source ENC API is required for future
study and software development. Given its design philosophy with regard to readability,
object oriented structure, and readily available libraries for management of GIS resources,
Python was chosen as the preferred programming language for the development of an
open-source API for elegant visualization and straightforward manipulation of spatial
data. The user must first construct an instance of the SeaCharts main class in order
to see the environment. This class must be created by supplying an origin or center
coordinate pair (easting and northing), as well as a bounding box size in meters and any
region of interest. Other characteristics such as vessels and other physical structures, as
well as abstract entities such as path references, pointer arrows, enclosing rings, or area
overlays of interest, can be depicted on top of the environment polygon data derived from
depth measurements. Furthermore, by setting the multiprocessing initialization option to
True, the method for charting each ship location may be constructed and performed in
parallel with the primary calling process, allowing the visualization plot to be updated
in real time. This separate operation reads the CSV file for each individual ship plot
to be presented in the environment, which contains all coordinate pairs, heading angles,
color names, and other variables. Finally, by supporting the combined use of convenient
spatial computing and visualization methods for autonomous navigation, this API may
be valuable for high-level autonomous path planning, control, obstacle avoidance, and
simulation in marine situations [BJ21].
1.4.2
Automatic Identification System - AIS
Controlling ship traffic in a busy harbor water region is a critical challenge for ensuring
safety and increasing port efficiency. Operators of the Vessel Traffic Service (VTS) pay
close attention to caution zones, such as ship route junctions or traffic congestion regions,
where there is a possibility of a ship accident. They aim to keep the traffic in the warning
zone at a reasonable level to reduce risk. Rapid changes in direction and speed are difficult
due to the inertial movement of the ship. As a result, it’s critical to forecast future
traffic in the danger zone ahead of time so that control measures on ship movements
may be implemented in a timely manner. VTS stations in the harbor region capture
a considerable amount of Automatic Identification Service (AIS) sensor data, which is
then analyzed. It contains data about the ship’s movement and characteristics. The
Ship Traffic Extraction Network (STENet) is a novel deep neural network model that
predicts the caution area’s medium- and long-term traffic. AIS sensor data is used to
train the STENet model. The outputs of the movement and contextual feature extraction
Chapter 1. Introduction[Mukesh]
5
modules are concatenated and fed into a prediction module in the STENet model, which
is structured into a hierarchical architecture. With a convolutional neural network, the
movement module retrieves the characteristics of overall ship motions. The contextual
modules are made up of five completely linked neural networks, each of which has its
own property. By avoiding unrelated characteristics from crosstalking, the separation of
feature extraction modules at the front phase aids in the extraction of effective features.
Over the benchmark approach, i.e., the SVR-based method, the suggested model showed
a 50.65 % relative performance increase for medium-term forecasts and a 57.65 % relative
performance improvement for long-term predictions on the real AIS sensor dataset. It’s
also shown how to make the training data out of AIS sensor data. The model forecasts
projected traffic in the caution zone over the medium and long term. The STENet model
outperformed the reference model by a significant margin in trials on a real ship traffic
dataset. The following are some of the benefits of the STENet model: First, it learns the
prediction model using a vast amount of AIS sensor data, which is easily acquired on the
VTS stations, rather than using mathematical modeling for traffic forecasts. Second, it
can identify effective traffic forecast models for the medium and long future. Third, both
the ship movement and ship attribute data encoding schemes are excellent at capturing
traffic characteristics. That can be supported by the fact that the STENet model has
demonstrated good performance in the caution area when it comes to traffic prediction.
If the output attribute is changed to another indication, such as vessel collision risk or
congestion rate, it is predicted that STENet will discover a satisfactory model for the
indicator [KL18].
Lu, Ma, Smart, Vuksanovic, Chiverton et al. [LMS+ 21] develop a fusion framework
that uses a deep learning-based vision detection and Automatic Identification System
(AIS) as input to obtain accurate and robust ship identification, aiming to address the
shortcomings of existing deep learning -based ship classification methods in the problem
of identifying detailed ship information (e.g. ship identity) under poor visual conditions.
The deep learning algorithm, YOLO5, is chosen as the recognition framework, while the
region of interest is delineated with the help of ship azimuth and camera-to-ship distance
estimates obtained from AIS, effectively improving the ship recognition rate. The test
was conducted on 3976 images and the efficiency rate is improved to 75.7%.
Liu, Yao, et al. [LYXZ19] develop a trajectory fusion algorithm designed to solve the
problems encountered when relying only on satellites and only on AIS to track vessels.
Satellites are susceptible to weather and cannot penetrate clouds for identification, and
some vessels choose to turn off their AIS systems at certain moments. To address this
issue, a new orbit-to-orbit correlation algorithm based on the well-known Iterative Closest
Point (ICP) and Global Nearest Neighbor (GNN) with multiple features are used to
improve the correlation between trajectories, effectively improving the ability to track
Chapter 1. Introduction[Mukesh]
6
ships with test at sea. The satellite used here is a geostationary orbit optical remote
sensing satellite with medium resolution.
Li, Zhang, et al. [LZMJ19] develop a method for predicting long-term ship motion trajectories based on a combination of trajectory-based classification and Long and Short
Term Memory (LSTM) to facilitate earlier detection of potential hazards. Considering
the ship’s movement, the Longest Common Subsequence (LCS) algorithm is first introduced to define the similarity of the trajectories, clustering was achieved with the help of
LCS metrics to group the trajectories, and then the LSTM algorithm combined with the
sliding window method is used to achieve a fast modelling prediction. Compared to other
methods, such as the Kalman method, it achieves very good performance in the long-term
prediction of trajectories. The long term predictions of trajectories can be used as input
for waterway management, water safety and water traffic planning, and are more useful
than short term predictions.
Tang, Cao and Li [TCL19] apply the JPPF framework to handle AIS big data. Then AIS
data are over 1 billion rows from the port of Tianjin. To analyse the spatial-temporal
characteristics contained in these data, a JPPF-based analysis framework is built, which is
a high-performance, distributed, highly reliable and secure distributed computing framework that can run on an arbitrary number of computers. The authors successfully build
a high performance computing framework that enables the analysis and summarisation of
huge amounts of data, laying the foundation for subsequent ship traffic simulation, safe
ship navigation, maritime sector regulation and macro management.
Wang, Ren, Li [WRL20]develop a deep learning ship berthing trajectory prediction model
based on a bidirectional Gated Recurrent Unit (Bi-GRU) network. The model acquires
data on the berthing trajectories of hundreds of container ships in Tianjin port waters,
and train, test and validate the data. Compared with other ship trajectory prediction
models, this recurrent neural network model focuses more on predicting ship trajectories
at the berthing stage, which solves the problem of difficult to predict ship movement
trends in crowded port waters and can provide a reference for the regulation of ship traffic
services. Using the error between the predicted value and the original real trajectory as
the criterion, this prediction model has been tested in the field and outperformed LSTM
and GRU.
The Automatic Identity System (AIS) is a VHF communication system that exchanges
data such as position, identification, course, speed, and others to provide identification/location information to boats and shore stations. The demand for long-range applications such as better management of hazardous cargo, increased security, and fighting
illicit operations has fueled interest in detecting and monitoring ships at distances greater
than what can be performed by standard terrestrial VHF communications. The goal is
Chapter 1. Introduction[Mukesh]
7
to use a constellation of LEO satellites to extend AIS to long-range positional reporting.
It discusses technical issues such as the high rate of message collisions from ships in a
satellite’s area of vision. To solve these issues, technical solutions are provided at the
system and sub-system level. To evaluate system performance and execute high-level system sizing, a computer-based system simulator is employed. Small constellation of LEO
satellites may offer good ship location detection probability and a reporting time interval
of a few hours with a comparatively small constellation of LEO satellites. It’s possible
to detect terrestrial AIS signals from space using a satellite-based AIS communications
system. Decoding possibilities are improved by using a deployable Helix antenna with 6
twists. The performance of the innovative satellite AIS receiver has significantly improved,
enabling for a more cost-effective system with a 3-hour target ship location reporting interval. Indeed, a constellation of 510 LEO satellites orbiting at 600 km altitude with a
properly designed orbital constellation has been proven to provide such performance with
a goal probability of ship detection of 90 %. The traffic forecast errors contribute to the
ambiguity in the constellation’s actual size [CGE11].
Meyer, Schwegmann and Kleynhans [MSK17] develop a Bayesian algorithm based on
combining historical AIS data and Synthetic Aperture Radar (SAR) images to distinguish between different types of vessels. Since the AIS system is not mandatory and
transponders can falsify signals, there could be fishing vessels disguised as cargo vessels to
steal in the exclusive economic zone. To solve this problem, this paper combines features
extracted from SAR images with historical AIS data in order to derive a list of category
probabilities that flag vessels that do not match historical category behaviour based on
their position, bearing and length. Since SAR is not very good at distinguishing similar
objects, this algorithm is not very expert in distinguishing between categories with similar
behaviour and size, such as cruise ships and container ships.
1.4.3
Robot Operating System - ROS
Demands in Industry 4.0 have emphasized the development of autonomous driving for
automated guided vehicles (AGVs). AGVs’ present driving control relies on established
pathways marked on the road by tapes or magnetic markers. An autonomous driving
framework for AGV is required to meet the demands of frequent route modifications, to
enable AGVs to work smarter, and to have flexible access to Internet of Things (IoTs).
The goal is to build a new framework based on Robot Operating System for all-wheels
powered AGVs with mecanum wheels (ROS). Robot Operating System (ROS) is a BSDlicensed open-source operating system. Many firms and colleges, such as Yujin Robots,
Clearpath Robotics, JAXA, and others, are now using ROS. Many open-source robotics
libraries, such as sensor driver libraries, SLAM, robot navigation, motion control, and so
Chapter 1. Introduction[Mukesh]
8
on, are included in ROS. Messages held by topics based on the TCP/IP layer are the most
fundamental communication technique in ROS. Many tiny programs (called nodes) can be
run at the same time and allocated to various computers, allowing a distributed computing
system to be built. Simultaneous Localization and Mapping (SLAM), object detection,
and path planning are crucial technologies enabling autonomous driving for mobile robots.
The technologies that will be employed in this study are detailed below. SLAM is a branch
of mobile robotics that studies how to generate a map and pinpoint a robot’s location.
When a robot is placed in an unfamiliar area, the basic idea behind the SLAM approach
is to allow it to move while simultaneously building a map of its surroundings. In ROS,
there are numerous SLAM techniques accessible, such as ”gmapping,” ”RTAB,” and so
on. The capacity of a robot to recognize its surroundings is much more necessary in order
to increase its ability to cope with external impediments such as moving individuals,
moving cars, and the comprehension ability of words, traffic signal bulbs, and so on.
This may be accomplished using the YOLO v3 open source object detection technology.
There are now three control modes in use to answer various and adaptable needs during
the plant and debug phases: manual mode, instructional mode, and autonomous mode.
The manual mode simulates a human driver operating a vehicle. The human operator
is in charge of a controller and a monitor that captures the front scene from behind
or within the vehicle. The AGV goes by itself along the path pre-instructed in the
manual mode in the teaching mode, which is accomplished using the ”rosbag” function
included into ROS. In autonomous mode, the vehicle autonomously plots the shortest way,
recognizes and avoids static and dynamic impediments after a human operator chooses the
start and target points. ”GMAPPING”, ”MAP SERVER,” ”AMCL,” ”MOVE BASE,”
”JOY,” ”REALSENSE,” ”ROSSERIAL ARDUINO,” ”RVIZ,” ”DARKNET ROS,” and
other programs are combined in this mode [ZWMS19].
As technology advances in the field of autonomous driving, the demand for more dependable and resilient systems that can perceive a vehicle’s surroundings in all weather
conditions and at any time of day has grown. Aside from that, there’s a requirement
for a system that can accurately measure metrics like distance and velocity of objects
in its range of vision. With these considerations in mind, a robust model to fuse camera and radar data was developed, which is a cost-effective option with a track record
of success. The Robot Operating System (ROS) environment is used to develop this
paradigm, which is an open source middleware for autonomous driving applications. The
sensor technologies that we have at our disposal each have their own set of advantages
and disadvantages. In varied weather and light circumstances, some function better than
others. mmWave Radars, for example, are becoming increasingly important in ADAS and
autonomous driving applications. Instead of using only one type of sensor, such as vision,
radar, or lidar, the traditional approach to autonomous driving has changed significantly.
Chapter 1. Introduction[Mukesh]
9
Now, instead of using only one type of sensor, such as vision, radar, or lidar, we are using
a combination of two sensors or all of them together to create a more robust system. We
can plainly observe how varied situations affect the performance of these sensors, as well
as the cost feasibility. The premise behind combining radar and camera sensors is that
radar provides precise distance, radial velocity, longitudinal velocity, azimuth, and elevation measurements of objects inside its field of view (FOV). A monocular camera produces
a perceptible output and is an excellent input for recognizing and categorizing items. Sensor ranges of at least 150 to 200 meters are required for autonomous driving applications.
We’ll be able to identify and categorize things at this range thanks to the combination of
camera and radar. Sensor fusion is a technique for avoiding the autonomy stack’s reliance
on a single sensor and making the system more resilient by combining data from several
sensors. Even if a component of the autonomy stack fails, the remaining sensors will keep
the system operational. ROS (Robot Operating System), an open source middleware, was
used to connect the radar. The measurements of the radar-detected objects were recorded
in PointCloud format. In addition, the radar’s 3D points were translated to picture pixel
coordinates. The radar’s pixel coordinates were then transferred to the objects observed
in the camera frame with the help of a classifier. After then, the objects were tracked in
both the camera and the radar frame. The present state of automobile radar technology
allows for extremely exact measurements of characteristics like as distance, velocity, azimuth, and elevation of objects in its field of view (FOV), whether moving or stationary.
However, it has a shortcoming in that it cannot determine the actual nature of the item.
This system is able to manage the synchronization of the two sensors easily because to its
implementation in ROS. Implementing a tracking algorithm for camera-detected objects
as well as a clustering method in conjunction with a tracking algorithm for radar points.
Finally, a feed is created, with each item being assigned a class and a distance. This
system’s benefits include its great reliability and precision in accurately recognizing and
categorizing items. The addition of unknown items increases the system’s dependability
by detecting things even when the camera fails to categorize them. This system can be
used in real-world cars. The target and calibration method employed here are both simple and readily available. The calibration target can be updated in the future to simplify
the calibration procedure and increase its accuracy. It may also be used with numerous
radar and video sensors to provide a 360-degree picture of the vehicle. An updated deep
learning classification model built on the driving dataset may be utilized to increase the
system’s performance and resilience [KJ19].
Robots are commonly utilized to service or assist people in a variety of industries, ranging
from everyday tasks to industrial applications. Due to current demands from individuals
and business, robot science has advanced tremendously. Mobile robots are one of the
most important aspects of robot research. Mobile robots can traverse across an area
Chapter 1. Introduction[Mukesh]
10
and interact with it using sensors and actuators. Autonomous mobile robots (AMR)
and autonomous guided vehicles (AGV) are two types of mobile robots (AGV). The
autonomous mechanism of these robots is said to constitute the difference between AMR
and AGV. An AGV navigates in a pre-defined environment on a pre-defined course using
physical guidance. AGVs are well-suited to repetitive activities, such as line follower
robots, and are frequently developed and manufactured for specialized purposes. Because
AGVs cannot make decisions and lack a decision system based on artificial intelligence,
the work that they will execute must be meticulously planned and all specifics must be
communicated to them by the programmer. AGVs operate according to pre-programmed
systems and processes, making fast modification challenging. The inability of AGVs
to execute dynamically changing jobs is its primary flaw. An AMR can navigate in an
unpredictably changing environment. AMRs can detect the environment’s characteristics,
build a model of the environment, and position themselves inside that model. As a result
of this behavior, AMR is able to create a navigation plan and optimize it using a unique
planning algorithm. In a nutshell, an AMR has no predetermined navigation strategy. An
AMR may also use sensor data to construct a map of the environment and then locate
itself within it. Simultaneous localization and mapping is the term for this (SLAM).
SLAM allows the programmer to construct a very sensitive navigation strategy that can
be dynamically altered and enhanced. Due to their superior characteristics, AMRs have
become widely employed in industry and several disciplines in recent years. In compared
to AGVs, AMRs have a more complicated hardware and software architecture since they
have more sensors and actuators. As a result, they feature more complex control systems
that function in concert with the data stream at the same time. This type of operation
is only feasible if and only if the hardware is smooth and compatible with the default
software. Designing an AMR that works properly takes time and effort. The sensitivity
of the sensors, sample rate, and related filters and algorithms were shown to be important
deterministic aspects in the navigation and mapping process. The mapping quality of the
robot dropped in a nonlinear fashion when the robot’s velocity was raised, as predicted,
due to the low sensor resolution and sensitivity. The issues that might arise during the
design and implementation phases of a ROS-based AMR [KÇP17].
Chapter 2. Map visualization using ENC data[Mukesh]
2
11
Map visualization using ENC data[Mukesh]
For nautical charts, the S-57 format is utilized, particularly for ENC electronic chart
displays used in ships. S-57 files are usually found in a folder with other files. The
extension ”.000” is usually used for S-57 (ENC) files. A file with the .000 extension is
a type of electronic navigational chart (ENC) that provides vector format data based on
the S-57 object model and is specified by the International Hydrographic Organisation
(IHO). It includes navigational data including sea depth, soundings, contours, and other
details. All of this information is stored in a vector format within the file and is completely
independent of how it is shown by S-57 readers. S-57 files can be opened using software
like ESRI ArcGIS, OpenCPN, and APIs like GDAL.
2.1
Definition of ENC data
To ensure the safety of navigation assistance Electronic navigation chart (ENC) is widely
used. Electronic navigation chart consist of geographical information about lands, bridges,
fairways, waterways, canals, lakes and other areas.
ENC database is created by national hydrographic office. ENC is created according to the
standards of International Hydrographic Organisation (IHO). ENC data is mainly used in
electronic chart system. ENC data is mostly encode in data formats like ISO/IEC 8211.
This format cannot be used so it has to be converted according to the system requirement.
ENC provides different information and automation features like possible danger, Routes,
Maritime limits, Depth, nature of seabed, elevation and configuration and characteristic
of coast, etc [PPK+ 13].
There are different types of ENC data which are available. Some are suitable for oceans
and some are suitable for inland waterways. Inland Electro Navigation Chart (IENC) are
developed specifically for inland areas like rivers, canals, lakes and others. By using ENC
information of the selected areas can be obtained and its visualisation can be realized (cf.
Figure 2.1).
Chapter 2. Map visualization using ENC data[Mukesh]
12
Figure 2.1: Representation of Duisburg port (778 km - 781 km) using electronic navigational chart [ITZ22].
The ENC data of the Rhine river can be obtained from Electronic Waterway Information
Service (ELWIS). Which provides the Enc data for Rhine, Ruhr and Rhein-Herne-Kanal
in .zip format.
2.2
Converting ENC data to shapefile
The ENC file will be in .000 format which is not useful to visualise. A python-code is
developed and allows to convert the shape-files to useful data. A shape-file is characterized
as polygon consisting of points. For this python packages such as fiona, ogr, os, GDAL/ogr
can be used. (cf. Figure 2.2).
Chapter 2. Map visualization using ENC data[Mukesh]
13
Figure 2.2: Converting ENC to Shapefile.
Geospatial Data Abstraction Library , also known as GDAL is a library of tools used for
manipulating geospatial data. GDAL works on both raster and vector data types, and is
an incredible useful tool to be familiar with when working with geospatial data. GDAL is
a translator library for raster and vector geospatial data formats [FW+ 22] GDAL is used
for opening the ENC files. Fiona enables to create shapefile it is designed to be simple
and dependable. It focuses on reading and writing data in standard Python IO style (cf.
Figure 2.3).
Chapter 2. Map visualization using ENC data[Mukesh]
14
Figure 2.3: Visualisation of the polygon ”HRBBSN” of the considered area Duisburg.
2.3
Layers in Shapefile
A shapefile is a basic, non-topological format for recording geographic feature geometric
location and attribute information. Points, lines, and polygons can be used to represent
geographic features in a shapefile (areas) [Fou20]. A single S-57 file may contain more
than 100 layers. For instance, the airport (AIRARE) object class can have the AIRARE,
CATAIR, CONDTN, CONVIS, NOBJNM, OBJNAM, STATUS, INFORM, NINFOM,
NTXTDS, PICREP, SCAMAX, SCAMIN, TXTDSC, RECDAT, RECIND, SORDAT,
and SORIND attributes. Layers, sometimes known as web layers, are logical groups of
geographic data used to build maps and scenes, as well as the foundation for geographic
analysis. A building layer, for example, might represent a collection of campus buildings
and include attributes that specify each structure’s qualities, such as the building’s name,
kind of building, size, and other possible attributes. Some important layers in shapefile’s
are BRIDGE, CANALS, FAIRWY, HRBBSN, LAKARE and LNDARE [Com21].
Chapter 2. Map visualization using ENC data[Mukesh]
2.3.1
15
Fairway Layer
Fairway is a channel either from offshore, in a river, or in a harbor that has enough depth
to accommodate the draft of large vessels [Org18a]. They encoded using the object class
FAIRWY (cf. Figure 2.4).
Figure 2.4: Visualisation of the layer ”Fairway” of the considered area Duisburg.
2.3.2
Land Layer
Land areas that are never covered by water must be encoded using the object class
LNDARE. Rivers, canals, lakes, basins and docks, which are not navigable at compilation
scale, must be encoded on top of LNDARE [Org18a]. LNDARE is usually of type area;
it may, however, be of type point for example a small island, rock that does not cover, or
of type line for example islet, offshore bar, isthmus (a narrow strip of land with sea on
either side, forming a link between two larger areas of land.). There must be no LNDARE
object in the area covering the waterway (cf. Figure 2.5).
Chapter 2. Map visualization using ENC data[Mukesh]
16
Figure 2.5: Visualisation of the layer ”LNDARE” of the considered area Duisburg.
2.3.3
Bridge Layer
Bridges are encoded using the object class ”BRIDGE” [Org18a]. If it is required to encode
a bridge, it must be done using the object class BRIDGE (cf. Figure 2.6).
Figure 2.6: Visualisation of the layer ”BRIDGE” of the considered area Krefeld.
2.3.4
Harbour Layer
A harbour is an area of the sea at the coast which is partly enclosed by land or strong
walls, so that boats can be left there safely. If it is required to encode a harbour facility, it
Chapter 2. Map visualization using ENC data[Mukesh]
17
must be done using the object class ”HRBBSN” [Org18a]. Depending on the navigational
purpose, harbour facilities are defined by: an area including docks, basins, and dockside
equipment; or a point (cf. Figure 2.7).
Figure 2.7: Visualisation of the layer ”HRBBSN” of the considered area Krefeld.
2.3.5
Lake Layer
A lake is a body of water located in a basin and surrounded by land, separate from any
river or other outlet that feeds or drains the lake. Lakes are not part of the ocean and
are located on land. If it is required to encode a lake that is not navigable at compilation
scale, it must be done using the object class ”LAKARE” [Org18a]. If the lake is navigable
at compilation scale, it must be encoded using the object classes ”DEPARE” [Org18a].
2.3.6
Canal Layer
Canals, often known as artificial waterways, are canals or engineered channels constructed
for drainage management (such as flood control and irrigation) or the conveyance of
water transport vehicles (e.g. water taxi). They can be thought of as artificial rivers
because they carry unimpeded, calm surface flow under air pressure. A canal usually has
a succession of dams and locks that generate low-speed current reservoirs. If it is required
to encode a canal that is not navigable at compilation scale, it must be done using the
object class ”CANALS”. If the canal is navigable at compilation scale, it must be encoded
using the object classes DEPARE or DRGARE [Org18b].
Chapter 2. Map visualization using ENC data[Mukesh]
2.4
18
Database consisting the infrastructure data
Shapefiles must be converted to Comma Separated Values (CSV) files. CSV is one of
the alternative file for storing spatial data. Python tool can be used to convert shapefile
to CSV file. The conversion of a shapefile to a .csv-file results two separate latitude
and longitude from a single shapefile. Latitude contains information about Y axis and
longitude contains information about X axis.
2.4.1
Comma Separated Values (CSV)
A Comma Separated Values (CSV) file is a text file with values separated by a comma.
A data record is represented by each line in the file. Each record has one or more fields,
which are separated by commas. The name for this file format comes from the use of
the comma as a field separator. A CSV file typically carries tabular data (numbers and
text) in plain text, with the same number of fields on each line. The CSV file format isn’t
completely standard. The basis is to use commas to separate fields, however commas
in the data or embedded line breaks must be treated differently. Some implementations
forbid such content, while others encircle the field with quotation marks, necessitating
escape if quotation marks are present in the data.
Any spreadsheet program, such as Microsoft Excel, Open Office, Google Sheets, and
others, can open CSV files. A simple text editor can also be used to open a CSV file.
Because it is simple and compatible with most platforms, it is a widely used and popular
file format for storing and accessing data. However, there are significant drawbacks to
this simplicity. CSV can only store a single sheet in a file and does not support formatting
or formulae. The distinction between CSV and XLS file formats is that CSV is a plain
text format with values separated by commas (Comma Separated Values), but XLS is an
Excel Sheets binary file format that contains information about all of the worksheets in
a file, including both content and formatting [Ltd22].
Longitude (X)
Latitude (Y)
2.5
Table 2.1: Structure of CSV file.
0
1
2
3
4
6.6605373 6.6604835 6.6604384 6.6603295 6.6602702
51.5629905 51.5629696 51.5629515 51.5629197 51.5629041
Setting Thresholds
For selecting the particular area a threshold in latitude (Y) and longitude (X) has to be
generated according to the size of the area. Without setting thresholds the areas will look
Chapter 2. Map visualization using ENC data[Mukesh]
19
like unfinished and opening points will appear. These open points are created because
in latitude and longitude CSV files the points are not in correct sequence. So threshold
selected accordingly to avoid the problem (cf. Figure 2.8).
Figure 2.8: Visualisation of the layer Fairway in area Duisburg, left with open points
before setting threshold, right with closed points after setting threshold.
For some areas setting thresholds along either longitude (X) or latitude (Y) is necessary.
But for some other areas both X and Y points must be needed to set threshold along
both latitude (Y) and longitude (X). Thresholds are also useful in selecting only necessary
things needed. For example in areas there will be more than 70 bridge layers it will take
long time to visualize each bridge and selecting the bridge needed in the considered area.
But by setting the threshold for particular area the bridge of the considered area can be
easily selected by in that area by removing unwanted other bridges which are larger or
smaller than the threshold (cf. Figure 2.9).
Chapter 2. Map visualization using ENC data[Mukesh]
20
Figure 2.9: Visualization of the area Krefeld (760 km - 769 km), left from Atlas map
[ITZ22], right the visualisation with the Tool in Rviz.
2.6
Publish Information to Rviz
A ROS node is essentially a compute process. It’s an executable program that runs inside
the tool. It enables the realization of a large number of nodes packages. To connect with
other nodes, ROS nodes use the ROS client library. A Topic can be published or subscribed
to by nodes. A Service can be provided by a node or used by it. It’s worth noting that
no two nodes can have the same name. At some point of time many instances has to run
on same node, a prefix or suffix needed to be given, or declare them anonymous. The
utilization of nodes in ROS has a number of advantages for the overall system. Because
crashes are localized to specific nodes, there is more fault tolerance. The main mechanism
used by ROS nodes to communicate is by sending and receiving messages. The messages
are organized into specific categories called topics. Nodes may publish messages on a
particular topic or subscribe to a topic to receive information [Hab18].
Chapter 2. Map visualization using ENC data[Mukesh]
21
Figure 2.10: Layers of Bridges, Harbours, and Fairway published by single noded ROS.
Each node can publish information about one part only. For example one node will be
able to publish information about only one bridge or harbour or fairway (cf. Figure 2.10).
Multiple ROS nodes act together to give information about one particular areas. These
multiple Ros nodes can be run separately one by one. But it will take lot of time so these
multiple ROS nodes can be operated under one single ROS launch file. ROS launch is a
tool for easily launching multiple ROS nodes. The settings to set and nodes to launch,
as well as the machines on which they should be run, are specified in one or more XML
configuration files with the .launch extension.
For examble area Krefeld (760 km) consist of many layers like BRIDGE, FAIRWY,
HRBBSN and LNDARE (Bridges, Fairway, Harbour and Land). All this information
cannot be published under one single node. To overcome this problem a launch file has
to be created and path of all layers in area krefeld should be added under one launch file
(cf. Figure 2.11).
Chapter 2. Map visualization using ENC data[Mukesh]
22
Figure 2.11: Static map of area Krefeld (760 km -769 km) image published by Rviz using
the implemented tool.
Algorithm 1 Publishing static map information to Rviz
Initialization in ROS:
Step1: Identify subscribers (Rviz) and publishers (points and lines) and establish communication
Step2: Establishing the framework of visualization
Step3: Establish the name-space in which the ROS node will run
Data input:
Read ENC data from CSV
for i = 0; i + +; i <length of CSV do
Publish the information of static map to Rviz
end for
Chapter 3. Dynamic visualization with AIS data[Xu]
3
23
Dynamic visualization with AIS data[Xu]
This chapter describes how the AIS dataset is used to achieve dynamic visualisation of
the vessel’s movement in ROS. Faced with this task, firstly a simplified kinematic model
of the boat is derived. Secondly, the AIS data is processed and filtered. Then algorithms
are written in C++ to realise the motion of the boat in ROS. After that, the way to solve
the synchronisation of dynamic visualization is introduced. Finally, the automatic code
generator is developed to write all the code automatically.
3.1
Background of model motion
The movement of the model consists of two parts. One part is the dynamic and simultaneous display of the vessel’s motion attitude (sailing angle) and position (latitude and
longitude) based on AIS data and a simplified kinematic model of the vessel in the static
map (cf. Figure 3.1). The other part is that the vessel is treated as a mass and its historical motion path is recorded within a static map (cf. Figure 3.2).
Figure 3.1: Example of motion attitude and position.
The kinematic model of vessel is simplified considering that the focus in this work is the
visualization of vessel’s movement based on kinematic states. The simplified three-degreeof-freedom kinematic model is shown in the Figure 3.3.
The 3-DOF simplified vessel kinematic model is centered at (x, y), x refers to longitude
and latitude is described by y. The coordinate system used in this case is the North-EastDown (NED) reference frame. φ is the COG (Course Over Ground).
Chapter 3. Dynamic visualization with AIS data[Xu]
Figure 3.2: Example of historical motion path.
Figure 3.3: 3-DOF simplified vessel kinematic model [YXF18].
24
Chapter 3. Dynamic visualization with AIS data[Xu]
25
With the help of this model, the position and attitude of vessel can be plotted within
the Rviz module of ROS by reading and generating of the latitude, longitude and COG
information from the AIS dataset, thus enabling the dynamic visualisation of the vessel’s
movement.
3.2
AIS data
The Automatic Identification System (AIS) is an automatic tracking system that uses a
transceiver on board the vessel and is used by the Vessel Traffic Service (VTS) [ITU98].
The information provided by the AIS equipment, such as unique identification, position,
course and speed, can be displayed on screen or on the Electronic Chart Display and
Information System (ECDIS) [WB99]. In the simplest case, the AIS operates between a
pair of radio transceivers, one of which is always on board. The other may be on board,
ashore (on the ground), or on a satellite. These represent vessel-to-vessel, vessel-to-shore,
and vessel-to-satellite operations respectively [Res01]. Vessels can switch off their AIS
transceivers for various reasons.
Table 3.1: Structure of AIS data in this project.
ID
22004008
22003990
22004186
1
Time
2021-09-29 13:43:30
2021-09-29 13:43:32
2021-09-29 13:43:35
Longitude Latitude
6.707810 51.241675
6.707805 51.241671
6.707807 51.241663
COG
252.3
252.4
252.5
Other parameters1
...
...
...
For example, true heading, position accuracy, navigation status etc.
AIS is currently used in many ways, most notably to avoid collisions between vessels
[HMWBW07]. AIS systems are also used in other applications, such as maritime security,
aids to navigation, search and rescue.
3.3
Processing big data
The AIS data are stored in database consisting of CSV (Comma Separated Values)files. The data are divided in days and structured as table accoroding to the Table
3.1. Therefore, both the vessel’s travel area and vessel ID are unordered. So the first
step of data processing is to summarize the vessel information according to the driving
area without destroying the chronological order, and the second step is to summarize the
driving information of each vessel according to the vessel ID in the selected driving area.
Chapter 3. Dynamic visualization with AIS data[Xu]
26
However, the amount of AIS data for a single day is huge (over 1.3 million rows) and
cluttered, making it a challenge to handle the huge amount of data.
Faced with this challenge, the option of using the Microsoft office Excel to process is first
ruled out because the maximum number of rows processed by the Excel tool is 220 , i.e.,
1048576 rows, and the AIS data for a single day exceeds this number of rows, so using
Excel would cause data loss and corruption. Secondly, if the AIS data are read directly
in python using a loop statement, the loop executes more than a million times and still
takes a long time to process. Therefore, an optimized method of reading and filtering data
must be considered. After consideration, the Pandas library for python is finally selected
Figure 3.4: Data processing procedures.
to process the AIS data. Pandas library is Python’s core data analysis support library
which provides fast, flexible and explicit data structures designed to handle relational,
tokenized data simply and intuitively [McK12]. With the optimized underlying code of
Pandas and its unique vectorized data structure, it is possible to significantly reduce the
data reading and filtering time by writing the algorithm based on the keyword matching
Chapter 3. Dynamic visualization with AIS data[Xu]
27
Figure 3.5: Trajectories of all vessels in Neuss before data filtering.
method of the built-in functions. Using the Time module in python to time the program
runtime, the time required to read a single day of AIS data and filter the vessels by region
is reduced to about 15 seconds.
After completing the extraction and separation of data, in order to verify the robustness
of the data, the AIS data of a whole day in this area of Neuss is selected to plot the
trajectories of all vessels (cf. Figure 3.5).
Where the arrows point, there are wrong paths, because the vessels cannot travel on land,
so there is wrong data within the AIS data set and data filtering must be performed.
The reason may be that, as mentioned before, the AIS system is not forced to be on, so
some vessels may have turned off the AIS system at some point. Also, the GPS signal
may be unstable in some areas and the vessel cannot provide the accurate position to the
AIS. By analyzing the data, two types of filtering can be used to reject erroneous data,
namely time filtering and distance filtering.
• The time filtering method is: Considering that the data update frequency specified
by the AIS system is once every 10 seconds, if more than 15 seconds have passed
since the acquisition of the previous data frame, but the AIS system has still not
acquired the next data frame, it is decided that there is erroneous data on this
vessel.
Chapter 3. Dynamic visualization with AIS data[Xu]
28
• The distance filtering method is: Taking into account the maximum speed of the
vessel and the update frequency of the AIS data, if the distance travelled by the
same vessel within the adjacent time stamps exceeds a certain threshold, the vessel
is judged to have incorrect data.
With filtering methods, anomalous data within the AIS dataset can be effectively eliminated (cf. Figure 3.6).
Figure 3.6: Trajectories of all vessels in Neuss after data filtering.
3.4
Converting static to dynamic
In Chapter 2.6, the algorithm of generation of static maps is explained in detail. In this
section, the algorithms of the dynamic visualisation of vessel movements are explained.
3.4.1
Attitude and position
Figure 3.1 illustrates the motion attitude and position of 2 vessels. Position is defined
here as the relative position of the vessel within the static map after dynamic visualization
Chapter 3. Dynamic visualization with AIS data[Xu]
29
based on its longitude and latitude at the specified moment. Attitude here refers to the
display of the vessel’s driving pose, such as bow orientation, within a static map based on
the Course Over Ground (COG) read from the AIS dataset at a given moment, combined
with the boat’s kinematic model.
In consideration of reducing the amount of computation within the ROS and visualizing the motion of the vessel more intuitively, the shape of the boat is simplified to an
axisymmetric pentagon with the bottom two angles at right angles.
Figure 3.7: The model of vessel in static map in ROS.
Within the static map, the parameters of the vessel are set as follows. The width of the
boat W is 2 units. The length of the hull L1 is 2 units and the length of the bow L2 is 1
unit.
Considering that the installation position of the AIS receiver on the vessel is unknown,
and the error of the length of the vessel relative to the latitude and longitude, the AIS
receiver is set to be installed at the center of the vessel’s stern at point O. That is, the
latitude and longitude data received by the AIS system is the geographic coordinates of
point O. Point O is also the reference point for calculating and visualizing the vessel’s
position.
Since the latitude and longitude of the reference points are known and the model of the
Chapter 3. Dynamic visualization with AIS data[Xu]
30
vessel is determined, the coordinates of each point of the pentagon (x1 , y1 ) ... (x5 , y5 ) can
be easily deduced and connected in turn to visualize the attitude and position of the
vessel. The pseudo code implemented within ROS is as Algorithm 2.
Algorithm 2 Dynamic visualization of the vessel’s position and attitude in ROS
Initialization in ROS:
Step1: Identify subscribers (Rviz) and publishers (points and lines) and establish communication
Step2: Establishing the framework of visualization
Step3: Establish the namespace in which the ros node will run
while Initialization in ROS is sucessful do
Determine parameters of points and lines
Colours and size of points and lines
Read AIS data from CSV
Longitude, latitude, time, etc.
while N (Initial value is 0)<length of CSV do
Step1: Time proportional synchronization
The movement of the vessel is visualized in 1/12 of real time.
Step2: Ensure the location and attitude in for-loop of the vessel at the Nth point
for i = 0; i + +; i < 6 do
Step1: Determine the boundary points of the vessel based on the kinematic
model
Step2: Publish the infomation of boundary points to Rviz
Step3: Connect the boundary points to form a pentagon describing the vessel’s
attitude in Rviz
end for
Step3: Output the ROS infomation
Step4: Show the vessel’s random ID
Step5: N = N + 1, update the vessel’s coordinate points
end while
Stop the ROS node
end while
3.4.2
Historical trajectory
The previous section introduces the visualization of the position and attitude of the vessel,
and this section focuses on the visualization of the history trajectory. The visualization
of history trajectory is very important to provide technical support for path prediction,
Chapter 3. Dynamic visualization with AIS data[Xu]
31
collision prevention, etc. Figure 3.2 illustrates historical trajectory of the vessels. Also, the
visualization of the vessel as a mass point can be used as a complement to the visualization
of the vessel in attitude and position. Because in the visualization of the previous section,
the size of the vessel’s model is enlarged within the static map in order to make it more
intuitive, which may make a part of the model out of the travelable area, so the error
caused by the different scale can be compensated here.
Historical trajectory visualization implemented in ROS means that only the geographic
location information and time stamp of the vessel are considered, and the vessel is treated
as a mass point and its dynamic motion trajectory is recorded. A dynamic historical
trajectory means that the trajectory is updated over time, rather than a static complete
path.
The pseudo code implemented within ROS is as Algorithm 3.
Algorithm 3 Dynamic visualization of the vessel’s historical trajectory in ROS
Initialization in ROS:
Step1: Identify subscribers (Rviz) and publishers (trajectory and mass point) and
establish communication
Step2: Establishing the framework of visualization
Step3: Establish the namespace in which the ros node will run
Step4: Read AIS data from CSV
while Initialization in ROS is sucessful do
Step1: Time proportional synchronization
The historical trajectory and he movement of mass point which represents vessel is
visualized in 1/12 of real time.
Step2: Publish the infomation of mass point to Rviz
Step3: Rviz automatically generates the trajectory
Step4: N = N + 1, update the position of the mass point
if N >length of CSV then
Stop the ROS node and while-loop
end if
end while
3.5
Multi-node visualization with time sequence synchronization
In this project, the time dimension of the vessel’s movement has to be taken into account
in order to better visualise and replay the vessel’s movement. Only by taking the time
Chapter 3. Dynamic visualization with AIS data[Xu]
32
dimension into account can the relative movement between vessels, the length of time
spent in port, the number of vessels in a given channel at a given time of day and some
other key information be better displayed.
To achieve this requirement, each vessel has two separate ROS nodes. One of them is
responsible for visualizing the position and attitude, and the other one is responsible for
the history path. The two nodes take similar time synchronization algorithms, so the
following section will introduce the time proportional synchronization algorithm using
position-attitude node as an example.
First of all, the principle of AIS data recording of ship movements in real situations is as
Table 3.1 and the upper part of Figure 3.8. As AIS records each line of data, it records
its acquisition time to the second. With the characteristic of AIS data logging, which
starts from the initial time and records new parameters at regular intervals, it is possible
to simulate a similar approach when visualizing within ROS (cf. Lower part of Figure
3.8).
Figure 3.8: The algorithm of time sequence synchronization.
In short, the time synchronization algorithm used in this project is that the vessel travels
across the same river in ROS for one twelfth of the actual navigation time. A 24-hour
navigation record of a river segment is visualized as a 2-hour replay in ROS.
The first step of the algorithm is to calculate the time interval between adjacent data
collection times of the vessels, starting from 00:00:00. For example, the first data collection
Chapter 3. Dynamic visualization with AIS data[Xu]
33
is
is 00:00:10, and the time interval treal interval
between the initial time and treal
time treal
1
1
0
interval
calculated as 10 seconds. In the same way it can be derived that treal 1
is 11 seconds.
From this, the interval between each adjacent data collection point for a particular vessel
can be calculated.
The second step is to set the start time of replay in ROS as the realistic data acquisition
start time, i.e., replay in ROS starts from 00:00:00 in real world and at this point system
starts all ROS nodes (the number of nodes is about 400-600, twice the number of vessels).
The third step is to visualize the attitude and position of the vessel in ROS every twelfth
of the actual time interval. This enables the visualization of proportional time synchronization.
Figure 3.9: The algorithm of time sequence synchronization with multi-node.
However, this method (cf. Figure 3.9) causes a load problem for the ROS system executing
600 nodes in parallel because all ROS nodes are started at the same time as the ROS
node starting time tros
0 . The high load causes large latency and negatively affects the
performance of time synchronization. In order to solve this problem, the following two
methods (cf. Figure 3.10) are adopted.
• Visualization process optimization: Within Algorithms 2 and 3 in the previous section, the separation of reading AIS data and dynamically publishing the data to the
Chapter 3. Dynamic visualization with AIS data[Xu]
34
Figure 3.10: Optimized time sequence synchronization in ROS.
Rviz module was implemented, i.e., reading the data before the time synchronization algorithm, publishing the data after the time synchronization, and reading all
the data in when all the nodes are started, so that only the synchronization of the
first few nodes is affected.
• Node-dormancy strategy: Due to the excellent performance of C++, it takes only a
few milliseconds to execute the code that publishes visual information to the Rviz
module, and the remaining time causes a waste of computing resources. Therefore,
our method is to put this ROS node directly into ”sleep” state during the time after
executing the current information publishing task, and wake up to execute the next
release information task after this time interval is over.
The optimized algorithm enables more accurate visualization of vessel motion with accurate timing information with the error level of second. Since ROS itself is not a real-time
operating system, the complicated programming of real-time operating system and the
data update frequency of AIS dataset (about 0.1Hz) , it is not possible to visualize the
ship movement with the error level of millisecond.
Chapter 3. Dynamic visualization with AIS data[Xu]
3.6
35
Auto code generator for multiple ROS nodes
Considering the concurrency and synchronization of node execution, as well as the simplicity and reliability of the code, the visualization of each ship consists of position-attitude
nodes and path node respectively. Each ship needs 2 ROS nodes, and there will be about
200-300 ships in each region every day, so manually building a large number of ROS
nodes will be a tedious and repetitive task. To solve this problem in the project and
to improve the automation of the visualization tool, an automatic code generator is designed and implemented. The code autogenerator consists of three parts, namely C++
code autogenerator, template code, and compile and execute file autogenerator.
The implementation idea is as follows (cf. Figure 3.11): C++ code file is also a kind of text
file before being compiled and executed, so it is possible to write C++ files directly with
the help of code. In section 3.4, the code for node visualization has the same structure,
so the node code of each vessel is only a little different from the node code of other
vessels, such as node name, number of loops, etc. So firstly, a template code is built,
the fixed parts and the parts that need to be changed are abstracted and then the ship
specific information from AIS dataset is used to write the changing parts to complete the
automation of node writing. The generator of the compilation and execution files can be
done directly from the ship information in the AIS dataset. Finally, the automatically
generated C++ code is combined with the compiled file to get the executable files and
which are executed by launch file.
Chapter 3. Dynamic visualization with AIS data[Xu]
Figure 3.11: Structure of the automatic code generator.
36
Chapter 4. Results[Mukesh and Xu]
4
37
Results[Mukesh and Xu]
In this chapter the results obtained from the implemented tool are shown. In section 4.1,
the static map of the considered area are shown and described. In section 4.2, the results
of the the dynamic visualisation are illustrated.
4.1
Static map
Figure 4.1: Static map of the Neuss area of the region 736 km to 745 km according to the
Atlas notation [Geo22] with the implemented Tool (cf. Chapter 2).
Figure 4.1 illustrates the static map of the Neuss areas obtained from the implemented
Tool (cf. Chapter 2). The used polygons refer to the shape-file LNDARE layer (blue),
FAIRWY layer (red) and the BRIDGE (green) layer [STA06]. This section of the river
passes mainly through Neuss and a part of Düsseldorf, with the port of Düsseldorf on the
left and right of the figure. This refers to the region between 736 km to 745 km according
to the Atlas description [Geo22].
Figure 4.2 illustrates the the static map of the Haus-Knipp-Eisenbahnbrücke and Bundesautobahnbrücke (HKEB) areas obtained from the implemented Tool (cf. Chapter 2).
The used polygons refer to the shape-file LNDARE layer (blue), FAIRWY layer (red) and
Chapter 4. Results[Mukesh and Xu]
38
Figure 4.2: Static map of the of Haus-Knipp-Eisenbahnbrücke and Bundesautobahnbrücke (HKEB) areas of the region 782 km to 788 km according to the
Atlas notation [Geo22] with the implemented Tool (cf. Chapter 2).
the BRIDGE (green) layer. The bridge on the lower part of the Figure is the Haus-KnippEisenbahnbrücke, the other one is the Bundesautobahnbrücke. This refers to the region
between 782 km to 788 km according to the Atlas description [Geo22].
Chapter 4. Results[Mukesh and Xu]
39
Figure 4.3: Static map of the Krefeld area of the region 760 km to 769 km according to
the Atlas notation [Geo22] with the implemented Tool (cf. Chapter 2).
Figure 4.3 illustrates the static map of the Krefeld areas obtained from the implemented
Tool (cf. Chapter 2). The used polygons refer to the shape-file LNDARE layer (blue),
FAIRWY layer (red) and the BRIDGE (green) layer [STA06]. This section of the river
passes mainly through area Krefeld, with the port of Krefeld hafen and Yatch hafen on
the left of the figure. Also with Rheinbrcke and Drehbrcke bridges. This refers to the
region between 760 km to 769 km according to the Atlas description [Geo22].
Chapter 4. Results[Mukesh and Xu]
40
Figure 4.4: Static map of the Duisburg area of the region 778 km to 783 km according to
the Atlas notation [Geo22] with the implemented Tool (cf. Chapter 2).
Figure 4.4 illustrates the static map of the Duisburg areas obtained from the implemented
Tool (cf. Chapter 2). The used polygons refer to the shape-file LNDARE layer (blue),
FAIRWY layer (red) and the BRIDGE (green) layer [STA06]. This section of the river
passes mainly through area Duisburg, with Bundesautobahnbrcke and feste Brcke on the
left side. Also with two bridges and five harbours in its right side. This refers to the
region between 778 km to 783 km according to the Atlas description [Geo22].
Chapter 4. Results[Mukesh and Xu]
41
Figure 4.5: Static map of the Rheinvorland im Orsoyer Rheinbogen area of the region 796
km to 803 km according to the Atlas notation [Geo22] with the implemented
Tool (cf. Chapter 2)
Figure 4.5 illustrates the static map of the Rheinvorland im Orsoyer Rheinbogen areas
obtained from the implemented Tool (cf. Chapter 2). The used polygons refer to the
shape-file LNDARE layer (blue), FAIRWY layer (red) and the BRIDGE (green) layer
[STA06]. This section of the river passes mainly through area Rheinvorland im Orsoyer
Rheinbogen, with no bridges in it. This refers to the region between 796 km to 803 km
according to the Atlas description [Geo22].
4.2
Dynamic visualization
All the visualization scenes of the vessels in the five regions that are included within the
AIS dataset are recorded as 50 videos. This section selects specific scenarios for each
region to explain.
Figure 4.6 and Figure 4.7 illustrate the scene obtained from the implemented Tool (cf.
Chapter 2 and 3) that the vessel with random ID 9702855 entered the port in Neuss areas
between 02:43 and 02:48 in 2001.02.01. The vessel with random ID 9702855 traveled
out of the river area (red polygon) before entering the harbor, and when it entered the
harbor it significantly reduced its speed compared to other vessels traveling normally and
traveled close to the left bank of the river in its direction of travel.
Chapter 4. Results[Mukesh and Xu]
42
Figure 4.6: The scene of the vessel before entering the port in Neuss area.
Figure 4.7: The scene of the vessel after entering the port in Neuss area.
Figure 4.8 and Figure 4.9 illustrate the scene that the vessel with random ID 5938551
and random ID 7203476 met and passed in HKEB areas between 20:36 and 20:49 in
2001.02.02. Before the two ships met, both vessels reduced their speed and both had
Chapter 4. Results[Mukesh and Xu]
Figure 4.8: The scene of the vessels before meeting in HKEB area.
Figure 4.9: The scene of the vessel after meeting in HKEB area.
43
Chapter 4. Results[Mukesh and Xu]
44
evasive maneuvers. The direction of bow of the vessel with random ID 5938551 was
deflected to the slightly west and the other vessel was slightly deflected to the east.
Figure 4.10: The scene of the vessel before departing the port in Duisburg area.
Chapter 4. Results[Mukesh and Xu]
45
Figure 4.11: The scene of the vessel after departing the port in Duisburg area.
Figure 4.10 and Figure 4.11 illustrate the scene obtained from the implemented Tool
(cf. Chapter 2 and 3) that the vessel with random ID 2582870 departed from the port
in Duisburg areas between 03:54 and 03:55 in 2001.02.02. The vessel with random ID
2582870 entered into the river area (red polygon) after departing the harbor, and when
it entered in the river it almost stopped in the river and prepared to turn around.
Chapter 4. Results[Mukesh and Xu]
46
Figure 4.12: The scene of the vessels decelerate at river-bend in area Rheinvorland im
Orsoyer Rheinbogen.
Figure 4.12 ilustrate the scene obtained from the implemented Tool (cf. Chapter 2 and 3)
that the vessels with random ID 6466619, 8536723, 1336038, and 4710429 entering river
bend in area Rheinvorland im Orsoyer Rheinbogen between 05:30 and 05:58 in 2001.02.08.
All vessels always decelerate to avoid collusion or loss of motor control when it entered in
the river bend area.
Figure 4.13: The scene of the vessels anchorage at krefeld area day 1.
Chapter 4. Results[Mukesh and Xu]
47
Figure 4.14: The scene of the vessels anchorage at krefeld area day 2.
Figure 4.13 and Figure 4.14 illustrate the scene obtained from the implemented Tool (cf.
Chapter 2 and 3) that the vessel anchorage area for many vessels are inferred from the
two figures and validated in Atlas map [Geo22].
Chapter 5. Conclusion[Xu]
5
48
Conclusion[Xu]
The visualization and replay of vessel motion plays an important role in many ways, for example, it helps researchers to better collect of different possible driving behaviors/actions
sequences, study the human decision making process in the spatial-temporal dimension,
etc. The vessel motion visualization and replay within the ROS framework provides researchers with a universal visualization process and a highly versatile interface for future
development based on an open source and widely used framework.
The chapter 1 describes the motivation and purpose of the project and the structure. The
chapter 2 is a literature review. The development of the static maps considered in this
work is explained in the chapter 3. First the ENC dataset is processed, converted to a
shape-file, then the desired areas are selected, and finally publish them to Rviz. Chapter
4 describes the method that visualizes the ship motion dynamically. The first step is to
process and filter the AIS big data, the second step is to write code to visualize attitude,
position based on the ship kinematic model and history trajectory, the third step is to
design an algorithm for time synchronization of multiple ship nodes, and finally a code
auto generator is designed to realize automatic code generation. In chapter 5, example
scenarios are selected for validation and analysis of the generated static maps and ship
motion visualization. The validation results show that the generated static maps are
accurate and reliable after comparison with Altas maps. And dynamic visualization can
accurately show the driving behavior and action.
In addition to the scenarios included in the report, a total of 50 videos of vessel movements
are recorded within the ROS for ten days in each region. This project has successfully
visualize the vessel motion and develope an automated tool chain, realize the previously
scheduled requirements fully, provides a solid basis for analyzing driving behavior and
developing an autonomous vessel.
REFERENCES
49
References
[BJ21]
Blindheim, Simon ; Johansen, Tor A.: Electronic Navigational Charts
for Visualization, Simulation, and Autonomous Ship Control. In: IEEE
Access (2021)
[CGE11]
Cervera, Miguel A. ; Ginesi, Alberto ; Eckstein, Knut: Satellitebased vessel Automatic Identification System: A feasibility and performance analysis. In: International Journal of Satellite Communications
and Networking 29 (2011), Nr. 2, S. 117–142
[Com21]
Community, Esri. ArcMap: Creating a layer package. August 2021
[Fou20]
Foundation., OpenStreetMap. Map data from OpenStreetMap, ODbL
1.0. Juni 2020
[FW+ 22]
Frank Warmerdam, Even R. [u. a.]. Open Source Geospatial Foundation. Juli 1998-2022
[Geo22]
Bundesamt für Kartographie und Geodäsie, BKG. Online altas
map of Germany. https://atlas.wsv.bund.de/clients/desktop/. 2022
[Hab18]
HabibOladepo. ROS Graph Concepts. Dezember 2018
[HMWBW07] Harati-Mokhtari, Abbas ; Wall, Alan ; Brooks, Philip ; Wang, Jin:
Automatic Identification System (AIS): Data Reliability and Human Error
Implications. In: Journal of Navigation 60 (2007), Nr. 3, S. 373389
[HY15]
Haiyan, Wang ; Youzhen, Wang: Vessel traffic flow forecasting with
the combined model based on support vector machine. In: 2015 International Conference on Transportation Information and Safety (ICTIS)
IEEE, 2015, S. 695–698
[ITU98]
ITU: Technical characteristics for a universal shipborne automatic identification system using time division multiple access in the VHF maritime
mobile band. (1998)
[ITZ22]
ITZBund. https://atlas.wsv.bund.de. 2022
[KÇP17]
Köseoğlu, Murat ; Çelik, Orkan M. ; Pektaş, Ömer: Design of an
autonomous mobile robot based on ROS. In: 2017 International Artificial
Intelligence and Data Processing Symposium (IDAP) IEEE, 2017, S. 1–5
REFERENCES
50
[KJ19]
Kumar, Rahul ; Jayashankar, Sujay: Radar and camera sensor fusion
with ROS for autonomous driving. In: 2019 Fifth International Conference
on Image Information Processing (ICIIP) IEEE, 2019, S. 568–573
[KL18]
Kim, Kwang-Il ; Lee, Keon M.: Deep learning-based caution area traffic
prediction with automatic identification system sensor data. In: Sensors
18 (2018), Nr. 9, S. 3172
[KYK08]
Kim, Choong N. ; Yang, Kyung H. ; Kim, Jaekyung: Human decisionmaking behavior and modeling effects. In: Decision Support Systems 45
(2008), Nr. 3, S. 517–527
[LMS+ 21]
Lu, Yongqiang ; Ma, Hongjie ; Smart, Edward ; Vuksanovic, Branislav
; Chiverton, John ; Prabhu, Shanker R. ; Glaister, Malcolm ; Dunston, Eric ; Hancock, Chris: Fusion of Camera-based Vessel Detection
and AIS for Maritime Surveillance. In: 2021 26th International Conference
on Automation and Computing (ICAC), 2021, S. 1–6
[Ltd22]
Ltd, BigCommerce P. Ecommerce How Tos. Februar 2003 - 2022
[LYXZ19]
Liu, Yong ; Yao, Libo ; Xiong, Wei ; Zhou, Zhimin: GF-4 Satellite and
Automatic Identification System Data Fusion for Ship Tracking. In: IEEE
Geoscience and Remote Sensing Letters 16 (2019), Nr. 2, S. 281–285
[LZMJ19]
Li, Wenkai ; Zhang, Chunwei ; Ma, Jie ; Jia, Chengfeng: Long-term
Vessel Motion Predication by Modeling Trajectory Patterns with AIS Data.
In: 2019 5th International Conference on Transportation Information and
Safety (ICTIS), 2019, S. 1389–1394
[McK12]
McKinney, Wes: Python for data analysis: Data wrangling with Pandas,
NumPy, and IPython. ” O’Reilly Media, Inc.”, 2012
[MSK17]
Meyer, R.G. V. ; Schwegmann, C.P. ; Kleynhans, W.: Vessel classification features using spatial Bayesian inference from historical ais data.
In: 2017 IEEE International Geoscience and Remote Sensing Symposium
(IGARSS), 2017, S. 2629–2632
[Org18a]
Organization, International H. ENC Product Specification - S-57 Appendix B.1. Januar 2018
[Org18b]
Organization, International H. ENC Product Specification - S-57 Appendix B.1. Januar 2018
REFERENCES
51
[PPK+ 13]
Park, Daewon ; Park, Jaeyang ; Kwon, Doowy ; Kwon, Hyuk-Chul
; Park, Suhyun: Design of data structure for system ENC kernel. In:
2013 International Conference on IT Convergence and Security (ICITCS)
IEEE, 2013, S. 1–3
[Res01]
Resolution, A. 917 (22), Guidelines for the onboard operational use of
shipborne automatic identification systems (AIS). 2001
[STA06]
STANDARD, IHO T.: for DIGITAL HYDROGRAPHIC DATA. (2006)
[TCL19]
Tang, Guolei ; Cao, Qiankai ; Li, Xiang: Analysis of Vessel Behaviors in Costal Waterways Using Big AIS Data. In: 2019 IEEE 4th International Conference on Cloud Computing and Big Data Analysis (ICCCBDA), 2019, S. 290–294
[WB99]
Wright, Dawn J. ; Barlett, Darius J.: Marine and coastal geographical
information systems. CRC press, 1999
[WRL20]
Wang, Chang ; Ren, Hongxiang ; Li, Haijiang: Vessel trajectory prediction based on AIS data and bidirectional GRU. In: 2020 International Conference on Computer Vision, Image and Deep Learning (CVIDL), 2020, S.
260–264
[WZC+ 17]
Wang, Chengbo ; Zhang, Xinyu ; Chen, Xiang ; Li, Ruijie ; Li, Gaocai:
Vessel traffic flow forecasting based on BP neural network and residual
analysis. In: 2017 4th International Conference on Information, Cybernetics and Computational Social Systems (ICCSS) IEEE, 2017, S. 350–354
[YXF18]
Yu, Wen-Zhao ; Xu, Hai-Xiang ; Feng, Hui: Robust adaptive faulttolerant control of dynamic positioning vessel with position reference system faults using backstepping design. In: International Journal of Robust
and Nonlinear Control 28 (2018), Nr. 2, S. 403–415
[ZWMS19]
Zhang, Haohao ; Watanabe, Kyosuke ; Motegi, Kazuhiro ; Shiraishi,
Yoichi: ROS Based Framework for Autonomous Driving of AGVs. In:
Proceedings of the IPS6-04, ICMEMIS, Kiryu, Japan (2019), S. 4–6