Complex & Intelligent Systems
https://doi.org/10.1007/s40747-023-01196-z
ORIGINAL ARTICLE
An overview of developments and challenges for unmanned surface
vehicle autonomous berthing
Gongxing Wu1
· Debiao Li1 · Hao Ding2 · Danda Shi1 · Bing Han3
Received: 13 January 2023 / Accepted: 15 July 2023
© The Author(s) 2023
Abstract
With the continuous progress of contemporary science and technology and the increasing requirements for marine vehicles
in various fields, the intelligence and automation of ships have become a general trend. The autonomous control of surface
Unmanned Surface Vessel (USV) generally covers the USV path planning, path tracking control, and autonomous collision
avoidance control. But in the whole navigation process of USV, autonomous berthing is also a crucial part. And the research
on the algorithm of the automatic berthing process of the USV is less. Mature USV autonomous berthing technology can
effectively reduce the cost of human and material resources and financial resources while reducing the accident rate reasonably
and safely. Therefore, it is of great importance to comprehensively promote the development of USV autonomous berthing
technology.
Keywords Autonomous berthing · Unmanned Surface Vessel · Berthing control algorithm · Path planning
Introduction
The USV, with their low cost, small size, fast action, intelligence, and other advantages, play a very important role in
daily life, emergency response, and scientific research. USV
generally consist of a platform system and a mission payload
system [1]. It can be used in civil fields such as environmental
monitoring, search and rescue and salvage, and hydrological exploration, as well as intelligence collection, regional
B
Gongxing Wu
wugx@shmtu.edu.cn
Debiao Li
202030410121@stu.shmtu.edu.cn
Hao Ding
202230110152@stu.shmtu.edu.cn
Danda Shi
ddshi@shmtu.edu.cn
Bing Han
han.bing@coscoshipping.com
1
College of Ocean Science and Engineering, Shanghai
Maritime University, Shanghai 201306, China
2
Merchant Marine College, Shanghai Maritime University,
Shanghai 201306, China
3
Shanghai Ship and Shipping Research Institute Co., Ltd.,
Shanghai 200135, China
inspection, Mine clearance, anti-submarine, and other military fields [2]. With the development of intelligence, higher
requirements are put forward for the "autonomy" of USV, and
the autonomous berthing technology of USV is an important
link to realize the real "unmanned" of USV.
The International Maritime Organization (IMO) and intelligent shipping companies have conducted some research
on the autonomous berthing function of USVs [3]. In 2018
IMO divided the level of ship autonomy into 4 classes as
shown in Table 1; The fourth class is a fully autonomous
ship, which can realize autonomous navigation from shore
to shore under completely unmanned conditions, including
autonomous berthing control. In the same year, the Norwegian operator Norled’s ferry "Folgefonn" has become the
first ferry in the world to be equipped with an automatic
berthing system, which was tested during several stops during a round trip to the port. During the test, the captain did
not perform any manual control operations. Nippon Yusen
Kabushiki Kaisha (NYK), NYK Group companies MTI Co.
Ltd. And Japan Marine Science Inc (JMS) have jointly developed a system to assist ships in berthing in 2019; In May of
the same year, China’s first unmanned autonomous navigation system experimental vessel—"Zhiteng" was launched
in Qingdao for sea trial; the vessel tested remote control
and automatic berthing and other functions [4]; In 2020,
123
Complex & Intelligent Systems
Table 1 IMO ship autonomy class and berthing method
Autonomy
level
Level 1
Level 2
Level 3
Level 4
Grade
description
Staffing
Berthing
method
Ships with
automation
and decision
support
Crew on
board
Manual
berthing
maneuvering
Remote
control of
the ship with
a crew on
board
A small
number of
crew on
board,
shore-based
monitoring
Remote
berthing
maneuverable
Remote
control of
ships
without
crew on
board
No crew on
board,
completely
monitored
from the
shore base
Remote
monitoring of
berthing
Fully
autonomous
ships
Emergency
intervention
without
crew and
shore-based
monitoring
Autonomous
berthing
Development
of
2022
Wang et al.
Autonomous berthing method based
on berth shoreline detection
2021
Oka Ship
Automatic slide, magnetic entry
when close
2020.10
Huang et al.
Fuzzy LOS based berthing path
tracking control method
2020.07
2020
2019.05
2019
2017
2016.06
2014
Jia et al.
Establishment of data training
under berth coordinate system
MIT USV
Detect shore QR codes
Conducting guidance
"Zhiteng" tests automatic berthing
function in Qingdao
NYKGroup companies MTI Co.
Ltd.Joint development of a system
to assist ships in berthing
Zhang et al.
Design of artificial potential field
based
Solve autonomous berthing
Joohyun Woo et al.
Vector field-based
berthing planning method
Jong-Yong et al.
PID-based control design
System with berthing rope
Mizuno et al.
2015 Based on multiple beating algorithms
MPC combined to accomplish
automatic berthing
researchers at the Computer Science and Artificial Intelligence Laboratory(CSAIL) at the Massachusetts Institute of
Technology (MIT) said they had created a self-driving boat
that could move autonomously, carrying passengers across
a river and guiding berthing by detecting QR codes on the
shore; in the same year, the South Korean KASS project conducted testing and evaluation of autonomous ships. In 2021,
Ouka Smart Hublot has also carried out research to upgrade
its equipment, making it possible for USV to berth automatically by means of automatic magnetic suction. In January
2022, the U. S. Navy tested an autonomous berthing and
recovery test of the Explorer USV in the Persian Gulf. In
the same month, MEGURI2040, an USV project supported
by the Nippon Foundation, completed its first demonstration
test, successfully conducting the world’s first autonomous
navigation demonstration of a small sightseeing boat in Saru
Island, Yokosuka. In June 2022, China’s first fully domestic
100-ton USV successfully completed its first autonomous
voyage at sea in the waters off Zhejiang.
On the theoretical side, in 2011, Van Phuoc Bui et al.
used autonomous tugs for berthing navigation [5]; In 2015,
Mizuno et al. designed experiments based on a variety of
beating algorithms combined with Model Predictive Control (MPC) to complete automatic berthing [6]; In June
2016, Joohyun Woo et al. designed a vector field-based
berthing planning method [7]; In 2017, Park, Jong-Yong,
et al. proposed a berthing system with berthing rope based
on Proportion Integral Differential (PID) control design [8];
123
2011
Van Phuoc Bui et al.
Use of autonomous tugs
Performing berthing
Fig. 1 Development of Autonomous Ships and USV
In the same year, Zhang et al. designed an artificial potential field-based solution to the autonomous berthing problem
[9]; In July 2020, Jia et al. conducted experiments on data
training under a berthing coordinate system [10]; In October
2020, Huang et al. designed a fuzz Line Of Sigh(LOS)based berthing path tracking control system [11]; In 2022,
Wang et al. proposed an unmanned berthing water shoreline detection-based boat autonomous berthing method [12].
From the above-mentioned current popular research projects
(As shown in Fig. 1), the research on autonomous berthing
of USV has important scientific and technological value and
economic value.
Aiming at the review of the research of USV autonomous
berthing methods, the subsequent chapters of this paper are
arranged as follows: the second part introduces the now popular USV berthing architecture; the third part introduces the
existing methods of USV berthing key technologies; the
fourth part introduces the opportunities and challenges of
USV autonomous berthing; the fifth part summarizes the key
technologies of USV autonomous berthing and looks forward
to the extensive use of USV in the future.
Complex & Intelligent Systems
Fig. 2 Schematic diagram of yacht berthing
The architecture of USV berthing
Fig. 4 Large ship-assisted stowage [14]
At present, most ships need manual assistance to leave the
dock, such as the captain driving the yacht to complete the
dock berthing (Fig. 2), and the USV needs manual assistance
to leave the shore (Fig. 3 [13]); the USV needs manual assistance to put away the boat on large ships [14] (Fig. 4). With
the development of artificial intelligence, if there is a control
system that can copy or transplant the rich driving experience
of the "old captain", the maneuvering control of USV will
become easy; if all USV can leave the dock independently,
then a lot of manpower and financial resources for operating USV will be saved; if all USV If all the USV can leave
the dock or mother ship independently, then the "unmanned"
management of USV will be truly realized.
Most existing USV are intelligent platforms that rely on
multiple shipboard sensors and navigate on the water surface in an autonomous or semi-autonomous manner [15].
They are only equipped with paddle, rudder, or double
propeller devices, which are typical under-driven control
systems, especially when berthing and unberthing maneuvers are performed in ports, where the speed is low; the
interference of external forces such as wind, waves, and currents are relatively increased, and the crowded and narrow
port environment significantly reduce the hull maneuvering
performance, and the USV control system shows strong nonlinear characteristics [16]. For these problems that cannot be
avoided when the USV is berthing, a set of architecture with
deliberate thinking is needed to organize the various functional modules in an orderly and strict manner to realize the
autonomous berthing function of the USV.
For the autonomous berthing problem of unmanned craft
under complex environmental conditions, to improve the
robustness and practicability of the entire berthing process
of unmanned craft, the classic OODA control cycle [17] is
generally used as the system architecture, which is composed
of observation, judgment, decision, and action. The USV
berthing system [18] includes USV situational awareness,
USV berthing modeling, USV berthing path planning, and
USV berthing control. Connecting their functions in tandem,
the parts always react to unknown environments, dangerous situations, and various emergencies, perform uncertainty
event monitoring, and subsequently autonomously perform
hierarchy and decision-making, prompting the computer into
manual warning and human intervention.
Then autonomous machine learning will be carried out
through manual modeling, manual decision-making, and
Fig. 3 USV rely on
human-assisted berthing [13]
123
Complex & Intelligent Systems
Succeed
Enter the berthing
area
Yes
Hull motion
modeling
Environment
modeling
Berthing
decisions
No
To the environment and
Special scenario response
Choosing the right decision
method
Integrated situational
awareness devices
Radar
Sonar
Weather
Stations
Portfolio
Navigation
...
Whether the bow and stern
are consistent with the
distance from the berth
shoreline
Berthing path
planning
No
Enter the
berth range
Whether the berthing
behavior can be carried
out normally and safely
Yes
Yes
Perceptual processing
of the current
environment
Select the berthing
method
Whether the predetermined
berth range has been
successfully entered
No
Fig. 5 The general process of autonomous berthing of the USV
remote control throughout the systems. The USV will also
always be for the input instructions to output and feedback
to the information acquisition system for cyclic processing
to get the optimal operation process, that is, the completion
of the USV’s autonomous berthing. Among them, berthing
equipment includes shore-based equipment and shipboard
equipment. The shore-based equipment includes intelligent USV shore-based monitoring computers and remote
communication equipment. On-board equipment includes
an autonomous driving control subsystem, navigation subsystem, intelligent planning subsystem, audio processing
subsystem, and optical visual processing subsystem. The
berthing system starts with human–computer interaction,
opens the whole berthing process, and sets up the autopilot
program; the planning system carries out path planning and
obstacle avoidance for the area, an emergency treatment for
the unexpected environment, and updates the environmental information in real-time; the navigation and intelligent
control system:carries out real-time berthing control for the
ship, adjusts its heading and position. The audio and image
processing system converts the detected sound and image
information and then gets the required information. After
converting and then getting the required important information, the USV carries out real-time navigation and completes
the berthing task. In this process, the on-board equipment
always dynamically collects audio and images of the environment where the USV is located and fuses the obtained
information to continuously make multi-loop action determination in real-time, after which the previously obtained
information is transmitted in a unified manner to present the
123
HIL(Hardware In the
Loop) debug
interface
HCI(human-compute
r interaction)
Automatic driving
settings
USV info display
Planning system
Path planning and
collision avoidance
Marine environment info
Berthing control
Boat-posture info
Control signal
Multi-dimensional info fusion
Navigation-intelligent
control system
Acoustic-vision
processing system
Basic controlacquisition system
USV-motion
simulation system
Engine
USV sports
emulation
Servo
Fig. 6 The semi-physical berthing system
whole USV’s movement process on the computer. The general process of autonomous berthing of the USV is shown in
Fig. 5. And the semi-physical berthing system is illustrated
in Fig. 6.
USV situational awareness
Situational awareness is the beginning of the USV berthing
process. When a USV berths at a terminal, it first needs to
sense the port environment [19, 20] and locate the target
Complex & Intelligent Systems
Berthing
Visual
position
target
info
location info
Berthing target
fusion
Monitor for
unknown
nfo
f
info
Interactive
Obstacle
info
Chart
info
Situation
forecasting
Berthing target
positioning
Windwave
flow info
Map info
fusion
Environment
map modeling
Beidou
GPS
Navigation Navigation
info
info
Pose info fusion
Self-pose info
Eg
study
Auto
hierarchical
decision-making
Berthing
situation
assessment
Fig. 7 Block Diagram of USV Berthing Situation Awareness and
Assessment
berth. There are many USV sensing devices such as Millimeter Wave Radar (MMW-radar), sonar, Global Positioning
System (GPS), ultrasound, camera [21], etc., which work in
concert [22] to realize the modeling of the berthing environment.
The situational awareness system is an integrated hardware and software system that fuses sensor information with
different sensing capabilities and evaluates the berthing posture. The system is generally equipped with navigation radar
for collision avoidance and path planning [23], automatic
positioning of ships, target detection, and signal processing
during navigation [24], etc.; Automatic Identification System (AIS) [25] is used to improve dynamic navigation track
planning of ships [26], to assess the risk of collision of ships,
to provide early warning and guarantee for ships’ navigation
[27], and to combine with other equipment for ship speed
judgment [28], etc.; GPS is used for positioning of ships [29];
The electronic nautical charts are used for navigation of ships
[30]; The forward-looking sonar is used to identify underwater targets [31], which is combined with other systems to
improve the stability and accuracy of target tracking [32]; the
electronic chart information system is used for route processing and radar processing; the accurate and timely acquisition
of specified position tidal information is used to draw up
navigation routes [33], etc.; The information from the Beidou Navigation System (BDS) and GPS will be integrated to
improve the accuracy and reliability of ship positioning; the
picture information from the visible camera and the infrared
camera [34] will be integrated to enhance the detection of
obstacles in the surrounding environment of USV; LIDAR
and MMW-radar are used to compensate for the blind spots
of nautical radar, which is the obstacle information at close
range. The block diagram of USV berthing situational awareness and evaluation is shown in Fig. 7.
Through these systems, the equipment can complete
berthing operations, using microwave, laser, MMW-Radar
carried by USV to obtain the distance between the ship and
the shoreline, then combined with the ship’s GPS, BDS terminal data to determine the ship’s position relative to the berth,
and get the ship’s position and attitude relative to the shore;
MMW-radar, attitude sensor, and front sonar are used for realtime feedback of the ship’s position information and heading;
Due to the limited accuracy of GPS, BDS and other devices,
when the ship is close to the berth, the judgment and processing of environmental information are carried out by the ship
carrying devices with higher accuracy; real-time dynamic
local planning [35] is carried out to come up with feasible
paths; finally, relying on the linkage between devices and
the implementation of commands by the ship, the computer
carries out the fusion of each data to realize the automatic
berthing of the ship.
In the field of situational awareness of USV, Sarang Thombre et al. [36] proposed that in order to ensure the perception
function of the unmanned ship itself and the accurate cooperation between individual sensors, by combining sensors
and AI technologies with each other to achieve adaptability and compatibility with the environment; in the process
of perception for the environment and targets, Chen et al.
[37] conducted an integrated framework for automated ship
identification and behavioral analysis to improve the functionality of video detection infrastructure, thus increasing
the safety and accuracy of perception of the surrounding
environment; Won-Jae Lee et al. [38] record images to understand other ships around them through cameras installed on
the ship and improve the performance of detection, localization, and tracking execution throughout the voyage through
deep learning models, which play an important role in the
perception of the surrounding environment also plays an
important role. Sharma et al. [39] and Wang et al. [40] investigated ship navigation situational awareness, Rowen et al. [41]
investigated enhanced navigation environment awareness,
and Zhang [42] investigated target detection and obstacle
autonomous avoidance of unmanned ships by monitoring the
visible light possessed by the tracked targets.
Multisensor information fusion
The sensors in the unmanned sensing system are the top priority, in which there is a clear division of labor among the
various layers, and the integration of complex and diverse
environmental information is the most essential requirement
for information fusion. Information fusion is the complementary combination of information from various sensors at
multiple levels and spaces, and processing optimization to
ultimately produce a consistent interpretation of the berthing
environment [43]. Information and data from multiple sensors or multiple sources are automatically analyzed and
synthesized under certain guidelines using computer technology to accomplish the required information processing
and estimation. In this process, multiple sources of data are
123
Complex & Intelligent Systems
to be fully utilized for rational domination and use, while the
ultimate goal of information fusion is to derive more useful
information by combining multiple levels and aspects of the
information based on the separated observations obtained
from each sensor. This not only takes advantage of multiple sensors operating in concert with each other but also
synthesizes data from other information sources to improve
the intelligence of the overall situational awareness system.
In nearly 50 years of scientific research, many effective
approaches have emerged.
Weighted Average Methodis one of the most used scenarios
in the fusion algorithm and has been improved through continuous improvement in the process to obtain better results
[44].
Kalman Filter Method is suitable for the real-time fusion of
redundant sensor information in dynamic environments and
can effectively eliminate external noise interference when
applying Kalman filter method to wireless sensors [45, 46].
Structural Layer Fusion Processing is a multi-sensor data
fusion algorithm model, which can fuse different information
levels of complex situations at three levels [47].
Neural Network a neural network is a nonlinear, adaptive
information processing system composed of a large number of interconnected processing units, proposed based on
modern neuroscience research results. Neural networks are a
new jinn approach to information processing by simulating
the way the brain’s neural networks process and remember
information [48].
Fuzzy Set Theory Fuzzy logic is a multi-valued logic that
specifies a real number from 0 to 1 to represent the degree of
affiliation and will perform well for mathematically model
agnostic systems [49]; Bayesian theory: Bayesian estimation is a common method for fusing multi-sensor underlying
data in static environments. Its information is described as a
probability distribution and is suitable for uncertainties with
addable Gaussian noise [50].
D–S evidence Theory Dempster–Shafer (D–S) evidence
inference is an extension of Bayesian methods, and its basic
principle is to describe sensor information using trust intervals [51].
Genetic Template Method The genetic algorithm is a population optimization process that consists of a set of initial
values. The optimization process is the process of continuous
reproduction, competition and genetic and mutation of this
population [47].
Cluster Analysis Method The so-called clustering is the
aggregation of a large number of d-dimensional data samples into n classes so that the similarity of samples within
the same class is maximized and the similarity of samples
123
within different classes is minimized. The cluster analysis
method is a heuristic algorithm that is effective in landmark
applications where the number of pattern classes is not precisely known, and it involves grouping (clustering) the data
according to some clustering criterion and interpreting each
data group as the corresponding target class [52, 53].
Logical Template Method The logical template method is
essentially a matching identification method that matches a
predetermined pattern (template) of the system with observation data to match and determine whether the conditions
are satisfied to perform inference [47, 54].
KohonenFeatureMap Kohonen proposes a self-organizing
system based on a network composed of adaptively laterally
interconnected neurons that form a dimensionally simplified
projection of a multidimensional input space [55]. The aforementioned multi-sensor information fusion methods [56], are
shown in Table 2.
The information that needs to be sensed during the
berthing of the USV mainly includes information about
environmental factors, dynamic obstacle information and
static obstacle information [57], and all other factors that
affect the safety of ship navigation. Environmental factors
mainly include meteorological information, hydrological
information, and navigational information. Dynamic obstacles mainly include the navigation of large ships, small
fishing vessels, and large floating objects and the speed,
heading, and real-time position information of these obstacles. Static obstacles mainly include moored ships, islands,
piers, and water structures. There are many shortcomings in
the single sensing device of ships, therefore, the research of
integrated sensing system that fuses multiple sensing devices
and realizes multiple sensing devices to compensate for each
other’s shortcomings is one of the main research directions
at present (Fig. 8). The 3D-aware autonomous berthing system mentioned later is also an important derivative research
direction of this approach.
The fused sensor information is characterized by redundant, complementary, timely, and low-cost information,
which helps to improve the shortcomings of single sensor observation with insufficient information, as well as
the resulting target monitoring, identification, and tracking
defects [58].
With the current social development, various sensors are
updated, multi-sensor information fusion has a better solution, various sensors are more perfectly matched together,
and better and faster access to the data of USV in the course
of travel is the key to USV berthing technology.
Complex & Intelligent Systems
Category
Methods
Randomized
algorithm
Weighted average
method
Kalman filter
method
Least squares method
Maximum
likelihood
estimation
method
AI technology
Structural layer
fusion processing
Expert systems
Neural network
Fuzzy set
theory
Parameter
method
Bayes’ theorem
Recognition
algorithm
Genetic template
method
Perception System
Anemoclinograph
Navigation Radar
AIS
GPS
Electro-Rotary
BDS
MTU(MaximumTransfer-Unit)
Laser Radar
MMW radar
Infrared camera
Sonar
Cluster analysis
method
Feedback on the
current environment
and observe again
Conventional navigation devices
ECDIS
D-S evidence theory
Operation Control
Shore-based Data
Center
Ship-to-shore
communication system
Boat-side Data Center
Autonomous berthing
decision system
Perceptual
Information Fusion
Feedback
autonomy
level
Table 2 Information fusion
methods for multiple sensors
Logical template
method
Kohonen
feature
map
Environment Watch
Environmental
Information Access
Determine if it happens
No
Yes
Decision Action Rules
Transmission decision
related information
Executive Action
Autonomous grade change module
Fig. 9 Autonomous grade link process
Fig. 8 Block diagram of information fusion of multiple sensors
Study of berthing posture assessment method based
on variable autonomy level
Due to the variability and complexity of the marine environment, coupled with the limited sensing capability of the
existing sensors, from time to time there will be raw data that
cannot be processed, and if left unprocessed, adverse consequences such as USV collisions and berthing failures will
occur.
Autonomy is one of the many capabilities possessed
by USV [59]: sensing, observing, analyzing, interacting,
planning, deciding, and executing to accomplish the tasks
delivered by humans. For variable autonomy levels as evident from Table 1 above, dividing the evaluation of unmanned
systems for their autonomous capabilities, the autonomy of
USV under different levels corresponds to different manning
and berthing methods. USV generally have the following four components: observation, judgment, decision, and
action. The paper [60] proposes a berthing posture evaluation
method based on variable autonomy levels. The problematic sensor data, including anomalies such as unidentifiable
dynamic obstacles, the uncertainty of berthing targets, and
missing information about its positioning, are uploaded to
the shore-based server to request manual decisions from the
monitoring personnel. Most of the anomalies can be solved
based on the human–machine interaction information. However, if no manual feedback is received, the USV will increase
its autonomy level (as shown in the flow of the autonomy
level link in Fig. 9 [60]) to ensure its safety by assessing
the current environment accurately in real-time through the
berthing posture assessment method with variable autonomy
level. Selecting the autonomy level of the system. The robustness of the USV berthing can be improved by the berthing
posture assessment method based on variable autonomy levels. The link of the whole variable autonomy level berthing
posture assessment method (Fig. 10).
Improving the efficiency of USV in performing their
tasks, optimizing the structure process of berthing posture
assessment method with variable autonomy level, satisfying
the information requirements of USV system in working, improving the accuracy and versatility of the system,
123
Complex & Intelligent Systems
Environmental Information Fusion
Multi-sensor
Berthing Control
Variable Autonomy Level Decision
Uncertainty Event
Monitoring
No risk for
normal
events
Exception
Event
Control Example
Learning
Berthing Speed and
Heading Control
Berthing Control
Assessment
User Decision
Extraction
Autonomous Hierarchical
Decision Making
Fig. 10 Architecture of multiple example learning and variable autonomy levels for berthing potential awareness
enhancing its ability to handle risky environment and communication, making the USV berthing with better effect and
improving the efficiency of the system.
The situational awareness system of USV is similar to
human eyes, ears, nose, and other senses, which is the
information basis for USV to make autonomous berthing
decisions. With the rapid development of dynamic sensing
fusion, high-performance computing decisions, high broadband information interaction, and high-speed USV platform
technology, the future development trend of intelligent and
unmanned maritime USV has become the industry consensus.
Judgment of USV berthing
The modeling of the berthing process, namely includes global
modeling of the map, target point modeling, environment
modeling, modeling of external natural conditions (wind,
waves, natural objects.), and modeling of the ship itself.
For the modeling of the ship itself, Wang [61] proposed a
4-degree-of-freedom ship maneuvering motion system identification modeling based on support vector machines; Zhang
[62] used virtual reality technology to simulate the state of
the ship 3D model in different scenarios and proposed a study
on the 3D modeling design of the ship based on virtual reality
technology; Ye [63] analyzed and improved the basic genetic
algorithm and used the algorithm to control the parameters
in the ship operation model to obtain a better ship model;
Ruiz et al. [64] studied the maneuvering test in two different water depths, in calm water and regular waves; Li et al.
[65] elaborated on the study of the effect of wind, waves
and tidal currents through Norrbin model on the influence of
ship turning ability by means of Norrbin model. Shi et al.
[66] proposed a regional multi-ship collision risk assessment modeling method based on fuzzy logic considering the
123
influence of ship crossing angle as well as navigation environment, which provides an important basis for the monitoring
of maritime regulatory authorities, crew’s perception of ship
risk and safe berthing of ships.
Multiscale raster modeling method based on electronic
nautical charts
Berthing path planning for an USV is how to find a collisionfree path from a given starting point to a berthing point in
an environment with obstacles so that the USV can safely
and collision-free bypass all obstacles during its motion. The
USV has to build a representation of the obstacles present in
the environment it is in, in its internal world model, before
planning. The first thing to do is to transform the description
of the environment from its original external form through a
series of processes into an internal world model suitable for
planning. A reasonable representation of the environment is
what facilitates the reduction of the search volume in planning and the reduction of the Spatio-temporal overhead. It is
on this different environment modeling that different planning approaches are based. The global path planning of an
unmanned surface boat rasterized by electronic charts [67]
is proposed based on a multi-scale raster modeling approach
(Fig. 11).
Due to a large amount of raw data on the marine environment (including layer data of continents, islands, reefs,
port facilities in electronic charts, data from AIS, navigation Radar, LIDAR, MMW-Radar, photoelectric equipment,
sonar equipment of obstacles. and also need to consider wind
and wave currents and other data), only reasonable environmental data representation can simplify the map model,
reduce the search volume of berthing paths, reduce data storage volume and search time. By analyzing the description
characteristics of electronic nautical charts, a multi-scale
marine environment modeling method is proposed. For
dynamic obstacle data, when modeling the unnavigable area,
the attitude prediction of the obstacle is applied to consider
the trend of its speed and steering, together with the speed
modeling and steering modeling of the dynamic obstacle, to
provide a more reasonable map model for the path search.
In summary, the traditional raster method has the advantages of easy maintenance, high adaptability, and direct
image; however, it cannot solve the direct contradiction
between accuracy, real-time, and time consuming; through
the improved multi-scale and variable-scale raster method
incorporating human physiological perceptual characteristics imitated, the cognitive effect on the surrounding environment can be accomplished more finely, in real-time and
swiftly [68].
Complex & Intelligent Systems
(a)
(b)
(a)General raster modeling method
(b)Different scale raster
(c)Multi-scale raster modeling method
(c)
Fig. 11 Comparison of ordinary raster and multi-scale raster modeling methods
Research on multi-scale modeling methods that consider
complex environmental information
reasonable map model for path search and realize the planning of berthing path with high efficiency.
Before planning, the USV needs to build a map model considering the marine environment and the surrounding obstacles.
The description of the environment is first transformed from
the external raw data through a series of processing to build
an internal world model suitable for optimal path search.
Regarding environment modeling, there is no lack of research
in this area in the last 20 years. The paper [69] discusses that
ship collisions pose a serious threat to navigation safety. An
ordered Probit model is used to fit the state equation showing
the degree of risk. And conclusions were drawn by interpolating the collision risk degree judged by humans and the
ordered probability model; the paper [70] proposed a ship
collision risk assessment method using AIS and historical
accident data. A collision candidate detection model based
on fuzzy quaternion ship domain theory is established; paper
[71] establishes a fuzzy quaternion ship collision field to better describe the risk condition near the bridge area; paper [72]
describes the ship risk assessment and identification field.
The method investigates the collision risk of ships from the
aspect of a single ship. It is concluded that the model of the
shipping field is closely related to the dynamic and static
characteristics of the ship.
Therefore, by analyzing the descriptive characteristics of
electronic nautical charts, a multi-scale-based marine environment modeling method is proposed. For dynamic obstacle
data, when modeling the unnavigable area, the posture prediction of the obstacle is applied to consider the trend of
its speed and steering, together with the speed modeling and
steering modeling of the dynamic obstacle, to provide a more
USV berthing decision
The behavior decision system of the USV is similar to the
human brain, which can comprehensively judge the berthing
situation and make a reasonable berthing behavior decision.
Decision-making is the premise of USV berthing control,
and only with deliberate behavioral decision-making can
the intelligence of USV be reflected. And the agility of
USV berthing action requires the design of a high-precision
berthing decision algorithm.
Collision hazard warning and intelligent collision avoidance
for USV in complex marine environments [73] research
When an USV performs a berthing task autonomously, it
is necessary to model the marine environment around the
USV and plan an optimal route to the berthing target. In
the process of autonomous piloting control, there exist both
negotiated collision avoidance (which requires course adjustment according to the "international maritime navigation
rules" [74], reactive collision avoidance (which will avoid
obstacles and try to approach the current waypoint or the
target point at the best speed and manner, which will not
consider the "international maritime It will take the fastest
speed and direction to avoid the obstacle regardless of the
original course or target point constraints) are not described
in detail here.
Comprehensive research on the impact of marine environmental loads such as strong winds , waves, high currents,
123
Complex & Intelligent Systems
and obstacles (dynamic obstacles such as islands, reefs, and
ships) on the navigation safety of USV, and propose collision hazard warning and intelligent collision avoidance
algorithms for USV; propose to divide collision hazard situations into no need to avoid obstacles, negotiated collision
avoidance situations, dangerous situations, and emergencies,
with emergencies having the highest priority, and through a
three-tier collision avoidance algorithm Improve the berthing
safety of USV in high sea state. Multi-level warning of
potential navigational hazards to achieve USV avoiding all
obstacles during navigation. The paper [75] proposes an
uninterrupted collision-free path planning system that facilitates sampling tasks in complex marine environments and
improves the operational performance of multipleUSV.
Research on autonomous decision-making methods
for USV under dynamic and multivariable scenarios
However, there can be a hundred and one cases, which require
the study of decision-making methods based on variable
autonomy levels. USV face dynamic and changing water
environments, how to ensure the safety of the boat and can
accurately berth to the designated berth, need to study the
autonomous behavior of the USV decision-making methods.
Based on the research of global route planning and intelligent collision avoidance of USV, the decision method based
on variable autonomy level is studied. When the berthing
path planning fails, berthing path tracking fails, and the
output heading speed is abnormal, the USV is required to
make a reasonable next-action decision autonomously and
prompt the shore crew to make an auxiliary decision through
the system. Through manual auxiliary decision-making, the
practicality of the USV in the actual marine environment can
be improved.
To achieve autonomous and safe berthing of the USV
in the dynamic and changing external environment, it is
necessary to be able to plan the optimal berthing path in
real-time according to the changing mission scenarios; and
to avoid dynamic obstacles in the environment and reach the
target point of berthing. The block diagram to realize the
autonomous berthing decision of the USV under dynamic
and variable task scenarios (Fig. 12).
Research on autonomous decision-making methods
for USV based on humanoid intelligent schema [58]
The structure diagram of the USV autonomous berthing control system (Fig. 13) is roughly divided into three rows and
three columns, three rows: motion planning control, information perception and understanding, and knowledge learning
and memory; three columns: central commander level, organization and coordination level, and basic control level. The
"motion planning and control" row is the functional module
123
of the USV to generate various actions; the "information perception and understanding" row is the progressive feedback
perception process of the USV, which compares the expected
value with the actual effect achieved at different levels, and
thus continuously generates It uses various feedback information to re-present the external world, thus forming the
"internal model" of the USV, through which the execution
process of the program can be controlled; "knowledge learning and memory" is the knowledge base of the unmanned
The "knowledge learning memory" is the knowledge base of
the USV at berth, which saves the maneuvering performance
parameters, various feature memories, feature models, control modes, thrust rules, various navigation rules, and various
optimized control parameters.
The idea based on hierarchical progressive control [76]
is mainly reflected in the following three columns: central commander level, organization and coordination level,
and basic control level. The central commander level is
mainly responsible for the coordination and decomposition of motion control tasks, and it is the "brain" of the
unmanned ship berthing, which is the embodiment of intelligence; the organization and coordination level can organize
different functional modules according to the motion tasks
of the higher level, and coordinate the role of each functional module to achieve the purpose of self-organization,
self-adaptation, and self-decision. The basic control level is
the lowest level of ship motion control, which has the highest
control accuracy.
Using an autonomous decision-making method for USV
based on humanoid intelligence schema, we realize the
autonomous decision-making for route planning and collision avoidance maneuvering of USV in dynamic and variable
ocean scenarios. It improves the ability of USV to cope
with dynamic and changing scenarios and then enhances the
robustness and reliability of USV for autonomous berthing
in the actual marine environment.
USV berthing path planning
In the whole berthing process of the USV [77], the path can
be divided into far-end berthing trajectory planning and end
trajectory planning at the near-shore terminal. Good path
planning is the basis for the success of the whole berthing process. The so-called distal berthing trajectory planning refers
to the process that the USV wants to arrive at the target berth
from the far shore. In the whole process of approaching the
berth, the influence of the external environment during the
trajectory needs to be fully considered; the first task in the
trajectory process is how to effectively carry out the path
planning, complete the navigation, and avoidance of collisions and various obstacles (Fig. 14).
Complex & Intelligent Systems
Decision input
model
Course
Speed
Berthing Speed and
heading heading
speed abnorma-li
emergency ties
obstacle
avoidance
Consultati
1 Negotiated Update local
on
routes
obstacle
Dangerou
Heading
2 Reactive
s
speed
obstacle
Pressing
3 Emergency Engine speed, rudder angle,
collision avoidance mode
situation
obstacle
Auto hierarchical decision-making
E. G
Study
Fig. 13 System structure of USV
autonomous berthing system
I nt er ac t i ve
Central Command Level
Pressing situations
Info
Perception
Knowledg
e Learning
Memory
Fig. 14 USV berthing planning schematic
Hull/mission status
Hull motion state info
processing
Emergency obstacle
avoidance
Environment/self state perception
Various sensors
Reactive obstacle
avoidance
Fixed speed
controller
USV
Dangerous situation
Directional
controller
Motion control actuators
Sailing Rules
Negotiated Obstacle
Avoidance
Basic control level
Obtain hull motion control
commands
Route Tracking
Directional fixed speed
coordination
Electronic
chart-based
path planner
Berthing Planning
Motion Planner
control
Organizational coordination level
Route Tracker
Planning
Variable autonomy level decision
Motion
Decision exception monitoring
Local map
No
obstacle
avoidance
ance
Path tracking control
info
Crash Hazard Determination
Variable autonomous level berthing decision
position
Self-positioning
Manipulation-based
performance path
Berthing target
Decision out put
Variable autonomy level berthing decision process
Path planning
failure
Route
Path planning
Hull
Berthing Map
Data
failure
Perform
Planning Model
Berthing path planning
Fig. 12 Block diagram of
autonomous berthing decision
for USV
Database of USV maneuvering performance, feature memory, feature models,
control modalities, inference rules, control parameters, knowledge
representationetc based on multiple example learning methods
Regarding berthing path planning, the dynamic path planning algorithm for unmanned ships based on deep reinforcement learning [78] can continuously improve the learning
ability and the ability to combine information so that a highprecision and the most intelligent path planning approach
can be obtained; the new algorithm obtained by improving
the A* algorithm [79] can achieve obstacle avoidance more
precisely and quickly to complete the collision avoidance
process; the artificial potential field is one of the most common paths planning methods, but its experimental process
also has more or fewer problems appear, fuzzy improvement
of the artificial potential field method [80], to obtain a more
effective planning approach; Ou et al. [81] studied the current
path planning algorithm and sampling-based autonomous
123
Complex & Intelligent Systems
navigation algorithm for autonomous mobile robots. The
paper [82] proposed a combination of water shoreline detection and visual odometry to, achieve accurate positioning of
berths and berthing control. The paper [83] improves the
ship berthing motion control based on improved simulation positioning and mapping algorithm, which improves the
problems of fast speed and high perception accuracy for ship
berthing requirements. Pape [84] proposes an improved network of YOLOv5s-CBAM to solve the real-time sea obstacle
detection problem of autonomous berthing in practical application, which makes accurate berthing at the end of berthing.
The positioning accuracy of the Global Navigation Satellite
System (GNSS) commonly used by ships is susceptible to
environmental impacts, which can lead to threats to the precise positioning of the last 10 m of the ship. To improve the
existing problems, Wei Liu et al. [85] improved the GNSS
vector tracking loop to enhance the USV navigation performance, resulting in an error rate reduction of 85. 4%;
Sakakibara et al. [86] monitored the ship-to-shore distance
and berthing speed by installing a docking sonar system on
the quay to interact with the ship’s forward-looking sonar to
provide an accurate final berthing Perkovic et al. [87] proposed a laser ranging docking system using more accurate
and responsive LIDAR and MMW- Radar, and applied it to
measure the ship’s distance and lateral velocity relative to
the quay; Wang et al. [88] associated the application of 3D
printing into the berthing system, where a 3D laser scanner
accurately captures and constructs the target object’s 3D surface geometry, which in turn allows the 3D laser scanner to
measure the distance from the bow and stern to the quay,
the angle of approach [89] and the speed; Martinsen et al.
[90] used Radar and ultrasonic distance sensors to measure
the distance to the ship’s berth for the next decision; P. Leite
et al. [91] used 3D Radar scanning to extract the geometric
features of the berth and estimate the position and orientation
of the ship relative to the berth to perform accurate berthing
actions.
The paper [92] provides a way to divide the automatic
berthing of a ship into two stages by using a path planning
algorithm according to the berthing rules of the ship: the
first stage is to calculate the rudder angle by calculating and
combining with the data processed by Radar, sonar and other
equipment to drive the ship to the turning point; the second
stage is to adjust the ship’s attitude after the ship reaches the
turning point and then drive the ship into the berth along the
wire. The specific process of automatic berthing is introduced
in the paper when the ship is berthing on the left and right
side of the berth centerline respectively, after reaching the
turning point. The relationship between the distance of the
ship and the berth, the relationship of the heading angle, and
the steering operation required when the ship reaches the
steering point are given in Fig. 15. (Six microwave Radars
are installed on the experimental ship in the paper, and the
123
distance of each Radar from the shoreline is Si, and is the
bow angle).
USV berthing path planning research based
on meteorological data and marine geographic
environment information
The difficulty of the path planning problem [93], the planning
process should consider the influence of the USV’s motion
characteristics and the natural environment in which it is
located (e. g., wind, waves, and currents). Nowadays, there
are studies such as the design of meteorological routes for
ships by A* algorithm [94]. To deal with the more complex marine environment, such as continents, islands, reefs,
and other unnavigable areas, as well as strong currents, large
waves, and other harsh sea conditions (which can have an
impact on the berthing safety of USV), the study of USV
berthing path planning methods based on meteorological
data and marine geographic environment information can
make USV automatically avoid all unnavigable areas in the
planning process and minimize the impact of time-varying
dynamic wind and wave currents on the USV The impact
of time-varying dynamic wind and wave currents on the
navigation of the USV can be minimized. The berthing decision system will start "collision avoidance hazard judgment"
when dynamic obstacles appear on the ferrying path.
USV berthing path planning based on heuristic A*
algorithm [95]
The A* (A-Star) algorithm is an optimal path planning algorithm proposed by Hart et al. [96] and others, which is based
on Dijkstra’s [97] algorithm, a heuristic, a classical graphical algorithm that is widely used to solve route planning
problems [98].
Berthing path planning for USV refers to how to find a
path from a given starting point to a berthing point in an
environment with obstacles so that the USV can safely and
touch-free bypass all obstacles during its movement. The
A* algorithm is applied as the theoretical basis to optimize
the route, and 8 search directions are used as an example
to realize the basic path planning. As shown in Fig. 16 [95]
below, it is a two-dimensional raster model with 8 search
directions, and each hollow circle represents a node. Each
grid point as a node, green dots represent the starting point,
pink dots represent the target point, blue dots represent the
8 search directions, black boxes represent obstacles, and the
three numbers in each square from top to bottom represent
the F value, G value, and H value of this node. The red line
is the final planning path.
Considering the complex and changeable sea conditions,
a path planning method considering wind and waves is proposed based on the A* algorithm, which can optimize the
Complex & Intelligent Systems
1
2
6
left
5
3
4
S3
(b)
S1
S5
S2
S4
Turning point
S6
Steering 䖢ੇാ
circle
Steering circle
Berth Midline
Fig. 15 Schematic diagram of
radar installation, ship berthing
and ship-related quantity
relationship
right
Speed regulating circle
Shore line
Berthing circle
(a)
Smin=S1 S5 S6 Ship Turning right
Smin=S3 S4
Ship Turning left
0° <¢<10°
Ship Turning right
Smin=S2
¢=10°
Ship Stay couse
10° <¢<180°
Ship Turning left
Berthing circle
Warning line
coastline
(c)
(a) Radar installation of the experimental vessel
(b) The ship is in the berth center line, the berthing process on the left and right
side respectively
(c) The relationship between the distance of the ship and the berth, the
relationship between the heading angle and the required steering operation
berthing route of the USV in real-time, so that it can avoid
the area where large wind and waves may be expected to
occur, and derive a navigation path with the shortest time
consumption while guaranteeing the safety of the USV. After
the weather information is imported, the wind and wave realities at each grid point can be known, so the weather influence
factor is added to the value function of the A* algorithm, and
according to the different weather realities at each grid point,
its influence on the ship’s speed is different, and the ship can
lead to serious stall conditions in a windy and rough environment, so the path is planned using the A* algorithm to select
a sailing path with the shortest The route is planned by using
A* algorithm to select a route with the shortest sailing time,
and at the same time effectively avoiding the area with high
wind and waves.
This algorithm, through multiple data calculations,
obtains the best heading from the departure point to the
target point, so that the ship can obtain an effective and reasonable route when traveling. Furthermore, the algorithm is
combined with the electronic chart and the influence of the
changes of wind and wave currents, so that the route can be
further and dynamically judged, to make the docking of the
ship more intelligent, and the combination of this method
with more external data may also be a research direction in
the future.
Berthing path planning considering USV maneuvering
constraints and environmental disturbances
Based on the nonlinear mathematical model of USV maneuvering motion, the effects of wind pressure and wind pressure
moment, tidal current, and other disturbing forces on the
maneuvering motion of USV are studied. By parameterizing
the maneuvering data such as the minimum slew diameter,
tactical diameter, longitudinal distance and inverse transverse
distance of the USV, the maneuvering performance of the
USV can be quantified so that the planned path is consistent with the maneuvering motion characteristics of the USV.
When berthing, the general speed is low, and the USV heading is easily disturbed by wind and wave currents, and the
factors of environmental disturbance need to be fully considered when planning the berthing path (Fig. 17).
Due to the interference of the marine environment, when
berthing, the USV needs to choose the direction with the least
influence of wind and wave currents for berthing, generally,
the top wind mode is chosen for berthing, which is beneficial
for the USV to control the heading, as shown in Fig. 18
Under the north wind conditions, the USV needs to turn to
the north direction for berthing operations, to allow the USV
to autonomously choose the best direction for berthing, the
method of adding virtual berthing obstacles in the map is
123
Complex & Intelligent Systems
Fig. 16 Path planning method
considering wind and waves
Target Points
Starting Point
Selected Points
Abandon Points
Best Path
Barrier
(a) Shortest path planning without consideration of wind and waves
Windy area
Target Points
Starting Point
Selected Points
Abandon Points
Best Path
Barrier
(b) Path planning considering wind and wave effects
proposed, using the previous raster modeling and A* search
algorithm, which is to realize the autonomous choice of
berthing direction.
Binding
Map modeling
range
Meteorological and
hydrological
information
The best route
to reach the
target point
Estimated arrival
time
Smooth path planning for USV based on an ant colony
algorithm
In the path planning before berthing, how to accurately and
precisely navigate the USV from the complex environment,
overcome the problems of large steering angle, many paths
inflection points, the high energy consumption of navigation,
and finally navigate to the target berth. A comprehensive
study of the improvement of the ant colony algorithm and
the problems in the algorithm. In the path planning before
berthing, how to accurately and precisely navigate the USV
from the complex environment, overcome the problems of
large steering angle, many path turning points, and high
123
Route
Avoiding
obstacles
Mooring
Posture
Berthing
optimization
algorithm
USV Mathematical
model of motion
Shortest path
planning
Maximum
navigation safety
Fig. 17 Berthing path planning considering multi-objective constraints
energy consumption of navigation, and finally navigate to
the target berth. A comprehensive study of the improvement
of the ant colony algorithm and the problems in the algorithm.
Complex & Intelligent Systems
Synthesis
direction
PB2
DR
PE
P
E
Berthing
position
Mooring
Planning point
(a) North wind berthing path planning
Mooring
Planning point
Synthesis
direction
Berthing
position
PB2
2
DR
has better effectiveness and more stable convergence after
simulation tests.
The paper [104] is an USV smooth path planning method
based on the ant colony algorithm. The method is exemplified by its use of the raster method for environment
modeling; setting the initial point information; through calculation, deriving the transfer probability to proceed to the
next step; updating the local pheromone; judging the correctness or incorrectness of reaching the target raster; deriving
the current optimal path and performing global information
update; judging whether the maximum number of iterations
is reached, and finally obtaining the optimal path through
post-processing, and performing path smoothing The optimal path is finally obtained through post-processing, and path
smoothing is performed. The changed ant colony algorithm
can effectively improve the problems of the traditional ant
colony algorithm, improve the efficiency of path planning;
the ability of path search; and the effect of path smoothing
processing (see Fig. 19).
The key technology of USV autonomous
berthing control
PE
P
E
(b) South wind berthing path planning
Figure 18 Schematic diagram of berthing path planning under different
environmental influences
Fig. 18 Schematic diagram of berthing path planning under different
environmental influences
Paper [99] on fusing particle swarm and ant colony
algorithms to obtain a new path planning method that can substantially improve the initial path finding efficiency as well
as the global search capability, thus significantly and effectively reducing the preliminary time for path planning; paper
[100] by fusing A* algorithm and wolf swarm mechanism,
thus improving the ant colony algorithm, which makes excellent reliability and convergence; papers [101, 102] improve
the algorithm with better global search capability and convergence by extensively collecting important factors such as
distance traveled, road smoothness, elevation gradient, and
steepness during the travel path and a newly designed information update mechanism. Paper [103] by combining the ant
colony algorithm with the Bayesian network and introducing the maximum association length parameter, the method
For the autonomous berthing problem of this USV, there are
five main research aspects: USV berthing situation sensing and evaluation under complex sea conditions, USV
autonomous berthing decision, intelligent berthing control
algorithm, variable autonomous level decision, and USV
autonomous berthing test validation (Fig. 20).
As an intelligent mobile body on the water’s surface,
an USV can be regarded as a kind of robotic system that
can sense the external environment through various sensor
systems to realize autonomous movement and accomplish
certain tasks in the complex and unknown marine environment. Since the actual marine environment is dynamic,
uncertain, and complex, the traditional control methods and
the architecture based on the cognitive model have been difficult to meet theoperational requirements of the USV in the
complex marine environment. The internal structure of the
USV berthing control system (Fig. 21).
The control of the hull is also an aspect worth considering
during the berthing trajectory of the ship, and the instability
of the hull itself can cause the whole trajectory to face problems. This leads to more or fewer difficulties. The constrained
nature of the ship can lead to the navigation of the ship and
may cause significant safety problems. By using a nonlinear genetic algorithm [105, 106], the small local problems
are optimized and improved; facing the collision avoidance
path problem of the ship in the course of navigation [107],
the changes in the ship load and navigation environment
are added to be considered thus improving the maneuverability of the ship; the influence of the natural environment
123
Complex & Intelligent Systems
parameter optimization method, plus in the actual marine
environment, the hull slew index may change rapidly and the
change magnitude is relatively large, which may lead to system instability, so better control design methods are needed
to ensure the global stability of the system.
Start
Raster Method
Environmental
Modeling
Set initial point
Target point
information parameters
Design of berthing controller for USV using
model-referenced adaptive method
Placement of initial points
Calculate the transfer
probability and proceed to
the next step
Local pheromone
update
NO
Calculate the transfer
probability
Proceed to the next
step
NO
Whether to reach
Next target grid
YES
Find the current
optimal path, global
pheromone update
Whether to reach
Maximum number of
iterations
Post-processing to output the
best path
Smoothing processing
END
Fig. 19 Smooth path planning method of USV based on ant colony
algorithm
on the ship navigation is also obvious, and the paper [95]
considers the complex meteorological changes to improve
the intelligent ship dynamic route planning to improve the
maneuverability of the ship; using a single ship group and
multiple ship groups, loading the conditions constrained with
the ship navigation to reduce the limitations arising from the
maneuvering, thus obtaining excellent route planning functions [108].
The slewing performance of the hull of the small USV is
good and easily affected by wind and wave currents; in terms
of the power system, its slewing index will change with the
change in engine speed. In other words, the USV system is a
complex system with time-varying, non-linear and vulnerable to disturbance. It is difficult to ensure the global stability
of the USV bow adaptive system designed by using the local
123
Model-referenced adaptive control techniques have been
widely used in aircraft autopilots, ship autopilot systems,
photoelectric tracking telescope follow-on systems, siliconcontrolled speed control systems, and robotic control systems. The basic principle of using the model-referenced
approach to design an adaptive controller is that the control
parameters are modified according to some function criterion
of the difference between the actual output of the controlled
object and the output of the reference model, trying to make
the generalized error between the actual output of the controlled object and the output of the reference model tend to
zero so that the controlled system achieves or approaches the
desired dynamic behavior. The principle structure of a typical
model-referenced adaptive control system (Fig. 22).
In the model-referenced adaptive control system, the ideal
reference model represents the designer’s desired dynamic
characteristics of the actual system. When external conditions change or disturbances occur, the dynamic response
of the controlled object also changes accordingly, deviating
from the designer’s desired dynamic response characteristics.
To compensate for the impact of the external environment or
other disturbances on the actual system, an adaptive mechanism needs to be designed to adjust the parameters of the
adjustable system to reduce the error between the response
of the reference model and the response of the actual system
and the response of the ideal model.
Model-referenced adaptive berthing control
algorithm based on Lyapunov stability theory [109]
According to the designed adaptive control law of USV heading and speed, an auxiliary signal generator is introduced,
which constitutes a feedback gain adaptive mechanism, and
together with the feedforward gain adaptive mechanism, it
constitutes the whole adaptive mechanism and then forms the
model reference-based adaptive USV heading controller and
speed controller, through the continuous learning of the controller and optimization of the control parameters, and finally
achieves that regardless of the actual USV Hongde Qin et al.
[110] combined Liapunov stability theory, back-stepping
method, and tan-type BLF to obtain a control algorithm with
smaller error.
Complex & Intelligent Systems
Fig. 20 Technology roadmap to
be adopted for the study of
autonomous berthing control
methods for USV
Control objects and
existing problems
Research methods
Berthing situational awareness, assessment
Uncertain events in the
berthing process. USV
systems are underdriven,
strongly nonlinear, inertialetc
USV Hull
Motion actuator
with time lag,
Electric
non-linearity,
wind, waves,
Propeller
Currents and
other disturrudder
bancesuncertainty
Marine Environment
Complexity
Ports, docks
Hull
Control
Environment
model
Berthing
planning
Research content
Path tracing Collision avoidance Berthing control
control
sensor
Oars, rudders, hulls
Port
environment
Fig. 21 Internal structure of USV berthing control system
USV reference model
Feed-forward
regulator
+
-
Feedback regulator
Parameter adjustment
Adaptive institutions
Fig. 22 Schematic structure of model reference adaptive control
Automatic berthing control method based
on the artificial neural network [111]
Artificial Neural Networks (ANN) [112], is a research
hotspot that has emerged in the field of artificial intelligence
since the 1980s. The use of ANN in intelligent combination
with other theories has received a great deal of attention in the
Situational Awareness
Berthing paths
Planning and Collision
Avoidance
Heuristic A* algorithm, considering the
maneuvering performance of USV, based
on three-layer collision hazard
The model references the adaptive method
Berthing Control
Algorithm
Model-referenced adaptive berthing
cont-rol algorithm based on Lyapunov
stabili-ty theory
Multi-example
learning method
Monitoring method of abnormal events
based on event triggering mechanism,
support vector machine based method
Variable autonomy
hierarchy
Architecture research based on multiple
example learning and variable autonomy
levels
marine world. FL (Fuzzy logic) and artificial neural networks
have been proposed, discussed, and evaluated in conjunction with each other intelligently as "neuro-fuzzy autopilots"
[113–115].
Artificial neural networks abstract the network of human
brain neurons from the perspective of information processing and build some kind of simple model to form different
networks according to different connections. An artificial
neural network is a nonlinear, adaptive information processing system composed of a large number of interconnected
processing units. It is proposed based on modern neuroscience research results and tries to process information by
simulating the way the brain’s neural network processes and
remembers information. The experimental results of artificial neural networks [116] have four basic characteristics:
nonlinear, nonlimited, nonconstant, and nonconvex. Thus,
the neural network is an obvious breakthrough for solving
the problems faced by USV control and travel path: nonlinearity, unknown model design, and weak anti-interference
ability. The ship first collects the real-time dynamic environment outside the path of travel, and after data processing
by the neural network, passes and calculates the network
weights and deviations through the BP neural network algorithm [117], followed by the mathematical model of the ship
berthing process established. Of course, if the initial real-time
variables are input, different output algorithms for automatic
berthing control will be derived. This design process (Fig. 23)
[118]. In the paper [119], by improving the problem of large
deviation of the ship’s heading from the shoreline at the end
of berthing of the previous ANN controller, some data from
the berthing process is used as controller input instead of
extracting all information from the berthing process, as a
123
Complex & Intelligent Systems
Training
data
Initial
berthing
conditions
Servo
features
BP neural
algorithm
Oar
characteristics
Ship action
model
Error
discrimination
Automatic
berthing
output
Weigh
bias
The captain is
experienced in
control
Vision servo-based method for autonomous
berthing of USV [124]
Fig. 23 Neural network berthing control process
Point cloud data
Image
CNN(Convolutional Neural Networks)
Image Feature extraction
Data preprocessing
Feature fusion of spatially continuous
convolutional networks
CNN
Point clouds
Feature extraction
Detection generates network
3D detection frame object
recognition and localization
Detection generates a 2D detection
frame for target recognition and
localization
Fig. 24 Construction of the perception algorithm
way to improve the training of the controller and thus obtain
excellent berthing results. However, the neural network control algorithm, which needs to pass a large number of training
and the difficulty ensuring real-time in the control process,
makes the neural network control method needs more practice [120].
Deep learning-based 3D perception algorithm
for autonomous berthing applications [121]
The 3D perception algorithm is a deep fusion based on combining multi-sensor point cloud detection technology [122],
image data detection technology [102], and sensor data detection technology [123] to improve the accuracy of target
detection and perception, which is applied to find a suitable berthing location. Accurate environment perception and
scene construction are performed in deep learning by cameras, Radar, sonar, and GPS, which are applied to the ship to
enable fast detection and discrimination of the surrounding
environment and improve the rate and accuracy of berthing.
The construction of the perception algorithm is shown in
Fig. 24, and the expected accuracy is obtained through its
architecture with the ship control scenario and the situational
awareness network, and finally based on the autonomous
123
berthing control system, it is successfully applied in the actual
berthing test, where the data and time cannot be accurately
matched due to the low efficiency of the algorithm operation,
which will make the error of the unmanned ship slewing.
However, the test verified the reliability and practicality of
the method, and the 3D perception algorithm based on deep
learning will become a new research direction and idea for
autonomous berthing.
Vision servo, in general, refers to the behavior of automatically receiving and processing images of real objects through
optical devices and non-contact sensors, and through image
feedback information, the machine system can perform further control or corresponding adaptive adjustment behavior
of the machine. The simulated scenes of the berths docked are
obtained on the unmanned surface boat by the vision acquisition system on board, from which the markers of the target
berth are extracted as tracking objects. The distance and orientation between the USV and the target berth are judged by
the size of the marker image, the different positions of the
geometric center point of the image in the vision system, and
the change of the marker image shape during the motion of
the USV, and a route is simulated and planned to guide the
USV to complete the autonomous berthing task. When the
markers of the target berth are easily disturbed by the outside
world, the berthing process will be affected and the berthing
and path planning will not be accurate. However, this will
become an important direction for future research.
Model reference-based adaptive berthing control
To achieve high-precision berthing control, the control input
required to bring the tracking error to zero is calculated by
comparing the error that exists between the berthing path
point and the current USV motion information and the trend
of the error change using a model-referenced adaptive control
algorithm. Through a large number of berthing control practices, the controller will continuously optimize the control
parameters to form a high-precision adaptive berthing control system suitable for this USV. To cope with the sudden
control failure problem, the USV berthing control architecture with variable autonomy level is proposed to improve the
reliability of the USV berthing navigation. The block diagram of the model reference-based adaptive berthing control
(Fig. 25).
The basic principle of using the model reference method to
design an adaptive controller is that the control parameters
are modified according to some functional criterion of the
difference between the actual output of the controlled object
and the output of the reference model, trying to make the
Complex & Intelligent Systems
Fig. 25 Block diagram of
berthing control based on model
reference adaption
Auto hierarchical
Interactive decision-making
E. g learning
Control abnmonitoring
Speed
USV Embedded control system
+
Course
+
-
Theberthing speed
model refers to the
adaptive controller
Engine throttle
control
+
Theberthing course
model refers to the
adaptive controller
-
Rudder angle
controller
motor
Electro-hydraulic servo
Rudder angle
Angle sensor
Wind
Wave
Flow
Marine disturbances
generalized error between the actual output of the controlled
object and the output of the reference model converge to
zero so that the controlled system achieves or approaches the
desired dynamic behavior.
Combining all these functions in one package gives a perfect berthing system. Therefore, future research still needs
to explore the ability to improve the anti-disturbance capability and deal with nonlinear problems, to improve stability
and swiftness when responding to problems, to be resilient
in the face of different and changing dynamic environments,
to completely disengage from human or assistance vessel
participation.
Opportunities and challenges for USV
berthing technology
There is still a big difference between theory and practice,
the actual navigation environment is complex and changeable, and there are various problems; there is a big difference
between artificial intelligence, computer programming, and
human brain intelligence. Therefore, USV berthing technology also faces many challenges and opportunities. The past
research work is compared with future work as shown in
Table 3.
(1) Control algorithms, as an important development direction for unmanned navigators, to improve operational
Haiteng 01 USV
Table 3 Past research work comparison with future work
Past research work
Future work
Exploration of the berthing system
of unmanned boats
Control algorithm
development
Multi-faceted study of
complex environments
Exploration of relevant berthing
methods for unmanned boats
Software and hardware
coordination of unmanned
boats
An in-depth exploration of
ship models
Exploration of key berthing
technologies related to unmanned
boats
Collaboration of multiple
algorithms
Learn from the past and
develop the future
effectiveness and achieve tasks that cannot be accomplished by a single navigator, will have great potential
for application in military and civilian fields [125].
(2) For environmental perception in complex scenarios, the
complex sea state environment is divided into static and
dynamic environments. The static environment includes
channel environment; berth; signal position; navigation
facilities. the dynamic environment includes incoming
and outgoing docked ships; sudden celestial events, and
meteorological environment. At present, these are only
carried out under simple experiments, for this complex
123
Complex & Intelligent Systems
(3)
(4)
(5)
(6)
environment navigation or have limitations. Therefore,
there is still a long way to go for direct input into life,
production, and the military.
The processing of big data, it is also a major difficulty
for the USV to berth. Need to enhance the ability to
handle the four key aspects of the USV observation,
judgment, decision-making, and action. Therefore, the
interactivity and synergy of hardware and software on
the USV is also an important link, which also needs a
long time to explore.
The uncertainty of the ship model [111], has a potential
impact on the berthing of the ship due to its nonlinearity
during the travel path. The current common modeling
methods are computational fluid dynamics simulation
method, ship empirical formula method. All of the above
methods have their advantages and disadvantages. For
the accurate modeling of the ship and the model processing of the real-time dynamics of the ship, it will also
become an important research direction for the research
in the future.
Algorithm research should be carried out on the cooperative work of multiple unmanned surface boats [126],
so that the boats can have the advantages of multi-ship
cooperative work.
Make full use of the existing theoretical, technical basis,
and simulation test results to avoid unnecessary duplication of work, resulting in the waste of human, material,
and financial resources. Achieve high efficiency and
high results [127].
Funding This work was funded by National Engineering Research
Center of Ship & Shipping Control System and then by National Natural Science Foundation of China to Gongxing Wu with grant number
52271322.
Declarations
Conflict of interest The authors declare that they have no conflict of
interests.
Open Access This article is licensed under a Creative Commons
Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as
long as you give appropriate credit to the original author(s) and the
source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material
in this article are included in the article’s Creative Commons licence,
unless indicated otherwise in a credit line to the material. If material
is not included in the article’s Creative Commons licence and your
intended use is not permitted by statutory regulation or exceeds the
permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecomm
ons.org/licenses/by/4.0/.
123
References
1. Wahlström M, Hakulinen J, Karvonen H, Lindborg I (2015)
Human factors challenges in unmanned ship operations—insights
from other domains. Proc Manuf 3:1038–1045
2. Smalley D (2014) The future is now: navy’s autonomous swarmboats can overwhelm adversaries. ONR Press Release
3. Wang L, Wu Q, Liu JL, Li SJ, Negenborn RR (2019) State-of-theart research on motion control of maritime autonomous surface
ships. J Mar Sci Eng 7(12):438
4. Meijuan Z (2021) Research on nonlinear automatic berthing
control of underactuated surface vessels. Shandong Jiaotong University, Berlin
5. Bui P, Kim Y-B (2011) Development of constrained control allocation for ship berthing by using autonomous tugboats. Int J Control
Autom Syst 9(6):1203–1208
6. Mizuno N, Uchida Y, Okazaki T (2015) Quasi real-time optimal control scheme for automatic berthing. IFAC-PapersOnLine.
48:305–312
7. Woo J, Kim N (2016) Design of guidance law for docking of
unmanned surface vehicle. J Ocean Eng Technol 30:208–213
8. Park J-Y, Kim N (2014) Design of an adaptive backstepping controller for auto-berthing a cruise ship under wind loads. Int J Naval
Arch Ocean Eng 6(2):347–360
9. Weibin Z (2017) Study on autonomous berthing methods of
unmanned surface vehicles. Harbin Engineering University
10. Yupeng J, Yong Y, Zhongxian Z (2020) Research on autonomous
berthing for unmanned ship based on berth coordinates. J Syst
Simul 32(10):1910–1917
11. Yesheng H, Zhengpan X, Yuanming C, Qi P, Xiaobin H (2020)
Design of automatic berthing control system of USV. China Meas
Test 46(10):111–117
12. Yubo W, Jianhua W, Xiang Z, Ju F, Hongyan J, Zhuya W (2022)
Autonomous berthing method for unmanned surface vessel based
on berth shoreline detection. Ship Eng 44(4):30–37
13. Du Z, Reppa V, Negenborn RR (2020) Cooperative control of autonomous tugs for ship towing. IFAC-PapersOnLine.
53:14470–14475
14. Xiaodong Z, Shiliang L, Yu L, Xiaofang H, Chao G (2018) Review
on development trend of launch and recovery technology for USV.
Chin J Ship Res 13(6):50–57
15. Yan P, Lei G, Xiaomao L, Yuxua Z, Xin Z (2019) Research status
and development trend of unmanned surface vehicle. J Shanghai
Univ (Natural Science) 25(5):645–654
16. Yulei L (2012) Nonlinear motion control methods of unmanned
surface vehicle. Harbin Engineering University
17. Huang Y (2015) Modeling and simulation method of the emergency response systems based on OODA. Knowl-Based Syst
89:527–540
18. Fanming Z, Shjie C (2016) Design and Trial research on the
autonomous control system of unmanned surface vehicle. Shipbuild China 57(02):164–174
19. Zhou X, Liu Z, Wu Z, Wang F (2019) Quantitative processing of
situation awareness for autonomous ships navigation. TransNav
Int J Mar Navig Saf Sea Transport 13:25–31
20. Thieme C, Utne I, Haugen S (2018) Assessing ship risk model
applicability to marine autonomous surface ships. Ocean Eng
165(Oct 1):140–154
21. Sotnikova MV (2018) Control system design for visual positioning of a ship based on NMPC and multi-objective structure.
IFAC-PapersOnLine. 51(32):445–450
22. Wu Y, Chu X, Deng L, Lei J, He W, Królczyk G, Li Z (2022)
A new multi-sensor fusion approach for integrated ship motion
perception in inland waterways. Measurement 200:111630