Robotic Ship Hull Inspection For Damage Repair
Luke Fitzgerald, ANTHONY WEIR, Dinesh Babu Duraibabu, EDIN OMERDIC, GERARD DOOLY, DANIEL
TOAL
Publication date
23-01-2024
Published in
OCEANS 2022, pp. 1-5
Licence
This work is made available under the CC BY-NC-SA 4.0 licence and should only be used in accordance with
that licence. For more information on the specific terms, consult the repository record for this item.
Document Version
1
Citation for this work (HarvardUL)
Fitzgerald, L., WEIR, A., Duraibabu, D.B., OMERDIC, E., DOOLY, G.and TOAL, D. (2024) ‘Robotic Ship Hull
Inspection For Damage Repair’, available: https://doi.org/10.34961/researchrepository-ul.25046582.v1.
This work was downloaded from the University of Limerick research repository.
For more information on this work, the University of Limerick research repository or to report an issue, you can
contact the repository administrators at ir@ul.ie. If you feel that this work breaches copyright, please provide
details and we will remove access to the work immediately while we investigate your claim.
Robotic Ship Hull Inspection For Damage Repair
*Luke FitzGerald, Anthony Weir, Dinesh Babu Duraibabu, Edin Omerdic, Gerard Dooly, Daniel Toal
Centre for Robotics & Intelligent Systems
University of Limerick
Limerick, Ireland
luke.fitzgerald@ul.ie
Abstract— In this paper, we investigate the methods to be used
in the inspection of damaged ship hulls in water at sea, as part of
the European Union (EU) Robotic Survey, Repair and Agile
Manufacture (RESURGAM) Project. Employing underwater
robotics equipped with camera or laser sensors, along with robust
control and positioning systems, 3D models of the damaged
section(s) can be produced with a goal of providing sufficient
information for the subsequent design of repair patch(es), and
ultimately the application of this repair to the ship. This work will
allow damaged ships at sea to seek repair without the need for
access to a dry dock. Preliminary results from recent field
operations are included in this paper, with further field research
to be carried out later this year.
II. EXPERIMENT & ROBOT SETUP
To simulate the structure of a damaged ship hull, a steel
plate with an appropriately damaged area was fabricated and
submerged in the test site quarry. The perimeter of the plate is
248cm by 152 cm.
Keywords—underwater inspection, laser scanning, optical
imaging, marine robotics
I. INTRODUCTION
Many of the challenges faced by small and medium sized
EU shipyards can be addressed by improving their productivity
for fabricating new, high technology vessels and increasing
their access to the specialist repair and maintenance market, the
RESURGAM project aims to address these challenges[1].
Friction Stir Welding (FSW) is a high integrity, low distortion,
environmentally benign, welding technique. Traditionally, it
has only been possible to use FSW in aluminium[2]. A recent
break-through in the tooling available for this technique shows
the potential to enable this process for welding of steel
structures, representing a huge opportunity to improve the
productivity of shipyards[3]–[5]. Using this method, with the
assistance of computer vision systems and underwater robotics,
ships can potentially be repaired at sea without the need for
access to a suitable dry dock which may be a considerable
distance away. Repair of damaged ships after transiting to
suitable dry docks can be very costly and inefficient with long
periods spent on manually intensive survey and repair
activities, exacerbating the loss of revenue whilst the ship is not
fulfilling its purpose. Considering this, it is clear that the
proposed streamlined approach will be of great benefit,
allowing for rapid survey, repair and maintenance of ships in
water.
Fig. 1: Steel plate submerged in quarry at test site
The inspection was performed with imaging sensors
mounted on a Remotely Operated Vehicle (ROV). ROV Étaín
(modified ROV Comanche by Sub-Atlantic) is a light-weight
work class ROV with a depth rating of 2000m, and is deployed
with a Tether Management System (TMS) allowing it to
perform deep operations.
This paper is structured as follows. The second section
details the experiment setup and equipment used to carry out
the inspection, section 3 outlines the acquisition of data,
resulting models of the damaged section are displayed in
section 4, and in the last section we conclude and discuss future
goals.
Fig. 2: ROV Étaín at test site
Prior to inspection, a cleaning operation is necessary in
order to allow for an accurate scan free of marine growth, rust,
etc. Two pieces of equipment were used for this. First, a
FlexiClean Subsea 7[6] flexible abrasive brush cleaning tool
was integrated with the ROV, mounted on one of the ROV
Schilling Orion Manipulators and powered from onboard HPU
pack. This tool is designed to remove stubborn marine growth
from subsea installed hardware and structures whilst leaving the
original hardware surfaces unscathed. Next, a high-pressure
water jetting tool that employs the cavitation phenomenon is
similarly used to clear the surface of the target after the brush
cleaning. Again, this is mounted on a Schilling ROV
Manipulator and powered from onboard HPU. Hull cleaning is
an operation carried out regularly at sea by industry so these
operations can be considered as standard.
Fig. 5: Water jetting tool integrated with ROV
Fig. 6: Water jetting tool in operation
Fig. 3: FlexiClean Subsea 7 integrated with ROV
For laser scanning, a Voyis (formerly 2G Robotics) Insight
Pro (formerly ULS-500 PRO)[7] was integrated with the ROV
and used to capture real-time point cloud data. This system
offers a maximum range of over 20m with a 4000m depth
rating. The laser scanner uses a laser plane generator and a
subsea camera sensor to build up a 3D point cloud model of the
target. It emits a laser swath that projects a line onto the target’s
surface, and these laser line reflections are captured by the
camera with data points being calculated along the laser line,
generating a laser profile which is a single line of points in 3D
space. This data is transmitted to the surface through a subsea
cable, then through a paired surface control unit, after which it
is displayed to the user.
Fig. 4: FlexiClean Subsea 7 in operation
Fig. 7: ULS-500 PRO laser system integrated with ROV
For optical imaging, a single FLIR Blackfly GigE Vision
Camera (BFLY-PGE-23S6C-C)[8] was used. It has a global
shutter Sony IMX249 1/1.2” CMOS Sensor, with a resolution
of 2.3MP, equipped with a Kowa LM6HC wide angle lens,
enclosed in an underwater housing rated to a water depth of
1000m. The camera relies on Power over Ethernet (PoE),
injected through the Gigabit Ethernet lines onboard the ROV.
This was mounted on the Sub-Atlantic Pan & Tilt Unit of the
ROV, allowing for easy adjustment when needed.
Laser data is best captured in dark environments. As
explained in section 2, the system operates by emitting a laser
swath, and then capturing the reflections through a camera.
Other sources of light acting in the same area will introduce
noise to the captured dataset. Given the fact that this damaged
ship survey operation must be carried out near the surface of the
water, daylight can be a problem.
Fig. 9: ROV laser scanning
Fig. 8: Vision camera in underwater housing mounted on ROV
The primary navigation system installed on the ROV is
PHINS 6000 with the following aiding sensors: GNSS (for
surface operations), Nortek DVL 500kHz, CTD/Pressure
(Depth).
III. ACQUIRING DATA
A. Laser Scanning
QINSy (Quality Integrated Navigation System)[9] was used
for laser data acquisition and processing. This software is a
suite of applications that can be used for various types of
surveys, ranging from simple single beam surveys to complex
offshore construction works. All systems (INS, USBL, GPS,
Multibeam, Laser, etc.) are interfaced with QINSy for data
acquisition, data processing and QA & QC of the data acquired.
Prior to acquiring data, mounting angle errors (roll, pitch
and heading) of the laser scanner must be determined through
calibration. A latency check is carried out by sailing a line twice
at differing speeds. If there is a latency issue, there will be a
mismatch in object positions when comparing the two lines.
Patch testing for roll, pitch and heading is carried out as
follows: for roll, two lines over a flat area in opposite directions
at the same speed, for pitch, two lines over an area with slopes
or objects in opposite direction at the same speed (in both of
these cases, the transducer tracks need to be aligned), for
heading, two lines over an area with slopes or objects in the
same direction at the same speed. In the case of heading, the
lines need to overlap half a swath width[10].
Fig. 10: Laser scanning view from ROV camera
B. Camera Scanning
The image capturing process was completed through
control of the ROV using the OceanRINGS software developed
within the Centre for Robotics and Intelligent Systems research
group[11], [12]. 3D reconstruction and photogrammetry
methods require overlap between consecutive images to allow
for the tracking of image features. To guarantee this overlap,
using the FlyCap2 software, one frame was captured for every
second of the camera survey. This resulted in a high quality
dataset of 1920x1200 pixel images covering all of the necessary
angles needed to reconstruct a realistic view of the steel plate
fracture.
In the process of capturing both laser and camera data, the
use of the OceanRINGS control software was crucial. This
allowed for smooth and stable autonomous point to point
movements, as a result of real time adjustments based on input
from the various sensors on board the ROV.
IV. RESULTS
A. Laser Scanning
While laser data was being acquired, a real-time 3D point
cloud was generated using the QINSy 3D Point Cloud Viewer
for quality assurance and control purposes. If there were errors
in positional information, or if there was too much ambient light
interfering in the scene, it was clearly visible at this stage.
B. Camera Scanning
The camera data was processed using the Agisoft
Metashape software[13]. 245 of the captured images were
selected for the photogrammetry process. This workflow began
with aligning the images, inferring their position to one another
in a local coordinate frame and generating a sparse point cloud.
At this stage, preliminary cleaning was carried out to remove
outlier/unwanted points, and any additional data unrelated to
the steel plate, leaving a total of 55,619 points. Next, a dense
point cloud was generated along with further cleaning to leave
only the plate structure, resulting in a total of 2,300,267 points
of the dense cloud. Lastly, a 3D mesh of the plate was created,
along with a texture generated from the image dataset, giving a
visually realistic model. See Fig. 12 and Fig. 14.
Fig. 11: Real-time display of laser scanning data
It can be seen in the above figure that while the steel plate
is easily identifiable, there were still some scanning artefacts
due to noise, multiple reflections, etc. The generated point
cloud data thus requires further post processing. Basic cleaning
of the point cloud was performed to remove the erroneous
points, the resulting model is shown in Fig. 13.
Fig. 12: Front view of damaged steel plate model
Fig. 13: Laser scan model after cleaning
Fig. 12: Angled side views of steel plate model
REFERENCES
V. CONCLUSION
This paper presented preliminary results of underwater ship
hull inspection demo field trials performed by the Centre for
Robotics and Intelligent team, University of Limerick, in the
period 18th – 22nd July 2022 in Portroe quarry, Ireland as part of
the RESURGAM project. As described, the work consisted of
initially cleaning the simulated damaged ship hull section,
performing laser and camera scanning, and finally post
processing the captured data to produce 3D models. To enable
the FSW system’s deployment and repair procedure, it is
necessary to identify and understand the type and degree of
damage sustained. The inspection process outlined in this paper
will influence the planning and execution of the repair.
In terms of future work, further testing of this process will
be carried out in December 2022 on board a ship for a realistic
environment. Issues confronted in these initial tests will be
ironed out resulting in a more streamlined procedure.
[1]
[2]
[3]
[4]
[5]
[6]
[7]
[8]
ACKNOWLEDGMENT
This work has received funding from the European Union’s
Horizon 2020 Research and Innovation programme under
Grant Agreement No. 101007005, and a research grant from
Science Foundation Ireland (SFI) through the National Centre
for Marine and Renewable Energy Ireland (MaREI) Phase 2
(Grant No. SFI/12/RC/2302_P2).
[9]
[10]
[11]
[12]
[13]
“ROBOTIC SURVEY, REPAIR & AGILE MANUFACTURE |
RESURGAM Project | Fact Sheet | H2020 | CORDIS | European
Commission.” https://cordis.europa.eu/project/id/101007005 (accessed
Sep. 05, 2022).
D. G. Mohan and C. Wu, “A Review on Friction Stir Welding of Steels,”
Chin. J. Mech. Eng., vol. 34, no. 1, p. 137, Dec. 2021, doi:
10.1186/s10033-021-00655-3.
S. Campbell, “A comparison between single sided and double sided
friction stir welded 8mm thick DH36 steel plate,” p. 7.
N. A. McPherson, A. M. Galloway, S. R. Cater, and S. J. Hambling,
“Friction stir welding of thin DH36 steel plate,” Sci. Technol. Weld. Join.,
vol.
18,
no.
5,
pp.
441–450,
Jul.
2013,
doi:
10.1179/1362171813Y.0000000122.
A. Toumpis, A. Galloway, S. Cater, and N. McPherson, “Development of
a process envelope for friction stir welding of DH36 steel – A step
change,” Mater. Des. 1980-2015, vol. 62, pp. 64–75, Oct. 2014, doi:
10.1016/j.matdes.2014.04.066.
Subsea 7, “Marine Growth Removal Tool (Flexi-Clean),” Subsea 7, Sep.
05, 2022. https://catalogue.subsea7.com/cleaning-tools/marine-growthremoval-tool (accessed Sep. 05, 2022).
“Insight
Pro
Underwater
Laser
Scanner,”
Voyis.
https://voyis.com/insight-pro/ (accessed Sep. 05, 2022).
“Blackfly
GigE
Camera
|
Teledyne
FLIR.”
https://www.flir.com/products/blackflygige?vertical=machine+vision&segment=iis (accessed Sep. 05, 2022).
“Qinsy 9,” QPS. https://www.qps.nl/qinsy/ (accessed Sep. 05, 2022).
“ULS-500 PRO User Manual.” 2G Robotics, Dec. 15, 2017.
R. Capocci, E. Omerdic, G. Dooly, and D. Toal, “Fault-Tolerant Control
for ROVs Using Control Reallocation and Power Isolation,” J. Mar. Sci.
Eng., vol. 6, no. 2, Art. no. 2, Jun. 2018, doi: 10.3390/jmse6020040.
E. Omerdic and D. Toal, “OceanRINGS: System concept & applications,”
in 2012 20th Mediterranean Conference on Control & Automation
(MED), Jul. 2012, pp. 1391–1396. doi: 10.1109/MED.2012.6265833.
“Agisoft Metashape: Agisoft Metashape.” https://www.agisoft.com/
(accessed Sep. 05, 2022).