A Vision-Based System for On-Board Identification
and Estimation of Discarded Bio-mass: A Tool for
Contributing to Marine Resources Sustainability
Luís T. Anteloa, Tatiana Ordóñeza, Iñaki Miniñob, Joaquín Graciab, Emilio Ribesc, Juan Hervásc, Santiago Simónc,
Antonio A. Alonsoa.
a
Process Engineering Group, IIM-CSIC. C/ Eduardo Cabello 6, 36208 Vigo, Spain.
b
MAREXI Marine Technology Co. C/ del Redondo 53, 36212 Vigo, Spain.
c
AIDO – Technological Institute. C/ Nicolás Copérnico 7-13 - Valencia Technology Park, 46980 Paterna, Valencia, Spain.
Abstract- In marine harvesting, fish waste due to the discards
of non-targeted species represent a risk for the sustainability of
fisheries, a loss of potentially valuable living resources and a
threat for the ecological equilibrium. The project FAROS aims to
develop and implement an efficient, integral management
network for discards, to reduce global impact of this established
practice, while putting in value and optimising the use of the bycatch. The development of an on-ship automated system for
discards evaluation and species biomass estimation is proposed in
this contribution. The Biomass Estimation Optical System
(BEOS) integrates machine vision technologies and optical
information processing and feature extraction by means of
nonlinear modeling based on artificial neural networks. A first
functional prototype and a methodology for system
implementation and operation are presented here.
I.
INTRODUCTION
The global harvesting of marine products has increased
from around 17 million tons in the 1950s to a current average
amount of 85 million tons. The Food and Agriculture
Organization of the United Nations (FAO) estimates that an
annual average of 27 million tons of non targeted species are
caught and thrown back into the sea, what means that near a
third of the fish volume captured every year is wasted. This is
the so called discards what, in itself, represent a purposeless
waste of valuable living resources which could risk the
sustainability of fisheries [1]. In addition, the large amounts of
organic waste thrown into the sea may produce severe adverse
effects on the ecological equilibrium of marine communities.
These discarding practices relate to the hardcore of fishing
operations, from an economic, legal, and biological point of
view. However, despite all these difficulties, there is a
common and positive agreement (among citizens, NGOs, the
fishing sector, policymakers, scientist, etc.) that perceives
discards as very negative and that solutions have to be
implemented. In this sustainability framework, the FAROS
Project (co-funded under the LIFE+ Environmental Program
of the European Union) has as its main objective the
development and implementation of an efficient and integral
discards and by-catch management network, implying all
actors present in the fishing sector (fleets, harbors, auctions,
industries, etc.), which both aims the minimization of
discards/by-catches as well as their optimal valorization to
recover and to produce valuable chemicals of interest in the
food and pharmaceutical industry.
In order to fulfill these core objectives, the development and
implementation of new on-board technologies on selected
fleets (Gran Sol and coastal trawlers) has been revealed as a
key issue in order to on-line identify volume and
types/fractions of species with higher levels of discards and to
transmit this data to land for the considered fleets. One of
these equipments is a vision-based system called BEOS
(Biomass Estimator Optical System). It is based on capturing
images which are analysed on real time, classifying and
quantifying the amount of captures per haul made in a given
métier. The use of species detection and identification
algorithms allows to both obtain complete advanced data
regarding the geometry as well as to make highly precise
estimations about the amount of capture of the species
collected in each haul.
The aim of BEOS (in combination with the data
transmitting device denoted as RED BOX) is to feed with real
catch/discards data the developed GIS models in order to get a
global map of the fishing activity in the European Atlantic
coast. This tool will be very useful: a) to assess which areas
are more suitable to host a particular fishing gear or fishing
operation; b) to perform a spatial rating of the fishing areas
based on the proportion target/by-catch at cell-based scale; c)
to use spatial statistics to estimate areas with higher proportion
target-by-catch on a spatial-temporal scale and; d) to quantify
the spatial and temporal distribution and abundance of
discards, which allow to apply statistical methods in order to
derive density maps. With this information, fishing fleets will
have complete information to plan in advance (in port) their
future daily activity, minimizing the amount of discards,
fishing pressure or other negative environmental impacts (like
fuel consumption, that can be also minimized) or legal
restrictions (bans, quotas, etc.) over stocks while maximizing
their profit .
The system proposed here is currently being developed by
the joint research team of FAROS. While a first demonstrator
has been built to test operability and functionality, a second
prototype will be put together for methodology testing and
performance validation. This system will be based on image
capturing, pre-processing, body shape information extraction
and colour modeling. In addition, once the species have been
identified from each image, an estimator based on the existing
correlation between body shape with expected weight will be
used to calculate biomass.
The application of different algorithms and tools for marine
species modeling by means of machine vision has been done
previously in the past [2,3,4]. However variability of target
species, as well as scalability of system functionality imposes
the use of heterogenic model structures based on 2D
morphological analysis and the application of nonlinear
modeling such as artificial neural networks (NN) for pattern
recognition [5,6,7].
This work is structured as follows: In Section II, the
selected methodology for image capturing and processing is
presented. The main characteristics (in terms of shape and
color) of most discarded species in the selected fleets on
FAROS are summarized in Section III. Finally, in Section IV,
the implementation of the proposed technical approach and the
development of the BEOS device are described in detail.
II. MACHINE VISION-BASED TECHNICAL APPROACH
A. Optical Sensing Technologies
Following the initial requirements established for the BEOS
system – to be versatile, robust, automated and cost-effective
–, different sensing techniques have been considered for
prototype development. The advantages of such approaches
and their complementarity have being decisive for the choice
of technology. A key aspect of prototype development has
been the consideration of the harsh working conditions in
which the device will perform. The prototype will be installed
on a fishing ship, and will inspect the capture on a conveyor
belt where the fish is transported. A brief explanation of each
of the considered sensing technique for species identification,
individual counting and biomass estimation is presented next.
Line- and Area-Scan Camera
Conventional RGB line- and area-scan cameras offer only
2D information of the scene of interest. This means that
volumetric information is somehow lost in each image. It must
be noted that installation for optimal camera performance
varies from one type of camera to other. Area-scan cameras
can work independently from external movement conditions.
It suffices only to set appropriate parametric information such
as exposure time and gain, so that image is acceptable, i.e.
well in focus and with enough contrast. However, line-scan
requires the implementation of an encoder which helps in the
synchronisation of the camera line image capturing. A poor
encoding strategy will result in deformed composite images,
requiring the need of pre-processing in order to alleviate the
effect on image information extraction. While species
classification can be implemented from 2D images, mass
estimation requires the use of models which relate the 2D
animal shape with the expected weight.
Stereoscopic Camera Arrangement
The use of 2 area-scan cameras for stereoscopic imaging is
being considered at present within the development of the
second prototype. Proper calibration of the camera
arrangement can offer extended functionality for volume
estimation for each species individual. However, the use of
stereoscopic devices is still under evaluation since hardware
integration and computational effort for volume estimation
may not pay off in terms of robustness and automation. Once
again, species identification can be done from one of the
cameras by using 2D images, while de correlation between
stereoscopic information and weight can be performed in a
more accurate way.
Laser Profilometry
Cameras based on laser profilometry can estimate object
volume by projecting a linear laser bean on the object surface
while recording the spatial variation of the laser reflection due
to the object morphology. In order to obtain a 2D
representation of a volumetric item, relative movement
between this and the recording device must happen. The use
of this technique can be particularly useful in species
identification (due to shape extraction), but also for biomass
estimation as fish volume can be well approximated, allowing
a proper weight calculation. On the other hand, some type of
surface such as black, glossy, metallic or aqueous can affect
the reflected laser beam capture, producing wrong recordings
of laser variations. Besides, and due to the relative position
between the laser source, the object and the recording device,
the laser beam and the optical axis of the camera must have an
angle different from 0, which makes this approach less
compactable than others. The implementation of this
technique precludes de use of 2D colour images, hence
discarding the pigmentation information for classification
purposes. A more detailed species classification could be
obtained by 3D model matching, which may require more
computational resources. Volumetric calculus and weight
estimation could be again achieved by relational multivariate
models.
B. Modeling for Species Classification and Weight Estimation
Morphological characteristics have been selected for fish
classification and biomass estimation. These are body shape
and pigmentation. Some species will be directly identified
only by shape, whereas for those cases of body similarity,
pigmentation will be used as the feature for classification.
Some work has been previously done in the field of species
classification by means of computer vision, and some
problems like fish body have being addressed. However there
are still technical issues for discrimination, as is the case of
similar species, where only fine body differences exist.
Once the body shape determination cannot be used as a
differentiation criterion between species – as it happens for
many fish types – colour and pigmentation may be used as the
classification strategy. The colour pattern identification must
be done in such a way that different species can be uniquely
identified.
Body weight estimation can be done by relating 2D and 3D
dimensional shape information to weight. The appropriateness
of this approach will depend on the approximations and the
complexity that the occurrence space may have. As a first
instance, a relationship between the 2D-dimensional volume
(surface) and the weight can be modeled. This relation can be
a-priori considered nonlinear. Nevertheless, a better
understanding of this problem should be achieved in order to
relate suitable morphological characteristics to weight, over
the different stages of growth and/or size of individuals, so
that more accurate models can be trained.
In this work, and regarding these three identification
approaches (classification by shape, by pigmentation and
biomass estimation), a number of strategies are proposed by
FAROS research team:
Species classification by body shape pattern matching
A model (normally a 2D closed graph) of the body shape of
each species can be produced and then matched/compared
with resulting shapes obtained after captured image processing
(by means of translations, rotations and scaling).
Classification is made after comparing a likeliness parameter
(LP) which gives the confidence on the matching result and a
pre-set threshold.
Species classification by spatial colour modeling
Spatial distribution of pigmentation can be employed to
distinguish between species. RGB spatial samples can be
extracted from the animal body represented in a captured
image. Spatial colour sampling must be chosen so that an
individual can be uniquely associated to one species or
another. In this approach, classification must include both
RGB and spatial information. For this purpose, artificial
neural network models can be used to map the spatial – colour
map into the classification space.
Biomass calculation by weight estimation modeling
As previously mentioned, a first approach for weight
estimation is currently being developed. To that purpose,
nonlinear modeling, also by means of artificial neural
networks, can be carried out. In this approach, trained
networks map the input space – surface – into the output space.
More complex input spaces can be taken into account for
precise modeling, such as local morphological parameters
related to animal areas of dissimilar weight (i.e. head, main
body, tail area, fins, etc.).
C. Inspection Methodology
The task of species classification and weight estimation is
complex mainly due to the diversity of species. In this work, a
novel methodology is stated based on the estimation of desired
results and the previously discussed available technologies.
The basis of the proposed methodology is the construction of a
model structure for each species that must be identified and
developed. Such structure is made up of three model parts: (i)
a body shape model; (ii) a spatial colour distribution model
and; (iii) a weight estimator. Each of these models must be
trained and validated. Once the model structure has been
constructed, each picture taken from a captured species will be
compared against each of the model structures created. As a
result, an output for classification and biomass estimation will
be produced. A first approach regarding the inspection
challenge has been done by considering the information flow
diagram represented in Figure 1. The algorithm can be
described as follows: Once a valid image has been recorded, it
is processed so the body shape for the species is identified.
Then, the image shape is correlated with every species body
shape model, until a match is found. If two or more matches
are found – i.e. image information could be associated with
more than one species –, then the species area in the image is
sampled for colour model testing. It is expected that both
shape and colour (pigmentation) models can uniquely identify
every species of interest. Once a species match is obtained, the
corresponding weight estimator is used next to calculate an
approximate weight for the species individual.
Figure
Methodology
workflow.
III. S1.PECIES
TAXONOMY
During the last decades, research on the field of marine
stocks monitoring and management stated that multivariate
techniques, like partitioning around medoids (PAM) and
multivariate regression tree (MRT), are shown to be
appropriate tools to find homogeneous groups either taken
only catch-profile information or also using explanatory
variables, respectively. As part of FAROS project actions,
clusters obtained through these techniques can be contrasted
with the knowledge of fisheries, in order to achieve an
appropriate segmentation that reflects coherent and
manageable grouping on terms of catch, discard practices,
seasonality and area of operation, i.e., métiers. These units
include: a) Galician bottom otter trawl fleet vessels authorized
to fish in Community waters targeting flat fish, and basically
operating in Gran Sol; b) Galician coastal bottom otter trawl
fleet vessels targeting a variety of demersal species. and; c)
Portuguese coastal bottom trawl vessels for demersal fish that
operate along the year, with hauls directed to a variety of
species. Main species discarded by each of them include:
Figure 2. Examples of variability of species in shape: Sea potato, (top left),
greater silver smelt (top right), boarfish (medium left), megrim (medium
right), Henslow’s swimming crab (bottom left) and squat lobster (bottom
right).
a) Horse mackerel (Trachurus trachurus), sea potato
(Actinauger richardi), boarfish (Capros aper), sea cucumber
(Holothuroidea), haddock (Melanogrammus aeglefinus),
small-spotted catshark (Scyliorhinus canicula), blue whiting
(Micromesistius poutassou), megrim (Lepidorhombus
whiffiagonis), Atlantic mackerel (Scomber scombrus), red
gurnard (Aspitrigla cuculus), hake (Merluccius merluccius)
and greater silver smelt (Argentina silus).
b) Henslow’s swimming crab (Polybius henslowii), smallspotted catshark, blue whiting, and Atlantic mackerel, blackmouth dogfish (Galeus melastomus), squat lobster (Munida
spp.), hake, grey gurnard (Eutrigla gurnardus) and four spot
megrim (Lepidorhombus boscii)
c) Chub mackerel (Scomber colias), hake, blue jack
mackerel (Trachurus picturatus), boarfish, Atlantic mackerel,
bogue (Boops boops), gurnards (Triglidae), pouting
(Trisopterus luscus) and Henslow´s swimming crab.
In order to properly identify these most discarded species
(those that represent higher volume fractions) in the
considered areas through the defined methodology described
in previous Section, their main morphologic and coloration
characteristics are summarized in Table 1.
TABLE I: MAIN MORPHOLOGICAL AND COLORATION CHARACTERISTICS OF
THE DISCARDED SPECIES IDENTIFIED IN FAROS
Species
Morphometry
Sea potato
Heart-shaped. Its size can be up
to 9 cm. in diameter
Sea
cucumber
The body ranges from almost
spherical to worm-like, and lacks
the arms found in many other
echinoderms. They are typically
10 to 30 cm.
Horse
mackerel
They exhibit a 30 to 60 cm,
elongated body, with a
characteristic lateral line on the
back
Atlantic
mackerel
They present an up to 65 cm,
elongated body.
Grey
gurnard
45 to 60 cm, conic body, with a
big quadrangular head (more
than ¼ of total length)
Blackmouth
dogfish
Males are up to 75 cm length
while females are up to90 cm.
Elongated body with flat head
Red
gurnard
Up to 50 cm length, conic body,
with big head (almost a quarter
of total length)
Blue
whiting
It has a long, narrow body. The
fish can attain a length of 40 cm
Smallspotted
catshark
Around 75 cm length, elongated
body with flat head.
Squat
lobster
Henslow’s
swimming
crab
The edge of the carapace also is
lined with small
spines. Carapace length up to 7
cm. The chelipeds are much
longer than the other walking
legs, slender, and spiny
Carapace nearly circular, flat, a
little wider (45 cm) than long (40
cm), postero-lateral margin a
little contracted, dorsal surface
smooth
Color
It looks like a hairy
potato (brown) due to its
spines
It can vary from brown to
reddish-brown or even to
light orange or white
Grey or blue to green on
the back, clear on the
side, blue-coloured fins,
with a black spot in the
superior angle of the
operculum
Blue and green back, with
dark winding bands.
Silver abdomen, light
green sides, with purple
and silver reflects. Grey
fins
Brownish-grey, pale
abdomen, back and sides,
usually with white spots.
Characteristic black spot
on the first dorsal fin
Longitudinal brownish
spots on the sides and
back, surrounded by a
lighter circle
Pink, lighter on the
bottom
Blue-gray dorsally,
grading to white
ventrally. Sometimes
with a small black blotch
at the base of the pectoral
fin
Yellowish to brownishgrey with numerous black
small dots
Overall color is orange or
brick red
Reddish brown, underside
pale.
Species
Morphometry
Hake
Elongated body, with lengths up
to 120 cm, usually 30-70 cm
Megrim
Flat fish with an a length up to
40 cm
Color
Usually slate grey above,
lighter on sides and belly
white
Brown, with four blank
rounded sport on anal and
dorsal fins
Finally, in Figures 2 and 3 some of the FAROS selected
species’ variability in shape and pigmentation are shown.
IV. METHODOLOGY IMPLEMENTATION AND SYSTEM
DEVELOPMENT
Regarding the methodology implementation at real scale,
intense work has been carried out so far for system
development in three main lines of activity. The first one is the
hardware integration, in which electronic, optical and
mechanical components have been identified and selected for
the construction of the first prototype BEOS-P1. The objective
is to establish an optimal system performance by analyzing
environmental working conditions, functionality and
operability. Secondly, efforts have been focused on species
modeling, where model structures as the one previously
described, are being constructed. Finally, the implementation
of the methodology in a software control application is also
being developed.
Hardware prototype development
A first version of BEOS (denoted as BEOS-P1) has been
developed to evaluate the technical design, capabilities and
operability of an on-board
automatic system for
estimating
discards
biomass. The specific
working
conditions
(space limitations on the
fishing
park,
inappropriate
light
conditions, oscillations,
humidity, etc.), in which
this equipment and future
updates have to work,
impose restrictions on the
development of hardware
and
software
requirements. BEOS-P1
provides an open and
modular architecture that
allows the adaptation of
the equipment to any
conveyor belts present in
Figure 4. The BEOS-P1 prototype as
installed in the “Vizconde de Eza”
different fishing ships. It
ship.
is composed of four
distinct elements for
image capturing, subsequent processing and data generation
by haul: a) a computer vision box with IP-67, including a
Basler scA1000-30gc CCD with a 12-6mm variable focal lens,
a micro-pc Fit-PC2 by CompuLab, for storage and image
processing, and the corresponding power supplies and
electrical circuitry for surge protection; b) a lighting chamber,
which contains the lighting system made up of four 18W
75cm cold neon tubes, using electronic reactance and enclosed
in IP-67 protective cases; c) a support structure to fix the
position of the system over the conveyor belt and; d) a control
application software, that allows the selection of camera
settings (exposure time and gain) and makes possible the
automatic setting for image registration, by selecting periods
of capture. Figure 5 shows a detailed view of the vision box
interior and a snapshot of the application interface.
The system, as shown in Figure 4, has been installed and
calibrated within the Oceanographic Campaign of “Vizconde
de Eza” (a research ship of the Secretaría General del Mar)
during a scientific campaign during September 2010 on the
Gran Sol fishing ground.
Figure 5. Detail of the BEOS-P1 vision head interior (with the areascan camera and control micro-pc (left). A snapshot of the BEOS-P1
software application (right).
Figure 6. Some details of registered images taken by BEOS-P1 (left top
and bottom) and first processing for identification of species body
shape and colour (right top and bottom).
During this time, BEOS-P1 carried out the capture of a high
number of images from different hauls. After this action, the
use of edge detection filters, which recognise the outlines of
individuals of given species (classification by body shape)
together with the use of a RGB camera to estimate color at
each point of the scene (classification by colour) are being
employed to identify the different species present – Figure 6 –.
Finally, in order to determine the mass for each discard
previously identified, the shape enclosed area in the image
may be related to available biomass estimation tables.
Species modeling
By using the images obtained during the BEOS-P1 test onboard “Vizconde de Eza”, a novel modeling structure
implementation is currently being put into practice, by which
each species selected for future analysis is associated to a
model structure based on body shape, pigmentation and
weight estimation. For body shape model construction, the
MATROX MIL9.0 [8] libraries are being used. These image
processing libraries allow the user to select patterns form an
image to become a model. By means of posterior correlation,
the model pattern can be translated, rotated and dilated, in
order to match occurrences in further captured images. As an
example Figure 7 shows the body shape extraction from a
individual image.
Figure 7. Extraction of body shape from an image of a raja montagui
member.
The MATROX MIL9.0 libraries allow the implementation
of automated matching, given the information on the matching
results (an LP as means of fit error, model shifting within the
image, rotation and dilation required). In order to obtain the
most general body shape model for each species, different
images from the same and several individuals from a single
species are being captured, and, after normalization, an
average of all the model images is calculated as the final
general species model.
For each model matching, the likeliness parameter can be
defined as the average quadratic distance in pixels between the
edges in the occurrence shape and the corresponding active
edges in the model, such as in Equation (1),
  x    y 
2
e
LP 
e
Nc
Nc
2


(1)
where LP is the likeliness parameter, xe is the error (shift
distance between the occurrence and model x co-ordinates) in
the x dimension, ye (shift distance between the occurrence and
model y co-ordinates) is the error in y dimension and Nc is the
number of total pixels involved in the matching form the
occurrence and the model (i.e. the pixels which form the shape
and which are used for the matching).
Figure 8. Body colour profile sampling from the capros aper
member (top) and chelidonichthys cuculus member (bottom). For
each member image, tops graphs represent the across colour profile
(top right in each sub-image) and the transversal colour profile (top
left in each sub-image). Graph colour codes correspond to the red
component (red), green component (green) and blue component
(blue).
Pigmentation is also being used for distinguishing different
species members. A first analysis over a selected number of
discarded species has being performed, by which colour
profile patterns have been extracted. It has been observed that
selecting patterns from specific body sections can be a means
of uniquely classifying each species. This is particularly
observed in the pigmentation patterns for fish which may be
similar in shape. Figure 8 shows a transversal and cross colour
profiles from two different species (Capros aper and
Chelidonichtys cuculus). Color model construction is
performed by selecting colour sample points from captured
images and by feeding the RGB information to a pre-defined
neural network for species identification. The model
construction strategy is explained next.
The construction of NN training and validation sets are
made from RGB points from species member images. The
sampling technique currently followed is colour profile
extraction (one transversal to the species body and the second
one across the body section). However, other sampling
strategies such as grid sampling are also being considered, as
seen in Figure 9. RGB samples are taken so that most of the
variability in colour pigmentation for all studied species
should be properly represented in the data sets. In many cases,
oversampling can occur, so a method for dimensional
reduction, such as principal component analysis (PCA) is used
to limit the number of 2D sampling points from the image.
PCA can identify
those RGB points
(or
whichever
combination
of
colour bands) which
better represent that
variability. Once the
PCA
is
being
applied to the data
Figure 9. A possible pattern with 70 grid
training
set,
a
sample points for colour extraction applied
number of inputs
to a capros aper member.
can be determined
for the neural network (Figure 10). Since this network is
employed as a classifier, the outputs must be vector tags. Such
vectors have the dimensionality corresponding to the number
of species that the NN will identify. The neural network is
chosen to be a multilayer perceptron (MLP) [9] with sigmoid
functions as neurons in the hidden and output layer. The
training algorithm to be employed has been selected to be the
back propagation (BP) algorithm [10,11], although some
forms of the Broyden-Fletcher-Goldfarb-Shanno (BFGS)
algorithm [12,13] with limited memory are also being
considered, due to the fact of its suitability when dealing with
problems with high dimensionality. For the sake of simplicity
in training and structure and as first approximation, the
number of outputs is restricted to a maximum of 5 outputs –
i.e. each neural network can only classify up to 5 species.
Nevertheless, other techniques for optimal structure
determination are also being considered [14]. For the
classification case, each species is associated with a tag vector,
whose entries are all 0, but that which determines the label for
the class, which is set to 1.
Weight estimation, still to be implemented, will be done
also by means of an MLP, whose input will be the surface
contained in the species member shape identified in the image,
and the output the estimated weight (Figure 11). A-priori, this
is a nonlinear modeling problem, which should be easily
solved by the NN approach.
In order to establish proper sequential model use, it is
necessary to establish the criteria by which one or other model
– whether body shape of colour – are applied. It is required to
identify which species may give problems when predicting a
member based on its body shape. These species will be the
ones whose shapes are similar. In order to establish a
quantitative estimation on shape similarity, all models will be
R
P1
G
B
α2,1
I1
β1
α3,1
I2
H1
λ1
α1,M
I3
O1
ω1,1
α1,M
ω1,L
α3N-1,1
ωM,1
S1
α1,M
α3N-2,1
R
PN
G
B
I3N-2
I3N-1
α3N,1
α3N-2,M
I3N
α3N-1,M
SL
ωM,L
OL
λL
HM
βM
α3N,M
Figure 10. Neural network architecture for spatial colour modeling. Pi =
RGB sample point from the image – N points chosen. Ii = Input neuron
(in this case, it is a mere signal distributor) – 3N input neurons. Hi =
hidden neuron (sigmoid function) – M hidden neurons. Oi = output
neuron (sigmoid function) – L output neurons. αi,j = weight applied to
signal from Ii to neuron Hj. ωi,j = weight applied to signal from Hi to
neuron Oj. βi = bias applied to neuron Hi. λi = bias applied to neuron Oj.
cross-tested. The calculation of the likeliness parameters
between different species models will be done, and those
models whose related likeliness parameters are high will be
marked as species with high body shape similarity. In such
cases, and when an image shape has high matching likeliness
with two or more different shape models, colour matching will
be employed, and the NN estimators corresponding to each
possible species’ model will have to be applied.
δ1
H1
ρ1
γ1
I
µ
O
VS
HK
γK
ρK
δK
Figure 11. Neural network architecture for weight and biomass
estimation. VS = Value of the surface enclosed in the body shape of a
member image. I = unique input neuron, distributing the value I to the
hidden layers. Hi = hidden neuron (sigmoid function) – K hidden
neurons. O = single output neuron (in this case, a summation). γi =
weight applied to signal from I to neuron Hi. ρi = weight applied to
signal from Hi to neuron O. δi = bias applied to neuron Hi. µ = bias
applied to neuron Oj.
Practical methodology considerations
Sound optical information flow is achieved by suitable
imaging capturing and processing. As a consequence, it is
necessary to ensure a good picture quality by homogeneous
illumination and high performance sensors. On the other side,
specimens in a given batch (haul) must be properly separated,
so that a clear picture of each species member can be taken.
In order to guarantee robustness, images must be pre-treated
to remove noise. For that purpose, wavelet methods, such as
the discrete wavelet transform (DWT) [16] can be used to
eliminate undesired noise while keeping critical image
information. Also, changes in specim sizes require the
implementation of normalization techniques for system.
V. CONCLUSIONS AND FURTHER WORK
In this contribution, and in order to help fleets to minimize
and optimal manage generated discards, a methodology and
first prototype implementation for discards analysis,
identification of species and biomass estimation have been
presented. However, and since the FAROS project (in which
this research is being developed) is currently ongoing, most of
the work described here is still being carried out. This means
that both models and methodologies implemented will have to
be practically validated on-board real fishing vessels, in order
to draw quantitative conclusions on the goodness of the
technological approach.
On the other side, there are still some issues which need to
be addressed. For example, discards specimen separation on
the conveyor belt for proper species imaging must be
considered. This may mean changing handling procedures on
the ship, such as the installation of separators on the conveyor
belts which deliver the capture to the inspection area. In
addition, all the procedures explained here must be
computationally evaluated so that the system can work in realtime. Introducing too many prediction stages may slow down
the estimation. On the other side, members can present a
bended posture, which will make body shape identification
more difficult. For this reason, model shape transformations to
simulate torsion may be put in place.
A further problem, regarding system scalability must also
be analysed. The inclusion of new species for detection and
estimation will require new model structures development.
Existing models may have to be combined, updated or
discarded in favor of new ones, so the prototype will handle
species variability with confidence.
Some of the practical implementations of a second
prototype (such as the choice of inspection hardware –
cameras, lenses, control pc, illumination) are still being
evaluated. The final prototype should be flexible for
installation, but also robust in performance. Preprocessing
strategies, such as de-noising by means of DWT, and
normalizing methods, as well as sampling colour strategies,
still need further work and validation.
ACKNOWLEDGEMENT
The authors acknowledge the financial support received from
the LIFE+ Program of the European Union (FAROS Project –
LIFE08 ENV/E/000119).
REFERENCES
[1] Kelleher, K.. Discards in the World's marine Fisheries: An Update. FAO
Fisheries Technical Paper No. 470. Food and Agricultural Organization
of the United Nations, Rome, Italy, 2005.
[2] N. J. C. Strachan, Length Measurement of Fish by Computer Vision,
Computers and Electronics in Agriculture, 1993a, vol. 8, pp. 93-104.
[3] N. J. C. Strachan, Recognition of fish species by colour and shape.
Image and Vision Computing 11(1), 1993a, pp. 2-10.
[4] D. J. White, C. Svellingen, N. J. C. Strachan, Automated Measurement
of Species and Length of Fish by Computer Vision, Fisheries Research,
2006, vol.8, pp. 203.210.
[5] A. Bartkowiak, Neural Networks and Pattern Recognition, Lectures
notes, Institute of Computer Science, University of Wroclaw, 2004.
[6] B. D. Ripley, Pattern Recognition and Neural Networks, Cambridge
University Press, 1996, ISBN 0-521-46086-7
[7] C.-F. Tsai, K. McGarry, J. Tait, Image Classification Using Hybrid
Neural Networks, Proceedings, SIGIR’03, ACM 1-58113-6463/03/0007.
[8] Matrox Imaging, Matrox Imaging Library 9 – User Guide, Manual no.
Y10513-301-0900, 2008, pp. 307-382.
[9] G. P. Zhang, Neural Networks for Classification: A Survey, IEEE
Transactions on Systems, Man and Cybernetics – Part C: Applications
and Reviews, 2000, vol. 30, no. 4, pp. 451-462.
[10] A. Mirzaaghazadeh, H. Motameni, M. Karshenas, and H. Nematzadeh,
Learning Flexible Neural Networks for Pattern Recognition, World
Academy of Science, Engineering and Technology, 2007, vol. 33, pp.
88-91.
[11] Z. Zainuddin, N. Mahat, and Y. Abu Hassan, Improving the
Convergence of the Backpropagation Algorithm Using Local Adaptive
Technologies, World Academy of Science, Engineering and Technology,
2005, vol. 1, pp. 79-82.
[12] Y.-H. Dai, Convergence Properties of the BFGS Algorithm, Society for
the Industrial and Applied Mathematics, 2002, vol. 13, no. 3, pp. 693701.
[13] M. H. A. Hassan, M. B. Monsi, L. W. June, Limited Modified BFGS
Method for Large-Scale Optimization, Matematika, 2001, vol. 17, no. 1,
pp. 15-23.
[14] K. Z. Mao, K.-C. Tan, W. Ser, Probabilistic Neural-Network Structure
Determination for Pattern Classification, IEEE Transactions on Neural
Networks, 2000, vol. 11, no. 4, pp. 1009-1016.
[15] S. K. Mohideen, S. A. Perumal, M. M. Sathik, Image De-noising Using
Discrete Wavelet Transform, International Journal of Computer Science
and Network Security, 2008, vol. 8, no. 1, pp. 213-216.