Non-Destructive Testing of
Fruit Firmness with Real-Time
constraints
Christopher Mills
Supervisors: Dr. Andrew Paplinski
Mr Charles Greif
Contents

Research aims

Fruit Firmness

Non-destructive testing (NDT)

Methods

Completed work

Future work

Conclusions

References
2
Project Aims



With our background research in Ultrasonic imaging, the
aim is to design a simple system that will grade fruit
firmness using NDT
And as part DigSys we are interested in an ASIC application
of these algorithms. They can execute up to one hundred
times faster in hardware.
Ensure that the system could be used in an industrial
setting, i.e. testing fruit on a rapidly moving conveyer belt.


Work within hard real time constraints (ie 10 fruit/sec)
Be able to test fruit without actual contact with the skin of fruit (is this
possible?)
3
Fruit Firmness
Definition of fruit firmness – mechanical rigidity of fruit cell
structure. It can be measured by conventional means;
stress testing, Magness-Taylor Probing
Measurement of Fruit Firmness is important because



Firmness affects the perception of enjoyment of food.
Perception of firmness is linked to freshness and the ripeness of
fruit.
Such perception may be of greater importance for the preparation
of fruit for later consumption. (Preservation: canning,
preserve/jam, etc)
Humans decide fruit firmness in a variety of ways


Feel/look as fruit is consumed.
Response to preparation/cooking.
4
Fruit Firmness (cont)

Biological factors of Fruit Firmness




Firmness varies with




Cell size/shape
Cell water content
Cell organization
Fruit type (apple, orange)
Fruit Age (under ripe, over ripe)
Conditions during maturation and storage
Image of boiled apple cells at 100x magnification
The image on the right, shows what apple cells look like at high
magnification, the boundaries between the cells are visible.
5
Fruit Firmness (cont)

Ultrasonic reflection can be used to measure
firmness.


it will be the gaps between cells that will best
respond to ultrasound and describe firmness.
The image to the right is a representation of a
fruits internal structure.

Fruit firmness varies with ripeness and time,
going from firm and unripe to soft and ripe
or overripe.

The reason for this is that chemical changes
within the fruit change the way the cells
inside interact and the chemical composition
within the fruit, eg starch being converted
into sugars.
6
Fruit Firmness (cont)
Fruit firmness testing is critical to
industries involved in the
sorting and grading of fruit. As
sorting can be done based on
fruit firmness measures.
 For the duration of this
project, a company called
Colour Vision Systems (CVS)
will be providing sponsoring
for this project.

CVS build large scale fruit
sorting machines, including
computational circuits for
automated sorting based on
vision for blemish detection
and near infrared for sugar
content evaluation.
7
Non-Destructive Testing

NDT methods of testing are used on mechanical structures
while they are in use or before use – and the structures can
continue to be used post testing.

Various modalities of NDT exist, such as





Sound methods (ultrasound, acoustic, etc)
Wave energy response (laser, infrared, x-ray)
Vision (Video camera’s)
Physical Response to small force (Laser air puff, bounce test, microdeformation)
Many researchers have attempted to develop methods for
fruit firmness testing. For the next few slides I will detail
some of these.
8
NDT-Examples, Laser scatter imaging
Kang et al attempted to use laser-scatter imaging to grade
quality of tomatoes.
The method is reasonably simple, a laser beam is fired
through a piece of fruit/vegetable, the scatter of the laser
beam is recorded by a camera, and the extent of the
scatter is an indication of quality.
9
NDT-Examples, Laser Air Puff
McGlone et al describes a method based on the laser air puff
test.
The laser air puff test uses deformation in the target caused
by air under pressure, this deformation is measured by a
laser.
It was found that while this method was reasonably accurate
on average, there was an issue with confidence and
resolution when testing firm fruit due to the decreased
measurable deformation.
10
NDT-Examples, Bounce Test
Delwiche et al attempted to build a fruit sorter based on the
impact force (or Bounce) testing method.
Based on previous work by the same researchers, built a
system where fruit would fall with a speed of 76.7 cm/s.
The force measurement was made by a force transducer
mounted vertically on a large steel mass or impact mass.
The fruit was dropped from a conveyor belt. Overall, the
system could process fruit at 5 fruit/s.
While the system was capable of sorting fruit based on
firmness, the error rate was high, 26% for peaches.
For this research, we will concentrate on ultrasonic methods to
measure fruit firmness due to our experience in the area.
11
NDT-Examples, Acoustic
Peleg et al built a fruit firmness sorter based on the principles of
acoustic energy.
 A small electrodynamic shaker, vibrates the bottom of the fruit
 The root mean square (RMS) level of the input signal Xi is
measured in the shaker head
 The output RMS signal level Xo is measured by a miniature
accelerometer attached to the top part of the fruit.
 A Firmness index PFT is defined by: PFT=X0/(X0-Xi).
Overall, the system performed well with reasonably high confidence
and repeatability (>80%).
12
NDT-Examples, Acoustic
The picture on the right
shows the ‘sensor wheel’.
 Fruit moves along the
conveyor
 Then it’s grabbed by the
acoustic transducers
 The fruit is held and tested
until it reaches the lower
conveyer
The fruit is tested at a rate of
7.5 fruit/s per lane.
13
NDT-Examples, Acoustic

The table to the right
shows some values of PFT
vs Penetrometer force


It shows that the measure
PFT is related to the force
measured by the
penetrometer
If the fruit is stored in a
Controlled Atmosphere,
the Penetrometer and PFT
show similar increase in
reading
14
NDT-Examples, Ultrasonic
Mizrach et al attempted to estimate fruit
qualities from a Ultrasonic measure of
fruit firmness
 The system used two transducers, one as
receiver, the other as a transmitter
 The resulting signal was processed



The Frequency response Analysed
And the speed of sound through the
target measured
The experiment focused on Mangos as
the test subject
Representation
of the system
15
NDT-Examples, Ultrasonic
The graphs on the right show
the received signal and the
Fourier transform that of
that signal.
The results were compared to
known values of firm and
soft fruits and a firmness
measure made based on
the comparison.
The accuracy of this method
is reasonably high.
16
NDT-Examples, Ultrasonic

The scatter plots here
represent the accuracy of the
system

The table below gives a value
called the Standard Error of
Calibration (SEC)
17
NDT - Ultrasound
Basics of Ultrasonic testing
 Required equipment




Transmitter and Receiver transducers
Pulsar/Receiver unit
External/internal microcomputer to store
results and control Pulsar/Receiver
Operation




Pulsar/receiver applies voltage to the
transmitter
Transmitter vibrates and creates high
frequency sound
Ultrasound reflects whenever a change in
density occurs.
Receiver responds to sound and sends a
voltage based on the amplitude of
received signal
18
NDT - Ultrasound
However, there is a problems with using Ultrasound. The most
common method of ultrasound is called ‘contact using liquid
immersion’. This is a problem because…
 In an automatic system, contact with the fruit could be
awkward and expensive.
 Application of conducting liquid could also be awkward.
One possible answer is to use Non-Contact Ultrasound (NCU).
The system is very similar to liquid contact except
 The Transducers do not contact the target
 Noise due to lack of contact


large reflections caused by sound waves entering target
considered as noise
To reduce reflection from transducer to air, an acoustic lens
is used.
19
NDT – Ultrasound (NCU)
The above image shows the behaviour of ultrasonic waves
using NCU.
20
Research Method


Empirically determine response of the cellular structure of fruit to
ultrasound
Possibly use Field 2, which can produce images based on
simulation values or real readings from an ultrasonic system
This is an example of Field 2
taking a source image and
simulating how it would look
through ultrasonic testing. The
same could be done with a
mock up of fruit internals.


However, we do not require images, just an overall
characterization of fruit firmness
Devise a Neural Network structure or other type of system that is
capable of determining fruit firmness (e.g. statistical methods)
based on the training data. Early testing of Neural Net to be done
in Matlab
21
Proposed system

Use Ultrasound on fruit via non-contact transducers to measure
fruit firmness.

Process Ultrasound response via a neural network that will require
training for each available fruit type, and evaluate fruit firmness.

Integrate with existing system manufactured by CVS


such as a vision system to detect blemishes (Some blemishes are
caused by fruit diseases that would effect firmness also)
Weight and volume information (fruit density could prove useful in
determining fruit firmness)
22
Proposed system
The card to the right is called the
OPCARD.
 It is a PCI add on card
 It is an Oscilloscope card designed
for ultrasound
 It has an 8bit DAC
 Highly Configurable


12.5MHz … 100MHz SampF
High pass and low pass filters
The Transducer shown here is the
AT50 from Airmar
 Air contact transducer
 Output signal Frequency of 50MHz
23
Work Completed

Research into Non-Contact Ultrasound (NCU)


Classification system



Based on what I have learned, NCU is a very appropriate technology
for this application. However, it is a relatively new method compared
to liquid immersion ultrasound, and apparently despite its advantages
not widely used so sourcing NCU transducers has been difficult.
At this stage, a neural network is the most likely system to use for
classification of Fruit Firmness
Other systems are possible, such as pattern recognition methods
including statistical analysis.
Physical arrangement of system


Some ideas have been discussed, such as the angle between the
emitter and receiver(s)
Angles of transducers to fruit surface
24
Future Work

Testing of various methods including

Acoustic/ultrasound






Determine accuracy of NCU
Machine Vision
Laser Air-puff
Non-destructive deformation
Sensor Fusion
Construction of a system based on results of testing
25
Conclusions

Ultrasonic testing can grade firmness with sufficient accuracy.

NCU is applicable in most situations where the more common
liquid contact Ultrasonic testing methods are used.

Sensor fusion is a sensible option in fruit firmness testing.
26
References
Texture - http://www.ba.ars.usda.gov/hb66/021texture.pdf
Evolution Of Piezoelectric Transducers To Full Scale NonContact Ultrasonic Analysis Mode http://www.ultrangroup.com/pdfs/WCNDT-NCU-64.pdf
Non-Contact Ultrasound: The Last Frontier In Non-Destructive
Testing And Evaluation http://www.ultrangroup.com/pdfs/esm1.pdf
Field 2 http://www.es.oersted.dtu.dk/staff/jaj/field/index.html
27