SUN
CH022.tex
18/7/2007
17: 15
Page 549
Quality Evaluation
and Control of Potato
Chips and French Fries
Franco Pedreschi1 , Domingo Mery2 and Thierry Marique3
1 Universidad de Santiago de Chile, Departamento de Ciencia y
Tecnología de Alimentos, Av. Ecuador 3769, Santiago de Chile, Chile
2 Pontificia Universidad Católica de Chile, Departamento de Ciencia de
la Computación, Av. Vicuña Mackenna 4860 (143), Santiago de
Chile, Chile
3 Centre pour l’Agronomie et l’Agro-Industrie de la Province de Hainaut
(CARAH), 11 rue P. Pastur, 7800 Ath, Belgium
1 Introduction
Potato chips have been popular salty snacks for 150 years, and retail sales in the US
are worth about $6 billion/year, representing 33 percent of the total sales of this market
(Garayo and Moreira, 2002; Clark, 2003). In 2001 about 50 percent of the US potato
crop was processed to produce 11 300 million kg of processed potatoes, of which
21.6 percent were made into chips. The worldwide trade over recent years indicates
that about 7.4 × 107 kg of potato chips were exported, with a value of ∼$165 million
annually (Economic Research Service, 2004).
Frying in hot oil at temperatures between 160◦ and 180◦ C is characterized by very
high drying velocities, which are critical to improve not only the mechanical but also the
structural properties of the potato chips (Baumann and Escher, 1995). Potato chips are
thin slices whose moisture content decreases from around 80 percent to almost 2 percent
when they are fried. However, the drying in oil inevitably leads to a considerable oil
uptake of around 35 percent, most of which is located on the surface of the chip
(there is almost no penetration during frying) and adheres to the surface at the end
of frying. Therefore, a high proportion of oil penetrates into the food microstructure
during the post-frying cooling stage (Ufheil and Escher, 1996; Aguilera and Gloria,
2000; Bouchon et al., 2003).
In the potato chip industry, the quality of each batch of potato tubers must be tested
before processing, and of the various quality criteria the visual aspect, especially the
Computer vision technology for food quality evaluation
ISBN: 978-0-12-373642-0
Copyright © 2007, Elsevier Inc.
All rights reserved
22
SUN
CH022.tex
18/7/2007
17: 15
Page 550
550 Quality Evaluation and Control of Potato Chips and French Fries
color, is of great importance (Marique et al., 2005). Color of potato chips is the first
quality parameter evaluated by consumers, and is critical in the acceptance of the
product (Pedreschi et al., 2006). Consumers tend to associate color with flavor, safety,
storage time, nutrition, and level of satisfaction, due to the fact that color correlates
well with physical, chemical, and sensorial evaluations of food quality. The color of
potato chips changes during frying, as the components of potatoes are restructured.
Consequently, the surface color reflects not only the heterogeneous surface formed as
a result of frying but also the non-homogeneous oil distribution. Visual aspects, such
as surface color and appearance, can be studied using computer vision techniques in
order to determine the potato chip.
Acrylamide, which is formed in potatoes during frying and is highly related to the
color of potato chips, is suspected to have critical implications for human health, since it
has recently been found to be a carcinogen in rats (Mottram and Wedzicha, 2002; Rosen
and Hellenäs, 2002; Stadler et al., 2002; Pedreschi et al., 2005). Potato chip color is
affected by the Maillard reaction, which depends on the content of reducing sugars,
amino acids, or proteins at the surface. It is also affected by the frying temperature
and time (Márquez and Añón, 1986). Generally, potato tubers that contain more than
2 percent of reducing sugars are discarded for frying, since they generate too dark a
coloration. Research has demonstrated that 2.5–3 mg of reducing sugar per gram of
potatoes should be the maximum value accepted for potato chip preparation (Lisinska
and Leszczynski, 1989).
In European factories, some computer vision systems are used for the on-line evaluation of potato chips, allowing chips to be sorted according to defects such as black
spots or blisters (Marique et al., 2005). Some researchers have been also working on
a promising device that is able both to classify chips according to color and to predict acrylamide levels using neural networks (Marique et al., 2003, 2005), and some
are currently evaluating devices based on this system with the expectation that it will
be fully operational very soon. Apart from the neural network, there is another device
based on statistical pattern recognition for color classification of potato chips (Marique
et al., 2003; Pedreschi et al., 2004). Researchers in this topic are routinely providing
classical visual evaluation against a standard chart, and have conducted a good amount
of work to testify which criteria (overall appearance, heterogeneity, contrasted extremities, etc.) should be taken into account by the operator to evaluate the surface of potato
chips. In this chapter, the application of computer vision to study the quality attributes
of potato chips is summarized.
2 Computer vision
Computer vision (CV) is a novel technology for acquiring and analyzing an image in
order to obtain information or to control processes. Basically, a computer vision system
(CVS) consists of a video camera for image acquisition, illuminants with standard
settings, and computer software for image analysis (Papadakis et al., 2000; Brosnan and
Sun, 2004). Image processing and image analysis are at the core of CV, with numerous
SUN
CH022.tex
18/7/2007
17: 15
Page 551
Computer vision 551
B
G
R
I
Preprocessing
Image
acquisition
B
G
R
Red
features
R
Segmentation
S
color
images
camera
chip
gray
image
Green
features
G
Blue
features
B
L*a*b*
features
L
Classification
Intensity
features
I
binary
image
Geometric
features
γ
“class A”
Figure 22.1 Schematic representation of the pattern-recognition process required for automatic
classification of potato chips. (Reprinted from Pedreschi et al., 2004©, by courtesy of the Institute of Food
Technologists.)
algorithms and methods being capable of objectively measuring and assessing the
appearance quality of agricultural products (Sun, 2004). Figure 22.1 shows a schematic
representation of a general CV pattern-recognition process required for the automatic
classification of potato chips, which involves the following four steps (Castleman,
1996; Mery et al., 2003): image acquisition, image pre-processing and segmentation,
feature extraction, and classification.
2.1 Image acquisition
A digital image of the potato chip is captured and stored in the computer. When acquiring images, it is important to consider the effect of illumination intensity and the
orientation of specimens relative to the illumination source, since the gray level of
the pixels is determined not only by the physical features of the surface but also by
these two parameters (Peleg, 1993; Chantler, 1995). Typically, a color digital camera
provides an image of which each pixel is associated with three digital values as red (R),
green (G), and blue (B). Figure 22.2 shows an image-acquisition system implemented
by Pedreschi et al. (2004) to measure the different quality attributes in potato chips.
This system is composed of:
1. A color digital camera with 4 megapixels of resolution (Power Shot A70, Canon,
Tokyo, Japan)
2. Four natural daylight 18-W fluorescent lights (60 cm in length) with a color
temperature of 6500 K (Philips, Natural Daylight, 18 W) and a color index (Ra)
close to 95 percent for proper illumination
3. A wooden box where the illuminating tubes and the camera are placed; the interior
walls of the box are painted black to minimize background light.
SUN
CH022.tex
18/7/2007
17: 15
Page 552
552 Quality Evaluation and Control of Potato Chips and French Fries
Lamps
Camera
Wooden box
Sample
Figure 22.2 Image-acquisition system developed to evaluate potato chip quality. (Reprinted from León
et al., 2006©, courtesy of Elsevier.)
2.2 Image pre-processing and segmentation
The digital images taken must be pre-processed to improve their quality before they are
analyzed. Using digital filtering, the noise in the image can be removed and the contrast
enhanced. Sometimes in this step the color image is converted to a gray-scale image,
called the intensity image (I). The intensity is used to divide the images into disjointed
regions with the purpose of separating the region of interest from the background.
This segmented image (S) is a binary image, where 0 (black) and 1 (white) indicate
background and object, respectively. In our case, such a region of interest corresponds
to the area where the potato chip is located for the test.
Segmentation is an essential step in computer vision, and the accuracy of this operation is critical in automatic pattern recognition for food image analysis. This is because
pattern recognition is based on the data subsequently extracted from the segmentation
process (Brosnan and Sun, 2004). Segmentation detects regions of interest inside the
image, or structural features of the object, and can be achieved by three different techniques: thresholding, edge-based, and region-based (Sonka et al., 1998; Sun, 2000)
segmentation. Mery and Pedreschi (2005) developed a robust algorithm implemented
in Matlab 6.1 software (The MathWorks, Inc., Natick, Mass., USA.) to segment potato
chips from the background. The segmentation has three steps:
1. Computation of a high-contrast gray-value image from an optimal linear
combination of the RGB components
2. Estimation of a global threshold using a statistical approach
3. A morphological operation in order to fill the possible holes presented in the
binary image (Figure 22.3).
2.3 Feature extraction
Image feature extraction is one of the most active research topics in computer vision (Du
and Sun, 2004). Following segmentation, feature extraction is concentrated principally
SUN
CH022.tex
18/7/2007
17: 15
Page 553
Computer vision 553
(a)
(b)
(c)
Figure 22.3 Potato images: (a) color image of a potato chip; (b) gray-scale image of (a); (c) segmented
image of (a). (Reprinted from Pedreschi et al., 2006©, by courtesy of Elsevier.)
around the measurement of the geometric properties (size and shape) and surface
characteristics of regions (color and texture) (Zheng et al., 2006). It is important to
know in advance which features provide relevant information for the classification. In
order to reduce the computational time required in the pattern-recognition process, it is
necessary to select the features that are relevant for the classification. For this reason,
feature selection must be performed in a training phase.
Features extracted from potato chip images by Pedreschi et al. (2004) are described
in Table 22.1, and are grouped into six types:
1.
2.
3.
4.
5.
6.
Geometric (γ)
Intensity (gray-scale image) (I)
Red component (R)
Green component (G)
Blue component (B)
Mean values of the L*a*b* components (L).
The details regarding now these features are calculated can be found in the references
cited in Table 22.1.
2.4 Classification
The extracted features of each region are analyzed and assigned to one of the defined
classes. A classifier is designed following supervised training. Simple classifiers can
be implemented by comparing the measured features with threshold values. However,
it is also possible to use more sophisticated classification techniques, such as statistical
and geometric analyses, neural networks, and fuzzy logic (Castleman, 1996; Jain et al.,
2000; Mery et al., 2003). For example, in statistical pattern recognition, classification
is performed using the concept of similarity – i.e. patterns that are similar are assigned
to the same class (Jain et al., 2000). In other words, a sample is classified as class “i”
if its features are located within the decision boundaries of class “I”. Furthermore, a
decision-tree classifier can be implemented to search for the feature that can separate
one class from the other classes with the most confidence (Safavian and Landgebe,
1991).
SUN
CH022.tex
18/7/2007
17: 15
Page 554
554 Quality Evaluation and Control of Potato Chips and French Fries
Table 22.1 Extracted features from images of potato.
Type
Feature
Description
Reference
γ
γ
γ
γ
γ
γ
γ
γ
γ
γ
(ī, j̄)
h, w, A, L, R
φ1 . . . φ7
|DF 0 |. . .|DF 7 |
FM1 . . . FM4
FZ 1 . . . FZ3
(ae , be )
ae /be
α (i 0 , j 0 )
Gd
Center of gravity
Height, width, area, roundness, and perimeter
Hu’s moments
Fourier descriptors
Flusser and Suk invariant moments
Gupta and Srinath invariant moments
Major and minor axis of fitted ellipse
Ratio of major to minor axis of fitted ellipse
Orientation and center of the fitted ellipse
Danielsson form factor
Castleman, 1996
Castleman, 1996
Sonka et al., 1998
Zahn and Roskies, 1971
Sonka et al., 1998
Sonka et al., 1998
Fitzgibbon et al., 1999
Fitzgibbon et al., 1999
Fitzgibbon et al., 1999
Danielsson, 1978
I, R, G, B
I, R, G, B
I, R, G, B
I, R, G, B
I, R, G, B
I, R, G, B
I, R, G, B
I, R, G, B
I, R, G, B
I, R, G, B
I, R, G, B
I, R, G, B
I, R, G, B
I, R, G, B
I, R, G, B
G
C
D
K1 . . . K3
Kσ
K
Q
Q
σQ
Q
Q̄
F 1 . . . F15
φ1 . . . φ7
σg2
Tx d
Castleman, 1996
Mery and Filbert, 2002
Mery and Filbert, 2002
Kamm, 1998
Mery and Filbert, 2002
Mery and Filbert, 2002
Mery, 2003
Mery, 2003
Mery, 2003
Mery, 2003
Mery, 2003
Mery, 2003
Sonka et al., 1998
Mery and Filbert 2002
Haralick et al., 1973
I, R, G, B
KL, DFT, DCT
Mean gray value
Mean gradient in the boundary
Mean second derivative
Radiographic contrasts
Deviation contrast
Contrast based on CLPa at 0◦ and 90◦
Difference between maximum and minimum of BCLP1
ln(Q + 1)
Standard deviation of BCLPa
Q normalized with average of the extreme of BCLP1
Mean of BCLPa
First components of DFT of BCLPa
Hu moments with gray value information
Local variance
Mean (M) and range () of 14 texture featuresb
with d = 1, 2, 3, 4, 5
64 first components of the KL, DFT,
and DCT transforma
L
L∗ a∗ b∗
Color components of the region
Hunt, 1991;
Papadakis et al., 2000
Castleman, 1996
Reprinted from Pedreschi et al., ©2004, by courtesy of the Institute of Food Technologists.
γ, geometric features; I, intensity features; R, red component features; G, green component features;
B, blue component features, L, L∗ a∗ b∗ features.
a
CLP: Crossing line profile, gray function value along a line that crosses the region at its center of gravity.
The term BCLP refers to the best CLP – in other words, the CLP that represents the best homogeneity
at its extremes (Mery, 2003).
b
The following features are extracted based on a co-occurrence matrix of the whole image of the potato
chips: second angular moment, contrast, correlation, sum of squares, inverse difference moment, mean
sum, variance of the sum, entropy of the sum, variance of the difference, entropy of the difference, two
measures of correlation information, and maximum correlation coefficient, for a distance of d pixels.
c
The transformation takes a re-sized window of 32 × 32 pixels which includes the middle of the potato chips.
3 Applications
3.1 Sorting of potato chips
Recently, the different features of color, size, shape, and texture have been combined for
their applications in the food industry, because in this way they increase the performance
of the methods proposed. These features can be applied with various kinds of food,
SUN
CH022.tex
18/7/2007
17: 15
Page 555
Applications 555
L*: 58.743568
a*: 8.0011638
b*: 26.339806
L*: 38.099905
a*: 13.974827
b*: 19.029009
Figure 22.4 The color of a complete potato chip and a small circular browned region of it in L*a*b*
units. (Reprinted from Pedreschi et al., 2006©, courtesy of Elsevier.)
such as in fried potatoes for the detection and segmentation of surface defects, the
prediction and characterization of chemical and physical properties, and the evaluation
and determination of sensorial characteristics (Pedreschi et al., 2004).
3.1.1 Color
The color of potato chips is an extremely important quality attribute and a fundamental
criterion for the potato-processing industry, since it is strictly related to consumer
perception, the Maillard reaction, and acrylamide formation (Scanlon et al., 1994;
Mottram and Wedzicha, 2002; Stadler et al., 2002; Pedreschi et al., 2006). These
reasons make extremely important to have methods for measuring the color of potato
chips properly (Pedreschi et al., 2005; León et al., 2006).
In image analysis for food products, color is an influential attribute and powerful
descriptor that often implies object extraction and identification, and that can be used to
quantify the color distribution of non-homogeneous samples (Brosnan and Sun, 2004).
The color of fried potatoes is usually measured usually in the unit of L∗ a∗ b∗ , using either
a colorimeter or specific image-acquisition and processing systems. Parameter L∗ is
the luminance or lightness component, which ranges from 0 to 100, and parameters
a∗ (from green to red) and b∗ (from blue to yellow) are the two chromatic components,
which range from –120 to 120 (Papadakis et al., 2000). In the L∗ a∗ b∗ space the color
perception is uniform, which means that the Euclidean distance between two colors
corresponds approximately to the color differences perceived by the human eyes (Hunt,
1991). More recently, potato-chip color has been measured with computer vision techniques (Scanlon et al., 1994; Segnini et al., 1999; Marique et al., 2003; Pedreschi et al.,
2004). Computer vision (CV) is used to measure the color of potato chips objectively,
as it provides some obvious advantages over a conventional colorimeter – namely, the
possibility of simultaneously analyzing the whole surface and the details of the chip,
and quantifying characteristics such as brown spots and other appearance defects on
the surface (Figure 22.4).
SUN
CH022.tex
18/7/2007
17: 15
Page 556
556 Quality Evaluation and Control of Potato Chips and French Fries
R
G
B
Preprocessing
Segmentation
binary image
Image
formation
Fluorescent lamps
Camera
Color conversion
procedure
L*a*b* values
Figure 22.5 Schematic representation of a computer vision system used to covert color images from RGB
to L*a*b* space. (Reprinted from Pedreschi et al., 2006©, by courtesy of Elsevier.)
3.1.1.1 Color models
The use of CV for the color assessment of potato chips requires an absolute color
calibration technique based on a common interchange format of color data and the
knowledge of which features can be best correlated to the product quality. With a digital
camera it is possible to register the color of any pixel of the image of the object using
three color sensors per pixel, which depend on the color model being used (Forsyth and
Ponce, 2003). The most frequently used color model is the RGB model, in which each
sensor captures the intensity of the light in the red (R), green (G) and blue (B) spectra
respectively. There have been two trends recently in the application of image color
for food quality evaluation: one is to carry out a point analysis, encompassing a small
group of pixels for the purpose of detecting small characteristics of the object; the other
is to carry out a global analysis of the object under the study of the color histogram in
order to analyze its homogeneity (Brosnan and Sun, 2004; Du and Sun, 2004).
Pedreschi et al. (2006) recently designed and implemented a CV system to measure
representatively and precisely the color of highly heterogeneous food materials, such
as potato chips, in L∗ a∗ b∗ units from RGB images (Figure 22.5). Since RGB digital
cameras obtain information in pixels, León et al. (2006) developed a computational
color conversion procedure that allows the obtaining of digital images in L∗ a∗ b∗ color
units from the RGB images by testing five models: linear, quadratic, gamma, direct,
and neural network. After the evaluation of the performance of the models, the neural
network model was found to perform the best, with an error of only 0.96 percent. The
network architecture is shown in Figure 22.6.
Finally, in order to show the capability of the proposed method, León et al. (2006)
compared the color of a potato chip measured by this approach with that obtained by a
Hunter Lab colorimeter. The colorimeter measurement was obtained by averaging 12
measurements (at 12 different places on the surface of the chip), whereas the measurement using the digital color image was estimated by averaging all pixels of the surface
image. Measurement from the colorimeter was used as the standard measurement.
SUN
CH022.tex
18/7/2007
17: 15
Page 557
Applications 557
ˆ*
L
R
aˆ *
bˆ *
G
B
Figure 22.6 Architecture of the neural network used to estimate L∗ a∗ b∗ values from RGB images.
(Reprinted from León et al., 2006©, by courtesy of Elsevier.)
Instrument
Hunter lab
Digital image
(a)
(b)
L*
61.7
66.9
a*
1.7
1.3
b*
26.6
27.3
(c)
Figure 22.7 Estimatation of L∗ a∗ b∗ values of a potato chip: (a) RGB image; (b) segmented image by the
method proposed by Mery and Pedreschi (2005); (c) color measured in L∗ a∗ b∗ space using a commercial
colorimeter and the approach of León et al. (2006). (Reprinted from León et al., 2006©, by courtesy of
Elsevier.)
The results are summarized in Figure 22.7, and the error of the CV system was only
1.8 percent.
3.1.1.2 Classification techniques
A pattern-recognition approach was used for the classification of potato chips processed
under six different conditions, and good classification results were obtained (Pedreschi
et al., 2004). Pedreschi et al. (2004) implemented an approach to classifying potato
chips using pattern recognition from color images where more than 1500 features
were extracted from each of the 60 potato images tested. The feature selection was
carried out based on the Sequential Forward Selection (SFS) method (Jain et al., 2000).
Finally, 11 features were selected according to their classification attributes. Although
samples were highly heterogeneous, classification of the potato chips using a simple
classifier and just a few features was able to obtain a very good performance (accuracy
≥ 90 percent) in all cases. These authors showed that pattern-recognition techniques
could easily and successfully be applied to classify highly heterogeneous materials
such as fried potato chips processed under different conditions, as well as other food
products.
Marique et al. (2003) used an artificial neuronal network to classify fried potato
chips. In this approach, gray-level features of the apex, the center and the base of each
potato chip were obtained from a color image in order to determine the quality class
SUN
CH022.tex
18/7/2007
17: 15
Page 558
558 Quality Evaluation and Control of Potato Chips and French Fries
Acrylamide content (µm/kg)
2000
R2 0.9569
1600
1200
800
400
0
10
Acrylamide content 113.77a* 1062.23
5
0
a*
5
10
Figure 22.8 Acrylamide content vs color parameter a∗ of controlled and blanched potato chips (moisture
content of ∼1.8%, wet basis) fried at 120, 150, and 180◦ C. (Reprinted from Pedreschi et al., 2005©, by
courtesy of Elsevier.)
to which each chip belonged. Using a relatively small number of samples, the authors
obtained good agreement with human inspectors, yielding a classification performance
of around 90 percent.
3.1.1.3 Color and frying temperature
Color has been extensively used for evaluation of the effect of different temperatures
on the quality of fried potato chips. The kinetics of color changes in potato slices during frying at four temperatures were investigated using the CV system implemented by
Pedreschi et al. (2006). Furthermore, Pedreschi et al. (2005) found a good linear correlation (r 2 = 0.9569) between the acrylamide content of potato chips (moisture content
∼1.8 percent on a wet basis) and their color represented by the redness component a∗ in
the range of the temperatures studied. The redness component a∗ is an indicator of nonenzymatic browning; the lower a∗ value, the paler the potato chip (Figure 22.8). As the
frying temperature increased from 120◦ to 180◦ C, the resultant chips became redder
and darker as a result of non-enzymatic browning reactions that are highly dependent
on oil temperature. Blanching reduced the a∗ value of potato chips due to the leaching out of reducing sugars previous to frying, thus inhibiting non-enzymatic browning
reactions and leading to lighter and less-red chips. Figure 22.9 shows how the potato
chips increased in redness and became darker as the frying temperature increased from
120◦ to 180◦ C. At the same frying temperature, blanching pre-treatment led to paler
potato chips after frying.
3.1.2 Texture
Computer analysis of the surface texture of foods is of interest, because it affects the
processing of many food products. For instance, there is a known dependence between
the oil uptake and the surface properties of fried potatoes (Pedreschi et al., 2000;
Bouchon, 2002). Visual textures are generally formed by the interaction of light with
a rough surface, such as that of fried potatoes. Scale-sensitive fractal analysis has
been applied directly over topographical data sets (heights as a function of position) to
SUN
CH022.tex
18/7/2007
17: 15
Page 559
Applications 559
(a)
(b)
(c)
(d)
(e)
(f)
Figure 22.9 Images of potato chips (moisture content of ∼1.8%, wet basis): (a) control fried at 120◦ C;
(b) control fried at 150◦ C; (c) control fried at 180◦ C; (d) blanched fried at 120◦ C; (e) blanched fried at
150◦ C; and (f) blanched fried at 180◦ C. Controls are unblanched slices. Blanching treatment was in hot
water at 85◦ C for 3.5 min. (Reprinted from Pedreschi et al., 2005©, by courtesy of Elsevier.)
quantify the important changes in the surface texture of potatoes during frying, such
as the area-scale fractal complexity (Asfc) and the smooth-rough crossover (SRC).
Another way to perform fractal analysis or to quantify the textural properties of a
surface is by using the information contained in images (brightness as a function of
position), with the advantage that the topography of the sample is not necessarily
correlated with the texture of its surface image (Rubnov and Saguy, 1997; Quevedo
et al., 2002).
Texture has been used in the quality inspection of potato chips. First, textural features
are acquired from images taken of the surfaces of a set of potato chip samples by using
video cameras. An identical set of samples of potato chips is used to obtain the quality
attributes of the samples, using sensory panellists or instruments. Afterwards, learning
models (e.g. statistical learning, fuzzy logic, and neural networks) can be set up to
correlate the texture features to the potato chip quality. Based on the information
obtained from the learning models, the qualities of different categories of potato chips
can be predicted by using their texture features from the images (Pedreschi et al., 2004).
Different problems involving appearance are associated with frying potato pieces
(either slices or strips). One is the presence of defects such as black dots and necrosis.
Searching for such defects involves on-line screening and eventual rejection of every
defective chip (Marique et al., 2003). Another is the development of dark coloration
because of the Maillard reaction between reducing sugars and amino groups (Márquez
and Añón, 1986). This must be assessed at the laboratory for every sample in the potato
batch intended for processing, because tubers that look perfectly healthy can develop
intensive and heterogeneous browning or dark tips, which lead to consumer rejection.
Defective batches are refused, to the disadvantage of the producer.
SUN
CH022.tex
18/7/2007
17: 15
Page 560
560 Quality Evaluation and Control of Potato Chips and French Fries
3.2 Browning evaluation
Evaluation of browning of samples taken from the frying lines must also be performed
frequently at the laboratory, to police the process. The synthesis of acrylamide during frying increases brownness, due to the reaction of asparagine and reducing sugars
(Pedreschi et al., 2005). As this reaction pathway is clearly correlated with the Maillard
reaction (Mottram and Wedzicha, 2002; Stadler et al., 2002; Pedreschi et al., 2005), it
has been proposed by several authors that quick and easy measurement of browning can
be performed using image analysis rather than painstaking chromatographic methods
(Pedreschi et al., 2005). Acrylamide is suspected to be a molecule with significant
toxicological effects – carcinogenic, neurotoxic, and mutagenic (Rosen and Hellenäs,
2002).
3.2.1 Using artificial neural networks (ANN) by CARAH (Centre pour
l’Agronomie et l’Agro-industrie de la Province de Hainaut, Belgium)
To estimate the darkening of French fries during frying, a simple frying assay is performed for 3 minutes at 180◦ C on 20 French fries obtained from the central part of
20 different potatoes. Each of the French fries is then assigned a category by visual
examination under standard white light (Marique et al., 2003). The assessors build
their evaluation with the help of a standard reference card (Figure 22.10), determined
from both the overall darkening of individual French fries, and the contrast between the
extremities (apex and base) and the center of the fries. Heterogeneous dark coloration
is also penalized.
There are, of course, problems associated with this subjective procedure. In particular, estimations may vary with the assessor. Even for a given assessor, sample
variability can influence results, since narrow distributions tend to be spread over the
scale. It is thus of great interest to develop a model that allows reproducible estimation
of the color category of French fries (Marique et al., 2003).
Artificial neural networks (ANNs) can attain very good performance when used to
predict values for complex non-linear systems (Mittal and Zhang, 2000; Wilkinson and
PF 00
Figure 22.10
PF 0
PF 1
PF 2
PF 3
A reading card of French fries for browning categories.
PF 4
PF 5
SUN
CH022.tex
18/7/2007
17: 15
Page 561
Applications 561
Yuksel, 1997). Moreover, they are endowed with a broad capacity for generalization,
so that they can give useful information for cases that are not part of their training
set (Schalkoff, 1997; Wilkinson and Yuksel, 1997; Marique and Wérenne, 2001; Yang
et al., 2002). They appear to be the logical choice for achieving successful prediction
of the darkening index for fried potatoes.
Marique and colleagues (2003) used image analysis to extract gray-level intensities
from an image data bank gathered from the routine frying assays of 12 different mealy
potato cultivars (Annabelle, Bintje, Cantate, Charmante, Cyclone, Daisy, Farmer, Innovator, Lady Olympia, Liseta, Markies, Victoria). Three values were computed for
individual French fries, corresponding to the mean gray values at the apex, center, and
base of the specimen, respectively. The ANN is a feed-forward network consisting of
three inputs, a hidden layer of four neurons with sigmoid transfer functions and bias
(Figure 22.11), and an output layer presenting a single linear neuron with bias that is
issued to the estimated value of the color category (from 0 to 4).
The ANN was trained with a Lenvenberg–Marquardt algorithm (Schalkoff, 1997)
and the output values were compared to the corresponding color categories estimated
by human operators, who assigned each of the French fries to a color category ranging from 0 (very pale) to 4 (very dark). The Lenvenberg–Marquardt algorithm gave
fast convergence, as is usually the case for small networks. Figure 22.12 shows how
the assessor assigns a particular color value to individual French fries. The different
color categories are distributed throughout the gray-value scale, and there is partial
overlapping. This comes from the fact that when a particular specimen stands exactly
Hidden
1
Apex
Hidden
2
Center
Output
Color
class
Hidden
3
Base
Hidden
4
Figure 22.11 Structure of a two-layer feed-forward artificial neural network with three inputs, four hidden
neurons with bias, and one output neuron with bias. (Reprinted from Marique et al., 2003©, courtesy of
Institute of Food Technologists).
SUN
CH022.tex
18/7/2007
17: 15
Page 562
562 Quality Evaluation and Control of Potato Chips and French Fries
300
300
250
250
200
200
150
150
100
Center
100
50
50
0
Apex
0
Figure 22.12 Mean gray values of the center and apex of French fries. The color code indicates the class of
color: very pale gray, 0; pale gray, 1; mid-gray, 2; dark gray, 3; black, 4. The ellipsoids contain the two
sub-populations of class 1: dark gray, globally darker fries; pale gray, paler fries with contrasted dark ends.
(Reprinted from Marique et al., 2003©, courtesy of Institute of Food Technologists).
between two categories, the assessor will select one at random and either undervalue
or overvalue it, which hence leads to the overlapping.
French fries are assigned to category 0 if they appear both very pale (gray levels over
150) or rather paler at the extremities than in the center. A specimen will be assigned
to category 1 for one of the following two reasons: it appears paler in the center but
has contrasting dark ends (global appearance), or it appears dark in the center with
paler ends. Category 1 is thus clearly split in two subpopulations (see ellipsoids in
Figure 22.12) flanking both sides of category 0. Categories 2, 3, and 4 then progressively regroup darker French fries, which are generally pale in the center with more
or less contrasted dark ends. Thus, it is only for the two lower color categories that
the assessor will overvalue a specimen possessing dark contrasted extremities. For
higher color categories, estimations are based mostly on the global (center) appearance
of French fries, where dark contrasted ends are considered to be “normal” (Marique
et al., 2003).
The trained ANN was used to generate a complete set of predictions for the different
possible combinations of gray levels of the center and the apex of the French fries. This
is illustrated in Figure 22.13, where computation was performed using equal gray values
for both the base and the apex of the fries. The ANN showed a good performance, with
correlation coefficients of 0.972 for the training data and 0.899 for the validation data.
The network displays complex and continuous behavior for the low color categories 0, 1,
and 2, but operates a discrete classification between categories 3 and 4. This could be a
SUN
CH022.tex
18/7/2007
17: 15
Page 563
Applications 563
4
3.5
3
2.5
2
6
1.5
Color class
4
0
2
1
0.5
0
100
2
0
0
0.5
200
100
1
200
300
300
Center
Apex
Figure 22.13 Response of the artificial neural network: color class categories are described as a function
of all possible combinations of gray levels of the center and the apex of French fries. (Reprinted from
Marique et al., 2003©, by courtesy of Institute of Food Technologists).
consequence of both the greatest number of data points for high color categories, and the
more complex behavior of the assessor for low color categories (Marique et al., 2003).
A more complete simulation is shown in Figure 22.14, illustrating the discrete color
categories (the values predicted from the ANN are approximated by the nearest integer)
obtained for all the possible combinations of gray levels of the three regions of the
fries. Again, more complex behavior is observed for the lower color categories. The
intermediate categories 2, 3, and 4 also extend between two “wings,” being either
globally paler with dark ends or globally darker with pale ends. Color classification
varies most with the apex and center gray values (Marique et al., 2003).
3.2.2 Using other methods (Walloon Agricultural Research Center, Belgium)
This research center, affiliated to the GemblouxAgricultural University in Belgium, has
developed a home-made system for the quality evaluation of French fries (Figure 22.15).
In the system, 20 French fries are cooked and arranged on a tray with a reference tongue
(Figure 22.16). The reference tongue is directly extracted from the USDA reference
card in order to represent the seven reference colors on each sample. For each image,
a tray contains 20 French fries and a reference tongue. The tray is then placed in the
box and an image is taken.
Image analysis is performed using Image Pro Plus software 6 (Media Cybernetics,
USA). The program checks the number of references, which are represented by the
seven colored squares arranged on each tray, and offers a manual correction in cases
SUN
CH022.tex
18/7/2007
17: 15
Page 564
300
300
250
250
200
200
Apex
Apex
564 Quality Evaluation and Control of Potato Chips and French Fries
150
100
50
50
300
250
200
(a)
150 100
Center
50
200 400
Base
0
300
250
200
200
100
50
50
250
200
(c)
150 100
Center
50
300
200 300
0 100
Base
300
200
Apex
200
100
50
50
200
150 100
Center
50
200 300
0 100
Base
200
150 100
Center
50
4
0
300
(f)
200 300
0 100
Base
150
100
250
250
300
250
300
200 300
0 100
Base
2
(d)
250
0
50
0
3
150
150 100
Center
150
100
300
200
300
250
150
250
(b)
1
0
Apex
0
Apex
Apex
300
(e)
150
100
0
0
250
200
150 100
Center
50
200 300
0 100
Base
Figure 22.14 Discrete color categories obtained from all the possible combinations of gray levels for the
three regions of the French fries: (a) general presentation; (b)–(f) partial views of each color category,
from 0 (b) to 4 (f). (Reprinted from Marique et al., 2003©, by courtesy of Institute of Food Technologists).
where this number is different from seven. The references are then scanned. The French
fries are also scanned and counted. Once this has been done, macro sequences successively comply in a few seconds (via sending data to Excel) to compare the colors of the
French fries with the reference tongue in order to determine the color parameters. For
this comparison, RGB images are first converted into gray-scale and their luminous
intensity is then estimated. These stages are repeated until there is automatic generation
of a detailed written (containing photographs of the sample, the date, color parameters
etc.) and archived report.
SUN
CH022.tex
18/7/2007
17: 15
Page 565
Applications 565
Figure 22.15 Device for quality evaluation of French fries (Gembloux Agricultural University in Belgium).
Figure 22.16 Tray ready for image analysis with 20 French fries and 7 colored squares.
In order to compare the image analysis method with sensory analysis (visual comparison), a series of 100 samples is first analyzed using both methods. The correlation
of the results between the two methods is 0.951, which is a very good result compared
with the weak repeatability of the reference method (Figure 22.17).
3.2.3 Browning-sorting and acrylamide estimation using ANN by CARAH
Determination of the acrylamide concentration nowadays appears to be necessary since
very high concentrations of this potentially toxic molecule are detected in amylaceous
SUN
CH022.tex
18/7/2007
17: 15
Page 566
566 Quality Evaluation and Control of Potato Chips and French Fries
6.00
Judge index 0.7491(Image analysis index) 0.6221
R 0.9510
5.00
Judge index
4.00
3.00
2.00
1.00
0.00
0.00
Figure 22.17
1.00
2.00
3.00
4.00
Image analysis index
5.00
6.00
Correlation between the visual measurements and those by image analysis.
Figure 22.18 Artificial neural network imaging system for the determination of color class and acrylamide
concentration in French fries.
fried foodstuffs (Rosen and Hellenäs, 2002). However, standard procedures for acrylamide determination involve slow and expensive methods of chromatography and mass
spectroscopy, and thus cannot be used for routine analysis. It was therefore a logical step
to develop alternative techniques based on image analysis of the browning of French
fries to measure the acrylamide concentration. It was known that there would be good
correlation between non-enzymatic browning development and acrylamide formation,
since several studies had reported a strong linear relationship between browning and
SUN
CH022.tex
18/7/2007
17: 15
Page 567
References 567
acrylamide accumulation in fries (Mottram and Wedzicha, 2002; Stadler et al., 2002;
Pedreschi et al., 2005).
CARAH and a Belgian industrial partner, Rovi-Tech s.a. (Presles, Belgium), have
developed a high-speed imaging system incorporated with ANN. Snapshots of every
one of the French fries tested are taken and then results for both color category and
acrylamide concentration are obtained (Figure 22.18). The system is intended to analyze
incoming potato batches of pre-fried French fries for quality control in food distribution.
The heart of the system is Rovi-Tech’s ILB-25 (Image Learning Box), a very efficient
ANN that allows easy and powerful correlations of complex visual data analysis.
4 Conclusions
Both gray-scale and color images of potato chips are useful for extracting image features
for an appropriate classification. The most relevant features for potato-chip classification are either texture or color from L∗ a∗ b∗ space. Potato chips can be properly
classified to obtain very good values, despite their high heterogeneity. The automatic
classification methodology proposed for potato chips has a wide range of potential uses.
The computer vision system described in this chapter allows determination of the
color of potato slices in L∗ a∗ b∗ color space that is transformed from the RGB space
in an easy, precise, and objective way. To achieve this, image pre-processing and
segmentation is performed. The computer vision system allows easy measurement of
color not only over the entire potato chip surface, but also at small, specific regions of
interest.
Five models that can measure the color in potato chips using the color of each pixel
on the target surface, which cannot be accessed with conventional colorimeters, have
been built. The best results were achieved by the quadratic and neural network models,
both with small errors of close to 1 percent.
For both control and blanched potato chips, acrylamide formation decreases dramatically as the frying temperature decreases from 180◦ to 120◦ C. There is a linear
correlation between non-enzymatic potato-chip browning quantified by computer
vision and the corresponding acrylamide values.
Acknowledgments
The authors acknowledge financial support from the FONDECYT Project N◦ 1030411.
References
Aguilera JM, Gloria H (2000) Oil absorption during frying of frozen par-fried potatoes.
Journal of Food Science, 65, 476–479.
SUN
CH022.tex
18/7/2007
17: 15
Page 568
568 Quality Evaluation and Control of Potato Chips and French Fries
Baumann B, Escher E (1995) Mass and heat transfer during deep fat frying of potato
slices. I. Rate of drying and oil uptake. Lebensmittel Wissenschaft und Technologie,
28, 395–403.
Bouchon P (2002) Modeling oil uptake during frying. PhD Thesis, Reading University,
England.
Bouchon P, Aguilera JM, Pyle DL (2003) Structure oil-absorption relationships during
deep-fat frying. Journal of Food Science, 68, 2711–2716.
Brosnan T, Sun D-W (2004) Improving quality inspection of food products by computer
vision – a review. Journal of Food Engineering, 61, 3–16.
Castleman K (1996) Digital Image Processing. Englewood Cliffs: Prentice Hall, p. 667.
Chantler M (1995) Why illuminant direction is fundamental to texture analysis. IEE
Procedures Visual Image Signal Process, 142, 199–206.
Clark JP (2003) Happy birthday, potato chip! And other snack developments. Food
Technology, 57(5), 89–92.
Danielsson PY (1978) A new shape factor. Computing Graph Image Processing, 7,
292–299.
Du C-J, Sun D-W (2004) Recent developments in the applications of image processing
techniques for food quality evaluation. Trends in Food Science & Technology, 15,
230–249.
Economic Research Service (ERS) of the US Department of Agriculture (2004) Briefing
room potatoes. Available at http://www.ers.usda.gov./briefing/potatoes/.
Fitzgibbon A, Pilu M, Fisher R (1999) Direct least square fitting ellipses. IEEE
Transactions on Pattern Analysis, 21(5), 476–480.
Forsyth D, Ponce J (2003) Computer Vision: A Modern Approach. Englewood Cliffs:
Prentice Hall.
Garayo J, Moreira R (2002) Vacuum frying of potato chips. Journal of Food Engineering,
55, 181–191.
Haralick RM, Shanmugan K, Dinstein L (1973) Textural features for image classification.
IEEE Transactions on Systems, Man, and Cybernetics, 3, 610–621.
Hunt RWG (1991) Measuring Color, 2nd edn. Chichester: Ellis Horwood, Ltd, p. 313.
Jain AK, Duin RPW, Mao J (2000) Statistical pattern recognition: a review. IEEE
Transactions Pattern Analysis Machines Intelligence, 22, 4–37.
Kamm KF (1998) Grundlagen der Röntgenabbildung. In: Moderne Bildgebung: Physik,
Gerätetechnik, Bildbearbeitung und -kommunikation, Strahlenschutz, Qualitätskontrolle (Ewwen K ed.). Stuttgart: Georg Thieme Verlag, pp. 45–61.
León K, Mery D, Pedreschi F, León J (2006) Color measurements in L∗ a∗ b∗ units from
RGB digital images. Food Research International, 39(10), 1084–1091.
Lisinska G, Leszczynski W (1989) Potato Science and Technology. London: Elsevier
Applied Science.
Marique T, Wérenne J (2001) A general artificial neural network for the modeling of culture
kinetics of different CHO strains. Cytotechnology, 36, 55–60.
Marique T, Kharoubi P, Bauffe P, Ducatillon C (2003) Modelling of fried potato chips
color classification using image analysis and artificial neural network. Journal of
Food Science, 68, 2263–2266.
SUN
CH022.tex
18/7/2007
17: 15
Page 569
References 569
Marique T, Pennincx S, Kharoubi A (2005) Image segmentation and bruise identification
on potatoes using a Kohonen’s self-organizing map. Journal of Food Science, 70,
415–417.
Márquez G, Añón MC (1986) Influence of reducing sugars and amino acids in the color
development of fried potatoes. Journal of Food Science, 51, 157–160.
Mery D (2003) Crossing line profile: a new approach to detecting defects in aluminium
castings. Proceedings of the Scandinavian Conference on Image Analysis (SCIA 2003)
(Bigun J, Gustavsson T, eds). Lecture Notes in Computer Science, 2749, 725–732.
Mery D, Filbert D (2002) Classification of potential defects in automated inspection
of aluminium castings using statistical pattern tecognition. In Proceedings of the
8th European Conference on Non-Destructive Testing (ECNDT 2002), June 17–21,
Barcelona, p. 14.
Mery D, Pedreschi, F (2005) Segmentation of color food images using a robust algorithm.
Journal of Food Engineering, 66, 353–360.
Mery D, da Silva R, Caloba LP, Rebolledo JMA (2003) Pattern recognition in the automatic
inspection of aluminium castings. Insight, 45, 475–483.
Mittal GS, Zang J (2000) Use of artificial neural network to predict temperature, moisture,
and fat in slab-shaped foods with edible coatings during deep-fat frying. Journal of
Food Science, 65, 978–983.
Mottram DS, Wedzicha BL (2002) Acrylamide is formed in the Maillard reaction. Nature,
419, 448–449.
Papadakis SE, Abdul-Malek S, Kamdem RE, Yam KL (2000) A versatile and inexpensive
technique for measuring color of foods. Food Technology, 54(12), 48–51.
Pedreschi F, Aguilera JM, Brown CA (2000) Characterization of food surfaces
using scale-sensitive fractal analysis. Journal of Food Process Engineering, 23,
127–143.
Pedreschi F, Mery D, Mendoza F, Aguilera JM (2004) Classification of potato chips using
pattern recognition. Journal of Food Science, 69, 264–270.
Pedreschi F, Moyano PC, Kaack K, Granby K (2005) Color changes and acrylamide
formation in fried potato slices. Food Research International, 38, 1–9.
Pedreschi F, León J, Mery D, Moyano P (2006) Implementation of a computer vision system
to measure the color of potato chips. Food Research International, in press.
Peleg M (1993) Fractals and foods. Critical Review Food Science & Nutrition, 33, 149–165.
Quevedo R, López C, Aguilera JM, Cadoche L (2002) Description of food surfaces
and microstructural changes using fractal image textura analysis. Journal of Food
Engineering, 53, 361–371.
Rosen J, Hellenäs KE (2002) Analysis of acrylamide in cooked foods by liquid chromatography tandem mass spectrometry. Analyst, 127, 880–882.
Rubnov M, Saguy IS (1997) Fractal analysis and crust water diffusivity of a restructured
potato product during deep-fat frying. Journal of Food Science, 62, 135–137, 154.
Safavian S, Landgrebe D (1991) A survey of decision tree classifier methodology. IEEE
Transactions on Systems, Man, and Cybernetics, 21(3), 660–674.
Scanlon MG, Roller R, Mazza G, Pritchard MK (1994) Computerized video image analysis
to quantify color of potato chips. American Potato Journal, 71, 717–733.
SUN
CH022.tex
18/7/2007
17: 15
Page 570
570 Quality Evaluation and Control of Potato Chips and French Fries
Segnini S, Dejmek P, Öste R (1999) A low cost video technique for color measurement of
potato chips. Lebensmittel Wissenschaft und Technologie, 32, 216–222.
Schalkoff RJ (1997) Artificial Neural Networks. New York: McGraw-Hill, p. 448.
Sonka M, Hlavac V, Boyle R (1998) Image Processing, Analysis and Machine Vision, 2nd
edn. Pacific Grove: PWS Publishing.
Stadler RH, Blank I, Varga N, Robert F, Hau J, Guy A, Robert P, Riediker MC (2002)
Acrylamide from Maillard reaction products. Nature, 419, 449–450.
Sun D-W (2000) Inspecting pizza topping percentage and distribution by a computer vision
method. Journal of Food Engineering, 44, 245–249.
Sun D-W (2004) Computer vision – an objective, rapid and non-contact quality evaluation
tool for the industry. Journal of Food Engineering, 61, 1–2.
Ufheil G, Escher F (1996) Dynamics of oil uptake during deep-fat frying of potato slices.
Lebensmittel Wissenschaft und Technologie, 29, 640–644.
Wilkinson C,Yuksel D (1997) Using artificial neural networks to develop prediction models
for sensory-instrumental relationships: an overview. Quality Preferences, 8, 439–445.
Yang CC, Prasher SO, Landry J, Ramaswamy HS (2002) Development of neural networks
for weed recognition in corn fields. Transactions of the ASAE, 45, 859–864.
Zahn C, Roskies R (1971) Fourier descriptors for plane closed curves. IEEE Transactions
on Computing, 21, 269–281.
Zheng C, Sun D-W, Zheng L (2006) Recent developments and applications of image features for food quality evaluation and inspection – a review. Trends in Food Science
and Technology, 17(12), 642–655.