Food Control 12 (2001) 229±234
www.elsevier.com/locate/foodcont
The use of quantitative risk assessment in HACCP
E. Hoornstra *, M.D. Northolt, S. Notermans, A.W. Barendsz
TNO Nutrition and Food Research Institute, P.O. Box 360, 3700 AJ Zeist, Netherlands
Abstract
During the hazard analysis as part of the development of a HACCP-system, ®rst the hazards (contaminants) have to be identi®ed
and then the risks have to be assessed. Often, this assessment is restricted to a qualitative analysis. By using elements of quantitative
risk assessment (QRA) the hazard analysis can be transformed into a more meaningful managerial tool. In this way the e€ect of
control measures can be quanti®ed, so the occurrence of contaminants in the endproducts can be estimated. Also, the quantitative
risk assessment is a tool to derive or validate control measures and critical limits at process steps (CCPs). The practical use of
quantitative risk assessment is demonstrated by two examples: the risk of raw fermented sausages and the risk of a pressurized meat
product. It can be concluded that quantitative risk assessment is a powerful combination of food microbiology, modelling and
applied statistics. It is recommended as the input for managing food safety issues as an extension or validation of the HACCPsystem. Ó 2001 Elsevier Science Ltd. All rights reserved.
1. Introduction
In the HACCP-plan, hazard analysis is the collection
and evaluation of information, characteristics and data
of contaminants and conditions leading to food safety
risks. The result of the hazard analysis is the identi®cation of control measures which are essential to (i) prevent
contamination, (ii) prevent unacceptable increase of
contaminants and (iii) reduce contaminants to acceptable levels. Next, the process steps at which control can
be applied and is essential to reduce a food safety risk are
de®ned as the critical control points (CCPs) in a HACCP-plan.
The identi®cation of risk factors is an important step
in hazard analysis. Risk factors contribute to the probability of occurrence of hazards in a product. At a risk
factor the hazard is introduced or there is a probability of
increase or decrease. To know the impact of a risk factor
the e€ect can be determined in a qualitative or in a
quantitative way. In most HACCP-systems a qualitative
approach is used. This results in a poor underpinning of
the need and e€ect of control measures. Using a quantitative approach, either a worst-case, what-if or statistical approach can be applied. The disadvantage of a
worst-case approach is the assumption that all unfa*
901.
Corresponding author. Tel.: +31-30-6944-729; fax: +31-30-6944E-mail address: hoornstra@voeding.tno.nl (E. Hoornstra).
vourable events occur at the same time. This will lead to
an overestimation of the risk and thus to conservative
control measures. Using what-if scenarios, di€erent values (e.g. averages) can be chosen resulting in a more realistic approach. However, no account is made for
variability. In most cases it is the variability of a parameter that determines the risk. In the statistical approach monitoring results, expert knowledge, literature
data and well-reasoned assumptions about the various
risk factors are incorporated in probability distribution
functions. This means that calculations are not based
on one value, e.g. the average or an extreme value, but on
several values each with a probability. In the calculations
values are sampled from the probability distribution
functions (Monte Carlo sampling). A quantitative approach gives more insight in the need and e€ect of control measures than a qualitative approach.
Using risk assessment, it can be calculated how well
the endproduct meets the speci®cations, and also the
probability of exceeding a criterion can be calculated.
The criteria and speci®cations are set by governments
(Food Safety Objectives), by clients or by the company
itself. The producer has to translate these endproduct
criteria to subcriteria and critical limits in order to adhere to the criteria set. For the design and/or validation
of the critical limits use can be made of quantitative
risk assessment. If food producers want to perform a
risk assessment use can be made of the method of risk
assessment as de®ned by the Codex Alimentarius
0956-7135/01/$ - see front matter Ó 2001 Elsevier Science Ltd. All rights reserved.
PII: S 0 9 5 6 - 7 1 3 5 ( 0 1 ) 0 0 0 1 9 - 6
230
E. Hoornstra et al. / Food Control 12 (2001) 229±234
Commission. In principle, this approach is used by
governments for the development of Food Safety Objectives (criteria). These formal risk assessments take a
lot of e€ort, in terms of several man years. In the literature a few formal risk assessments are described (Cassin, Lammerding, Todd, Ross, & McColl, 1998; Marks,
Coleman, Jordan-Lin, & Roberts, 1998; Notermans et
al., 1997; Notermans, Dufrenne, Teunis, & Chackraborty, 1998; Whiting & Buchanan, 1997).
2. Risk assessment
Risk assessment according to the Codex Alimentarius
Commission is a scienti®c evaluation of known or potential adverse health e€ects resulting from exposure to
(food borne) hazardous agents. The process consists of
the following steps: (i) hazard identi®cation, (ii) hazard
characterization, (iii) exposure assessment and (iv) risk
characterization.
In hazard identi®cation, the signi®cant hazards for
the endproduct are identi®ed systematically. Not all
hazards can be addressed in detail in the risk assessment
or in the HACCP-plan. Therefore, only the most important hazards have to be selected. In a risk assessment
for governments mostly use is made of epidemiological
data. In a risk assessment for food companies mostly use
is made of product and process information. It can be
useful to combine hazards in a group depending on the
type of product and production process used. Examples
of groups are: sporeforming bacteria, toxin producing
bacteria, etc. Sometimes a hazard has to be de®ned
speci®cally, if it has certain speci®c characteristics (e.g.,
acid tolerance, heat stability). This systematical identi®cation of signi®cant hazards should also be done in
hazard analysis, but from experience it is known that in
HACCP-systems it is often done insucient. However,
it is the basis of a clear organized and transparant risk
assessment and should therefore be documented well
(Van Gerwen, de Wit, Notermans, & Zwietering, 1997).
Hazard characterization is the qualitative and/or
quantitative evaluation of the nature of the adverse
health e€ects associated with biological, chemical, and
physical agents which may be present in the food. Factors that are considered include virulence, microbial
variability and consumer sensitivity. For micro-organisms the so far developed dose-response-relations are
poor, with a large uncertainty (Buchanan, Damert,
Whiting, & van Schothorst, 1997; Coleman & Marks,
1998; Teunis, Medema, Kruidenier, & Havelaar, 1997).
In a HACCP-system it cannot be expected to make use
of quantitative health e€ects. In principle, only the
speci®cations (endproduct criteria) are used which are in
general based on adverse health e€ects. It is assumed as
a starting point that concentrations of hazards below the
criteria do not result in signi®cant health e€ects and
concentrations above the limit lead to an increased
probability of an adverse health e€ect. If criteria are
absent they have to be set, which might take much more
e€ort.
Exposure assessment is the qualitative or quantitative
evaluation of the likely intake of hazardous biological,
chemical and physical agents via food. The risk factors
in exposure assessment depends on the quality of the
raw materials, the process steps and the process environment, as well as the composition, packaging and
storage conditions of the product. Many of these risk
factors can and should be controlled by the food company. To evaluate the consumer's risk, it is also necessary to calculate the concentration of contaminants until
the moment of consumption and to know the quantity
of consumption. Determining this is far too complicated
for a HACCP-plan especially if a food company is not at
the end of a food chain. It suces to evaluate the
probability of occurrence of contaminants in the endproduct of the food company.
Risk characterization is the combination of hazard
characterization and exposure assessment leading to a
risk estimate. The risk estimate is the probability of
occurrence of hazardous agents in food, in quantities
leading to an adverse health e€ect. Quantities exceeding
the criteria are considered to result in an adverse health
e€ect.
Traditionally risk assessment has mostly been viewed
to set criteria for endproducts. The objective was to
protect the consumer. However, it also allows the understanding of a production process and to provide insights how to best manage the risk, and identify what
critical information is lacking in the understanding of
the risk situation. Therefore, the results of risk assessment have to be communicated clear and transparent to
the risk manager. A description of the risk pro®le and
the results of scenario analysis might support the decision making by the risk manager (i.e., government or
food company).
The objective of risk assessment for food companies
is to derive or validate critical limits at critical control
points. Although HACCP is meant for food safety, also
spoilage risks can be incorporated in risk assessment.
Spoilage micro-organisms are mostly more resistant to
heat, acid, etc. Therefore, spoilage risks, also potentially
leading to claims and recalls, are food company risks as
well. In the following examples the two types of risk
assessment are worked out. In the ®rst case of raw fermented sausages the risk manager is the government. In
the second case of a pasteurized meat product the risk
manager is the food company. In that case also spoilage
risks are taken into account.
Example 1 (Quantitative risk assessment of raw fermented meat products). In the recent past in the USA
and Australia a few outbreaks of E. coli O157:H7 in raw
E. Hoornstra et al. / Food Control 12 (2001) 229±234
fermented sausages have occurred (Centers for Disease
Control & Prevention, 1995a,b). In the USA a 5D performance standard (at least 5 log reduction) is accepted
as criterion for the fermentation process. In order to
underpin if this could be a realistic criterion for the
Dutch meat industry, a quantitative risk assessment is
done, based on literature data, challenge tests and other
microbiological information and assumptions together
with the use of applied statistics. For two di€erent types
of sausages a risk assessment is done. For the sausage
with a relative low water activity and high pH the results
are described below.
Hazard identification.E. coli O157:H7 is identi®ed as
most signi®cant hazard for raw fermented sausages,
based on the outbreaks and on the fact that E. coli
O157:H7 can be relative acid tolerant.
Hazard characterization. Very little information is
known about the infective dose of E. coli O157:H7.
Therefore, the dose-response relation of Shigella dysenteriae is used which has a comparable infection
mechanism. This dose-response relation contains a lot of
uncertainty, because it is based on a human feeding
study with healthy volunteers who consumed high
numbers of Shigella.
Exposure assessment. First, the risk contributing
factors in the food chain are identi®ed. Most important
are the prevalence and concentration of E. coli O157:H7
in feces of bulls, (cross-)contamination during slaughtering, reduction during fermenting and during storage
and the amount of consumption. The impact of every
single risk factor is quanti®ed using probability distribution functions. The probability distribution of E. coli
O157:H7 on meat trimmings has a variability between 0
and 1000 per g strongly skewed to the left. Because the
sausages have a certain limited mass, not all sausages
will contain E. coli O157:H7. The positive sausages will
have a heterogeneous contamination. During the fermentation and storage, from the challenge tests a reduction of 2±3 D was observed (Fig. 1). The number of
E. coli O157:H7 in the product can be calculated from
the probability distribution functions by performing socalled Monte Carlo simulations. This is done by running
100,000 iterations in @RISKe using the Latin hypercube method. Fig. 2 shows the probability distribution
of E. coli O157:H7 on meat from positive bulls before
and after fermentation. It can be seen that not only the
numbers of E. coli O157:H7 decrease (X-axis) but also
the probability that numbers occur decrease (Y-axis).
The time of consumption determines the achieved reduction during storage. Finally, the probability distribution of E. coli O157:H7 in the product at the time of
consumption is calculated (results in Table 1). This
probability of occurrence is multiplied with the amount
of consumption to assess the exposure.
Risk characterization. The dose±response relation is
combined with the exposure data to estimate the risk.
231
Fig. 1. Reduction/survival of E. coli O157:H7 in raw fermented sausage
during processing (14 days) and storage (from 14 to 78 days). The
numbers at day 35 are between 0 and 2 log (negative with counting
method, positive after enrichment of 1 g). The numbers at day 78 are
between )1.4 and 0 log (negative after enrichment of 1 g, positive after
enrichment of 25 g).
Because of the high variability in dose±response relation
the variability of risk was also high.
Risk management. The hazard identi®cation was
based on some incidental epidemiological data. Because
the data describing the hazard characterization was
poor, no well-underpinned risk characterization was
possible. Therefore, the exposure assessment was the
most important element of this risk assessment. Decision making was only justi®ed based on the estimation
of the probability of occurrence of E. coli O157:H7 in
the endproduct at the time of consumption. Because of
the low percentage of positive sausages and the low
numbers potentially present in positive sausages, it was
decided that there was no need for a performance
standard of 5D. Another important feature of the model
is the fact that the data can be held up-to-date and that
measures for improvement can be incorporated into the
Fig. 2. Probability distribution of E. coli O157:H7 on meat from
positive bulls before () and after (+) fermentation (for a better
readability the numbers lower than 10 E. coli O157:H7 per sausage
(with a high probability) and higher than 100 E. coli O157:H7 per
sausage (with a low probability) are not presented in the graph).
232
E. Hoornstra et al. / Food Control 12 (2001) 229±234
Table 1
Probability of occurrence of number of E. coli O157:H7 (Nt) in raw
fermented sausages at time of consumption
Number of E. coli O157:H7 in a sausage
Probability
Nt ˆ 0
Nt P 1
Nt P 10
99.7 %
0.3 %
0.002 %
(1 in 50,000)
<1 in 109
Nt P 100
model. The quantitative e€ect of these measures can be
predicted, which is essential for risk management. This
scenario-analysis can be combined with a cost±bene®t
calculation to improve the situation if necessary.
Example 2 (Quantitative risk assessment of a pasteurized
meat product). A food company wants to validate their
hazard analysis and derive a critical limit for the pH of
the meat product. The meat product consists of meat,
spices and herbs and other ingredients. It contains nitrite
salt and has a water activity of 0.97. The product is
pasteurized (P-value of 40 min at 70°C). The pH used to
be 6.3, which gave a self-stable product at a maximum
storage temperature of 7°C. The food company would
like to consider storage at ambient temperatures for 5
days. In order to prevent spoilage and safety risks a risk
assessment is conducted. Two pH levels are chosen, 5.7
and 5.3. Also a treatment of the spices and herbs as an
addition measurement is taken into account. It is assumed that the storage temperature is 20°C.
Hazard identification. Raw meat can be contaminated
with a lot of pathogens. Spices and herbs can have high
levels of sporeforming bacteria. It can be concluded that
the initial contamination level is high. The pasteurization process is sucient to inactivate all vegetative micro-organisms. To underpin this statement, a relatively
heat-stable vegetative pathogenic and spoilage bacterium should be taken into account in the risk assessment: Listeria monocytogenes and Streptococcus faecalis.
The sporeforming microorganisms are able to survive
the pasteurization process. However, the combination of
pH and salt (nitrate) prevent the pathogens Bacillus
cereus, Clostridium perfringens and Clostridium botulinum from growing. The pH and salt content are determined as CCPs and should be well-controlled according
to the HACCP-system. The important micro-organisms
with respect to spoilage are di€erent Bacillus spp. of
which Bacillus subtilis is frequently occurring and relatively fast-growing. This bacterium is therefore signi®cant to determine whether or not there are any risks
during storage at higher temperatures.
Hazard characterization. For Listeria monocytogenes
a criterion of 100 per g is assumed, taken from the Dutch
Food Commodity Act. For spoilage bacteria, normally it
suces to take an assumed spoilage limit into account,
e.g., 7 log cfu per g. However, Bacillus subtilis can also
act as pathogen if present in high numbers. Therefore,
the critical limit at the time of consumption is set together with the risk manager at 5 log cfu per g.
Exposure assessment. For Listeria monocytogenes
even in the worst case situation there is no probability of
survival during the pasteurization process, with a theoretical number of decimal reductions of 133D (ICMSF,
1996). The most heat resistant vegetative micro-organism Streptococcus faecalis will be reduced by 13D in the
mean situation and 5D in the worst-case (Magnus,
McCurdy, & Ingledew, 1988; Incze, K
ormendy,
K
ormendy, & Zsarn
oczay, 1999). Therefore, no further
assessment is done for vegetative micro-organisms. For
Bacillus subtilis a detailed risk assessment is done.
The use of probabilities gives insight in the distribution of Bacillus subtilis in the raw materials, resulting in
a probability of occurrence in the mixed raw product
(Fig. 3). The growth characteristics of the organism is
modelled, resulting in a lag-time of 48 h (mean) and a
generation time of 1.7 h (mean) at 20°C (Food MicroModel, 1999). From these inputs the number of Bacillus
subtilis during storage at 20°C can be calculated at the
pH of 6.3. Monte Carlo simulations are done in
@RISKe by running 10,000 iterations (Latin Hypercube) from the probability density functions.
Risk characterization. From the growth data of Bacillus subtilis at 20°C the probability of exceeding the
critical limit (spoilage by Bacillus subtilis) after ®ve days
is calculated. The situation will result in spoilage of
every meat product (risk is 100%) during storage at 20°C
within ®ve days. The company wants to know the e€ect
of a change of the pH of the product to 5.3 or 5.7. A
lower pH increases the lag time and generation time.
Another option is to treat the spices which contribute to
the initial contamination level with Bacillus subtilis. The
results of the measures for improvement are given in
Table 2.
Risk management. An acceptable risk can be achieved
by lowering the pH to 5.3 or lower. This might have an
Fig. 3. Probability distribution of Bacillus subtilis in the mixed raw
meat product directly after processing.
E. Hoornstra et al. / Food Control 12 (2001) 229±234
233
Table 2
Probability of spoilage by Bacillus subtilis of a meat product during ®ve days storage at 20°C with a pH of 6.3, 5.7 and 5.3, and with and without
treatment of the spices
Current situation
Option 1
Option 2
Option 3
Option 4
Option 5
Treatment spices and herbs
pH of the product
Probability of spoilage
Spoilage: 1 on . . .
No
Yes
No
Yes
No
Yes
6.3
6.3
5.7
5.7
5.3
5.3
100%
99.99%
99.18%
0.08%
0.06%
<10 7 %
1
1
1
1250
1667
>10,000
adverse e€ect on the ¯avour of the product. In combination with the treatment of the spices and herbs the pH
can be set higher. The risk manager, i.e., food company,
can make a decision based on the risk relating to costs of
the treatment of the spices and herbs and the acidity of
the product (¯avour).
The risk assessment model can be extended with other
scenarios. It may be interesting to predict the e€ect of
unfavourable events, like a higher storage temperature
(e.g., during transport) or the risks of contamination
with vegetative bacteria after pasteurization. Also, other
product and process development activities can be predicted, like the e€ect of modi®ed atmosphere packaging.
agement. This is more accurate than the worst-case approach, but still a manager can decide which approach
he thinks is best. From the exposure assessment it can be
concluded which risk factors are most important. Also
the impact of measures of improvement can be predicted
so a cost±bene®t analysis of options is possible. All these
advantages make it possible to manage food safety
problems in a quantitative way. This makes the quantitative assessment of microbiological risks a powerful
tool for food companies as an extension or validation of
the HACCP-system.
References
3. Discussion
The production of safe food is increasingly based on
the use of risk analysis. It is used to set (internationally)
Food Safety Objectives. It is also used by food companies to guarantee that the criteria are met and to derive
critical limits in order to meet the criteria in a cost-ecient way. Although food companies have the aim to
come close to a zero risk, it is known that this can never
be achieved. Carrying out a formal risk assessment as
de®ned by Codex Alimentarius Commission may take
years. Some elements can be used by food companies to
estimate the probability of occurrence of contaminants
in the endproducts. This will lead to an assessment of
the food company's risks, which can be done in a period
of weeks or months. Most important elements are a
systematic hazard identi®cation and an assessment of
the exposure. A worst-case and what-if approach is
necessary for a ®rst start in risk assessment. From these
results it can be concluded for which hazards a more
detailed assessment should be done. The statistical approach is recommended when quantitative information
about the risk is desirable. Although there is not always
much data at a risk factor, uncertainty can also be taken
into account. Further, information in literature, in existing models and from experts can all be incorporated
in the probability distribution functions. In this way it is
possible to use all the information there is in risk man-
Buchanan, R. L., Damert, W. G., Whiting, R. C., & van Schothorst,
M. (1997). Use of epidemiologic and food survey data to estimate a
purposefully conservative dose±response relationship for Listeria
monocytogenes levels and incidence of listeriosis. Journal of Food
Protection, 60(8), 918±922.
Cassin, M. H., Lammerding, A. M., Todd, E. C. D., Ross, W., &
McColl, R. S. (1998). Quantitative risk assessment for Escherichia
coli O157:H7 in ground beef hamburgers. International Journal of
Food Microbiology, 41, 21±44.
Centers for Disease Control and Prevention. (1995a). Escherichia coli
O157:H7 outbreak linked to commercially distributed dry-cured
salami. Morbidity and Mortality Weekly Report, 44(9), 157±160.
Centers for Diseese Control and Prevention. (1995b). Community
outbreak of hemolytic uremic syndrome attributable to Escherichia
coli O111:NM - South Australia. Morbidity and Mortality Weekly
Report, 44(29), 550±551, 557±558.
Coleman, M., & Marks, H. (1998). Topics in dose±response modeling.
Journal of Food Protection, 61(11), 1550±1559.
Food MicroModel. (1999). version 3.02, Ministry of Agriculture
Fisheries and Food.
Van Gerwen, S. J. C., de Wit, J. C., Notermans, S., & Zwietering, M.
H. (1997). An identi®cation procedure for foodborne microbial
hazards. International Journal of Food Microbiology, 38, 1±15.
ICMSF. (1996). Listeria monocytogenes In Micro-organisms in foods 5.
ICMSF.
Incze, K., K
ormendy, L., K
ormendy, I., & Zsarn
oczay, G. (1999).
Considerations of critical microorganisms and indicator enzymes in
connection with the pasteurization of meat products. Meat Science,
51, 115±121.
Magnus, C. A., McCurdy, A. R., & Ingledew, W. M. (1988). Further
studies on the thermal resistance of Streptococcus faecium and
Streptococcus faecalis in pasteurized ham. Canadian Instititute of
Food Science and Technology Journal, 21, 209±212.
234
E. Hoornstra et al. / Food Control 12 (2001) 229±234
Marks, H. M., Coleman, M. E., Jordan-Lin, L. C. T., & Roberts, T.
(1998). Topics in microbial risk assessment: dynamic ¯ow tree
process. Risk Analysis, 18(3), 309±328.
Notermans, S., Dufrenne, J., Teunis, P., Beumer, R., te Gi€el, M., &
Peeters-Weem, P. (1997). A risk assessment study of Bacillus cereus
present in pasteurized milk. Food Microbiology, 14(2), 143±151.
Notermans, S., Dufrenne, J., Teunis, P., & Chackraborty, T. (1998).
Studies on the risk assessment of Listeria monocytogenes. Journal of
Food Protection, 61(2), 244±248.
Teunis, P. F. M., Medema, G. J., Kruidenier, L., & Havelaar, A. H.
(1997). Assessment of the risk of infection by Cryptosporidium or
Giardia in drinking water from a surface water source. Water
Research, 31(6), 1333±1346.
Whiting, R. C., & Buchanan, R. L. (1997). Development of a
quantitative risk assessment model for Salmonella enteritidis in
pasteurized liquid eggs. International Journal of Food Microbiology,
(36), 111±125.