Improving the Organic Certification System
Workshop in Brussels, October 14, 2011
Raffaele Zanoli
Università Politecnica delle Marche, IT
Seventh Framework
Programme
Grant Agreement No.
207727

The goal of Risk Based Inspections (RBIs) is to develop a cost-effective inspection and maintenance program that provides assurance of acceptable integrity and reliability of a control system
A risk based approach to inspection planning is used to:
Ensure risk is reduced as low as reasonably practicable
Optimize the inspection schedule
Focus inspection effort onto the most critical areas
Identify and use the most appropriate methods of inspection

1. Assessment and description of the current inspection practices in terms of risk and efficiency
2. Define a probabilistic model to increase the efficiency of the system based on probability theory
3. Optimisation of enforcement measures to reduce the occurrence of objectionable organic production

In order to predict the risk of non-compliance
At farmer/operator level
Depending on crop type, farm type, geographic location, operators characteristics, etc.
We need data on: detected non-compliances; structural, financial and managerial information at operator level

Collected during CERTCOST EU project
Data from 6 different European CBs (from CH,
CZ, DE, DK, IT, UK)
Three years covered (2007-2009)
We used standard data that is routinely recorded by inspection bodies

Databases mainly contain structural data
CBs collect NC data with non-homogenised textual descriptions: hard to rank NC severity
Sanction data are more standardised, but: they are only a proxy of NC no common definition of sanctions across CBs / countries no clear relationship between NC and sanctions (with some exceptions); no information available about why an operator receives a sanction (e.g. use of pesticides in wheat production, use of unauthorised feed for livestock, etc.)

IT, CZ (and UK) CBs use a similar 4 sanction category (UK:
NC)classification
Further aggregation in terms of slight and severe sanction categories
IT, CZ, UK straightforward interpretation; DE, DK, CH: input from CBs to correctly classify sanctions

Econometric models
Parametric approach: a-priori distributional assumptions
Statistical testing of each risk factor, count models
Good for standardised analysis across countries/CBs
Bayesian
Networks
Semiparametric approach: more flexible, no individual factor testing allowed
Single and multiple risk factors impact evaluation
Good for farm type simulations

46 hypothesis concerning factors affecting the probability for an operator to get a sanction has been generated with collaboration from partners
The hypothesis refers to the following aspects: general risk, structural / managerial for farms, structural/managerial for processors, specific crop, livestock and product variables, control related issues
Some of the hypothesis cannot be tested for all countries/years due to missing data (eg processor turnover, risk class)

Few risk factors found relevant for all countries: Past behaviour, Farm
Size, Bovine livestock
History dependence : operators who are not compliant tend to continue to be so if one operator has been non compliant the previous year is more likely to be non compliant in the next year
If one operator has committed minor irregularities is more likely to be found to have committed major infringements
No overall risk pattern for crop types, though country specific risks
For livestock, bovines and pigs entail higher risk
In countries where (slight) non compliances are more numerous (DK,
UK, partly CH) there might be a higher farms homogeneity, hence lower discrimination effects of explanatory variables
Personal, farmer-specific variables are probably crucial in explaining risk but we have VERY limited data on these

We can say with some confidence which factors contribute to risk, but we cannot rule out those who don ’t
As a consequence, we cannot define low risk operator types
To implement more efficient Risk Based Inspection procedures
CBs would need better or different datasets
RBI based on past experience can limit predictable risk, but cannot avoid potential ‘catastrophic’ events uncertainty is an essential factor that should inform inspection procedures (black swans): think what can impact (the sector, the consumer, the CB, etc.) most, even if the risk (probability) of occurrence is low (but maybe the cost of detection is also low)

Harmonised RBI is fundamental to guarantee integrity, improve efficiency and reduce the cost of inspection: a growing body of small
“organic” farmers and growers are refusing certification and inspection schemes and selling on alternative short supply-chains – this creates further confusion among consumers
Without clear and uniform criteria for classifying non-compliances as irregularities or infringement AND without better data and better information systems, no RBI system can work on a global scale
Without global trust on certification and inspection procedures no global organic trade can survive

Seventh Framework
Programme
Grant Agreement No.
207727
zanoli@agrecon.univpm.it

Data issues:
Data suffer from censoring (i.e. missing data ): we only have information on NCs that were detected by the CBs, but we have no idea how many and what kind of NCs have NOT been detected
Inspection data contain varying quality/quantity of management & structural data, but little/no personal information on operators
All operators should be inspected at least once per year (legal requirement), but the share of subsequent inspections (either unannounced or follow ups) varies across countries and CBs
Data are little/no harmonised both within a country and across various countries

Epistemological/methodological issues:
What is the data generating process (DGP)? Since CBs are actually using some form of internal RBI protocol to inform timing of compulsory announced inspections as well as follow-ups and unannounced inspections, the risk factors that we have observed may simply depend on their inspection planning and NOT actual risk ( confirmation bias )
Due to limited amount of severe NCs and related sanctions in the database, the reliability of the analysis of factors influencing severe risks is limited by statistical reasons