1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
TWO ON-FARM TESTS TO EVALUATE IN-LINE SENSORS FOR
MASTITIS DETECTION
C. Kamphuis, B. Dela Rue, and J. Jago
DairyNZ Ltd.
Hamilton, Waikato, New Zealand
ABSTRACT
To date, there is no independent and uniformly presented information available
regarding detection performance of automated in-line mastitis detection systems.
This lack of information makes it hard for farmers or their advisors to make
informed investment decisions. This paper describes two on-farm tests that will
provide farmers with an indicative performance of in-line mastitis sensors using
data from early adopters of sensors of interest. The first test provides insight into a
system’s ability to identifying cows treated for clinical mastitis. The second test
provides insight into a system’s ability to identify cows with a high somatic cell
count. This partial evaluation was applied to data from a New Zealand research
farm with an in-line mastitis detection system. Results showed this system had
63% sensitivity with 87 false alerts per 1000 cow milkings for identifying cows
treated for clinical mastitis. It also suggested that 10-36% of the herd should be
excluded from the bulk tank to decrease bulk milk somatic cell count by 25%.
These results can be used by farmers to determine if sensors of interest are likely
to meet their on-farm requirements and expectations.
Keywords:
Automatic mastitis detection, On-farm evaluation
INTRODUCTION
Mastitis is one of the most common and costly diseases in dairy cows (Halasa
et al., 2007; Huijps et al., 2008). Cows with clinical mastitis (CM) are identified
by visually inspecting the udder and foremilk for abnormalities. Cows with
subclinical mastitis, or a high somatic cell count (SCC), can be manually
identified by applying the California Mastitis Test. These manual detection
practices, however, are not ideal due to larger herds, more reliance on less skilled
labour and an increasing emphasis on lowering bulk milk SCC (BMSCC).
In-line mastitis sensing systems have been developed to automate the manual
practices of identifying cows with mastitis. Because of their simplicity and low
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
costs, sensing systems measuring electrical conductivity (EC) at the cow or the
quarter milk level are the most widespread and common method of automated
mastitis detection (Mottram et al., 2007). Sensors developed more recently
analyse electrical properties (or ‘complex impedance’) at the cow level, or they
provide an estimate of the SCC at the cow or the quarter level by automating the
California Mastitis Test (Whyte et al., 2004) or using NIR spectroscopy
(Tsenkova et al., 2001). Little is known about the performance of these mastitis
detection systems in the field and information that is available does not allow
comparison between systems. This makes it hard for farmers interested in buying
a detection system to make well-informed investment decisions.
A framework for evaluating in-line mastitis sensing systems has been proposed
by Kamphuis et al. (2011). This framework describes gold standards, evaluation
protocols and performance targets for three practical requirements that a mastitis
detection system should fulfil to be useful on farm: (1) identify cows with CM
promptly and accurately, (2) identify cows with high SCC for managing BMSCC
and (3) provide the infection-status of a cow at the end of lactation to support
individual drying off decisions (Fig. 1.). The proposed protocols involve costly
and rigorous testing procedures to fully evaluate a mastitis detection system. This
paper describes how a partial evaluation on the first two requirements (Fig. 1.)
will provide farmers or their advisors with an indicative performance estimate of
the sensor of interest.
MATERIALS AND METHODS
Data from the 2011/12 milking season were obtained from a New Zealand
research farm (DairyNZ, Lye Farm, Hamilton, New Zealand). Data collection
started on 24 June 2011, when cows were seasonally calved, and continued until
17 April 2012. The herd of 330 cows (85% Holstein Friesian and 15% Jersey and
Fig. 1. Flowchart demonstrating how the full and partial evaluation for inline mastitis detection systems are related, the data required to conduct the
partial evaluation, and hypothetical outcome.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
Holstein Friesian x Jersey crossbreds) were grazed rotationally on 95 ha
(predominantly perennial ryegrass/white clover-dominant pasture) and
supplemented with pasture silage when post-grazing residuals were less than
target. Cows were milked between 0630 and 0900 h and between 1500 and 1630 h
in a 30-bail rotary parlour. During the collection period, the average milk
production was 410 kg of milk solids/cow and the average BMSCC was 167,000
cells/ml.
Health records related to CM treatments were retrieved from the management
software, as well as milk weight and cow EC for each cow milking (n = 175,801).
Electrical conductivity values were expressed in ‘units’ and used to generate
mastitis alerts using the supplier recommended alert thresholds for individual cow
data as follows: a cow milking receives a mastitis alert when the current EC value
exceeds +2.8 times the standard deviation of the 10-day EC mean or when EC of
the current and prior milking both exceed +2.0 times the standard deviation of the
10-day EC mean. A detection algorithm was applied retrospectively to generate
EC-based mastitis alerts which resulted in 14,956 (8.5%) cow milkings alerted for
CM. Composite milk samples were collected approximately every 2 weeks from
all cows to determine SCC (Fossomatic 5000; Foss Electric).
Test 1: A system’s ability to identify cows treated for clinical mastitis
This test requires treatment information and sensor mastitis alert data. During
the 2011/12 milking season, 62 cows were treated for 85 CM events. The starting
date of treatment was merged with milking data by cow, milking date and milking
session. Data from cows less than 10 days in milk were excluded from analyses as
the mastitis detection algorithm required data from at least 10 milking days. This
left 169,531 cow milkings available for further analyses; 14,848 (8.8%) of these
cow milkings received a mastitis alert and 46 CM events from 34 cows remained.
Mastitis alerts were compared with CM events: if a treated cow received a
mastitis alert on the same day or the day before treatment (48 h time period), that
treatment was assigned one true positive alert. If the treated cow received no EC
alert during this time period, that treatment was assigned one false negative alert.
Each single EC alert outside the 48 h time period was considered a false positive
alert. An alternative wider time period was analysed in which a mastitis alert was
expected at the day of treatment or up to three days prior to this date (96 h). The
total number of cow milkings and the true positive, false negative and false
positive alerts were used to evaluate performance by calculating sensitivity (SN)
and the number of false alerts per 1000 cow milkings (FAR/1000) as described by
Sherlock et al. (2008).
Test 2: A system’s ability to rank cows according to total SCC contribution
This test requires herd test SCC data and herd management software data for
milk weight and sensor output for the same cow milkings used for the herd test
SCC analysis. Cows were ranked according to their total SCC contribution
(calculated by multiplying herd test SCC values by milk weight). The gold
standard was determined by plotting the cumulative percentage decrease in
BMSCC as cows were sequentially excluded from the bulk tank based on their
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
ranking. Ranking curves were created based on sensor output for the same
milking and for milkings 24 h or 72 h earlier than that used for SCC analysis. The
system’s performance was determined by comparing the herd test SCC ranking
curve with the system’s ranking curve and the proportion of ranked cows that had
to be excluded from the bulk tank to decrease the BMSCC by 25%. For this study,
three herd tests were selected to represent SCC of cows during early lactation (20
September 2011), mid lactation (13 December 2011) and late lactation (20 March
2012) and ranking curves were created for each herd test separately.
RESULTS
Of the 46 CM events, 27 were alerted by the EC-based mastitis detection
algorithm within the 48 h time period (SN = 63%) and 14,814 cow milkings
created false alerts (FAR/1000 = 87). Extending the time period to 96 h increased
SN considerably (74%), with the FAR/1000 remaining at the same level of 87.
Ranking curves based on total SCC contribution using data from herd tests
consistently indicated that ~1.5% of the ranked cows should be excluded from the
bulk tank to decrease BMSCC by 25% (Fig. 2.; Table 1). This was irrespective of
the BMSCC level at that herd test (Table 1). In contrast, ranking cows according
to sensor output from the same milking used for herd testing was not as effective
and varied greatly between herd tests. In early lactation, 8.9% of the ranked cows
were be excluded and this increased up to 35.6% of the ranked cows in late
lactation (Fig. 2.; Table 1). Ranking cows based on sensor output of milkings 24 h
or 72 h prior to the milking that was used for herd testing also showed
considerable variation in the proportion of cows that should be excluded to
decrease BMSCC by 25% (Table 1).
Table 1. Percentage of ranked cows that had to be excluded from the bulk
tank to decrease BMSCC by 25% (or the nearest percentage below). Cows
were ranked based on total SCC contribution in early, mid and late lactation
using herd test data or on sensor output for the same milkings, and milkings
24 h or 72 h earlier.
Lactation
stage
Cows
tested (#)
BMSCC
(x103
cells/ml)
Early
Mid
Late
326
329
329
100
118
175
Percentage of cows to be excluded to decrease
BMSCC by 25%
Based on
Based on sensor output ranking
herd test
SCC
ranking
Same
24 h
72 h
milking
earlier
earlier
1.2
8.9
2.8
16.3
1.2
22.5
44.9
42.6
1.8
35.6
30.7
75.1
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
Fig. 2. Ranking curves when ranking of cows is done according to herd test
values (solid lines) or on EC values (dotted lines). Black curves represent
ranking in early lactation, dark grey curves represent ranking in mid
lactation and light grey curves represent ranking in late lactation.
DISCUSSION
The partial evaluation uses CM treatment data as a proxy for total CM events.
By doing this, the partial evaluation uses data that are easier and less costly to
obtain than data used in the full evaluation (Kamphuis et al., 2011) but still offers
an indicative performance of a sensor of interest given that complete records of
CM treatments are available. Treated cases of CM, however, may not be the same
as the total number of CM events on a farm; farmers may chose not to treat cows
with CM and instead do nothing, decide to dry off the infected quarter or cow
earlier or cull. As a result, treated cases of CM is likely to be an underestimation
of the total number of CM events on a farm, thus the outcome of the partial
evaluation is likely to be an overestimation of a system’s ability to identify cows
with CM. The system evaluated in the current study had a SN of 63% with a
FAR/1000 of 87. This means that 37% of all treated CM cases were identified by
other means than sensor alerts and that farm staff checked ~30 cows (9% of the
herd) for CM unnecessarily at each milking session. With this performance, the
system fails to meet performance targets for CM detection proposed by Kamphuis
et al. (2011; 80% SN with <10 FAR/1000). This indicates this system is likely to
be insufficient for farmers that want an accurate detection system to reduce
dependency on farm staff. On the other hand, this system may be sufficient to
assist unskilled staff in detecting cows with CM as the costs of labour associated
with checking large numbers of cows to locate one CM event may still outweigh
the costs of failure to identify a CM event.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
Data to evaluate a system’s ability to rank cows according to their total SCC
contribution in this partial evaluation were exactly the same as used in the full
evaluation (Kamphuis et al., 2011) and the performance target proposed by
Kamphuis et al. (2011) can be applied. That target suggests that a system should
not exclude twice the percentage of the herd as indicated by the gold standard to
decrease BMSCC by 25%. According to the system’s ranking, 10-36% of the herd
had to be excluded to decrease BMSCC by 25% compared to 1.5% when herd test
information was used. The consistent and large discrepancy between SCC and
sensor data, and thus the consistent failure in meeting the performance target,
suggests that this sensor will not be useful for farmers having difficulty with
managing BMSCC or for farmers that want to decrease their overall BMSCC.
Despite the fact that the partial evaluation tests a sensor for two out of three
practical requirements (Fig. 1.) and the fact that it uses a proxy measure for CM
events, the outcome of this partial evaluation can be used by farmers to retrieve an
indicative performance. This information can be used to identify the extent to
which a sensor is likely to meet a farmer’s on-farm requirements and
expectations.
CONCLUDING REMARKS
The partial evaluation was simple to perform, using data that are readily
available and less costly to retrieve than when carrying out a full evaluation for
mastitis detection systems. The outcome of this partial evaluation will provide an
indicative performance of a system’s ability to identify cows treated for CM and
of its ability to rank cows for BMSCC management purposes. This indicative
performance can be used by farmers to determine whether a particular detection
system will meet their on-farm requirements and expectations.
ACKNOWLEDGEMENT
This paper was funded by the Ministry for Primary Industries, Primary Growth
Partnership and by DairyNZ on behalf of New Zealand Dairy Farmers (Project
SY1006).
REFERENCES
Halasa, T., K. Huijps, O. Østerås, H. Hogeveen. 2007. Economic effects of bovine
mastitis and mastitis management: a review. Veterinary Quarterly. 29: 18-31.
Huijps, K., T.J.G.M. Lam, H. Hogeveen. 2008. Costs of mastitis: facts and
perception. Journal of Dairy Research. 75: 113-120.
Kamphuis, C., B. Dela Rue, G. Mein, J. Jago. 2011. Practical evalaution of
automatic in-line mastitis sensors. Pp. 100-104 in Proceedings of the 3rd
international symposium on mastitis and milk quality, 22-24 September 2011,
St. Louis, USA.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
Mottram, T., A. Rudnitskaya, A. Legin, J.L. Fitzpatrick, P.D. Eckersall. 2007.
Evaluation of a novel chemical sensor system to detection clinical mastitis in
bovine milk. Biosensors and Bioelectronics. 22: 2689-2693.
Sherlock, R., H. Hogeveen, G. Mein, M.D. Rasmussen. 2008. Performance
evalaution of systems for automated monitoring of udder health: analytical
issues and guidelines. Pp. 275-282 In Mastitis control – From science to
practice. T.J.G.M. Lam (ed.). 30 September – 2 October 2008, The Hague, the
Netherlands, Wageningen Academic Publishers, Wageningen, the Netherlands.
Tsenkova, R., S. Atanassova, S. Kawano, K. Toyoda. 2001. Somatic cell count
determination in cows’ milk by near-infrared spectroscopy; an new diagnostic
tool. Journal of Animal Science. 79: 2550-2557.
Whyte, D.S., R.G. Orchard, P.S. Cross, T. Frietsch, R.W. Claycomb, G.M. Mein.
2004. An on-line somatic cell count sensor. Pp. 235-240 In Automatic milking:
a better understanding. A. Meijering, H. Hogeveen and C.J.A.M. de Koning
(eds.). 24-26 March 2004, Lelystad, the Netherlands, Wageningen Academic
Publishers, Wageningen, the Netherlands.