Performance of human observers and an automatic 3-dimensional computer-vision-based locomotion scoring method to detect lameness and hoof lesions in dairy cows.

The objective of this study was to determine if a 3-dimensional computer vision automatic locomotion scoring (3D-ALS) method was able to outperform human observers for classifying cows as lame or nonlame and for detecting cows affected and nonaffected by specific type(s) of hoof lesion. Data collection was carried out in 2 experimental sessions (5 mo apart). In every session all cows were assessed for (1) locomotion by 2 observers (Obs1 and Obs2) and by a 3D-ALS; and (2) identification of different types of hoof lesions during hoof trimming (i.e., skin and horn lesions and combinations of skin/horn lesions and skin/hyperplasia). Performances of observers and 3D-ALS for classifying cows as lame or nonlame and for detecting cows affected or nonaffected by types of lesion were estimated using the percentage of agreement (PA), kappa coefficient (κ), sensitivity (SEN), and specificity (SPE). Observers and 3D-ALS showed similar SENlame values for classifying lame cows as lame (SENlame comparison Obs1-Obs2 = 74.2%; comparison observers-3D-ALS = 73.9-71.8%). Specificity values for classifying nonlame cows as nonlame were lower for 3D-ALS when compared with observers (SPEnonlame comparison Obs1-Obs2 = 88.5%; comparison observers-3D-ALS = 65.3-67.8%). Accordingly, overall performance of 3D-ALS for classifying cows as lame and nonlame was lower than observers (Obs1-Obs2 comparison PAlame/nonlame = 84.2% and κlame/nonlame = 0.63; observers-3D-ALS comparisons PAlame/nonlame = 67.7-69.2% and κlame/nonlame = 0.33-0.36). Similarly, observers and 3D-ALS had comparable and moderate SENlesion values for detecting horn (SENlesion Obs1 = 68.6%; Obs2 = 71.4%; 3D-ALS = 75.0%) and combinations of skin/horn lesions (SENlesion Obs1 = 51.1%; Obs2 = 64.5%; 3D-ALS = 53.3%). The SPEnonlesion values for detecting cows without lesions when classified as nonlame were lower for 3D-ALS than for observers (SPEnonlesion Obs1 = 83.9%; Obs2 = 80.2%; 3D-ALS = 60.2%). This was translated into a poor overall performance of 3D-ALS for detecting cows affected and nonaffected by horn lesions (PAlesion/nonlesion Obs1 = 80.6%; Obs2 = 78.3%; 3D-ALS = 63.5% and κlesion/nonlesion Obs1 = 0.48; Obs2 = 0.44; 3D-ALS = 0.25) and skin/horn lesions (PAlesion/nonlesion Obs1 = 75.1%; Obs2 = 75.9%; 3D-ALS = 58.6% and κlesion/nonlesion Obs1 = 0.35; Obs2 = 0.42; 3D-ALS = 0.10), when compared with observers. Performance of observers and 3D-ALS for detecting skin lesions was poor (SENlesion for Obs1, Obs2, and 3D-ALS <40%). Comparable SENlame and SENlesion values for observers and 3D-ALS are explained by an overestimation of lameness by 3D-ALS when compared with observers. Thus, comparable SENlame and SENlesion were reached at the expense high number of false positives and low SPEnonlame and SPEnonlesion. Considering that observers and 3D-ALS showed similar performance for classifying cows as lame and for detecting horn and combinations of skin/horn lesions, the 3D-ALS could be a useful tool for supporting dairy farmers in their hoof health management.

Relation between observed locomotion traits and locomotion score in dairy cows.

Lameness is still an important problem in modern dairy farming. Human observation of locomotion, by looking at different traits in one go, is used in practice to assess locomotion. The objectives of this article were to determine which individual locomotion traits are most related to locomotion scores in dairy cows, and whether experienced raters are capable of scoring these individual traits consistently. Locomotion and 5 individual locomotion traits (arched back, asymmetric gait, head bobbing, reluctance to bear weight, and tracking up) were scored independently on a 5-level scale for 58 videos of different cows. Videos were shown to 10 experienced raters in 2 different scoring sessions. Relations between locomotion score and traits were estimated by 3 logistic regression models aiming to calculate the size of the fixed effects on the probability of scoring a cow in 1 of the 5 levels of the scale (model 1) and the probability of classifying a cow as lame (locomotion score ≥3; model 2) or as severely lame (locomotion score ≥4; model 3). Fixed effects were rater, session, traits, and interactions among fixed effects. Odds ratios were calculated to estimate the relative probability to classify a cow as lame when an altered (trait score ≥3) or severely altered trait (trait score ≥4) was present. Overall intrarater and interrater reliability and agreement were calculated as weighted kappa coefficient (κw) and percentage of agreement, respectively. Specific intrarater and interrater agreement for individual levels within a 5-level scale were calculated. All traits were significantly related to the locomotion score when scored with a 5-level scale and when classified as (severely) lame or nonlame. Odds ratios for altered and severely altered traits were 10.8 and 14.5 for reluctance to bear weight, 6.5 and 7.2 for asymmetric gait, and 4.8 and 3.2 for arched back, respectively. Raters showed substantial variation in reliability and agreement values when scoring traits. The acceptance threshold for overall intrarater reliability (κw ≥0.60) was exceeded by locomotion scoring and all traits. Overall interrater reliability values ranged from κw=0.53 for tracking up to κw=0.61 for reluctance to bear weight. Intrarater and interrater agreement were below the acceptance threshold (percentage of agreement <75%). Most traits tended to have lower specific intrarater and interrater agreement in level 3 and 5 of the scale. In conclusion, raters had difficulties in scoring locomotion traits consistently, especially slight alterations were difficult to detect by experienced raters. Yet, the locomotion traits reluctance to bear weight, asymmetric gait, and arched back had the strongest relation with the locomotion score. These traits should have priority in locomotion-scoring-system guidelines and are the best to be used for the development of automated locomotion scoring systems.

Effect of merging levels of locomotion scores for dairy cows on intra- and interrater reliability and agreement.

Locomotion scores are used for lameness detection in dairy cows. In research, locomotion scores with 5 levels are used most often. Analysis of scores, however, is done after transformation of the original 5-level scale into a 4-, 3-, or 2-level scale to improve reliability and agreement. The objective of this study was to evaluate different ways of merging levels to optimize resolution, reliability, and agreement of locomotion scores for dairy cows. Locomotion scoring was done by using a 5-level scale and 10 experienced raters in 2 different scoring sessions from videos from 58 cows. Intra- and interrater reliability and agreement were calculated as weighted kappa coefficient (κw) and percentage of agreement (PA), respectively. Overall intra- and interrater reliability and agreement and specific intra- and interrater agreement were determined for the 5-level scale and after transformation into 4-, 3-, and 2-level scales by merging different combinations of adjacent levels. Intrarater reliability (κw) ranged from 0.63 to 0.86, whereas intrarater agreement (PA) ranged from 60.3 to 82.8% for the 5-level scale. Interrater κw=0.28 to 0.84 and interrater PA=22.6 to 81.8% for the 5-level scale. The specific intrarater agreement was 76.4% for locomotion level 1, 68.5% for level 2, 65% for level 3, 77.2% for level 4, and 80% for level 5. Specific interrater agreement was 64.7% for locomotion level 1, 57.5% for level 2, 50.8% for level 3, 60% for level 4, and 45.2% for level 5. Specific intra- and interrater agreement suggested that levels 2 and 3 were more difficult to score consistently compared with other levels in the 5-level scale. The acceptance threshold for overall intra- and interrater reliability (κw and κ ≥0.6) and agreement (PA ≥75%) and specific intra- and interrater agreement (≥75% for all levels within locomotion score) was exceeded only for the 2-level scale when the 5 levels were merged as (12)(345) or (123)(45). In conclusion, when locomotion scoring is performed by experienced raters without further training together, the lowest specific intra- and interrater agreement was obtained in levels 2 and 3 of the 5-level scale. Acceptance thresholds for overall intra- and interrater reliability and agreement and specific intra- and interrater agreement were exceeded only in the 2-level scale.

Manual and automatic locomotion scoring systems in dairy cows: a review.

The objective of this review was to describe, compare and evaluate agreement, reliability, and validity of manual and automatic locomotion scoring systems (MLSSs and ALSSs, respectively) used in dairy cattle lameness research. There are many different types of MLSSs and ALSSs. Twenty-five MLSSs were found in 244 articles. MLSSs use different types of scale (ordinal or continuous) and different gait and posture traits need to be observed. The most used MLSS (used in 28% of the references) is based on asymmetric gait, reluctance to bear weight, and arched back, and is scored on a five-level scale. Fifteen ALSSs were found that could be categorized according to three approaches: (a) the kinetic approach measures forces involved in locomotion, (b) the kinematic approach measures time and distance of variables associated to limb movement and some specific posture variables, and (c) the indirect approach uses behavioural variables or production variables as indicators for impaired locomotion. Agreement and reliability estimates were scarcely reported in articles related to MLSSs. When reported, inappropriate statistical methods such as PABAK and Pearson and Spearman correlation coefficients were commonly used. Some of the most frequently used MLSSs were poorly evaluated for agreement and reliability. Agreement and reliability estimates for the original four-, five- or nine-level MLSS, expressed in percentage of agreement, kappa and weighted kappa, showed large ranges among and sometimes also within articles. After the transformation into a two-level scale, agreement and reliability estimates showed acceptable estimates (percentage of agreement ≥ 75%; kappa and weighted kappa ≥ 0.6), but still estimates showed a large variation between articles. Agreement and reliability estimates for ALSSs were not reported in any article. Several ALSSs use MLSSs as a reference for model calibration and validation. However, varying agreement and reliability estimates of MLSSs make a clear definition of a lameness case difficult, and thus affect the validity of ALSSs. MLSSs and ALSSs showed limited validity for hoof lesion detection and pain assessment. The utilization of MLSSs and ALSSs should aim to the prevention and efficient management of conditions that induce impaired locomotion. Long-term studies comparing MLSSs and ALSSs while applying various strategies to detect and control unfavourable conditions leading to impaired locomotion are required to determine the usefulness of MLSSs and ALSSs for securing optimal production and animal welfare in practice.