Crowd sourcing remote comparative lameness assessments for dairy cattle.

Lameness assessments are rarely conducted routinely on dairy farms and when completed typically underestimate lameness prevalence, hampering early diagnosis and treatment. A well-known feature of many perceptual tasks is that relative assessments are more accurate than absolute assessments, suggesting that creating methods that allow for the relative scoring of which cow is more lame will allow for reliable lameness assessments. Here we developed and tested a remote comparative lameness assessment method: we recruited nonexperienced crowd workers via an online platform and asked them to watch 2 videos side-by-side, each showing a cow walking, and to identify which cow was more lame and by how much (on a scale of -3 to 3). We created 11 tasks, each with 10 video pairs for comparison, and recruited 50 workers per task. All tasks were also completed by 5 experienced cattle lameness assessors. We evaluated data filtering and clustering methods based on worker responses and determined the agreement among workers, among experienced assessors, and between these groups. A moderate to high interobserver reliability was observed (intraclass correlation coefficient, ICC = 0.46 to 0.77) for crowd workers and agreement was high among the experienced assessors (ICC = 0.87). Average crowd-worker responses showed excellent agreement with the average of experienced assessor responses (ICC = 0.89 to 0.91), regardless of data processing method. To investigate if we could use fewer workers per task while still retaining high agreement with experienced assessors, we randomly subsampled 2 to 43 (1 less than the minimum number of workers retained per task after data cleaning) workers from each task. The agreement with experienced assessors increased substantially as we increased the number of workers from 2 to 10, but little increase was observed after 10 or more workers were used (ICC > 0.80). The proposed method provides a fast and cost-effective way to assess lameness in commercial herds. In addition, this method allows for large-scale data collection useful for training computer vision algorithms that could be used to automate lameness assessments on farm.

Pairwise comparison locomotion scoring for dairy cattle.

Conventional locomotion scoring is a subjective, absolute, and discrete assessment of locomotion. Here we assess pairwise comparison scoring to improve upon the limited intra- and interobserver consistency typical of conventional locomotion scoring. Five observers performed conventional 4-level locomotion scoring using 50 video recordings of dairy cattle, and also assessed 90 pairs of videos (composed from the same 50 recordings) using relative pairwise scoring. Intra- and interobserver consistency of pairwise scores [intraobserver: percentage agreement (PA) = 82%, κ = 0.63; interobserver: PA = 79%, κ = 0.57] were greater than of 4-level absolute scores (intraobserver: PA = 72%, κw = 0.74; interobserver: PA = 56%, κw = 0.59). Pairwise scores were scaled with an optimization method to obtain the position of the 50 recordings on a continuous locomotion scale. These continuous locomotion scores (CLS) were compared with the conventional mean absolute visual locomotion scores (VLS). Correlation between CLS and VLS was strong (τ = 0.69), and consistency between binarized CLS and binarized VLS was high (PA = 84%, κ = 0.66 for threshold VLS ≥1). Just noticeable difference (JND) for locomotion scoring was 0.3 on a 4-level scale ranging from 0 to 3. Pairwise scoring and scaling had the scoring consistency of binary absolute scoring with finer continuous granularity than 4-level absolute scoring. The pairwise scoring method, and associated scaling, offer a more consistent and informative alternative to conventional absolute multilevel locomotion scoring.

Recommendations for describing statistical studies and results in general readership science and engineering journals.

This paper recommends how authors of statistical studies can communicate to general audiences fully, clearly, and comfortably. The studies may use statistical methods to explore issues in science, engineering, and society or they may address issues in statistics specifically. In either case, readers without explicit statistical training should have no problem understanding the issues, the methods, or the results at a non-technical level. The arguments for those results should be clear, logical, and persuasive. This paper also provides advice for editors of general journals on selecting high quality statistical articles without the need for exceptional work or expense. Finally, readers are also advised to watch out for some common errors or misuses of statistics that can be detected without a technical statistical background.

The misuse of statistics: concepts, tools, and a research agenda.

This paper presents concerns regarding misuse of statistics in scientific work, especially in biomedical research. The paper discusses what is meant by "misuse." It appears that misuse arises from various sources: degrees of competence in statistical theory and methods, honest error in the application of methods, egregious negligence, and deliberate deception (misconduct.) The incidence of error is partly due to a perceived need to meet artificial statistical criteria for acceptance of research reports for publication by journals. There has been no systematic research into the prevalence of misuse or its breakdown by type. Nonetheless, there are ways to encourage, or even to enforce, good statistical practice. These can be greatly supported by use of available statistical ethics documents. This article suggests lines of further research that could define the problem more explicitly and that might lead to additional corrective measures.

Shackling the shoulders of giants. A report on excerpts from the National Academies' Symposium on the Role of Scientific and Technical Data and Information in the Public Domain, Washington, DC, September 5-6, 2002.

This paper informally summarizes a two-day symposium held at the U.S. National Academy of Sciences in Washington, D.C., September 5-6, 2002. The issue was to what extent the progress of science and societal capacity for continued technological innovation are threatened by excessive protection of intellectual property. Excessive protection creates disadvantages not only for scientists and inventors but also for educators/students and for librarians/clientele. Speakers from a variety of disciplines and institutions agreed unanimously that scientific and technological progress is, indeed, under serious threat. Various opinions were expressed about the degree of threat, currently and prospectively, as well as what counter-measures are best suited to resist undue restrictions on creative uses of scientific and technical data and information. This summary is based entirely on the author's notes from the symposium, and the commentary offered is his alone. My apologies to the speakers if this paper does not accurately reflect the primary intent of their presentations. The "Suggested Readings" offered at the end are not specific to the speakers' statements but rather are offered as a general resource to aid further research. The definitive record of the symposium is planned to be available from the National Academies Press as a Proceedings publication in the summer of 2003.