Rapid turnover of sensor data to genetic evaluation for dairy cows in the cloud.

More and more sensor and automation data are available that enable animal breeders to define novel traits. However, sensor and automation data are often frequently measured differently (e.g., milk yield and different milk components are continuously measured during each milking). These differences are challenging animal breeders to define traits and use the most appropriate analytical models for genetic evaluation and breeding values. Traditionally, the process from raw data to breeding value estimations involves several steps: data curation, trait definition, variance component estimation, genetic evaluation, and validation of the estimated breeding values (EBV). All these steps often take many iterations and several research projects to optimize the final genetic evaluations. To make this entire process-from raw data to validated EBV-more efficient, we combined all these steps in a cloud environment that allows for faster processing and a faster data distribution time. We used real data (including 1,782,373,113 daily milk-yield records of 1,120,550 dairy cows) and a real trait (a resilience trait based on the deviations from expected milk yields) to demonstrate the functioning of this cloud environment. The daily milk-yield records were incorporated into our cloud solution, in which we have set up central binary large object storage. Subsequent steps were all performed in the cloud. The data set was preprocessed in approximately 6 h to obtain the resilience indicator for 352,871 cows in the first 3 lactations. Estimation of genetic parameters (heritabilities and genetic correlations) was performed by splitting the data into 5 subsets in ASReml, and prediction of subsequent EBV was performed on the entire data set using MiXBLUP. Together with the validation of breeding values, this process encompassed 16.5 h. By combining the different steps from preprocessing sensor data to genetic evaluation of new traits in one cloud environment, we generated EBV and validation plots in approximately 1 working day. Moreover, our setup is a flexible design and can be adapted easily to test new, longitudinal sensor-driven traits and compare the performance of these new traits to previous ones.

Invited review: Toward a common language in data-driven mastitis detection research.

Sensor technologies for mastitis detection have resulted in the collection and availability of a large amount of data. As a result, scientific publications reporting mastitis detection research have become less driven by approaches based on biological assumptions and more by data-driven modeling. Most of these approaches try to predict mastitis events from (combinations of) raw sensor data to which a wide variety of methods are applied originating from machine learning and classical statistical approaches. However, an even wider variety in terminologies is used by researchers for methods that are similar in nature. This makes it difficult for readers from other disciplines to understand the specific methods that are used and how these differ from each other. The aim of this paper was to provide a framework (filtering, transformation, and classification) for describing the different methods applied in sensor data-based clinical mastitis detection research and use this framework to review and categorize the approaches and underlying methods described in the scientific literature on mastitis detection. We identified 40 scientific publications between 1992 and 2020 that applied methods to detect clinical mastitis from sensor data. Based on these publications, we developed and used the framework and categorized these scientific publications into the 2 data processing techniques of filtering and transformation. These data processing techniques make raw data more amendable to be used for the third step in our framework, that of classification, which is used to distinguish between healthy and nonhealthy (mastitis) cows. Most publications (n = 34) used filtering or transformation, or a combination of these 2, for data processing before classification, whereas the remaining publications (n = 6) classified the observations directly from raw data. Concerning classification, applying a simple threshold was the most used method (n = 19 publications). Our work identified that within approaches several different methods and terminologies for similar methods were used. Not all publications provided a clear description of the method used, and therefore it seemed that different methods were used between publications, whereas in fact just a different terminology was used, or the other way around. This paper is intended to serve as a reference for people from various research disciplines who need to collaborate and communicate efficiently about the topic of sensor-based mastitis detection and the methods used in this context. The framework used in this paper can support future research to correctly classify approaches and methods, which can improve the understanding of scientific publication. We encourage future research on sensor-based animal disease detection, including that of mastitis detection, to use a more coherent terminology for methods, and clearly state which technique (e.g., filtering) and approach (e.g., moving average) are used. This paper, therefore, can serve as a starting point and further stimulates the interdisciplinary cooperation in sensor-based mastitis research.

Storing, combining and analysing turkey experimental data in the Big Data era.

With the increasing availability of large amounts of data in the livestock domain, we face the challenge to store, combine and analyse these data efficiently. With this study, we explored the use of a data lake for storing and analysing data to improve scalability and interoperability. Data originated from a 2-day animal experiment in which the gait score of approximately 200 turkeys was determined through visual inspection by an expert. Additionally, inertial measurement units (IMUs), a 3D-video camera and a force plate (FP) were installed to explore the effectiveness of these sensors in automating the visual gait scoring. We deployed a data lake using the IMU and FP data of a single day of that animal experiment. This encompasses data from 84 turkeys for which we preprocessed by performing an 'extract, transform and load' (ETL-) procedure. To test scalability of the ETL-procedure, we simulated increasing volumes of the available data from this animal experiment and computed the 'wall time' (elapsed real time) for converting FP data into comma-separated files and storing these files. With a simulated data set of 30 000 turkeys, the wall time reduced from 1 h to less than 15 min, when 12 cores were used compared to 1 core. This demonstrated the ETL-procedure to be scalable. Subsequently, a machine learning (ML) pipeline was developed to test the potential of a data lake to automatically distinguish between two classses, that is, very bad gait scores v. other scores. In conclusion, we have set up a dedicated customized data lake, loaded data and developed a prediction model via the creation of an ML pipeline. A data lake appears to be a useful tool to face the challenge of storing, combining and analysing increasing volumes of data of varying nature in an effective manner.