RESUMO
This article investigates the problem of communication-efficient and resilient multiagent reinforcement learning (MARL). Specifically, we consider a setting where a set of agents are interconnected over a given network, and can only exchange information with their neighbors. Each agent observes a common Markov Decision Process and has a local cost which is a function of the current system state and the applied control action. The goal of MARL is for all agents to learn a policy that optimizes the infinite horizon discounted average of all their costs. Within this general setting, we consider two extensions to existing MARL algorithms. First, we provide an event-triggered learning rule where agents only exchange information with their neighbors if a certain triggering condition is satisfied. We show that this enables learning while reducing the amount of communication. Next, we consider the scenario where some of the agents can be adversarial (as captured by the Byzantine attack model), and arbitrarily deviate from the prescribed learning algorithm. We establish a fundamental trade-off between optimality and resilience when Byzantine agents are present. We then create a resilient algorithm and show almost sure convergence of all reliable agents' value functions to the neighborhood of the optimal value function of all reliable agents, under certain conditions on the network topology. When the optimal Q -values are sufficiently separated for different actions, we show that all reliable agents can learn the optimal policy under our algorithm.
RESUMO
Precision livestock farming (PLF) offers a strategic solution to enhance the management capacity of large animal groups, while simultaneously improving profitability, efficiency, and minimizing environmental impacts associated with livestock production systems. Additionally, PLF contributes to optimizing the ability to manage and monitor animal welfare while providing solutions to global grand challenges posed by the growing demand for animal products and ensuring global food security. By enabling a return to the "per animal" approach by harnessing technological advancements, PLF enables cost-effective, individualized care for animals through enhanced monitoring and control capabilities within complex farming systems. Meeting the nutritional requirements of a global population exponentially approaching ten billion people will likely require the density of animal proteins for decades to come. The development and application of digital technologies are critical to facilitate the responsible and sustainable intensification of livestock production over the next several decades to maximize the potential benefits of PLF. Real-time continuous monitoring of each animal is expected to enable more precise and accurate tracking and management of health and well-being. Importantly, the digitalization of agriculture is expected to provide collateral benefits of ensuring auditability in value chains while assuaging concerns associated with labor shortages. Despite notable advances in PLF technology adoption, a number of critical concerns currently limit the viability of these state-of-the-art technologies. The potential benefits of PLF for livestock management systems which are enabled by autonomous continuous monitoring and environmental control can be rapidly enhanced through an Internet of Things approach to monitoring and (where appropriate) closed-loop management. In this paper, we analyze the multilayered network of sensors, actuators, communication, networking, and analytics currently used in PLF, focusing on dairy farming as an illustrative example. We explore the current state-of-the-art, identify key shortcomings, and propose potential solutions to bridge the gap between technology and animal agriculture. Additionally, we examine the potential implications of advancements in communication, robotics, and artificial intelligence on the health, security, and welfare of animals.
Precision technologies are revolutionizing animal agriculture by enhancing the management of animal welfare and productivity. To fully realize the potential benefits of precision livestock farming (PLF), the development and application of digital technologies are needed to facilitate the responsible and sustainable intensification of livestock production over the next several decades. Importantly, the digitalization of agriculture is expected to provide collateral benefits of ensuring audibility in value chains while assuaging concerns associated with labor shortages. In this paper, we analyze the multilayered network of sensors, actuators, communication, and analytics currently in use in PLF. We analyze the various aspects of sensing, communication, networking, and intelligence on the farm leveraging dairy farms as an example system. We also discuss the potential implications of advancements in communication, robotics, and artificial intelligence on the security and welfare of animals.