Plant science in the age of simulation intelligence.

Historically, plant and crop sciences have been quantitative fields that intensively use measurements and modeling. Traditionally, researchers choose between two dominant modeling approaches: mechanistic plant growth models or data-driven, statistical methodologies. At the intersection of both paradigms, a novel approach referred to as "simulation intelligence", has emerged as a powerful tool for comprehending and controlling complex systems, including plants and crops. This work explores the transformative potential for the plant science community of the nine simulation intelligence motifs, from understanding molecular plant processes to optimizing greenhouse control. Many of these concepts, such as surrogate models and agent-based modeling, have gained prominence in plant and crop sciences. In contrast, some motifs, such as open-ended optimization or program synthesis, still need to be explored further. The motifs of simulation intelligence can potentially revolutionize breeding and precision farming towards more sustainable food production.

Effective cloth folding trajectories in simulation with only two parameters.

Robotic cloth folding remains challenging for robots due to its highly deformable nature. In order to deal with these deformations, several strategies with varying amounts of adaptability have been proposed. We study robotic cloth folding by simulating and evaluating a trajectory search space with only two parameters: one parameter for the trajectory's height and one parameter to tilt it. We extensively analyzed folding a long-sleeved shirt in a high-fidelity simulator. To demonstrate that the trajectory is sufficiently adaptable and robust, we test several cloth shapes, cloth materials, an entire folding sequence and different folding velocities. We can deal with every folding scenario by tuning the two parameters correctly. The trajectories' simplicity and their robustness in simulation make them ideal candidates for future transfer to real-world robotic setups.

Leveraging plant physiological dynamics using physical reservoir computing.

Plants are complex organisms subject to variable environmental conditions, which influence their physiology and phenotype dynamically. We propose to interpret plants as reservoirs in physical reservoir computing. The physical reservoir computing paradigm originates from computer science; instead of relying on Boolean circuits to perform computations, any substrate that exhibits complex non-linear and temporal dynamics can serve as a computing element. Here, we present the first application of physical reservoir computing with plants. In addition to investigating classical benchmark tasks, we show that Fragaria × ananassa (strawberry) plants can solve environmental and eco-physiological tasks using only eight leaf thickness sensors. Although the results indicate that plants are not suitable for general-purpose computation but are well-suited for eco-physiological tasks such as photosynthetic rate and transpiration rate. Having the means to investigate the information processing by plants improves quantification and understanding of integrative plant responses to dynamic changes in their environment. This first demonstration of physical reservoir computing with plants is key for transitioning towards a holistic view of phenotyping and early stress detection in precision agriculture applications since physical reservoir computing enables us to analyse plant responses in a general way: environmental changes are processed by plants to optimise their phenotype.

MIRRA: A Modular and Cost-Effective Microclimate Monitoring System for Real-Time Remote Applications.

Monitoring climate change, and its impacts on ecological, agricultural, and other societal systems, is often based on temperature data derived from official weather stations. Yet, these data do not capture most microclimates, influenced by soil, vegetation and topography, operating at spatial scales relevant to the majority of organisms on Earth. Detecting and attributing climate change impacts with confidence and certainty will only be possible by a better quantification of temperature changes in forests, croplands, mountains, shrublands, and other remote habitats. There is an urgent need for a novel, miniature and simple device filling the gap between low-cost devices with manual data download (no instantaneous data) and high-end, expensive weather stations with real-time data access. Here, we develop an integrative real-time monitoring system for microclimate measurements: MIRRA (Microclimate Instrument for Real-time Remote Applications) to tackle this problem. The goal of this platform is the design of a miniature and simple instrument for near instantaneous, long-term and remote measurements of microclimates. To that end, we optimised power consumption and transfer data using a cellular uplink. MIRRA is modular, enabling the use of different sensors (e.g., air and soil temperature, soil moisture and radiation) depending upon the application, and uses an innovative node system highly suitable for remote locations. Data from separate sensor modules are wirelessly sent to a gateway, thus avoiding the drawbacks of cables. With this sensor technology for the long-term, low-cost, real-time and remote sensing of microclimates, we lay the foundation and open a wide range of possibilities to map microclimates in different ecosystems, feeding a next generation of models. MIRRA is, however, not limited to microclimate monitoring thanks to its modular and wireless design. Within limits, it is suitable or any application requiring real-time data logging of power-efficient sensors over long periods of time. We compare the performance of this system to a reference system in real-world conditions in the field, indicating excellent correlation with data collected by established data loggers. This proof-of-concept forms an important foundation to creating the next version of MIRRA, fit for large scale deployment and possible commercialisation. In conclusion, we developed a novel wireless cost-effective sensor system for microclimates.

Modular Piezoresistive Smart Textile for State Estimation of Cloths.

Smart textiles have found numerous applications ranging from health monitoring to smart homes. Their main allure is their flexibility, which allows for seamless integration of sensing in everyday objects like clothing. The application domain also includes robotics; smart textiles have been used to improve human-robot interaction, to solve the problem of state estimation of soft robots, and for state estimation to enable learning of robotic manipulation of textiles. The latter application provides an alternative to computationally expensive vision-based pipelines and we believe it is the key to accelerate robotic learning of textile manipulation. Current smart textiles, however, maintain wired connections to external units, which impedes robotic manipulation, and lack modularity to facilitate state estimation of large cloths. In this work, we propose an open-source, fully wireless, highly flexible, light, and modular version of a piezoresistive smart textile. Its output stability was experimentally quantified and determined to be sufficient for classification tasks. Its functionality as a state sensor for larger cloths was also verified in a classification task where two of the smart textiles were sewn onto a piece of clothing of which three states are defined. The modular smart textile system was able to recognize these states with average per-class F1-scores ranging from 85.7 to 94.6% with a basic linear classifier.

From leaf to label: A robust automated workflow for stomata detection.

Plant leaf stomata are the gatekeepers of the atmosphere-plant interface and are essential building blocks of land surface models as they control transpiration and photosynthesis. Although more stomatal trait data are needed to significantly reduce the error in these model predictions, recording these traits is time-consuming, and no standardized protocol is currently available. Some attempts were made to automate stomatal detection from photomicrographs; however, these approaches have the disadvantage of using classic image processing or targeting a narrow taxonomic entity which makes these technologies less robust and generalizable to other plant species. We propose an easy-to-use and adaptable workflow from leaf to label. A methodology for automatic stomata detection was developed using deep neural networks according to the state of the art and its applicability demonstrated across the phylogeny of the angiosperms.We used a patch-based approach for training/tuning three different deep learning architectures. For training, we used 431 micrographs taken from leaf prints made according to the nail polish method from herbarium specimens of 19 species. The best-performing architecture was tested on 595 images of 16 additional species spread across the angiosperm phylogeny.The nail polish method was successfully applied in 78% of the species sampled here. The VGG19 architecture slightly outperformed the basic shallow and deep architectures, with a confidence threshold equal to 0.7 resulting in an optimal trade-off between precision and recall. Applying this threshold, the VGG19 architecture obtained an average F-score of 0.87, 0.89, and 0.67 on the training, validation, and unseen test set, respectively. The average accuracy was very high (94%) for computed stomatal counts on unseen images of species used for training.The leaf-to-label pipeline is an easy-to-use workflow for researchers of different areas of expertise interested in detecting stomata more efficiently. The described methodology was based on multiple species and well-established methods so that it can serve as a reference for future work.

Gloxinia-An Open-Source Sensing Platform to Monitor the Dynamic Responses of Plants.

The study of the dynamic responses of plants to short-term environmental changes is becoming increasingly important in basic plant science, phenotyping, breeding, crop management, and modelling. These short-term variations are crucial in plant adaptation to new environments and, consequently, in plant fitness and productivity. Scalable, versatile, accurate, and low-cost data-logging solutions are necessary to advance these fields and complement existing sensing platforms such as high-throughput phenotyping. However, current data logging and sensing platforms do not meet the requirements to monitor these responses. Therefore, a new modular data logging platform was designed, named Gloxinia. Different sensor boards are interconnected depending upon the needs, with the potential to scale to hundreds of sensors in a distributed sensor system. To demonstrate the architecture, two sensor boards were designed-one for single-ended measurements and one for lock-in amplifier based measurements, named Sylvatica and Planalta, respectively. To evaluate the performance of the system in small setups, a small-scale trial was conducted in a growth chamber. Expected plant dynamics were successfully captured, indicating proper operation of the system. Though a large scale trial was not performed, we expect the system to scale very well to larger setups. Additionally, the platform is open-source, enabling other users to easily build upon our work and perform application-specific optimisations.

Century-long apparent decrease in intrinsic water-use efficiency with no evidence of progressive nutrient limitation in African tropical forests.

Forests exhibit leaf- and ecosystem-level responses to environmental changes. Specifically, rising carbon dioxide (CO2 ) levels over the past century are expected to have increased the intrinsic water-use efficiency (iWUE) of tropical trees while the ecosystem is gradually pushed into progressive nutrient limitation. Due to the long-term character of these changes, however, observational datasets to validate both paradigms are limited in space and time. In this study, we used a unique herbarium record to go back nearly a century and show that despite the rise in CO2 concentrations, iWUE has decreased in central African tropical trees in the Congo Basin. Although we find evidence that points to leaf-level adaptation to increasing CO2 -that is, increasing photosynthesis-related nutrients and decreasing maximum stomatal conductance, a decrease in leaf δ13 C clearly indicates a decreasing iWUE over time. Additionally, the stoichiometric carbon to nitrogen and nitrogen to phosphorus ratios in the leaves show no sign of progressive nutrient limitation as they have remained constant since 1938, which suggests that nutrients have not increasingly limited productivity in this biome. Altogether, the data suggest that other environmental factors, such as increasing temperature, might have negatively affected net photosynthesis and consequently downregulated the iWUE. Results from this study reveal that the second largest tropical forest on Earth has responded differently to recent environmental changes than expected, highlighting the need for further on-ground monitoring in the Congo Basin.

Advanced X-ray CT scanning can boost tree ring research for earth system sciences.

BACKGROUND AND AIMS: Tree rings, as archives of the past and biosensors of the present, offer unique opportunities to study influences of the fluctuating environment over decades to centuries. As such, tree-ring-based wood traits are capital input for global vegetation models. To contribute to earth system sciences, however, sufficient spatial coverage is required of detailed individual-based measurements, necessitating large amounts of data. X-ray computed tomography (CT) scanning is one of the few techniques that can deliver such data sets. METHODS: Increment cores of four different temperate tree species were scanned with a state-of-the-art X-ray CT system at resolutions ranging from 60 µm down to 4.5 µm, with an additional scan at a resolution of 0.8 µm of a splinter-sized sample using a second X-ray CT system to highlight the potential of cell-level scanning. Calibration-free densitometry, based on full scanner simulation of a third X-ray CT system, is illustrated on increment cores of a tropical tree species. KEY RESULTS: We show how multiscale scanning offers unprecedented potential for mapping tree rings and wood traits without sample manipulation and with limited operator intervention. Custom-designed sample holders enable simultaneous scanning of multiple increment cores at resolutions sufficient for tree ring analysis and densitometry as well as single core scanning enabling quantitative wood anatomy, thereby approaching the conventional thin section approach. Standardized X-ray CT volumes are, furthermore, ideal input imagery for automated pipelines with neural-based learning for tree ring detection and measurements of wood traits. CONCLUSIONS: Advanced X-ray CT scanning for high-throughput processing of increment cores is within reach, generating pith-to-bark ring width series, density profiles and wood trait data. This would allow contribution to large-scale monitoring and modelling efforts with sufficient global coverage.

Body Randomization Reduces the Sim-to-Real Gap for Compliant Quadruped Locomotion.

Designing controllers for compliant, underactuated robots is challenging and usually requires a learning procedure. Learning robotic control in simulated environments can speed up the process whilst lowering risk of physical damage. Since perfect simulations are unfeasible, several techniques are used to improve transfer to the real world. Here, we investigate the impact of randomizing body parameters during learning of CPG controllers in simulation. The controllers are evaluated on our physical quadruped robot. We find that body randomization in simulation increases chances of finding gaits that function well on the real robot.

A Differentiable Physics Engine for Deep Learning in Robotics.

An important field in robotics is the optimization of controllers. Currently, robots are often treated as a black box in this optimization process, which is the reason why derivative-free optimization methods such as evolutionary algorithms or reinforcement learning are omnipresent. When gradient-based methods are used, models are kept small or rely on finite difference approximations for the Jacobian. This method quickly grows expensive with increasing numbers of parameters, such as found in deep learning. We propose the implementation of a modern physics engine, which can differentiate control parameters. This engine is implemented for both CPU and GPU. Firstly, this paper shows how such an engine speeds up the optimization process, even for small problems. Furthermore, it explains why this is an alternative approach to deep Q-learning, for using deep learning in robotics. Finally, we argue that this is a big step for deep learning in robotics, as it opens up new possibilities to optimize robots, both in hardware and software.

Morphological Properties of Mass-Spring Networks for Optimal Locomotion Learning.

Robots have proven very useful in automating industrial processes. Their rigid components and powerful actuators, however, render them unsafe or unfit to work in normal human environments such as schools or hospitals. Robots made of compliant, softer materials may offer a valid alternative. Yet, the dynamics of these compliant robots are much more complicated compared to normal rigid robots of which all components can be accurately controlled. It is often claimed that, by using the concept of morphological computation, the dynamical complexity can become a strength. On the one hand, the use of flexible materials can lead to higher power efficiency and more fluent and robust motions. On the other hand, using embodiment in a closed-loop controller, part of the control task itself can be outsourced to the body dynamics. This can significantly simplify the additional resources required for locomotion control. To this goal, a first step consists in an exploration of the trade-offs between morphology, efficiency of locomotion, and the ability of a mechanical body to serve as a computational resource. In this work, we use a detailed dynamical model of a Mass-Spring-Damper (MSD) network to study these trade-offs. We first investigate the influence of the network size and compliance on locomotion quality and energy efficiency by optimizing an external open-loop controller using evolutionary algorithms. We find that larger networks can lead to more stable gaits and that the system's optimal compliance to maximize the traveled distance is directly linked to the desired frequency of locomotion. In the last set of experiments, the suitability of MSD bodies for being used in a closed loop is also investigated. Since maximally efficient actuator signals are clearly related to the natural body dynamics, in a sense, the body is tailored for the task of contributing to its own control. Using the same simulation platform, we therefore study how the network states can be successfully used to create a feedback signal and how its accuracy is linked to the body size.

Nanophotonic reservoir computing with photonic crystal cavities to generate periodic patterns.

Reservoir computing (RC) is a technique in machine learning inspired by neural systems. RC has been used successfully to solve complex problems such as signal classification and signal generation. These systems are mainly implemented in software, and thereby they are limited in speed and power efficiency. Several optical and optoelectronic implementations have been demonstrated, in which the system has signals with an amplitude and phase. It is proven that these enrich the dynamics of the system, which is beneficial for the performance. In this paper, we introduce a novel optical architecture based on nanophotonic crystal cavities. This allows us to integrate many neurons on one chip, which, compared with other photonic solutions, closest resembles a classical neural network. Furthermore, the components are passive, which simplifies the design and reduces the power consumption. To assess the performance of this network, we train a photonic network to generate periodic patterns, using an alternative online learning rule called first-order reduced and corrected error. For this, we first train a classical hyperbolic tangent reservoir, but then we vary some of the properties to incorporate typical aspects of a photonics reservoir, such as the use of continuous-time versus discrete-time signals and the use of complex-valued versus real-valued signals. Then, the nanophotonic reservoir is simulated and we explore the role of relevant parameters such as the topology, the phases between the resonators, the number of nodes that are biased and the delay between the resonators. It is important that these parameters are chosen such that no strong self-oscillations occur. Finally, our results show that for a signal generation task a complex-valued, continuous-time nanophotonic reservoir outperforms a classical (i.e., discrete-time, real-valued) leaky hyperbolic tangent reservoir (normalized root-mean-square errors=0.030 versus NRMSE=0.127).

Frequency modulation of large oscillatory neural networks.

Dynamical systems which generate periodic signals are of interest as models of biological central pattern generators and in a number of robotic applications. A basic functionality that is required in both biological modelling and robotics is frequency modulation. This leads to the question of whether there are generic mechanisms to control the frequency of neural oscillators. Here we describe why this objective is of a different nature, and more difficult to achieve, than modulating other oscillation characteristics (like amplitude, offset, signal shape). We propose a generic way to solve this task which makes use of a simple linear controller. It rests on the insight that there is a bidirectional dependency between the frequency of an oscillation and geometric properties of the neural oscillator's phase portrait. By controlling the geometry of the neural state orbits, it is possible to control the frequency on the condition that the state space can be shaped such that it can be pushed easily to any frequency.

Feedback control by online learning an inverse model.

A model, predictor, or error estimator is often used by a feedback controller to control a plant. Creating such a model is difficult when the plant exhibits nonlinear behavior. In this paper, a novel online learning control framework is proposed that does not require explicit knowledge about the plant. This framework uses two learning modules, one for creating an inverse model, and the other for actually controlling the plant. Except for their inputs, they are identical. The inverse model learns by the exploration performed by the not yet fully trained controller, while the actual controller is based on the currently learned model. The proposed framework allows fast online learning of an accurate controller. The controller can be applied on a broad range of tasks with different dynamic characteristics. We validate this claim by applying our control framework on several control tasks: 1) the heating tank problem (slow nonlinear dynamics); 2) flight pitch control (slow linear dynamics); and 3) the balancing problem of a double inverted pendulum (fast linear and nonlinear dynamics). The results of these experiments show that fast learning and accurate control can be achieved. Furthermore, a comparison is made with some classical control approaches, and observations concerning convergence and stability are made.