Data science competition for cross-site individual tree species identification from airborne remote sensing data.

Data on individual tree crowns from remote sensing have the potential to advance forest ecology by providing information about forest composition and structure with a continuous spatial coverage over large spatial extents. Classifying individual trees to their taxonomic species over large regions from remote sensing data is challenging. Methods to classify individual species are often accurate for common species, but perform poorly for less common species and when applied to new sites. We ran a data science competition to help identify effective methods for the task of classification of individual crowns to species identity. The competition included data from three sites to assess each methods' ability to generalize patterns across two sites simultaneously and apply methods to an untrained site. Three different metrics were used to assess and compare model performance. Six teams participated, representing four countries and nine individuals. The highest performing method from a previous competition in 2017 was applied and used as a baseline to understand advancements and changes in successful methods. The best species classification method was based on a two-stage fully connected neural network that significantly outperformed the baseline random forest and gradient boosting ensemble methods. All methods generalized well by showing relatively strong performance on the trained sites (accuracy = 0.46-0.55, macro F1 = 0.09-0.32, cross entropy loss = 2.4-9.2), but generally failed to transfer effectively to the untrained site (accuracy = 0.07-0.32, macro F1 = 0.02-0.18, cross entropy loss = 2.8-16.3). Classification performance was influenced by the number of samples with species labels available for training, with most methods predicting common species at the training sites well (maximum F1 score of 0.86) relative to the uncommon species where none were predicted. Classification errors were most common between species in the same genus and different species that occur in the same habitat. Most methods performed better than the baseline in detecting if a species was not in the training data by predicting an untrained mixed-species class, especially in the untrained site. This work has highlighted that data science competitions can encourage advancement of methods, particularly by bringing in new people from outside the focal discipline, and by providing an open dataset and evaluation criteria from which participants can learn.

A general deep learning model for bird detection in high-resolution airborne imagery.

Advances in artificial intelligence for computer vision hold great promise for increasing the scales at which ecological systems can be studied. The distribution and behavior of individuals is central to ecology, and computer vision using deep neural networks can learn to detect individual objects in imagery. However, developing supervised models for ecological monitoring is challenging because it requires large amounts of human-labeled training data, requires advanced technical expertise and computational infrastructure, and is prone to overfitting. This limits application across space and time. One solution is developing generalized models that can be applied across species and ecosystems. Using over 250,000 annotations from 13 projects from around the world, we develop a general bird detection model that achieves over 65% recall and 50% precision on novel aerial data without any local training despite differences in species, habitat, and imaging methodology. Fine-tuning this model with only 1000 local annotations increases these values to an average of 84% recall and 69% precision by building on the general features learned from other data sources. Retraining from the general model improves local predictions even when moderately large annotation sets are available and makes model training faster and more stable. Our results demonstrate that general models for detecting broad classes of organisms using airborne imagery are achievable. These models can reduce the effort, expertise, and computational resources necessary for automating the detection of individual organisms across large scales, helping to transform the scale of data collection in ecology and the questions that can be addressed.

A benchmark dataset for canopy crown detection and delineation in co-registered airborne RGB, LiDAR and hyperspectral imagery from the National Ecological Observation Network.

Broad scale remote sensing promises to build forest inventories at unprecedented scales. A crucial step in this process is to associate sensor data into individual crowns. While dozens of crown detection algorithms have been proposed, their performance is typically not compared based on standard data or evaluation metrics. There is a need for a benchmark dataset to minimize differences in reported results as well as support evaluation of algorithms across a broad range of forest types. Combining RGB, LiDAR and hyperspectral sensor data from the USA National Ecological Observatory Network's Airborne Observation Platform with multiple types of evaluation data, we created a benchmark dataset to assess crown detection and delineation methods for canopy trees covering dominant forest types in the United States. This benchmark dataset includes an R package to standardize evaluation metrics and simplify comparisons between methods. The benchmark dataset contains over 6,000 image-annotated crowns, 400 field-annotated crowns, and 3,000 canopy stem points from a wide range of forest types. In addition, we include over 10,000 training crowns for optional use. We discuss the different evaluation data sources and assess the accuracy of the image-annotated crowns by comparing annotations among multiple annotators as well as overlapping field-annotated crowns. We provide an example submission and score for an open-source algorithm that can serve as a baseline for future methods.

Evaluating probabilistic ecological forecasts.

Probabilistic near-term forecasting facilitates evaluation of model predictions against observations and is of pressing need in ecology to inform environmental decision-making and effect societal change. Despite this imperative, many ecologists are unfamiliar with the widely used tools for evaluating probabilistic forecasts developed in other fields. We address this gap by reviewing the literature on probabilistic forecast evaluation from diverse fields including climatology, economics, and epidemiology. We present established practices for selecting evaluation data (end-sample hold out), graphical forecast evaluation (times-series plots with uncertainty, probability integral transform plots), quantitative evaluation using scoring rules (log, quadratic, spherical, and ranked probability scores), and comparing scores across models (skill score, Diebold-Mariano test). We cover common approaches, highlight mathematical concepts to follow, and note decision points to allow application of general principles to specific forecasting endeavors. We illustrate these approaches with an application to a long-term rodent population time series currently used for ecological forecasting and discuss how ecology can continue to learn from and drive the cross-disciplinary field of forecasting science.

A remote sensing derived data set of 100 million individual tree crowns for the National Ecological Observatory Network.

Forests provide biodiversity, ecosystem, and economic services. Information on individual trees is important for understanding forest ecosystems but obtaining individual-level data at broad scales is challenging due to the costs and logistics of data collection. While advances in remote sensing techniques allow surveys of individual trees at unprecedented extents, there remain technical challenges in turning sensor data into tangible information. Using deep learning methods, we produced an open-source data set of individual-level crown estimates for 100 million trees at 37 sites across the United States surveyed by the National Ecological Observatory Network's Airborne Observation Platform. Each canopy tree crown is represented by a rectangular bounding box and includes information on the height, crown area, and spatial location of the tree. These data have the potential to drive significant expansion of individual-level research on trees by facilitating both regional analyses and cross-region comparisons encompassing forest types from most of the United States.

Estimating individual-level plant traits at scale.

Functional ecology has increasingly focused on describing ecological communities based on their traits (measurable features affecting individuals' fitness and performance). Analyzing trait distributions within and among forests could significantly improve understanding of community composition and ecosystem function. Historically, data on trait distributions are generated by (1) collecting a small number of leaves from a small number of trees, which suffers from limited sampling but produces information at the fundamental ecological unit (the individual), or (2) using remote-sensing images to infer traits, producing information continuously across large regions, but as plots (containing multiple trees of different species) or pixels, not individuals. Remote-sensing methods that identify individual trees and estimate their traits would provide the benefits of both approaches, producing continuous large-scale data linked to biological individuals. We used data from the National Ecological Observatory Network (NEON) to develop a method to scale up functional traits from 160 trees to the millions of trees within the spatial extent of two NEON sites. The pipeline consists of three stages: (1) image segmentation, to identify individual trees and estimate structural traits; (2) an ensemble of models to infer leaf mass area (LMA), nitrogen, carbon, and phosphorus content using hyperspectral signatures, and DBH from allometry; and (3) predictions for segmented crowns for the full remote-sensing footprint at the NEON sites. The R2 values on held-out test data ranged from 0.41 to 0.75 on held-out test data. The ensemble approach performed better than single partial least-squares models. Carbon performed poorly compared to other traits (R2 of 0.41). The crown segmentation step contributed the most uncertainty in the pipeline, due to over-segmentation. The pipeline produced good estimates of DBH (R2 of 0.62 on held-out data). Trait predictions for crowns performed significantly better than comparable predictions on pixels, resulting in improvement of R2 on test data of between 0.07 and 0.26. We used the pipeline to produce individual-level trait data for ~5 million individual crowns, covering a total extent of ~360 km2 . This large data set allows testing ecological questions on landscape scales, revealing that foliar traits are correlated with structural traits and environmental conditions.

A simulation study of the use of temporal occupancy for identifying core and transient species.

Transient species, which do not maintain self-sustaining populations in a system where they are observed, are ubiquitous in nature and their presence often impacts the interpretation of ecological patterns and processes. Identifying transient species from temporal occupancy, the proportion of time a species is observed at a given site over a time series, is subject to classification errors as a result of imperfect detection and source-sink dynamics. We use a simulation-based approach to assess how often errors in detection or classification occur in order to validate the use of temporal occupancy as a metric for inferring whether a species is a core or transient member of a community. We found that low detection increases error in the classification of core species, while high habitat heterogeneity and high detection increase error in classification of transient species. These findings confirm that temporal occupancy is a valid metric for inferring whether a species can maintain a self-sustaining population, but imperfect detection, low abundance, and highly heterogeneous landscapes may yield high misclassification rates.

Ten Simple Rules for a successful remote postdoc.

Postdocs are a critical transition for early-career researchers. This transient period, between finishing a PhD and finding a permanent position, is when early-career researchers develop independent research programs and establish collaborative relationships that can make a successful career. Traditionally, postdocs physically relocate-sometimes multiple times-for these short-term appointments, which creates challenges that can disproportionately affect members of traditionally underrepresented groups in science, technology, engineering, and mathematics (STEM). However, many research activities involving analytical and quantitative work do not require a physical presence in a lab and can be accomplished remotely. Other fields have embraced remote work, yet many academics have been hesitant to hire remote postdocs. In this article, we present advice to both principal investigators (PIs) and postdocs for successfully navigating a remote position. Using the combined experience of the authors (as either remote postdocs or employers of remote postdocs), we provide a road map to overcome the real (and perceived) obstacles associated with remote work. With planning, communication, and creativity, remote postdocs can be a fully functioning and productive member of a research lab. Further, our rules can be useful for research labs generally and can help foster a more flexible and inclusive environment.

Automated data-intensive forecasting of plant phenology throughout the United States.

Phenology, the timing of cyclical and seasonal natural phenomena such as flowering and leaf out, is an integral part of ecological systems with impacts on human activities like environmental management, tourism, and agriculture. As a result, there are numerous potential applications for actionable predictions of when phenological events will occur. However, despite the availability of phenological data with large spatial, temporal, and taxonomic extents, and numerous phenology models, there have been no automated species-level forecasts of plant phenology. This is due in part to the challenges of building a system that integrates large volumes of climate observations and forecasts, uses that data to fit models and make predictions for large numbers of species, and consistently disseminates the results of these forecasts in interpretable ways. Here, we describe a new near-term phenology-forecasting system that makes predictions for the timing of budburst, flowers, ripe fruit, and fall colors for 78 species across the United States up to 6 months in advance and is updated every four days. We use the lessons learned in developing this system to provide guidance developing large-scale near-term ecological forecast systems more generally, to help advance the use of automated forecasting in ecology.

A data science challenge for converting airborne remote sensing data into ecological information.

Ecology has reached the point where data science competitions, in which multiple groups solve the same problem using the same data by different methods, will be productive for advancing quantitative methods for tasks such as species identification from remote sensing images. We ran a competition to help improve three tasks that are central to converting images into information on individual trees: (1) crown segmentation, for identifying the location and size of individual trees; (2) alignment, to match ground truthed trees with remote sensing; and (3) species classification of individual trees. Six teams (composed of 16 individual participants) submitted predictions for one or more tasks. The crown segmentation task proved to be the most challenging, with the highest-performing algorithm yielding only 34% overlap between remotely sensed crowns and the ground truthed trees. However, most algorithms performed better on large trees. For the alignment task, an algorithm based on minimizing the difference, in terms of both position and tree size, between ground truthed and remotely sensed crowns yielded a perfect alignment. In hindsight, this task was over simplified by only including targeted trees instead of all possible remotely sensed crowns. Several algorithms performed well for species classification, with the highest-performing algorithm correctly classifying 92% of individuals and performing well on both common and rare species. Comparisons of results across algorithms provided a number of insights for improving the overall accuracy in extracting ecological information from remote sensing. Our experience suggests that this kind of competition can benefit methods development in ecology and biology more broadly.

Developing a modern data workflow for regularly updated data.

Over the past decade, biology has undergone a data revolution in how researchers collect data and the amount of data being collected. An emerging challenge that has received limited attention in biology is managing, working with, and providing access to data under continual active collection. Regularly updated data present unique challenges in quality assurance and control, data publication, archiving, and reproducibility. We developed a workflow for a long-term ecological study that addresses many of the challenges associated with managing this type of data. We do this by leveraging existing tools to 1) perform quality assurance and control; 2) import, restructure, version, and archive data; 3) rapidly publish new data in ways that ensure appropriate credit to all contributors; and 4) automate most steps in the data pipeline to reduce the time and effort required by researchers. The workflow leverages tools from software development, including version control and continuous integration, to create a modern data management system that automates the pipeline.

Energetic equivalence underpins the size structure of tree and phytoplankton communities.

The size structure of autotroph communities - the relative abundance of small vs. large individuals - shapes the functioning of ecosystems. Whether common mechanisms underpin the size structure of unicellular and multicellular autotrophs is, however, unknown. Using a global data compilation, we show that individual body masses in tree and phytoplankton communities follow power-law distributions and that the average exponents of these individual size distributions (ISD) differ. Phytoplankton communities are characterized by an average ISD exponent consistent with three-quarter-power scaling of metabolism with body mass and equivalence in energy use among mass classes. Tree communities deviate from this pattern in a manner consistent with equivalence in energy use among diameter size classes. Our findings suggest that whilst universal metabolic constraints ultimately underlie the emergent size structure of autotroph communities, divergent aspects of body size (volumetric vs. linear dimensions) shape the ecological outcome of metabolic scaling in forest vs. pelagic ecosystems.

Comparison of large-scale citizen science data and long-term study data for phenology modeling.

Large-scale observational data from citizen science efforts are becoming increasingly common in ecology, and researchers often choose between these and data from intensive local-scale studies for their analyses. This choice has potential trade-offs related to spatial scale, observer variance, and interannual variability. Here we explored this issue with phenology by comparing models built using data from the large-scale, citizen science USA National Phenology Network (USA-NPN) effort with models built using data from more intensive studies at Long Term Ecological Research (LTER) sites. We built statistical and process based phenology models for species common to each data set. From these models, we compared parameter estimates, estimates of phenological events, and out-of-sample errors between models derived from both USA-NPN and LTER data. We found that model parameter estimates for the same species were most similar between the two data sets when using simple models, but parameter estimates varied widely as model complexity increased. Despite this, estimates for the date of phenological events and out-of-sample errors were similar, regardless of the model chosen. Predictions for USA-NPN data had the lowest error when using models built from the USA-NPN data, while LTER predictions were best made using LTER-derived models, confirming that models perform best when applied at the same scale they were built. This difference in the cross-scale model comparison is likely due to variation in phenological requirements within species. Models using the USA-NPN data set can integrate parameters over a large spatial scale while those using an LTER data set can only estimate parameters for a single location. Accordingly, the choice of data set depends on the research question. Inferences about species-specific phenological requirements are best made with LTER data, and if USA-NPN or similar data are all that is available, then analyses should be limited to simple models. Large-scale predictive modeling is best done with the larger-scale USA-NPN data, which has high spatial representation and a large regional species pool. LTER data sets, on the other hand, have high site fidelity and thus characterize inter-annual variability extremely well. Future research aimed at forecasting phenology events for particular species over larger scales should develop models that integrate the strengths of both data sets.

The proportion of core species in a community varies with spatial scale and environmental heterogeneity.

Ecological communities are composed of a combination of core species that maintain local viable populations and transient species that occur infrequently due to dispersal from surrounding regions. Preliminary work indicates that while core and transient species are both commonly observed in community surveys of a wide range of taxonomic groups, their relative prevalence varies substantially from one community to another depending upon the spatial scale at which the community was characterized and its environmental context. We used a geographically extensive dataset of 968 bird community time series to quantitatively describe how the proportion of core species in a community varies with spatial scale and environmental heterogeneity. We found that the proportion of core species in an assemblage increased with spatial scale in a positive decelerating fashion with a concomitant decrease in the proportion of transient species. Variation in the shape of this scaling relationship between sites was related to regional environmental heterogeneity, with lower proportions of core species at a given scale associated with high environmental heterogeneity. Understanding this influence of scale and environmental heterogeneity on the proportion of core species may help resolve discrepancies between studies of biotic interactions, resource availability, and mass effects conducted at different scales, because the importance of these and other ecological processes are expected to differ substantially between core and transient species.

The prevalence and impact of transient species in ecological communities.

Transient species occur infrequently in a community over time and do not maintain viable local populations. Because transient species interact differently than non-transients with their biotic and abiotic environment, it is important to characterize the prevalence of these species and how they impact our understanding of ecological systems. We quantified the prevalence and impact of transient species in communities using data on over 19,000 community time series spanning an array of ecosystems, taxonomic groups, and spatial scales. We found that transient species are a general feature of communities regardless of taxa or ecosystem. The proportion of these species decreases with increasing spatial scale leading to a need to control for scale in comparative work. Removing transient species from analyses influences the form of a suite of commonly studied ecological patterns including species-abundance distributions, species-energy relationships, species-area relationships, and temporal turnover. Careful consideration should be given to whether transient species are included in analyses depending on the theoretical and practical relevance of these species for the question being studied.

Forecasting biodiversity in breeding birds using best practices.

Biodiversity forecasts are important for conservation, management, and evaluating how well current models characterize natural systems. While the number of forecasts for biodiversity is increasing, there is little information available on how well these forecasts work. Most biodiversity forecasts are not evaluated to determine how well they predict future diversity, fail to account for uncertainty, and do not use time-series data that captures the actual dynamics being studied. We addressed these limitations by using best practices to explore our ability to forecast the species richness of breeding birds in North America. We used hindcasting to evaluate six different modeling approaches for predicting richness. Hindcasts for each method were evaluated annually for a decade at 1,237 sites distributed throughout the continental United States. All models explained more than 50% of the variance in richness, but none of them consistently outperformed a baseline model that predicted constant richness at each site. The best practices implemented in this study directly influenced the forecasts and evaluations. Stacked species distribution models and "naive" forecasts produced poor estimates of uncertainty and accounting for this resulted in these models dropping in the relative performance compared to other models. Accounting for observer effects improved model performance overall, but also changed the rank ordering of models because it did not improve the accuracy of the "naive" model. Considering the forecast horizon revealed that the prediction accuracy decreased across all models as the time horizon of the forecast increased. To facilitate the rapid improvement of biodiversity forecasts, we emphasize the value of specific best practices in making forecasts and evaluating forecasting methods.

Iterative near-term ecological forecasting: Needs, opportunities, and challenges.

Two foundational questions about sustainability are "How are ecosystems and the services they provide going to change in the future?" and "How do human decisions affect these trajectories?" Answering these questions requires an ability to forecast ecological processes. Unfortunately, most ecological forecasts focus on centennial-scale climate responses, therefore neither meeting the needs of near-term (daily to decadal) environmental decision-making nor allowing comparison of specific, quantitative predictions to new observational data, one of the strongest tests of scientific theory. Near-term forecasts provide the opportunity to iteratively cycle between performing analyses and updating predictions in light of new evidence. This iterative process of gaining feedback, building experience, and correcting models and methods is critical for improving forecasts. Iterative, near-term forecasting will accelerate ecological research, make it more relevant to society, and inform sustainable decision-making under high uncertainty and adaptive management. Here, we identify the immediate scientific and societal needs, opportunities, and challenges for iterative near-term ecological forecasting. Over the past decade, data volume, variety, and accessibility have greatly increased, but challenges remain in interoperability, latency, and uncertainty quantification. Similarly, ecologists have made considerable advances in applying computational, informatic, and statistical methods, but opportunities exist for improving forecast-specific theory, methods, and cyberinfrastructure. Effective forecasting will also require changes in scientific training, culture, and institutions. The need to start forecasting is now; the time for making ecology more predictive is here, and learning by doing is the fastest route to drive the science forward.

No general relationship between mass and temperature in endothermic species.

Bergmann's rule is a widely-accepted biogeographic rule stating that individuals within a species are smaller in warmer environments. While there are many single-species studies and integrative reviews documenting this pattern, a data-intensive approach has not been used yet to determine the generality of this pattern. We assessed the strength and direction of the intraspecific relationship between temperature and individual mass for 952 bird and mammal species. For eighty-seven percent of species, temperature explained less than 10% of variation in mass, and for 79% of species the correlation was not statistically significant. These results suggest that Bergmann's rule is not general and temperature is not a dominant driver of biogeographic variation in mass. Further understanding of size variation will require integrating multiple processes that influence size. The lack of dominant temperature forcing weakens the justification for the hypothesis that global warming could result in widespread decreases in body size.

Skills and Knowledge for Data-Intensive Environmental Research.

The scale and magnitude of complex and pressing environmental issues lend urgency to the need for integrative and reproducible analysis and synthesis, facilitated by data-intensive research approaches. However, the recent pace of technological change has been such that appropriate skills to accomplish data-intensive research are lacking among environmental scientists, who more than ever need greater access to training and mentorship in computational skills. Here, we provide a roadmap for raising data competencies of current and next-generation environmental researchers by describing the concepts and skills needed for effectively engaging with the heterogeneous, distributed, and rapidly growing volumes of available data. We articulate five key skills: (1) data management and processing, (2) analysis, (3) software skills for science, (4) visualization, and (5) communication methods for collaboration and dissemination. We provide an overview of the current suite of training initiatives available to environmental scientists and models for closing the skill-transfer gap.

An extensive comparison of species-abundance distribution models.

A number of different models have been proposed as descriptions of the species-abundance distribution (SAD). Most evaluations of these models use only one or two models, focus on only a single ecosystem or taxonomic group, or fail to use appropriate statistical methods. We use likelihood and AIC to compare the fit of four of the most widely used models to data on over 16,000 communities from a diverse array of taxonomic groups and ecosystems. Across all datasets combined the log-series, Poisson lognormal, and negative binomial all yield similar overall fits to the data. Therefore, when correcting for differences in the number of parameters the log-series generally provides the best fit to data. Within individual datasets some other distributions performed nearly as well as the log-series even after correcting for the number of parameters. The Zipf distribution is generally a poor characterization of the SAD.