Gaussian processes retrieval of crop traits in Google Earth Engine based on Sentinel-2 top-of-atmosphere data.

The unprecedented availability of optical satellite data in cloud-based computing platforms, such as Google Earth Engine (GEE), opens new possibilities to develop crop trait retrieval models from the local to the planetary scale. Hybrid retrieval models are of interest to run in these platforms as they combine the advantages of physically- based radiative transfer models (RTM) with the flexibility of machine learning regression algorithms. Previous research with GEE primarily relied on processing bottom-of-atmosphere (BOA) reflectance data, which requires atmospheric correction. In the present study, we implemented hybrid models directly into GEE for processing Sentinel-2 (S2) Level-1C (L1C) top-of-atmosphere (TOA) reflectance data into crop traits. To achieve this, a training dataset was generated using the leaf-canopy RTM PROSAIL in combination with the atmospheric model 6SV. Gaussian process regression (GPR) retrieval models were then established for eight essential crop traits namely leaf chlorophyll content, leaf water content, leaf dry matter content, fractional vegetation cover, leaf area index (LAI), and upscaled leaf variables (i.e., canopy chlorophyll content, canopy water content and canopy dry matter content). An important pre-requisite for implementation into GEE is that the models are sufficiently light in order to facilitate efficient and fast processing. Successful reduction of the training dataset by 78% was achieved using the active learning technique Euclidean distance-based diversity (EBD). With the EBD-GPR models, highly accurate validation results of LAI and upscaled leaf variables were obtained against in situ field data from the validation study site Munich-North-Isar (MNI), with normalized root mean square errors (NRMSE) from 6% to 13%. Using an independent validation dataset of similar crop types (Italian Grosseto test site), the retrieval models showed moderate to good performances for canopy-level variables, with NRMSE ranging from 14% to 50%, but failed for the leaf-level estimates. Obtained maps over the MNI site were further compared against Sentinel-2 Level 2 Prototype Processor (SL2P) vegetation estimates generated from the ESA Sentinels' Application Platform (SNAP) Biophysical Processor, proving high consistency of both retrievals (R 2 from 0.80 to 0.94). Finally, thanks to the seamless GEE processing capability, the TOA-based mapping was applied over the entirety of Germany at 20 m spatial resolution including information about prediction uncertainty. The obtained maps provided confidence of the developed EBD-GPR retrieval models for integration in the GEE framework and national scale mapping from S2-L1C imagery. In summary, the proposed retrieval workflow demonstrates the possibility of routine processing of S2 TOA data into crop traits maps at any place on Earth as required for operational agricultural applications.

Prototyping Sentinel-2 green LAI and brown LAI products for cropland monitoring.

For agricultural applications, identification of non-photosynthetic above-ground vegetation is of great interest as it contributes to assess harvest practices, detecting crop residues or drought events, as well as to better predict the carbon, water and nutrients uptake. While the mapping of green Leaf Area Index (LAI) is well established, current operational retrieval models are not calibrated for LAI estimation over senescent, brown vegetation. This not only leads to an underestimation of LAI when crops are ripening, but is also a missed monitoring opportunity. The high spatial and temporal resolution of Sentinel-2 (S2) satellites constellation offers the possibility to estimate brown LAI (LAI G ) next to green LAI (LAI G ). By using LAI ground measurements from multiple campaigns associated with airborne or satellite spectra, Gaussian processes regression (GPR) models were developed for both LAI G and LAI B , providing alongside associated uncertainty estimates, which allows to mask out unreliable LAI retrievals with higher uncertainties. A processing chain was implemented to apply both models to S2 images, generating a multiband LAI product at 20 m spatial resolution. The models were adequately validated with in-situ data from various European study sites (LAI G : R2 = 0.7, RMSE = 0.67 m2/m2; LAI B : R2 = 0.62, RMSE = 0.43 m2/m2). Thanks to the S2 bands in the red edge (B5: 705 nm and B6: 740 nm) and in the shortwave infrared (B12: 2190 nm) a distinction between LAI G and LAI B can be achieved. To demonstrate the capability of LAI B to identify when crops start senescing, S2 time series were processed over multiple European study sites and seasonal maps were produced, which show the onset of crop senescence after the green vegetation peak. Particularly, the LAI B product permits the detection of harvest (i.e., sudden drop in LAI B ) and the determination of crop residues (i.e., remaining LAI B ), although a better spectral sampling in the shortwave infrared would have been desirable to disentangle brown LAI from soil variability and its perturbing effects. Finally, a single total LAI product was created by merging LAI G and LAI B estimates, and then compared to the LAI derived from S2 L2B biophysical processor integrated in SNAP. The spatiotemporal analysis results confirmed the improvement of the proposed descriptors with respect to the standard SNAP LAI product accounting only for photosynthetically active green vegetation.

Quantifying vegetation biophysical variables from the Sentinel-3/FLEX tandem mission: Evaluation of the synergy of OLCI and FLORIS data sources.

The ESA's forthcoming FLuorescence EXplorer (FLEX) mission is dedicated to the global monitoring of the vegetation's chlorophyll fluorescence by means of an imaging spectrometer, FLORIS. In order to properly interpret the fluorescence signal in relation to photosynthetic activity, essential vegetation variables need to be retrieved concomitantly. FLEX will fly in tandem with Sentinel-3 (S3), which conveys the Ocean and Land Colour Instrument (OLCI) that is designed to characterize the atmosphere and the terrestrial vegetation at a spatial resolution of 300 m. In this work we present the retrieval models of four essential biophysical variables: (1) Leaf Area Index (LAI), (2) leaf chlorophyll content (Cab), (3) fraction of absorbed photosynthetically active radiation (fAPAR), and (4) fractional vegetation cover (FCover). These variables can be operationally inferred by hybrid retrieval approaches, which combine the generalization capabilities offered by radiative transfer models (RTMs) with the flexibility and computational efficiency of machine learning methods. The RTM SCOPE (Soil Canopy Observation, Photochemistry and Energy fluxes) was used to generate a database of reflectance spectra corresponding to a large variety of canopy realizations, which served subsequently as input to train a Gaussian Process Regression (GPR) algorithm for each targeted variable. Three sets of GPR models were developed, based on different spectral band settings: (1) OLCI (21 bands between 400 and 1040 nm), (2) FLORIS (281 bands between 500 and 780 nm), and (3) their synergy. Their respective performances were assessed based on simulated reflectance scenes. Regarding the retrieval of Cab, the OLCI model gave good model performances (R2: 0.91; RMSE: 7.6 µg. cm -2), yet superior accuracies were achieved as a result of FLORIS' higher spectral resolution (R2: 0.96; RMSE: 4.8 µg. cm -2). The synergy of both datasets did not further enhance the variable retrieval. Regarding LAI, the improvement of the model performances by using only FLORIS spectra (R2: 0.87; RMSE: 1.05 m2.m-2) rather than only OLCI spectra (R2: 0.86; RMSE: 1.12 m2.m-2) was less evident but merging both data sets was more beneficial (R2: 0.88; RMSE: 1.01 m2.m-2). Finally, the three data sources gave good model performances for the retrieval of fAPAR and Fcover, with the best performing model being the Synergy model (fAPAR: R2: 0.99; RMSE: 0.02 and FCover: R2: 0.98; RMSE: 0.04). The ability of the models to process real data was subsequently demonstrated by applying the OLCI models to S3 surface reflectance products acquired over Western Europe and Argentina. Obtained maps showed consistent patterns and variable ranges, and comparison against corresponding Sentinel-2 products (coarsened to a 300 m spatial resolution) led to reasonable matches (R2: 0.5-0.7). Altogether, given the availability of the multiple data sources, the FLEX tandem mission will foster unique opportunities to quantify essential vegetation properties, and hence facilitate the interpretation of the measured fluorescence levels.

DATimeS: A machine learning time series GUI toolbox for gap-filling and vegetation phenology trends detection.

Optical remotely sensed data are typically discontinuous, with missing values due to cloud cover. Consequently, gap-filling solutions are needed for accurate crop phenology characterization. The here presented Decomposition and Analysis of Time Series software (DATimeS) expands established time series interpolation methods with a diversity of advanced machine learning fitting algorithms (e.g., Gaussian Process Regression: GPR) particularly effective for the reconstruction of multiple-seasons vegetation temporal patterns. DATimeS is freely available as a powerful image time series software that generates cloud-free composite maps and captures seasonal vegetation dynamics from regular or irregular satellite time series. This work describes the main features of DATimeS, and provides a demonstration case using Sentinel-2 Leaf Area Index time series data over a Spanish site. GPR resulted as an optimum fitting algorithm with most accurate gap-filling performance and associated uncertainties. DATimeS further quantified LAI fluctuations among multiple crop seasons and provided phenological indicators for specific crop types.

Fusing optical and SAR time series for LAI gap fillingwith multioutput Gaussian processes.

The availability of satellite optical information is often hampered by the natural presence of clouds, which can be problematic for many applications. Persistent clouds over agricultural fields can mask key stages of crop growth, leading to unreliable yield predictions. Synthetic Aperture Radar (SAR) provides all-weather imagery which can potentially overcome this limitation, but given its high and distinct sensitivity to different surface properties, the fusion of SAR and optical data still remains an open challenge. In this work, we propose the use of Multi-Output Gaussian Process (MOGP) regression, a machine learning technique that learns automatically the statistical relationships among multisensor time series, to detect vegetated areas over which the synergy between SAR-optical imageries is profitable. For this purpose, we use the Sentinel-1 Radar Vegetation Index (RVI) and Sentinel-2 Leaf Area Index (LAI) time series over a study area in north west of the Iberian peninsula. Through a physical interpretation of MOGP trained models, we show its ability to provide estimations of LAI even over cloudy periods using the information shared with RVI, which guarantees the solution keeps always tied to real measurements. Results demonstrate the advantage of MOGP especially for long data gaps, where optical-based methods notoriously fail. The leave-one-image-out assessment technique applied to the whole vegetation cover shows MOGP predictions improve standard GP estimations over short-time gaps (R2 of 74% vs 68%, RMSE of 0.4 vs 0.44 [m 2 m -2]) and especially over long-time gaps (R2 of 33% vs 12%, RMSE of 0.5 vs 1.09 [m 2 m -2]). A second assessment is focused on crop-specific regions, clustering pixels fulfilling specific model conditions where the synergy is profitable. Results reveal the MOGP performance is crop type and crop stage dependent. For long time gaps, best R2 are obtained over maize, ranging from 0.1 (tillering) to 0.36 (development) up to 0.81 (maturity); for moderate time gap, R2 = 0.93 (maturity) is obtained. Crops such as wheat, oats, rye and barley, can profit from the LAI-RVI synergy, with R2 varying between 0.4 and 0.6. For beet or potatoes, MOGP provides poorer results, but alternative descriptors to RVI should be tested for these specific crops in the future before discarding synergy real benefits. In conclusion, active-passive sensor fusion with MOGP represents a novel and promising approach to cope with crop monitoring over cloud-dominated areas.

Multi-Season Phenology Mapping of Nile Delta Croplands Using Time Series of Sentinel-2 and Landsat 8 Green LAI.

Space-based cropland phenology monitoring substantially assists agricultural managing practices and plays an important role in crop yield predictions. Multitemporal satellite observations allow analyzing vegetation seasonal dynamics over large areas by using vegetation indices or by deriving biophysical variables. The Nile Delta represents about half of all agricultural lands of Egypt. In this region, intensifying farming systems are predominant and multi-cropping rotations schemes are increasing, requiring a high temporal and spatial resolution monitoring for capturing successive crop growth cycles. This study presents a workflow for cropland phenology characterization and mapping based on time series of green Leaf Area Index (LAI) generated from NASA's Harmonized Landsat 8 (L8) and Sentinel-2 (S2) surface reflectance dataset from 2016 to 2019. LAI time series were processed for each satellite dataset, which were used separately and combined to identify seasonal dynamics for a selection of crop types (wheat, clover, maize and rice). For the combination of L8 with S2 LAI products, we proposed two time series smoothing and fitting methods: (1) the Savitzky-Golay (SG) filter and (2) the Gaussian Processes Regression (GPR) fitting function. Single-sensor and L8-S2 combined LAI time series were used for the calculation of key crop Land Surface Phenology (LSP) metrics (start of season, end of season, length of season), whereby the detection of cropland growing seasons was based on two established threshold methods, i.e., a seasonal or a relative amplitude value. Overall, the developed phenology extraction scheme enabled identifying up to two successive crop cycles within a year, with a superior performance observed for the seasonal than for the relative threshold method, in terms of consistency and cropland season detection capability. Differences between the time series collections were analyzed by comparing the phenology metrics per crop type and year. Results suggest that L8-S2 combined LAI data streams with GPR led to a more precise detection of the start and end of growing seasons for most crop types, reaching an overall detection of 74% over the total planted crops versus 69% with S2 and 63% with L8 alone. Finally, the phenology mapping allowed us to evaluate the spatial and temporal evolution of the croplands over the agroecosystem in the Nile Delta.

Quantifying Fundamental Vegetation Traits over Europe Using the Sentinel-3 OLCI Catalogue in Google Earth Engine.

Thanks to the emergence of cloud-computing platforms and the ability of machine learning methods to solve prediction problems efficiently, this work presents a workflow to automate spatiotemporal mapping of essential vegetation traits from Sentinel-3 (S3) imagery. The traits included leaf chlorophyll content (LCC), leaf area index (LAI), fraction of absorbed photosynthetically active radiation (FAPAR), and fractional vegetation cover (FVC), being fundamental for assessing photosynthetic activity on Earth. The workflow involved Gaussian process regression (GPR) algorithms trained on top-of-atmosphere (TOA) radiance simulations generated by the coupled canopy radiative transfer model (RTM) SCOPE and the atmospheric RTM 6SV. The retrieval models, named to S3-TOA-GPR-1.0, were directly implemented in Google Earth Engine (GEE) to enable the quantification of the traits from TOA data as acquired from the S3 Ocean and Land Colour Instrument (OLCI) sensor.Following good to high theoretical validation results with normalized root mean square error (NRMSE) ranging from 5% (FAPAR) to 19% (LAI), a three fold evaluation approach over diverse sites and land cover types was pursued: (1) temporal comparison against LAI and FAPAR products obtained from Moderate Resolution Imaging Spectroradiometer (MODIS) for the time window 2016-2020, (2) spatial difference mapping with Copernicus Global Land Service (CGLS) estimates, and (3) direct validation using interpolated in situ data from the VALERI network. For all three approaches, promising results were achieved. Selected sites demonstrated coherent seasonal patterns compared to LAI and FAPAR MODIS products, with differences between spatially averaged temporal patterns of only 6.59%. In respect of the spatial mapping comparison, estimates provided by the S3-TOA-GPR-1.0 models indicated highest consistency with FVC and FAPAR CGLS products. Moreover, the direct validation of our S3-TOA-GPR-1.0 models against VALERI estimates indicated with regard to jurisdictional claims in good retrieval performance for LAI, FAPAR and FVC. We conclude that our retrieval workflow of spatiotemporal S3 TOA data processing into GEE opens the path towards global monitoring of fundamental vegetation traits, accessible to the whole research community.

Green LAI Mapping and Cloud Gap-Filling Using Gaussian Process Regression in Google Earth Engine.

For the last decade, Gaussian process regression (GPR) proved to be a competitive machine learning regression algorithm for Earth observation applications, with attractive unique properties such as band relevance ranking and uncertainty estimates. More recently, GPR also proved to be a proficient time series processor to fill up gaps in optical imagery, typically due to cloud cover. This makes GPR perfectly suited for large-scale spatiotemporal processing of satellite imageries into cloud-free products of biophysical variables. With the advent of the Google Earth Engine (GEE) cloud platform, new opportunities emerged to process local-to-planetary scale satellite data using advanced machine learning techniques and convert them into gap-filled vegetation properties products. However, GPR is not yet part of the GEE ecosystem. To circumvent this limitation, this work proposes a general adaptation of GPR formulation to parallel processing framework and its integration into GEE. To demonstrate the functioning and utility of the developed workflow, a GPR model predicting green leaf area index (LAI G ) from Sentinel-2 imagery was imported. Although by running this GPR model into GEE any corner of the world can be mapped into LAI G at a resolution of 20 m, here we show some demonstration cases over western Europe with zoom-ins over Spain. Thanks to the computational power of GEE, the mapping takes place on-the-fly. Additionally, a GPR-based gap filling strategy based on pre-optimized kernel hyperparameters is also put forward for the generation of multi-orbit cloud-free LAI G maps with an unprecedented level of detail, and the extraction of regularly-sampled LAI G time series at a pixel level. The ability to plugin a locally-trained GPR model into the GEE framework and its instant processing opens up a new paradigm of remote sensing image processing.

Monitoring Cropland Phenology on Google Earth Engine Using Gaussian Process Regression.

Monitoring cropland phenology from optical satellite data remains a challenging task due to the influence of clouds and atmospheric artifacts. Therefore, measures need to be taken to overcome these challenges and gain better knowledge of crop dynamics. The arrival of cloud computing platforms such as Google Earth Engine (GEE) has enabled us to propose a Sentinel-2 (S2) phenology end-to-end processing chain. To achieve this, the following pipeline was implemented: (1) the building of hybrid Gaussian Process Regression (GPR) retrieval models of crop traits optimized with active learning, (2) implementation of these models on GEE (3) generation of spatiotemporally continuous maps and time series of these crop traits with the use of gap-filling through GPR fitting, and finally, (4) calculation of land surface phenology (LSP) metrics such as the start of season (SOS) or end of season (EOS). Overall, from good to high performance was achieved, in particular for the estimation of canopy-level traits such as leaf area index (LAI) and canopy chlorophyll content, with normalized root mean square errors (NRMSE) of 9% and 10%, respectively. By means of the GPR gap-filling time series of S2, entire tiles were reconstructed, and resulting maps were demonstrated over an agricultural area in Castile and Leon, Spain, where crop calendar data were available to assess the validity of LSP metrics derived from crop traits. In addition, phenology derived from the normalized difference vegetation index (NDVI) was used as reference. NDVI not only proved to be a robust indicator for the calculation of LSP metrics, but also served to demonstrate the good phenology quality of the quantitative trait products. Thanks to the GEE framework, the proposed workflow can be realized anywhere in the world and for any time window, thus representing a shift in the satellite data processing paradigm. We anticipate that the produced LSP metrics can provide meaningful insights into crop seasonal patterns in a changing environment that demands adaptive agricultural production.

Retrieving and Validating Leaf and Canopy Chlorophyll Content at Moderate Resolution: A Multiscale Analysis with the Sentinel-3 OLCI Sensor.

ESA's Eighth Earth Explorer mission "FLuorescence EXplorer" (FLEX) will be dedicated to the global monitoring of the chlorophyll fluorescence emitted by vegetation. In order to properly interpret the measured fluorescence signal, essential vegetation variables need to be retrieved concomitantly. FLEX will fly in tandem formation with Sentinel-3 (S3), which conveys the Ocean and Land Color Instrument (OLCI) that is designed to characterize the atmosphere and the terrestrial vegetation at a spatial resolution of 300 m. In support of FLEX's preparatory activities, this paper presents a first validation exercise of OLCI vegetation products against in situ data coming from the 2018 FLEXSense campaign. During this campaign, leaf chlorophyll content (LCC) and leaf area index (LAI) measurements were collected over croplands, while HyPlant DUAL images of the area were acquired at a 3 m spatial resolution. A multiscale validation strategy was pursued. First, estimates of these two variables, together with the combined canopy chlorophyll content (CCC = LCC × LAI), were obtained at the HyPlant spatial resolution and were compared against the in situ measurements. Second, the fine-scale retrieval maps from HyPlant were coarsened to the S3 spatial scale as a reference to assess the quality of the OLCI vegetation products. As an intermediary step, vegetation products extracted from Sentinel-2 data were used to compare retrievals at the in-between spatial resolution of 20 m. For all spatial scales, CCC delivered the most accurate estimates with the smallest prediction error obtained at the 300 m resolution (R2 of 0.74 and RMSE = 26.8 µg cm-2). Results of a scaling analysis suggest that CCC performs well at the different tested spatial resolutions since it presents a linear behavior across scales. LCC, on the other hand, was poorly retrieved at the 300 m scale, showing overestimated values over heterogeneous pixels. The introduction of a new LCC model integrating mixed reflectance spectra in its training enabled to improve by 16% the retrieval accuracy for this variable (RMSE = 10 µg cm-2 for the new model versus RMSE = 11.9 µg cm-2 for the former model).

Optimizing Gaussian Process Regression for Image Time Series Gap-Filling and Crop Monitoring.

Image processing entered the era of artificial intelligence, and machine learning algorithms emerged as attractive alternatives for time series data processing. Satellite image time series processing enables crop phenology monitoring, such as the calculation of start and end of season. Among the promising algorithms, Gaussian process regression (GPR) proved to be a competitive time series gap-filling algorithm with the advantage of, as developed within a Bayesian framework, providing associated uncertainty estimates. Nevertheless, the processing of time series images becomes computationally inefficient in its standard per-pixel usage, mainly for GPR training rather than the fitting step. To mitigate this computational burden, we propose to substitute the per-pixel optimization step with the creation of a cropland-based precalculations for the GPR hyperparameters θ . To demonstrate our approach hardly affects the accuracy in fitting, we used Sentinel-2 LAI time series over an agricultural region in Castile and Leon, North-West Spain. The performance of image reconstructions were compared against the standard per-pixel GPR time series processing. Results showed that accuracies were on the same order (RMSE 0.1767 vs. 0.1564 [m2/m2], 12% RMSE degradation) whereas processing time accelerated about 90 times. We further evaluated the alternative option of using the same hyperparameters for all the pixels within the complete scene. It led to similar overall accuracies over crop areas and computational performance. Crop phenology indicators were also calculated for the three different approaches and compared. Results showed analogous crop temporal patterns, with differences in start and end of growing season of no more than five days. To the benefit of crop monitoring applications, all the gap-filling and phenology indicators retrieval techniques have been implemented into the freely downloadable GUI toolbox DATimeS.

Urban planning and agriculture. Methodology for assessing rooftop greenhouse potential of non-residential areas using airborne sensors.

The integration of rooftop greenhouses (RTGs) in urban buildings is a practice that is becoming increasingly important in the world for their contribution to food security and sustainable development. However, the supply of tools and procedures to facilitate their implementation at the city scale is limited and laborious. This work aims to develop a specific and automated methodology for identifying the feasibility of implementation of rooftop greenhouses in non-residential urban areas, using airborne sensors. The use of Light Detection and Ranging (LIDAR) and Long Wave Infrared (LWIR) data and the Leica ALS50-II and TASI-600 sensors allow for the identification of some building roof parameters (area, slope, materials, and solar radiation) to determine the potential for constructing a RTG. This development represents an improvement in time and accuracy with respect to previous methodology, where all the relevant information must be acquired manually. The methodology has been applied and validated in a case study corresponding to a non-residential urban area in the industrial municipality of Rubí, Barcelona (Spain). Based on this practical application, an area of 36,312m2 out of a total area of 1,243,540m2 of roofs with ideal characteristics for the construction of RTGs was identified. This area can produce approximately 600tons of tomatoes per year, which represents the average yearly consumption for about 50% of Rubí total population. The use of this methodology also facilitates the decision making process in urban agriculture, allowing a quick identification of optimal surfaces for the future implementation of urban agriculture in housing. It also opens new avenues for the use of airborne technology in environmental topics in cities.