Combining Ground Based Remote Sensing Tools for Rockfalls Assessment and Monitoring: The Poggio Baldi Landslide Natural Laboratory.

Nowadays the use of remote monitoring sensors is a standard practice in landslide characterization and monitoring. In the last decades, technologies such as LiDAR, terrestrial and satellite SAR interferometry (InSAR) and photogrammetry demonstrated a great potential for rock slope assessment while limited studies and applications are still available for ArcSAR Interferometry, Gigapixel imaging and Acoustic sensing. Taking advantage of the facilities located at the Poggio Baldi Landslide Natural Laboratory, an intensive monitoring campaign was carried out on May 2019 using simultaneously the HYDRA-G ArcSAR for radar monitoring, the Gigapan robotic system equipped with a DSLR camera for photo-monitoring purposes and the DUO Smart Noise Monitor for acoustic measurements. The aim of this study was to evaluate the potential of each monitoring sensor and to investigate the ongoing gravitational processes at the Poggio Baldi landslide. Analysis of multi-temporal Gigapixel-images revealed the occurrence of 84 failures of various sizes between 14-17 May 2019. This allowed us to understand the short-term evolution of the rock cliff that is characterized by several impulsive rockfall events and continuous debris production. Radar displacement maps revealed a constant movement of the debris talus at the toe of the main rock scarp, while acoustic records proved the capability of this technique to identify rockfall events as well as their spectral content in a narrow range of frequencies between 200 Hz to 1000 Hz. This work demonstrates the great potential of the combined use of a variety of remote sensors to achieve high spatial and temporal resolution data in the field of landslide characterization and monitoring.

Unsupervised mutual transformer learning for multi-gigapixel Whole Slide Image classification.

The classification of gigapixel Whole Slide Images (WSIs) is an important task in the emerging area of computational pathology. There has been a surge of interest in deep learning models for WSI classification with clinical applications such as cancer detection or prediction of cellular mutations. Most supervised methods require expensive and labor-intensive manual annotations by expert pathologists. Weakly supervised Multiple Instance Learning (MIL) methods have recently demonstrated excellent performance; however, they still require large-scale slide-level labeled training datasets that require a careful inspection of each slide by an expert pathologist. In this work, we propose a fully unsupervised WSI classification algorithm based on mutual transformer learning. The instances (i.e., patches) from gigapixel WSIs are transformed into a latent space and then inverse-transformed to the original space. Using the transformation loss, pseudo labels are generated and cleaned using a transformer label cleaner. The proposed transformer-based pseudo-label generator and cleaner modules mutually train each other iteratively in an unsupervised manner. A discriminative learning mechanism is introduced to improve normal versus cancerous instance labeling. In addition to the unsupervised learning, we demonstrate the effectiveness of the proposed framework for weakly supervised learning and cancer subtype classification as downstream analysis. Extensive experiments on four publicly available datasets show better performance of the proposed algorithm compared to the existing state-of-the-art methods.

Magnifying Networks for Histopathological Images with Billions of Pixels.

Amongst the other benefits conferred by the shift from traditional to digital pathology is the potential to use machine learning for diagnosis, prognosis, and personalization. A major challenge in the realization of this potential emerges from the extremely large size of digitized images, which are often in excess of 100,000 × 100,000 pixels. In this paper, we tackle this challenge head-on by diverging from the existing approaches in the literature-which rely on the splitting of the original images into small patches-and introducing magnifying networks (MagNets). By using an attention mechanism, MagNets identify the regions of the gigapixel image that benefit from an analysis on a finer scale. This process is repeated, resulting in an attention-driven coarse-to-fine analysis of only a small portion of the information contained in the original whole-slide images. Importantly, this is achieved using minimal ground truth annotation, namely, using only global, slide-level labels. The results from our tests on the publicly available Camelyon16 and Camelyon17 datasets demonstrate the effectiveness of MagNets-as well as the proposed optimization framework-in the task of whole-slide image classification. Importantly, MagNets process at least five times fewer patches from each whole-slide image than any of the existing end-to-end approaches.

PodoCount: A Robust, Fully Automated, Whole-Slide Podocyte Quantification Tool.

Introduction: Podocyte depletion is a histomorphologic indicator of glomerular injury and predicts clinical outcomes. Podocyte estimation methods or podometrics are semiquantitative, technically involved, and laborious. Implementation of high-throughput podometrics in experimental and clinical workflows necessitates an automated podometrics pipeline. Recognizing that computational image analysis offers a robust approach to study cell and tissue structure, we developed and validated PodoCount (a computational tool for automated podocyte quantification in immunohistochemically labeled tissues) using a diverse data set. Methods: Whole-slide images (WSIs) of tissues immunostained with a podocyte nuclear marker and periodic acid-Schiff counterstain were acquired. The data set consisted of murine whole kidney sections (n = 135) from 6 disease models and human kidney biopsy specimens from patients with diabetic nephropathy (DN) (n = 45). Within segmented glomeruli, podocytes were extracted and image analysis was applied to compute measures of podocyte depletion and nuclear morphometry. Computational performance evaluation and statistical testing were performed to validate podometric and associated image features. PodoCount was disbursed as an open-source, cloud-based computational tool. Results: PodoCount produced highly accurate podocyte quantification when benchmarked against existing methods. Podocyte nuclear profiles were identified with 0.98 accuracy and segmented with 0.85 sensitivity and 0.99 specificity. Errors in podocyte count were bounded by 1 podocyte per glomerulus. Podocyte-specific image features were found to be significant predictors of disease state, proteinuria, and clinical outcome. Conclusion: PodoCount offers high-performance podocyte quantitation in diverse murine disease models and in human kidney biopsy specimens. Resultant features offer significant correlation with associated metadata and outcome. Our cloud-based tool will provide end users with a standardized approach for automated podometrics from gigapixel-sized WSIs.

Documenting Paintings with Gigapixel Photography.

Digital photographic capture of pictorial artworks with gigapixel resolution (around 1000 megapixels or greater) is a novel technique that is beginning to be used by some important international museums as a means of documentation, analysis, and dissemination of their masterpieces. This line of research is extremely interesting, not only for art curators and scholars but also for the general public. The results can be disseminated through online virtual museum displays, offering a detailed interactive visualization. These virtual visualizations allow the viewer to delve into the artwork in such a way that it is possible to zoom in and observe those details, which would be negligible to the naked eye in a real visit. Therefore, this kind of virtual visualization using gigapixel images has become an essential tool to enhance cultural heritage and to make it accessible to everyone. Since today's professional digital cameras provide images of around 40 megapixels, obtaining gigapixel images requires some special capture and editing techniques. This article describes a series of photographic methodologies and equipment, developed by the team of researchers, that have been put into practice to achieve a very high level of detail and chromatic fidelity, in the documentation and dissemination of pictorial artworks. The result of this research work consisted in the gigapixel documentation of several masterpieces of the Museo de Bellas Artes of Valencia, one of the main art galleries in Spain. The results will be disseminated through the Internet, as will be shown with some examples.

Review of bio-optical imaging systems with a high space-bandwidth product.

Optical imaging has served as a primary method to collect information about biosystems across scales-from functionalities of tissues to morphological structures of cells and even at biomolecular levels. However, to adequately characterize a complex biosystem, an imaging system with a number of resolvable points, referred to as a space-bandwidth product (SBP), in excess of one billion is typically needed. Since a gigapixel-scale far exceeds the capacity of current optical imagers, compromises must be made to obtain either a low spatial resolution or a narrow field-of-view (FOV). The problem originates from constituent refractive optics-the larger the aperture, the more challenging the correction of lens aberrations. Therefore, it is impractical for a conventional optical imaging system to achieve an SBP over hundreds of millions. To address this unmet need, a variety of high-SBP imagers have emerged over the past decade, enabling an unprecedented resolution and FOV beyond the limit of conventional optics. We provide a comprehensive survey of high-SBP imaging techniques, exploring their underlying principles and applications in bioimaging.

Gigapixel big data movies provide cost-effective seascape scale direct measurements of open-access coastal human use such as recreational fisheries.

Collecting data on unlicensed open-access coastal activities, such as some types of recreational fishing, has often relied on telephone interviews selected from landline directories. However, this approach is becoming obsolete due to changes in communication technology such as a switch to unlisted mobile phones. Other methods, such as boat ramp interviews, are often impractical due to high labor cost. We trialed an autonomous, ultra-high-resolution photosampling method as a cost effect solution for direct measurements of a recreational fishery. Our sequential photosampling was batched processed using a novel software application to produce "big data" time series movies from a spatial subset of the fishery, and we validated this with a regional bus-route survey and interviews with participants at access points. We also compared labor costs between these two methods. Most trailer boat users were recreational fishers targeting tuna spp. Our camera system closely matched trends in temporal variation from the larger scale regional survey, but as the camera data were at much higher frequency, we could additionally describe strong, daily variability in effort. Peaks were normally associated with weekends, but consecutive weekend tuna fishing competitions led to an anomaly of high effort across the normal weekday lulls. By reducing field time and batch processing imagery, Monthly labor costs for the camera sampling were a quarter of the bus-route survey; and individual camera samples cost 2.5% of bus route samples to obtain. Gigapixel panoramic camera observations of fishing were representative of the temporal variability of regional fishing effort and could be used to develop a cost-efficient index. High-frequency sampling had the added benefit of being more likely to detect abnormal patterns of use. Combinations of remote sensing and on-site interviews may provide a solution to describing highly variable effort in recreational fisheries while also validating activity and catch.

The gigapixel image concept for graphic SEM documentation. Applications in archeological use-wear studies.

In this paper, we propose a specific procedure to create gigapixel-like images from SEM (scanning electron microscope) micrographs. This methodology allows intensive SEM observations to be made for those disciplines that require of large surfaces to be analyzed at different scales once the SEM sessions have been completed (e.g., stone tools use-wear studies). This is also a very useful resource for academic purposes or as a support for collaborative studies, thus reducing the number of live observation sessions and the associated expense.

Wide-field, high-resolution Fourier ptychographic microscopy.

In this article, we report an imaging method, termed Fourier ptychographic microscopy (FPM), which iteratively stitches together a number of variably illuminated, low-resolution intensity images in Fourier space to produce a wide-field, high-resolution complex sample image. By adopting a wavefront correction strategy, the FPM method can also correct for aberrations and digitally extend a microscope's depth-of-focus beyond the physical limitations of its optics. As a demonstration, we built a microscope prototype with a resolution of 0.78 µm, a field-of-view of ~120 mm2, and a resolution-invariant depth-of-focus of 0.3 mm (characterized at 632 nm). Gigapixel colour images of histology slides verify FPM's successful operation. The reported imaging procedure transforms the general challenge of high-throughput, high-resolution microscopy from one that is coupled to the physical limitations of the system's optics to one that is solvable through computation.

Results and insights from the NCSU Insect Museum GigaPan project.

Pinned insect specimens stored in museum collections are a fragile and valuable resource for entomological research. As such, they are usually kept away from viewing by the public and hard to access by experts. Here we present a method for mass imaging insect specimens, using GigaPan technology to achieve highly explorable, many-megapixel panoramas of insect museum drawers. We discuss the advantages and limitations of the system, and describe future avenues of collections research using this technology.