An UltraMNIST classification benchmark to train CNNs for very large images.

Current convolutional neural networks (CNNs) are not designed for large scientific images with rich multi-scale features, such as in satellite and microscopy domain. A new phase of development of CNNs especially designed for large images is awaited. However, application-independent high-quality and challenging datasets needed for such development are still missing. We present the 'UltraMNIST dataset' and associated benchmarks for this new research problem of 'training CNNs for large images'. The dataset is simple, representative of wide-ranging challenges in scientific data, and easily customizable for different levels of complexity, smallest and largest features, and sizes of images. Two variants of the problem are discussed: standard version that facilitates the development of novel CNN methods for effective use of the best available GPU resources and the budget-aware version to promote the development of methods that work under constrained GPU memory. Several baselines are presented and the effect of reduced resolution is studied. The presented benchmark dataset and baselines will hopefully trigger the development of new CNN methods for large scientific images.

Characterizing the consistency of motion of spermatozoa through nanoscale motion tracing.

OBJECTIVE: To demonstrate nanoscale motion tracing of spermatozoa and present analysis of the motion traces to characterize the consistency of motion of spermatozoa as a complement to progressive motility analysis. DESIGN: Anonymized sperm samples videographed under quantitative phase microscope, followed by generating and analyzing super-resolution motion traces of individual spermatozoa. SUBJECTS: Centrifuged human sperm samples. MAIN OUTCOME MEASURES: Precision of motion trace of individual sperms, presence of helical pattern in the motion trace, and mean and standard deviations of helical periods and radii of sperm motion traces, speed of progression. RESULTS: Spatially sensitive quantitative phase imaging with a super-resolution computational technique MUltiple SIgnal Classification ALgorithm (MUSICAL) allowed achieving motion precision of 340 nm using 10×, 0.25 NA lens whereas the diffraction limited resolution at this setting was 1320 nm. The motion traces thus derived facilitated new kinematic features of sperm, namely the statistics of helix period and radii per sperm. Through the analysis, 47 sperms with speed >25 µm/sec randomly selected from the same healthy donor's semen sample, it is seen that the kinematic features did not correlate with the speed of the sperms. Also, it is noted that spermatozoa may experience changes in the periodicity and radius of the helical path over time. Further, some very fast sperms (for example >70 µm/sec) may demonstrate irregular motion, needs further investigation. Presented computational analysis can be used directly for sperm samples from both fertility patients with normal and abnormal sperm cell conditions. We note that MUSICAL is an image analysis technique which may vaguely fall under machine learning category, but the conventional metrics for reporting found in EQUATOR do not apply. Alternative suitable metrics are reported, and bias is avoided through random selection of regions for analysis. Detailed methods are included for reproducibility. CONCLUSION: Kinematic features derived from nanoscale motion traces of spermatozoa contain information complementary to the speed of the sperms, allowing further distinction among the progressively motile sperms. Some highly progressive spermatozoa may have irregular motion pattern, and whether irregularity of motion indicate poor quality regarding artificial insemination needs further investigation. Presented technique can be generalized for sperm analysis for a variety of fertility conditions.

Compact Linnik-type hyperspectral quantitative phase microscope for advanced classification of cellular components.

Hyperspectral quantitative phase microscopy (HS-QPM) involves the acquisition of phase images across narrow spectral bands, which enables wavelength-dependent study of different biological samples. In the present work, a compact Linnik-type HS-QPM system is developed to reduce the instability and complexity associated with conventional HS-QPM techniques. The use of a single objective lens for both reference and sample arms makes the setup compact. The capabilities of the system are demonstrated by evaluating the HS phase map of both industrial and biological specimens. Phase maps of exfoliated cheek cells at different wavelengths are stacked to form a HS phase cube, adding spectral dimensionality to spatial phase distribution. Analysis of wavelength response of different cellular components are performed using principal component analysis to identify dominant spectral features present in the HS phase dataset. Findings of the study emphasize on the efficiency and effectiveness of HS-QPM for advancing cellular characterization in biomedical research and clinical applications.

Molecular histopathology of matrix proteins through autofluorescence super-resolution microscopy.

Extracellular matrix diseases like fibrosis are elusive to diagnose early on, to avoid complete loss of organ function or even cancer progression, making early diagnosis crucial. Imaging the matrix densities of proteins like collagen in fixed tissue sections with suitable stains and labels is a standard for diagnosis and staging. However, fine changes in matrix density are difficult to realize by conventional histological staining and microscopy as the matrix fibrils are finer than the resolving capacity of these microscopes. The dyes further blur the outline of the matrix and add a background that bottlenecks high-precision early diagnosis of matrix diseases. Here we demonstrate the multiple signal classification method-MUSICAL-otherwise a computational super-resolution microscopy technique to precisely estimate matrix density in fixed tissue sections using fibril autofluorescence with image stacks acquired on a conventional epifluorescence microscope. We validated the diagnostic and staging performance of the method in extracted collagen fibrils, mouse skin during repair, and pre-cancers in human oral mucosa. The method enables early high-precision label-free diagnosis of matrix-associated fibrotic diseases without needing additional infrastructure or rigorous clinical training.

Taxonomy of hybridly polarized Stokes vortex beams.

Structured beams carrying topological defects, namely phase and Stokes singularities, have gained extensive interest in numerous areas of optics. The non-separable spin and orbital angular momentum states of hybridly polarized Stokes singular beams provide additional freedom for manipulating optical fields. However, the characterization of hybridly polarized Stokes vortex beams remains challenging owing to the degeneracy associated with the complex polarization structures of these beams. In addition, experimental noise factors such as relative phase, amplitude, and polarization difference together with beam fluctuations add to the perplexity in the identification process. Here, we present a generalized diffraction-based Stokes polarimetry approach assisted with deep learning for efficient identification of Stokes singular beams. A total of 15 classes of beams are considered based on the type of Stokes singularity and their associated mode indices. The resultant total and polarization component intensities of Stokes singular beams after diffraction through a triangular aperture are exploited by the deep neural network to recognize these beams. Our approach presents a classification accuracy of 98.67% for 15 types of Stokes singular beams that comprise several degenerate cases. The present study illustrates the potential of diffraction of the Stokes singular beam with polarization transformation, modeling of experimental noise factors, and a deep learning framework for characterizing hybridly polarized beams.

Fluorescence microscopy and correlative brightfield videos of mitochondria and vesicles in H9c2 cardiomyoblasts.

This paper presents data acquired to study the dynamics and interactions of mitochondria and subcellular vesicles in living cardiomyoblasts. The study was motivated by the importance of mitochondrial quality control and turnover in cardiovascular health. Although fluorescence microscopy is an invaluable tool, it presents several limitations. Correlative fluorescence and brightfield images (label-free) were therefore acquired with the purpose of achieving virtual labelling via machine learning. In comparison with the fluorescence images of mitochondria, the brightfield images show vesicles and subcellular components, providing additional insights about sub-cellular components. A large part of the data contains correlative fluorescence images of lysosomes and/or endosomes over a duration of up to 400 timepoints (>30 min). The data can be reused for biological inferences about mitochondrial and vesicular morphology, dynamics, and interactions. Furthermore, virtual labelling of mitochondria or subcellular vesicles can be achieved using these datasets. Finally, the data can inspire new imaging experiments for cellular investigations or computational developments. The data is available through two large, open datasets on DataverseNO.

Single-shot multispectral quantitative phase imaging of biological samples using deep learning.

Multispectral quantitative phase imaging (MS-QPI) is a high-contrast label-free technique for morphological imaging of the specimens. The aim of the present study is to extract spectral dependent quantitative information in single-shot using a highly spatially sensitive digital holographic microscope assisted by a deep neural network. There are three different wavelengths used in our method: λ=532, 633, and 808 nm. The first step is to get the interferometric data for each wavelength. The acquired datasets are used to train a generative adversarial network to generate multispectral (MS) quantitative phase maps from a single input interferogram. The network was trained and validated on two different samples: the optical waveguide and MG63 osteosarcoma cells. Validation of the present approach is performed by comparing the predicted MS phase maps with numerically reconstructed (F T+T I E) phase maps and quantifying with different image quality assessment metrices.

High-resolution visualization and assessment of basal and OXPHOS-induced mitophagy in H9c2 cardiomyoblasts.

Mitochondria are susceptible to damage resulting from their activity as energy providers. Damaged mitochondria can cause harm to the cell and thus mitochondria are subjected to elaborate quality-control mechanisms including elimination via lysosomal degradation in a process termed mitophagy. Basal mitophagy is a house-keeping mechanism fine-tuning the number of mitochondria according to the metabolic state of the cell. However, the molecular mechanisms underlying basal mitophagy remain largely elusive. In this study, we visualized and assessed the level of mitophagy in H9c2 cardiomyoblasts at basal conditions and after OXPHOS induction by galactose adaptation. We used cells with a stable expression of a pH-sensitive fluorescent mitochondrial reporter and applied state-of-the-art imaging techniques and image analysis. Our data showed a significant increase in acidic mitochondria after galactose adaptation. Using a machine-learning approach we also demonstrated increased mitochondrial fragmentation by OXPHOS induction. Furthermore, super-resolution microscopy of live cells enabled capturing of mitochondrial fragments within lysosomes as well as dynamic transfer of mitochondrial contents to lysosomes. Applying correlative light and electron microscopy we revealed the ultrastructure of the acidic mitochondria confirming their proximity to the mitochondrial network, ER and lysosomes. Finally, exploiting siRNA knockdown strategy combined with flux perturbation with lysosomal inhibitors, we demonstrated the importance of both canonical as well as non-canonical autophagy mediators in lysosomal degradation of mitochondria after OXPHOS induction. Taken together, our high-resolution imaging approaches applied on H9c2 cells provide novel insights on mitophagy during physiologically relevant conditions. The implication of redundant underlying mechanisms highlights the fundamental importance of mitophagy.Abbreviations: ATG: autophagy related; ATG7: autophagy related 7; ATP: adenosine triphosphate; BafA1: bafilomycin A1; CLEM: correlative light and electron microscopy; EGFP: enhanced green fluorescent protein; MAP1LC3B: microtubule associated protein 1 light chain 3 beta; OXPHOS: oxidative phosphorylation; PepA: pepstatin A; PLA: proximity ligation assay; PRKN: parkin RBR E3 ubiquitin protein ligase; RAB5A: RAB5A, member RAS oncogene family; RAB7A: RAB7A, member RAS oncogene family; RAB9A: RAB9A, member RAS oncogene family; ROS: reactive oxygen species; SIM: structured illumination microscopy; siRNA: short interfering RNA; SYNJ2BP: synaptojanin 2 binding protein; TEM: transmission electron microscopy; TOMM20: translocase of outer mitochondrial membrane 20; ULK1: unc-51 like kinase 1.

Viewing life without labels under optical microscopes.

Optical microscopes today have pushed the limits of speed, quality, and observable space in biological specimens revolutionizing how we view life today. Further, specific labeling of samples for imaging has provided insight into how life functions. This enabled label-based microscopy to percolate and integrate into mainstream life science research. However, the use of labelfree microscopy has been mostly limited, resulting in testing for bio-application but not bio-integration. To enable bio-integration, such microscopes need to be evaluated for their timeliness to answer biological questions uniquely and establish a long-term growth prospect. The article presents key label-free optical microscopes and discusses their integrative potential in life science research for the unperturbed analysis of biological samples.

3D full-wave multi-scattering forward solver for coherent microscopes.

A rigorous forward model solver for conventional coherent microscope is presented. The forward model is derived from Maxwell's equations and models the wave behaviour of light matter interaction. Vectorial waves and multiple-scattering effect are considered in this model. Scattered field can be calculated with given distribution of the refractive index of the biological sample. Bright field images can be obtained by combining the scattered field and reflected illumination, and experimental validation is included. Insights into the utility of the full-wave multi-scattering (FWMS) solver and comparison with the conventional Born approximation based solver are presented. The model is also generalizable to the other forms of label-free coherent microscopes, such as quantitative phase microscope and dark-field microscope.

Analyzing Mitochondrial Morphology Through Simulation Supervised Learning.

The quantitative analysis of subcellular organelles such as mitochondria in cell fluorescence microscopy images is a demanding task because of the inherent challenges in the segmentation of these small and morphologically diverse structures. In this article, we demonstrate the use of a machine learning-aided segmentation and analysis pipeline for the quantification of mitochondrial morphology in fluorescence microscopy images of fixed cells. The deep learning-based segmentation tool is trained on simulated images and eliminates the requirement for ground truth annotations for supervised deep learning. We demonstrate the utility of this tool on fluorescence microscopy images of fixed cardiomyoblasts with a stable expression of fluorescent mitochondria markers and employ specific cell culture conditions to induce changes in the mitochondrial morphology.

Impact of Blood Pressure Control on Graft Survival in Kidney Transplant Recipients.

BACKGROUND: Cardiovascular disease is the primary driver of morbidity and mortality in kidney transplant recipients. Hypertension is an important risk factor for development of cardiovascular disease in this population. Despite its important role in post-transplant outcomes, the blood pressure goals for kidney transplant recipients remain elusive. Current guidelines are based on observational data or data extrapolated from the chronic kidney disease population. METHODS: We followed 5-year blood pressure control of 378 kidney-alone transplant recipients at a single center and evaluated patient survival, graft survival, proteinuria, and rate of decline of kidney graft function. RESULTS: We found that a mean systolic blood pressure (BP) of 121 to 130 mm Hg was associated with better graft survival, slower decline of kidney allograft function, and lower degree of proteinuria when compared with a mean systolic BP ≤120 or >130 mm Hg. CONCLUSION: This study provides evidence for strict blood pressure control, systolic BP between 121 and 130 mm Hg, and also cautions against intensive control of systolic BP <120 mm Hg in kidney transplant recipients.

Scalable-resolution structured illumination microscopy.

Structured illumination microscopy suffers from the need of sophisticated instrumentation and precise calibration. This makes structured illumination microscopes costly and skill-dependent. We present a novel approach to realize super-resolution structured illumination microscopy using an alignment non-critical illumination system and a reconstruction algorithm that does not need illumination information. The optical system is designed to encode higher order frequency components of the specimen by projecting PSF-modulated binary patterns for illuminating the sample plane, which do not have clean Fourier peaks conventionally used in structured illumination microscopy. These patterns fold high frequency content of sample into the measurements in an obfuscated manner, which are de-obfuscated using multiple signal classification algorithm. This algorithm eliminates the need of clean peaks in illumination and the knowledge of illumination patterns, which makes instrumentation simple and flexible for use with a variety of microscope objective lenses. We present a variety of experimental results on beads and cell samples to demonstrate resolution enhancement by a factor of 2.6 to 3.4 times, which is better than the enhancement supported by the conventional linear structure illumination microscopy where the same objective lens is used for structured illumination as well as collection of light. We show that the same system can be used in SIM configuration with different collection objective lenses without any careful re-calibration or realignment, thereby supporting a range of resolutions with the same system.

Virtual labeling of mitochondria in living cells using correlative imaging and physics-guided deep learning.

Mitochondria play a crucial role in cellular metabolism. This paper presents a novel method to visualize mitochondria in living cells without the use of fluorescent markers. We propose a physics-guided deep learning approach for obtaining virtually labeled micrographs of mitochondria from bright-field images. We integrate a microscope's point spread function in the learning of an adversarial neural network for improving virtual labeling. We show results (average Pearson correlation 0.86) significantly better than what was achieved by state-of-the-art (0.71) for virtual labeling of mitochondria. We also provide new insights into the virtual labeling problem and suggest additional metrics for quality assessment. The results show that our virtual labeling approach is a powerful way of segmenting and tracking individual mitochondria in bright-field images, results previously achievable only for fluorescently labeled mitochondria.

Three-dimensional structured illumination microscopy data of mitochondria and lysosomes in cardiomyoblasts under normal and galactose-adapted conditions.

This three-dimensional structured illumination microscopy (3DSIM) dataset was generated to highlight the suitability of 3DSIM to investigate mitochondria-derived vesicles (MDVs) in H9c2 cardiomyoblasts in living or fixed cells. MDVs act as a mitochondria quality control mechanism. The cells were stably expressing the tandem-tag eGFP-mCherry-OMP25-TM (outer mitochondrial membrane) which can be used as a sensor for acidity. A part of the dataset is showing correlative imaging of lysosomes labeled using LysoTracker in fixed and living cells. The cells were cultivated in either normal or glucose-deprived medium containing galactose. The resulting 3DSIM data were of high quality and can be used to undertake a variety of studies. Interestingly, many dynamic tubules derived from mitochondria are visible in the 3DSIM videos under both glucose and galactose-adapted growth conditions. As the raw 3DSIM data, optical parameters, and reconstructed 3DSIM images are provided, the data is especially suitable for use in the development of SIM reconstruction algorithms, bioimage analysis methods, and for biological studies of mitochondria.

Chip-based multimodal super-resolution microscopy for histological investigations of cryopreserved tissue sections.

Histology involves the observation of structural features in tissues using a microscope. While diffraction-limited optical microscopes are commonly used in histological investigations, their resolving capabilities are insufficient to visualize details at subcellular level. Although a novel set of super-resolution optical microscopy techniques can fulfill the resolution demands in such cases, the system complexity, high operating cost, lack of multi-modality, and low-throughput imaging of these methods limit their wide adoption for histological analysis. In this study, we introduce the photonic chip as a feasible high-throughput microscopy platform for super-resolution imaging of histological samples. Using cryopreserved ultrathin tissue sections of human placenta, mouse kidney, pig heart, and zebrafish eye retina prepared by the Tokuyasu method, we demonstrate diverse imaging capabilities of the photonic chip including total internal reflection fluorescence microscopy, intensity fluctuation-based optical nanoscopy, single-molecule localization microscopy, and correlative light-electron microscopy. Our results validate the photonic chip as a feasible imaging platform for tissue sections and pave the way for the adoption of super-resolution high-throughput multimodal analysis of cryopreserved tissue samples both in research and clinical settings.