Deep learning-based size prediction for optical trapped nanoparticles and extracellular vesicles from limited bandwidth camera detection.

Due to its ability to record position, intensity, and intensity distribution information, camera-based monitoring of nanoparticles in optical traps can enable multi-parametric morpho-optical characterization at the single-particle level. However, blurring due to the relatively long (10s of microsecond) integration times and aliasing from the resulting limited temporal bandwidth affect the detected particle position when considering nanoparticles in traps with strong stiffness, leading to inaccurate size predictions. Here, we propose a ResNet-based method for accurate size characterization of trapped nanoparticles, which is trained by considering only simulated time series data of nanoparticles' constrained Brownian motion. Experiments prove the method outperforms state-of-art sizing algorithms such as adjusted Lorentzian fitting or CNN-based networks on both standard nanoparticles and extracellular vesicles (EVs), as well as maintains good accuracy even when measurement times are relatively short (<1s per particle). On samples of clinical EVs, our network demonstrates a well-generalized ability to accurately determine the EV size distribution, as confirmed by comparison with gold-standard nanoparticle tracking analysis (NTA). Furthermore, by combining the sizing network with still frame images from high-speed video, the camera-based optical tweezers have the unique capacity to quantify both the size and refractive index of bio-nanoparticles at the single-particle level. These experiments prove the proposed sizing network as an ideal path for predicting the morphological heterogeneity of bio-nanoparticles in optical potential trapping-related measurements.

Organelle-specific phase contrast microscopy (OS-PCM) enables facile correlation study of organelles and proteins.

Current methods for studying organelle and protein interactions and correlations depend on multiplex fluorescent labeling, which is experimentally complex and harmful to cells. Here we propose to solve this challenge via OS-PCM, where organelles are imaged and segmented without labels, and combined with standard fluorescence microscopy of protein distributions. In this work, we develop new neural networks to obtain unlabeled organelle, nucleus and membrane predictions from a single 2D image. Automated analysis is also implemented to obtain quantitative information regarding the spatial distribution and co-localization of both protein and organelle, as well as their relationship to the landmark structures of nucleus and membrane. Using mitochondria and DRP1 protein as a proof-of-concept, we conducted a correlation study where only DRP1 is labeled, with results consistent with prior reports utilizing multiplex labeling. Thus our work demonstrates that OS-PCM simplifies the correlation study of organelles and proteins.

Highly Sensitive, Portable Detection System for Multiplex Chemiluminescence Analysis.

Chemiluminescence (CL) has emerged as a critical tool for the sensing and quantification of various bioanalytes in virtually all clinical fields. However, the rapid nature of many CL reactions raises challenges for typical low-cost optical sensors such as cameras to achieve accurate and sensitive detection. Meanwhile, classic sensors such as photomultiplier tubes are highly sensitive but lack spatial multiplexing capabilities and are generally not suited for point-of-care applications outside a standard laboratory setting. To address this issue, in this paper, a miniaturized and versatile silicon-photomultiplier-based fiber-integrated CL device (SFCD) was designed for sensitive multiplex CL detection. The SFCD comprises a silicon photomultiplier array coupled to an array of high numerical aperture plastic optical fibers to achieve 16-plex detection. The optical fibers ensure efficient light collection while allowing the fixed detector to be mated with diverse sample geometries (e.g., circular or grid), simply by adjusting the fiber configuration. In a head-to-head comparison with a lens-based camera system featuring a cooled detector, the SFCD achieved a 14-fold improved limit of detection in both direct and enzyme-mediated CL reactions. The SFCD also features improved compactness and lower cost, as well as faster temporal resolution compared with camera-based systems while preserving spatial multiplexing and good environmental robustness. Thus, the SFCD has excellent potential for point-of-care biosensing applications.

Image reconstruction for low cost spatial light interference microscopy with fixed and arbitrary phase modulation.

During the past decade, spatial light interference microscopy (SLIM) has undergone rapid development, evidenced by its broadening applications in biology and medicine. However, the need for an expensive spatial light modulator (SLM) may limit its adoption, and the requirement for multiple images per plane limits its speed in volumetric imaging. Here we propose to address these issues by replacing the SLM with a mask fabricated from a low cost optical density (OD) filter, and recover high contrast images computationally rather than through phase-shifting. This is done using a specially constructed Wiener filter to recover the object scattering potential. A crucial part of the Wiener filter is estimating the arbitrary phase introduced by the OD filter. Our results demonstrate that not only were we able to estimate the OD filter's phase modulation in situ, but also the contrast of the reconstructed images is greatly improved. Comparisons with other related methods are also performed, with the conclusion that the combination of an inexpensive OD mask and modified Wiener filtering leads to results that are closest to the traditional SLIM setup. Thus, we have demonstrated the feasibility of a low cost, high speed SLIM system utilizing computational phase reconstruction, paving the way for wider adoption of high resolution phase microscopy.

Rapid Intracellular Detection and Analysis of Lipid Droplets' Morpho-Chemical Composition by Phase-Guided Raman Sampling.

Intracellular lipid droplets (LDs) are dynamic, complex organelles involved in nearly all aspects of cellular metabolism. In situ characterization methods are primarily limited to fluorescence imaging, which yields limited chemical information, or Raman spectroscopy, which provides excellent chemical profiling but very low throughput. Here, we propose a new paradigm where locations of both large and small droplets are obtained automatically from high-resolution phase images and fed into a galvomirror-controlled Raman sampling arm to obtain the full spectrum of each LD efficiently. Using this phase-guided Raman sampling, we can characterize hundreds of LDs within a single cell in minutes and easily acquire more than 40,000 high-quality spectra. The data set revealed strong, cell line-dependent, cell-dependent, and individual droplet-dependent composition changes to various culture conditions. In particular, we revealed a strong competitive relationship between mono- and polyunsaturated fatty acids, where supplementation with one led to a relative decrease in the other.

Fast Raman imaging through the combination of context-aware matrix completion and low spectral resolution.

Raman hyperspectral imaging is an effective method for label-free imaging with chemical specificity, yet the weak signals and correspondingly long integration times have hindered its wide adoption as a routine analytical method. Recently, low resolution Raman imaging has been proposed to improve the spectral signal-to-noise ratio, which significantly improves the speed of Raman imaging. In this paper, low resolution Raman spectroscopy is combined with "context-aware" matrix completion, where regions of the sample that are not of interest are skipped, and the regions that are measured are under-sampled, then reconstructed with a low-rank constraint. Both simulations and experiment show that low-resolution Raman boosts the speed and image quality of the computationally-reconstructed Raman images, allowing deeper sub-sampling, reduced exposure time, and an overall >10-fold improvement in imaging speed, without sacrificing chemical specificity or spatial image quality. As the method utilizes traditional point-scan imaging, it retains full confocality and is "backwards-compatible" with pre-existing traditional Raman instruments, broadening the potential scope of Raman imaging applications.

A critical evaluation of compressed line-scan Raman imaging.

The weak signal strength of Raman imaging leads to long imaging times. To increase the speed of Raman imaging, line scanning and compressed Raman imaging methods have been proposed. Here we combine both line scanning and compressed sensing to further increase the speed. However, the direct combination leads to poor reconstruction results due to the missed coverage of the sample. To avoid this issue, "full-coverage" Compressed Line-scan Raman Imaging (FC-CLRI) is proposed, where line positions are random but constrained to measure each line position of the sample at least once. In proof-of-concept studies of polymer beads and yeast cells, FC-CLRI achieved reasonable image quality while making only 20-40% of the measurements of a fully-sampled line-scan image, achieving 640 µm2 FOV imaging in <2 min with 1.5 mW µm-2 laser power. Furthermore, we critically evaluate the CLRI method through comparison with simple downsampling, and have found that FC-CLRI preserves spatial resolution better while naïve downsampling provides an overall higher image quality for complex samples.

Hybrid Principal Component Analysis Denoising Enables Rapid, Label-Free Morpho-Chemical Quantification of Individual Nanoliposomes.

Laser tweezers Raman spectroscopy enables multiplexed, quantitative chemical and morphological analysis of individual bionanoparticles such as drug-loaded nanoliposomes, yet it requires minutes-scale acquisition times per particle, leading to a lack of statistical power in typical small-sized data sets. The long acquisition times present a bottleneck not only in measurement time but also in the analytical throughput, as particle concentration (and thus throughput) must be kept low enough to avoid swarm measurement. The only effective way to improve this situation is to reduce the exposure time, which comes at the expense of increased noise. Here, we present a hybrid principal component analysis (PCA) denoising method, where a small number (∼30 spectra) of high signal-to-noise ratio (SNR) training data construct an effective principal component subspace into which low SNR test data are projected. Simulations and experiments prove the method outperforms traditional denoising methods such as the wavelet transform or traditional PCA. On experimental liposome samples, denoising accelerated data acquisition from 90 to 3 s, with an overall 4.5-fold improvement in particle throughput. The denoised data retained the ability to accurately determine complex morphochemical parameters such as lamellarity of individual nanoliposomes, as confirmed by comparison with cryo-EM imaging. We therefore show that hybrid PCA denoising is an efficient and effective tool for denoising spectral data sets with limited chemical variability and that the RR-NTA technique offers an ideal path for studying the multidimensional heterogeneity of nanoliposomes and other micro/nanoscale bioparticles.

Multicomponent Raman spectral regression using complete and incomplete models and convolutional neural networks.

With the advent of hyperspectral Raman imaging technology, especially the rapid and high-resolution imaging schemes, datasets with thousands to millions of spectra are now commonplace. Standard preprocessing and regression methods such as least squares approaches are time consuming and require input from highly trained operators. Here we propose a solution to this analytic bottleneck through a convolutional neural network trained fully on synthetic data and then applied to experimental measurements, including cases where complete spectral information is missing (i.e. an underdetermined model). An advantage of the model is that it combines background correction and regression into a single step, and does not require user-selected parameters. We compare our results with traditional least squares methods, including the popular asymmetric least squares (AsLS) approach. Our results demonstrate that the proposed CNN model boasts less sensitivity to parameter selection, and with a rapid processing speed, with performance equal to or better than comparison methods. The performance is validated on synthetic spectral mixtures, as well as experimentally measured single-vesicle liposome data.

High-Resolution Low-Power Hyperspectral Line-Scan Imaging of Fast Cellular Dynamics Using Azo-Enhanced Raman Scattering Probes.

Small-molecule Raman probes for cellular imaging have attracted great attention owing to their sharp peaks that are sensitive to environmental changes. The small cross section of molecular Raman scattering limits dynamic cellular Raman imaging to expensive and complex coherent approaches that acquire single-channel images and lose hyperspectral Raman information. We introduce a new method, dynamic azo-enhanced Raman imaging (DAERI), to couple the new class of azo-enhanced Raman probes with a high-speed line-scan Raman imaging system. DAERI achieved high-resolution low-power imaging of fast cellular dynamics resolved at ∼270 nm along the confocal direction, 75 µW/µm2 and 3.5 s/frame. Based on the azo-enhanced Raman probes with characteristic signals 102-104 stronger than classic Raman labels, DAERI was not restricted to the cellular Raman-silent region as in prior work and enabled multiplex visualization of organelle motions and interactions. We anticipate DAERI to be a powerful tool for future studies in biophysics and cell biology.

A Drop-in, Focus-Extending Phase Mask Simplifies Microscopic and Microfluidic Imaging Systems for Cost-Effective Point-of-Care Diagnostics.

Microscopic imaging and imaging flow cytometry have wide potential in point-of-care assays; however, their narrow depth of focus necessitates precise mechanical or fluidic focus control of a sample in order to acquire high-quality images that can be used for downstream analysis, increasing the cost and complexity of the imaging system. This complexity represents a barrier to miniaturization and translation of point-of-care assays based on microscopic imaging or imaging flow cytometry. To address this challenge, we present a simple drop-in phase mask with a physics-informed, circularly symmetric asphere phase profile that extends the depth of focus by >5-fold while largely preserving the image quality compared to other depth extending methods. We show that such a focus-extended system overcomes manufacturing tolerances in low-cost sample chambers, enlarges the useable field-of-view of low-cost objectives, and permits increased throughput and precision in flow imaging systems without the need for complex flow-focusing. As the image quality is preserved without the need for postacquisition image restoration, our solution is also highly appropriate for on-line applications such as cell sorting.

Label-free imaging of intracellular organelle dynamics using flat-fielding quantitative phase contrast microscopy (FF-QPCM).

Panoramic and long-term observation of nanosized organelle dynamics and interactions with high spatiotemporal resolution still hold great challenge for current imaging platforms. In this study, we propose a live-organelle imaging platform, where a flat-fielding quantitative phase contrast microscope (FF-QPCM) visualizes all the membrane-bound subcellular organelles, and an intermittent fluorescence channel assists in specific organelle identification. FF-QPCM features a high spatiotemporal resolution of 245 nm and 250 Hz and strong immunity against external disturbance. Thus, we could investigate several important dynamic processes of intracellular organelles from direct perspectives, including chromosome duplication in mitosis, mitochondrial fusion and fission, filaments, and vesicles' morphologies in apoptosis. Of note, we have captured, for the first time, a new type of mitochondrial fission (entitled mitochondrial disintegration), the generation and fusion process of vesicle-like organelles, as well as the mitochondrial vacuolization during necrosis. All these results bring us new insights into spatiotemporal dynamics and interactions among organelles, and hence aid us in understanding the real behaviors and functional implications of the organelles in cellular activities.

Simultaneous 3D deconvolution and halo removal for spatial light interference microscopy through a two-edge apodized Wiener filter.

As one of the most sensitive quantitative phase microscopy techniques, spatial light interference microscopy (SLIM) has undergone rapid development in the past decade and has seen wide application in both basic science and clinical studies. However, as with any other traditional microscope, the axial resolution is the worst among the three dimensions. This leads to lower contrast in the thicker regions of cell samples. Another common foe in the phase contrast image is the halo artifact, which can block underlying structures, in particular when high resolution is desired. Current solutions focus on either halo removal or contrast enhancement alone, and thus need two processing steps to create both high contrast and halo-free phase images. Further, raw images often suffer from artifacts that are both bright and slowly varying, dubbed here as cloud-like artifacts. After deconvolution, these cloud-like artifacts often dominate the image and obscure high-frequency information, which is typically of greatest interest. In this paper, we first analyzed the unique characteristics of the phase transfer function associated with SLIM to find the root of the cloud-like artifacts and halo artifacts. Then we designed a two-edge apodized deconvolution scheme as a counter measure. We show that even with a simple Wiener filter, the two-edge apodization (TEA) can effectively improve the contrast while suppressing the halo and cloud-like artifacts. Our algorithm, named TEA-Weiner, is non-iterative and thus can be implemented in real time. For low-contrast structures inside the cell such as the endoplasmic reticulum (ER), where ringing artifacts are more likely, we show that two-edge apodization can be combined with additional constraints such as total variation so that their contrast can be enhanced simultaneously with other bright structures inside the cell. Comparing our method with other state-of-the-art algorithms, our method has two advantages: First, deconvolution and halo removal are accomplished simultaneously; second, the image quality is highest using TEA-Weiner filtering.

Epi-illumination dark-field microscopy enables direct visualization of unlabeled small organisms with high spatial and temporal resolution.

Dark-field microscopy is known to offer both high resolution and direct visualization of thin samples. However, its performance and optimization on thick samples is under-explored and so far, only meso-scale information from whole organisms has been demonstrated. In this work, we carefully investigate the difference between trans- and epi-illumination configurations. Our findings suggest that the epi-illumination configuration is superior in both contrast and fidelity compared to trans-illumination, while having the added advantage of experimental simplicity and an "open top" for experimental intervention. Guided by the theoretical analysis, we constructed an epi-illumination dark-field microscope with measured lateral and axial resolutions of 260 nm and 520 nm, respectively. Subcellular structures in whole organisms were directly visualized without the need for image reconstruction, and further confirmed via simultaneous fluorescence imaging. With an imaging speed of 20 to 50 fps, we visualize fast dynamic processes such as cell division and pharyngeal pumping in Caenorhabditis elegans.

Super-resolved Raman imaging via galvo-painted structured line illumination.

Traditional line-scan Raman imaging features a rapid imaging speed while preserving complete spectral information, yet has diffraction-limited resolution. Sinusoidally structured line excitation can yield an improvement in the lateral resolution of the Raman image along the line's direction. However, given the need for the line and spectrometer slit to be aligned, the resolution in the perpendicular direction remains diffraction limited. To overcome this, we present here a galvo-modulated structured line imaging system, where a system of three galvos can arbitrarily orient the structured line on the sample plane, while keeping the beam aligned to the spectrometer slit in the detection plane. Thus, a two-fold isotropic improvement in the lateral resolution fold is possible. We demonstrate the feasibility using mixtures of microspheres as chemical and size standards. The results prove an improvement in the lateral resolution of 1.8-fold (limited by line contrast at higher frequencies), while preserving complete spectral information of the sample.

Organelle-specific phase contrast microscopy enables gentle monitoring and analysis of mitochondrial network dynamics.

Mitochondria are delicate organelles that play a key role in cell fate. Current research methods rely on fluorescence labeling that introduces stress due to photobleaching and phototoxicity. Here we propose a new, gentle method to study mitochondrial dynamics, where organelle-specific three-dimensional information is obtained in a label-free manner at high resolution, high specificity, and without detrimental effects associated with staining. A mitochondria cleavage experiment demonstrates that not only do the label-free mitochondria-specific images have the required resolution and precision, but also fairly include all cells and mitochondria in downstream morphological analysis, while fluorescence images omit dim cells and mitochondria. The robustness of the method was tested on samples of different cell lines and on data collected from multiple systems. Thus, we have demonstrated that our method is an attractive alternative to study mitochondrial dynamics, connecting behavior and function in a simpler and more robust way than traditional fluorescence imaging.

A sample-preparation-free, automated, sample-to-answer system for cell counting in human body fluids.

While many clinical laboratory tests are now highly automated, body fluid cell counting, particularly in low-cellularity samples such as cerebral spinal fluid (CSF), is often performed manually. Here, we report a simple, cost-effective method to obtain white and red blood cell counts from human body fluids such as CSF. The method consists of a compact, automated, and low-cost fluorescence microscope system, coupled to a sample chamber containing all of the necessary reagents in dry form to stain and prepare the sample. Sample focus and scanning are handled automatically, and the acquired multimodal images are automatically analyzed to extract cell counts. Comparison with manual counting on over 200 clinical samples shows excellent agreement. As the system counts a substantially larger image region than a standard manual cell count, we find our sensitivity to extremely low cellularity samples to potentially be higher than the manual gold standard, evidenced by our system recording images of cells in samples whose cell count was registered as "0" by a trained user. Thus, our system holds promise for routine, automated, and sensitive analysis of body fluids whose cellularity extends across a wide dynamic range.

Fast confocal Raman imaging via context-aware compressive sensing.

Raman hyperspectral imaging is a powerful method to obtain detailed chemical information about a wide variety of organic and inorganic samples noninvasively and without labels. However, due to the weak, nonresonant nature of spontaneous Raman scattering, acquiring a Raman imaging dataset is time-consuming and inefficient. In this paper we utilize a compressive imaging strategy coupled with a context-aware image prior to improve Raman imaging speed by 5- to 10-fold compared to classic point-scanning Raman imaging, while maintaining the traditional benefits of point scanning imaging, such as isotropic resolution and confocality. With faster data acquisition, large datasets can be acquired in reasonable timescales, leading to more reliable downstream analysis. On standard samples, context-aware Raman compressive imaging (CARCI) was able to reduce the number of measurements by ∼85% while maintaining high image quality (SSIM >0.85). Using CARCI, we obtained a large dataset of chemical images of fission yeast cells, showing that by collecting 5-fold more cells in a given experiment time, we were able to get more accurate chemical images, identification of rare cells, and improved biochemical modeling. For example, applying VCA to nearly 100 cells' data together, cellular organelles were resolved that were not faithfully reconstructed by a single cell's dataset.

Nanometer precise red blood cell sizing using a cost-effective quantitative dark field imaging system.

Because of the bulk, complexity, calibration requirements, and need for operator training, most current flow-based blood counting devices are not appropriate for field use. Standard imaging methods could be much more compact, inexpensive, and with minimal calibration requirements. However, due to the diffraction limit, imaging lacks the nanometer precision required to measure red blood cell volumes. To address this challenge, we utilize Mie scattering, which can measure nanometer-scale morphological information from cells, in a dark-field imaging geometry. The approach consists of a custom-built dark-field scattering microscope with symmetrically oblique illumination at a precisely defined angle to record wide-field images of diluted and sphered blood samples. Scattering intensities of each cell under three wavelengths are obtained by segmenting images via digital image processing. These scattering intensities are then used to determine size and hemoglobin information via Mie theory and machine learning. Validation on 90 clinical blood samples confirmed the ability to obtain mean corpuscular volume (MCV), mean corpuscular hemoglobin concentration (MCHC), and red cell distribution width (RDW) with high accuracy. Simulations based on historical data suggest that an instrument with the accuracy achieved in this study could be used for widespread anemia screening.

Optical volumetric projection with large NA objectives for fast high-resolution 3D imaging of neural signals.

One critical challenge in studying neural circuits of freely behaving model organisms is to record neural signals distributed within the whole brain, yet simultaneously maintaining cellular resolution. However, due to the dense packing of neuron cells in animal brains, high numerical aperture (NA) objectives are often required to differentiate neighboring neurons with the consequent need for axial scanning for whole brain imaging. Extending the depth of focus (EDoF) will be beneficial for fast 3D imaging of those neurons. However, current EDoF-enabled microscopes are primarily based on objectives with small NAs (≤0.3 ) such that the paraxial approximation can be applied. In this paper, we started from a nonparaxial approximation of the defocus aberration and derived a new phase mask that was appropriate for large NA microscopic systems. We validated the performance experimentally with a spatial light modulator (SLM) to create the designed phase mask. The performance was tested on different samples such as multilayered fluorescence beads and thick brain tissues, as well as with different objectives. Results confirmed that our design has extended the depth of focus about 10 fold and the image quality is much higher than those based on the most common EDoF method, the cubic phase method, popularly used to generate Airy beams. Meanwhile, our phase mask is rotationally symmetric and easy to fabricate. We fabricated one such phase plate and tested it on the pan-neuronal labeled Caenorhabditis elegans (C.elegans). The imaging performance demonstrated that we can capture all neurons in the whole brain with one snapshot and with cellular resolution, while the imaging speed is increased about 3 fold compared to the system using SLM. Thus we have shown that our method can not only provide the required imaging speed and resolution for studying neural activities in model animals, but also can be implemented as a low-cost, add-on module that can immediately augment existing fluorescence microscopes with only minor system modifications, and yielding substantially higher photon efficiency than SLM-based methods.