Deep learning-based size prediction for optical trapped nanoparticles and extracellular vesicles from limited bandwidth camera detection.

Due to its ability to record position, intensity, and intensity distribution information, camera-based monitoring of nanoparticles in optical traps can enable multi-parametric morpho-optical characterization at the single-particle level. However, blurring due to the relatively long (10s of microsecond) integration times and aliasing from the resulting limited temporal bandwidth affect the detected particle position when considering nanoparticles in traps with strong stiffness, leading to inaccurate size predictions. Here, we propose a ResNet-based method for accurate size characterization of trapped nanoparticles, which is trained by considering only simulated time series data of nanoparticles' constrained Brownian motion. Experiments prove the method outperforms state-of-art sizing algorithms such as adjusted Lorentzian fitting or CNN-based networks on both standard nanoparticles and extracellular vesicles (EVs), as well as maintains good accuracy even when measurement times are relatively short (<1s per particle). On samples of clinical EVs, our network demonstrates a well-generalized ability to accurately determine the EV size distribution, as confirmed by comparison with gold-standard nanoparticle tracking analysis (NTA). Furthermore, by combining the sizing network with still frame images from high-speed video, the camera-based optical tweezers have the unique capacity to quantify both the size and refractive index of bio-nanoparticles at the single-particle level. These experiments prove the proposed sizing network as an ideal path for predicting the morphological heterogeneity of bio-nanoparticles in optical potential trapping-related measurements.

Organelle-specific phase contrast microscopy (OS-PCM) enables facile correlation study of organelles and proteins.

Current methods for studying organelle and protein interactions and correlations depend on multiplex fluorescent labeling, which is experimentally complex and harmful to cells. Here we propose to solve this challenge via OS-PCM, where organelles are imaged and segmented without labels, and combined with standard fluorescence microscopy of protein distributions. In this work, we develop new neural networks to obtain unlabeled organelle, nucleus and membrane predictions from a single 2D image. Automated analysis is also implemented to obtain quantitative information regarding the spatial distribution and co-localization of both protein and organelle, as well as their relationship to the landmark structures of nucleus and membrane. Using mitochondria and DRP1 protein as a proof-of-concept, we conducted a correlation study where only DRP1 is labeled, with results consistent with prior reports utilizing multiplex labeling. Thus our work demonstrates that OS-PCM simplifies the correlation study of organelles and proteins.

Highly Sensitive, Portable Detection System for Multiplex Chemiluminescence Analysis.

Chemiluminescence (CL) has emerged as a critical tool for the sensing and quantification of various bioanalytes in virtually all clinical fields. However, the rapid nature of many CL reactions raises challenges for typical low-cost optical sensors such as cameras to achieve accurate and sensitive detection. Meanwhile, classic sensors such as photomultiplier tubes are highly sensitive but lack spatial multiplexing capabilities and are generally not suited for point-of-care applications outside a standard laboratory setting. To address this issue, in this paper, a miniaturized and versatile silicon-photomultiplier-based fiber-integrated CL device (SFCD) was designed for sensitive multiplex CL detection. The SFCD comprises a silicon photomultiplier array coupled to an array of high numerical aperture plastic optical fibers to achieve 16-plex detection. The optical fibers ensure efficient light collection while allowing the fixed detector to be mated with diverse sample geometries (e.g., circular or grid), simply by adjusting the fiber configuration. In a head-to-head comparison with a lens-based camera system featuring a cooled detector, the SFCD achieved a 14-fold improved limit of detection in both direct and enzyme-mediated CL reactions. The SFCD also features improved compactness and lower cost, as well as faster temporal resolution compared with camera-based systems while preserving spatial multiplexing and good environmental robustness. Thus, the SFCD has excellent potential for point-of-care biosensing applications.

Image reconstruction for low cost spatial light interference microscopy with fixed and arbitrary phase modulation.

During the past decade, spatial light interference microscopy (SLIM) has undergone rapid development, evidenced by its broadening applications in biology and medicine. However, the need for an expensive spatial light modulator (SLM) may limit its adoption, and the requirement for multiple images per plane limits its speed in volumetric imaging. Here we propose to address these issues by replacing the SLM with a mask fabricated from a low cost optical density (OD) filter, and recover high contrast images computationally rather than through phase-shifting. This is done using a specially constructed Wiener filter to recover the object scattering potential. A crucial part of the Wiener filter is estimating the arbitrary phase introduced by the OD filter. Our results demonstrate that not only were we able to estimate the OD filter's phase modulation in situ, but also the contrast of the reconstructed images is greatly improved. Comparisons with other related methods are also performed, with the conclusion that the combination of an inexpensive OD mask and modified Wiener filtering leads to results that are closest to the traditional SLIM setup. Thus, we have demonstrated the feasibility of a low cost, high speed SLIM system utilizing computational phase reconstruction, paving the way for wider adoption of high resolution phase microscopy.

Fast Raman imaging through the combination of context-aware matrix completion and low spectral resolution.

Raman hyperspectral imaging is an effective method for label-free imaging with chemical specificity, yet the weak signals and correspondingly long integration times have hindered its wide adoption as a routine analytical method. Recently, low resolution Raman imaging has been proposed to improve the spectral signal-to-noise ratio, which significantly improves the speed of Raman imaging. In this paper, low resolution Raman spectroscopy is combined with "context-aware" matrix completion, where regions of the sample that are not of interest are skipped, and the regions that are measured are under-sampled, then reconstructed with a low-rank constraint. Both simulations and experiment show that low-resolution Raman boosts the speed and image quality of the computationally-reconstructed Raman images, allowing deeper sub-sampling, reduced exposure time, and an overall >10-fold improvement in imaging speed, without sacrificing chemical specificity or spatial image quality. As the method utilizes traditional point-scan imaging, it retains full confocality and is "backwards-compatible" with pre-existing traditional Raman instruments, broadening the potential scope of Raman imaging applications.

Rapid Intracellular Detection and Analysis of Lipid Droplets' Morpho-Chemical Composition by Phase-Guided Raman Sampling.

Intracellular lipid droplets (LDs) are dynamic, complex organelles involved in nearly all aspects of cellular metabolism. In situ characterization methods are primarily limited to fluorescence imaging, which yields limited chemical information, or Raman spectroscopy, which provides excellent chemical profiling but very low throughput. Here, we propose a new paradigm where locations of both large and small droplets are obtained automatically from high-resolution phase images and fed into a galvomirror-controlled Raman sampling arm to obtain the full spectrum of each LD efficiently. Using this phase-guided Raman sampling, we can characterize hundreds of LDs within a single cell in minutes and easily acquire more than 40,000 high-quality spectra. The data set revealed strong, cell line-dependent, cell-dependent, and individual droplet-dependent composition changes to various culture conditions. In particular, we revealed a strong competitive relationship between mono- and polyunsaturated fatty acids, where supplementation with one led to a relative decrease in the other.

A critical evaluation of compressed line-scan Raman imaging.

The weak signal strength of Raman imaging leads to long imaging times. To increase the speed of Raman imaging, line scanning and compressed Raman imaging methods have been proposed. Here we combine both line scanning and compressed sensing to further increase the speed. However, the direct combination leads to poor reconstruction results due to the missed coverage of the sample. To avoid this issue, "full-coverage" Compressed Line-scan Raman Imaging (FC-CLRI) is proposed, where line positions are random but constrained to measure each line position of the sample at least once. In proof-of-concept studies of polymer beads and yeast cells, FC-CLRI achieved reasonable image quality while making only 20-40% of the measurements of a fully-sampled line-scan image, achieving 640 µm2 FOV imaging in <2 min with 1.5 mW µm-2 laser power. Furthermore, we critically evaluate the CLRI method through comparison with simple downsampling, and have found that FC-CLRI preserves spatial resolution better while naïve downsampling provides an overall higher image quality for complex samples.

Hybrid Principal Component Analysis Denoising Enables Rapid, Label-Free Morpho-Chemical Quantification of Individual Nanoliposomes.

Laser tweezers Raman spectroscopy enables multiplexed, quantitative chemical and morphological analysis of individual bionanoparticles such as drug-loaded nanoliposomes, yet it requires minutes-scale acquisition times per particle, leading to a lack of statistical power in typical small-sized data sets. The long acquisition times present a bottleneck not only in measurement time but also in the analytical throughput, as particle concentration (and thus throughput) must be kept low enough to avoid swarm measurement. The only effective way to improve this situation is to reduce the exposure time, which comes at the expense of increased noise. Here, we present a hybrid principal component analysis (PCA) denoising method, where a small number (∼30 spectra) of high signal-to-noise ratio (SNR) training data construct an effective principal component subspace into which low SNR test data are projected. Simulations and experiments prove the method outperforms traditional denoising methods such as the wavelet transform or traditional PCA. On experimental liposome samples, denoising accelerated data acquisition from 90 to 3 s, with an overall 4.5-fold improvement in particle throughput. The denoised data retained the ability to accurately determine complex morphochemical parameters such as lamellarity of individual nanoliposomes, as confirmed by comparison with cryo-EM imaging. We therefore show that hybrid PCA denoising is an efficient and effective tool for denoising spectral data sets with limited chemical variability and that the RR-NTA technique offers an ideal path for studying the multidimensional heterogeneity of nanoliposomes and other micro/nanoscale bioparticles.

Multicomponent Raman spectral regression using complete and incomplete models and convolutional neural networks.

With the advent of hyperspectral Raman imaging technology, especially the rapid and high-resolution imaging schemes, datasets with thousands to millions of spectra are now commonplace. Standard preprocessing and regression methods such as least squares approaches are time consuming and require input from highly trained operators. Here we propose a solution to this analytic bottleneck through a convolutional neural network trained fully on synthetic data and then applied to experimental measurements, including cases where complete spectral information is missing (i.e. an underdetermined model). An advantage of the model is that it combines background correction and regression into a single step, and does not require user-selected parameters. We compare our results with traditional least squares methods, including the popular asymmetric least squares (AsLS) approach. Our results demonstrate that the proposed CNN model boasts less sensitivity to parameter selection, and with a rapid processing speed, with performance equal to or better than comparison methods. The performance is validated on synthetic spectral mixtures, as well as experimentally measured single-vesicle liposome data.

High-Resolution Low-Power Hyperspectral Line-Scan Imaging of Fast Cellular Dynamics Using Azo-Enhanced Raman Scattering Probes.

Small-molecule Raman probes for cellular imaging have attracted great attention owing to their sharp peaks that are sensitive to environmental changes. The small cross section of molecular Raman scattering limits dynamic cellular Raman imaging to expensive and complex coherent approaches that acquire single-channel images and lose hyperspectral Raman information. We introduce a new method, dynamic azo-enhanced Raman imaging (DAERI), to couple the new class of azo-enhanced Raman probes with a high-speed line-scan Raman imaging system. DAERI achieved high-resolution low-power imaging of fast cellular dynamics resolved at ∼270 nm along the confocal direction, 75 µW/µm2 and 3.5 s/frame. Based on the azo-enhanced Raman probes with characteristic signals 102-104 stronger than classic Raman labels, DAERI was not restricted to the cellular Raman-silent region as in prior work and enabled multiplex visualization of organelle motions and interactions. We anticipate DAERI to be a powerful tool for future studies in biophysics and cell biology.

A Drop-in, Focus-Extending Phase Mask Simplifies Microscopic and Microfluidic Imaging Systems for Cost-Effective Point-of-Care Diagnostics.

Microscopic imaging and imaging flow cytometry have wide potential in point-of-care assays; however, their narrow depth of focus necessitates precise mechanical or fluidic focus control of a sample in order to acquire high-quality images that can be used for downstream analysis, increasing the cost and complexity of the imaging system. This complexity represents a barrier to miniaturization and translation of point-of-care assays based on microscopic imaging or imaging flow cytometry. To address this challenge, we present a simple drop-in phase mask with a physics-informed, circularly symmetric asphere phase profile that extends the depth of focus by >5-fold while largely preserving the image quality compared to other depth extending methods. We show that such a focus-extended system overcomes manufacturing tolerances in low-cost sample chambers, enlarges the useable field-of-view of low-cost objectives, and permits increased throughput and precision in flow imaging systems without the need for complex flow-focusing. As the image quality is preserved without the need for postacquisition image restoration, our solution is also highly appropriate for on-line applications such as cell sorting.

Microfluidic Paper-Based Preconcentration and Retrieval for Rapid Ribonucleic Acid Biomarker Detection and Visualization.

Microfluidic paper-based analytical devices (µPADs) have attracted significant attention in the field of point-of-care (POC) diagnostics. However, the heterogeneous structure of the paper often impairs the limit of detection (LOD) for low-abundance targets when those targets are directly analyzed. One viable solution to bypass this limitation is to elevate the target concentration above the LOD on-site to reach a valid readout. Here, we developed a 3D µPADs preconcentrator (3D-µP2) to increase sample concentration by electrokinetic trapping and demonstrated its application in increasing the LOD of a downstream colorimetric assay. The three-dimensional (3D) structure of this device was composed of a loading pad, a vertical fluid path formed by stacked absorbent pads, and an ion-selective membrane of PEDOT:PSS. This novel design facilitates fast preconcentration, high capacity in sample processing, and easy target retrieval. The concentration of an exemplary target, a single-stranded DNA sequence, was increased up to 170-fold within 80 s. The LOD of the colorimetric assay to verify the DNA target was increased 3 orders of magnitude with a preconcentrated sample compared to the control. The device and its analysis equipment used in this study were all cheap and portable. Thus, the 3D-µP2 can be a powerful POC tool for sample pretreatment in resource-limited areas.

Label-free imaging of intracellular organelle dynamics using flat-fielding quantitative phase contrast microscopy (FF-QPCM).

Panoramic and long-term observation of nanosized organelle dynamics and interactions with high spatiotemporal resolution still hold great challenge for current imaging platforms. In this study, we propose a live-organelle imaging platform, where a flat-fielding quantitative phase contrast microscope (FF-QPCM) visualizes all the membrane-bound subcellular organelles, and an intermittent fluorescence channel assists in specific organelle identification. FF-QPCM features a high spatiotemporal resolution of 245 nm and 250 Hz and strong immunity against external disturbance. Thus, we could investigate several important dynamic processes of intracellular organelles from direct perspectives, including chromosome duplication in mitosis, mitochondrial fusion and fission, filaments, and vesicles' morphologies in apoptosis. Of note, we have captured, for the first time, a new type of mitochondrial fission (entitled mitochondrial disintegration), the generation and fusion process of vesicle-like organelles, as well as the mitochondrial vacuolization during necrosis. All these results bring us new insights into spatiotemporal dynamics and interactions among organelles, and hence aid us in understanding the real behaviors and functional implications of the organelles in cellular activities.

Simultaneous 3D deconvolution and halo removal for spatial light interference microscopy through a two-edge apodized Wiener filter.

As one of the most sensitive quantitative phase microscopy techniques, spatial light interference microscopy (SLIM) has undergone rapid development in the past decade and has seen wide application in both basic science and clinical studies. However, as with any other traditional microscope, the axial resolution is the worst among the three dimensions. This leads to lower contrast in the thicker regions of cell samples. Another common foe in the phase contrast image is the halo artifact, which can block underlying structures, in particular when high resolution is desired. Current solutions focus on either halo removal or contrast enhancement alone, and thus need two processing steps to create both high contrast and halo-free phase images. Further, raw images often suffer from artifacts that are both bright and slowly varying, dubbed here as cloud-like artifacts. After deconvolution, these cloud-like artifacts often dominate the image and obscure high-frequency information, which is typically of greatest interest. In this paper, we first analyzed the unique characteristics of the phase transfer function associated with SLIM to find the root of the cloud-like artifacts and halo artifacts. Then we designed a two-edge apodized deconvolution scheme as a counter measure. We show that even with a simple Wiener filter, the two-edge apodization (TEA) can effectively improve the contrast while suppressing the halo and cloud-like artifacts. Our algorithm, named TEA-Weiner, is non-iterative and thus can be implemented in real time. For low-contrast structures inside the cell such as the endoplasmic reticulum (ER), where ringing artifacts are more likely, we show that two-edge apodization can be combined with additional constraints such as total variation so that their contrast can be enhanced simultaneously with other bright structures inside the cell. Comparing our method with other state-of-the-art algorithms, our method has two advantages: First, deconvolution and halo removal are accomplished simultaneously; second, the image quality is highest using TEA-Weiner filtering.

Epi-illumination dark-field microscopy enables direct visualization of unlabeled small organisms with high spatial and temporal resolution.

Dark-field microscopy is known to offer both high resolution and direct visualization of thin samples. However, its performance and optimization on thick samples is under-explored and so far, only meso-scale information from whole organisms has been demonstrated. In this work, we carefully investigate the difference between trans- and epi-illumination configurations. Our findings suggest that the epi-illumination configuration is superior in both contrast and fidelity compared to trans-illumination, while having the added advantage of experimental simplicity and an "open top" for experimental intervention. Guided by the theoretical analysis, we constructed an epi-illumination dark-field microscope with measured lateral and axial resolutions of 260 nm and 520 nm, respectively. Subcellular structures in whole organisms were directly visualized without the need for image reconstruction, and further confirmed via simultaneous fluorescence imaging. With an imaging speed of 20 to 50 fps, we visualize fast dynamic processes such as cell division and pharyngeal pumping in Caenorhabditis elegans.

Super-resolved Raman imaging via galvo-painted structured line illumination.

Traditional line-scan Raman imaging features a rapid imaging speed while preserving complete spectral information, yet has diffraction-limited resolution. Sinusoidally structured line excitation can yield an improvement in the lateral resolution of the Raman image along the line's direction. However, given the need for the line and spectrometer slit to be aligned, the resolution in the perpendicular direction remains diffraction limited. To overcome this, we present here a galvo-modulated structured line imaging system, where a system of three galvos can arbitrarily orient the structured line on the sample plane, while keeping the beam aligned to the spectrometer slit in the detection plane. Thus, a two-fold isotropic improvement in the lateral resolution fold is possible. We demonstrate the feasibility using mixtures of microspheres as chemical and size standards. The results prove an improvement in the lateral resolution of 1.8-fold (limited by line contrast at higher frequencies), while preserving complete spectral information of the sample.

Vibrational Fingerprint Analysis of an Azo-based Resonance Raman Scattering Probe for Imaging Proton Distribution in Cellular Lysosomes.

Due to the fundamental mechanism of vibrational state transitions for chemical bonds, the spectra of Raman scattering are narrow-banded and photostable signals capable of probing specific reactions. In the case of protonation/deprotonation reactions, certain chemical bonds are broken and new bonds are formed. Based on the changes of the vibrational modes for the corresponding bonds, fingerprint analysis of multiple Raman bands may allow for the in situ visualization of proton distribution in live cells. However, Raman scattering faces the well-known challenge of low sensitivity. To perform the vibrational fingerprint analysis of Raman scattering by overcoming this challenge, we developed an azo-based resonance Raman pH probe. It was an azobenzene-featured small molecule responsive to protons with the inherent Raman signal ∼104-fold more intense than that of the conventional alkyne-type Raman reporter 5-ethynyl-2'-deoxyuridine. Through the substitution of the electron-donating and -withdrawing entities to the azobenzene group, the effect of resonance Raman scattering and fluorescence quenching was obtained. This effect resulted in a significant Raman enhancement factor of ∼103 compared to the counterpart molecules without the molecular design. Based on the enhanced Raman sensitivity of the azo-based resonance Raman pH probe, the identification of vibrational fingerprint changes at the azo group was achieved during the protonation/deprotonation reactions, and the vibrational fingerprint analysis resolved a pH difference of less than 0.2 unit. The method enabled sensitive hyperspectral cell imaging that clearly visualized the change of proton distribution in autophagic cells.

Organelle-specific phase contrast microscopy enables gentle monitoring and analysis of mitochondrial network dynamics.

Mitochondria are delicate organelles that play a key role in cell fate. Current research methods rely on fluorescence labeling that introduces stress due to photobleaching and phototoxicity. Here we propose a new, gentle method to study mitochondrial dynamics, where organelle-specific three-dimensional information is obtained in a label-free manner at high resolution, high specificity, and without detrimental effects associated with staining. A mitochondria cleavage experiment demonstrates that not only do the label-free mitochondria-specific images have the required resolution and precision, but also fairly include all cells and mitochondria in downstream morphological analysis, while fluorescence images omit dim cells and mitochondria. The robustness of the method was tested on samples of different cell lines and on data collected from multiple systems. Thus, we have demonstrated that our method is an attractive alternative to study mitochondrial dynamics, connecting behavior and function in a simpler and more robust way than traditional fluorescence imaging.

Azo-Enhanced Raman Scattering for Enhancing the Sensitivity and Tuning the Frequency of Molecular Vibrations.

Raman scattering provides stable narrow-banded signals that potentially allow for multicolor microscopic imaging. The major obstacle for the applications of Raman spectroscopy and microscopy is the small cross section of Raman scattering that results in low sensitivity. Here, we report a new concept of azo-enhanced Raman scattering (AERS) by designing the intrinsic molecular structures using resonance Raman and concomitant fluorescence quenching strategies. Based on the selection of vibrational modes and the enhancing unit of azobenzenes, we obtained a library of AERS molecules with specific Raman signals in the fingerprint and silent frequency regions. The spectral characterization and molecular simulation revealed that the azobenzene unit conjugated to the vibrational modes significantly enhanced Raman signals due to the mechanism of extending the conjugation system, coupling the electronic-vibrational transitions, and improving the symmetry of vibrational modes. The nonradiative decay of azobenzene from the excited state quenched the commitment fluorescence, thus providing a clean background for identifying Raman scattering. The most sensitive AERS molecules produced Raman signals of more than 4 orders of magnitude compared to 5-ethynyl-2'-deoxyuridine (EdU). In addition, a frequency tunability of 10 distinct Raman bands was achieved by selecting different types of vibrational modes. This methodology of AERS allows for designing small-molecule Raman probes to visualize various entities in complex systems by multicolor spontaneous Raman imaging. It will open new prospects to explore innovative applications of AERS in interdisciplinary research fields.

A sample-preparation-free, automated, sample-to-answer system for cell counting in human body fluids.

While many clinical laboratory tests are now highly automated, body fluid cell counting, particularly in low-cellularity samples such as cerebral spinal fluid (CSF), is often performed manually. Here, we report a simple, cost-effective method to obtain white and red blood cell counts from human body fluids such as CSF. The method consists of a compact, automated, and low-cost fluorescence microscope system, coupled to a sample chamber containing all of the necessary reagents in dry form to stain and prepare the sample. Sample focus and scanning are handled automatically, and the acquired multimodal images are automatically analyzed to extract cell counts. Comparison with manual counting on over 200 clinical samples shows excellent agreement. As the system counts a substantially larger image region than a standard manual cell count, we find our sensitivity to extremely low cellularity samples to potentially be higher than the manual gold standard, evidenced by our system recording images of cells in samples whose cell count was registered as "0" by a trained user. Thus, our system holds promise for routine, automated, and sensitive analysis of body fluids whose cellularity extends across a wide dynamic range.