Fly-scan high-throughput coded ptychographic microscopy via active micro-vibration and rolling-shutter distortion correction.

Recent advancements in ptychography have demonstrated the potential of coded ptychography (CP) for high-resolution optical imaging in a lensless configuration. However, CP suffers imaging throughput limitations due to scanning inefficiencies. To address this, we propose what we believe is a novel 'fly-scan' scanning strategy utilizing two eccentric rotating mass (ERM) vibration motors for high-throughput coded ptychographic microscopy. The intrinsic continuity of the 'fly-scan' technique effectively eliminates the scanning overhead typically encountered during data acquisition. Additionally, its randomized scanning trajectory considerably reduces periodic artifacts in image reconstruction. We also developed what we believe to be a novel rolling-shutter distortion correction algorithm to fix the rolling-shutter effects. We built up a low-cost, DIY-made prototype platform and validated our approach with various samples including a resolution target, a quantitative phase target, a thick potato sample and biospecimens. The reported platform may offer a cost-effective and turnkey solution for high-throughput bio-imaging.

Digital-scRRBS: A Cost-Effective, Highly Sensitive, and Automated Single-Cell Methylome Analysis Platform via Digital Microfluidics.

Single-cell DNA methylation sequencing is highly effective for identifying cell subpopulations and constructing epigenetic regulatory networks. Existing methylome analyses require extensive starting materials and are costly, complex, and susceptible to contamination, thereby impeding the development of single-cell methylome technology. In this work, we report digital microfluidics-based single-cell reduced representation bisulfite sequencing (digital-scRRBS), the first microfluidics-based single-cell methylome library construction platform, which is an automatic, effective, reproducible, and reagent-efficient technique to dissect the single-cell methylome. Using our digital microfluidic chip, we isolated single cells in 15 s and successfully constructed single-cell methylation sequencing libraries with a unique genome mapping rate of up to 53.6%, covering up to 2.26 million CpG sites. Digital-scRRBS demonstrates a high capacity for distinguishing cell identity and tracking DNA methylation during drug administration. Digital-scRRBS expands the applicability of single-cell methylation methods as a versatile tool for epigenetic analysis of rare cells and populations with high levels of heterogeneity.

Structured modulation multi-height microscopy for high-resolution imaging.

Conventional multi-height microscopy techniques introduce different object-to-detector distances to obtain multiple measurements for phase retrieval. However, surpassing the diffraction limit imposed by the numerical aperture (NA) of the objective lens remains a challenging task. Here, we report a novel structured modulation multi-height microscopy technique for quantitative high-resolution imaging. In our platform, a thin diffuser is placed in between the sample and the objective lens. By translating the diffuser to different axial positions, a sequence of modulated intensity images is captured for reconstruction. The otherwise inaccessible high-resolution object information can thus be encoded into the optical system for detection. In the construction process, we report a ptychographic phase retrieval algorithm to recover the existing wavefront of the complex object. We validate our approach using a resolution target, a phase target, and various biological samples. We demonstrate a ∼4-fold resolution gain over the diffraction limit. We also demonstrate our approach to achieve a 6.5 mm by 4.3 mm field of view and a half-pitch resolution of 1.2 µm. The reported methodology provides a portable, turnkey solution for quantitative high-resolution imaging with potential applications in disease diagnosis, sample screening, and other fields.

Remote referencing strategy for high-resolution coded ptychographic imaging.

The applications of conventional ptychography are limited by its relatively low resolution and throughput in the visible light regime. The new development of coded ptychography (CP) has addressed these issues and achieved the highest numerical aperture for large-area optical imaging in a lensless configuration. A high-quality reconstruction of CP relies on precise tracking of the coded sensor's positional shifts. The coded layer on the sensor, however, prevents the use of cross correlation analysis for motion tracking. Here we derive and analyze the motion tracking model of CP. A novel, to the best of our knowledge, remote referencing scheme and its subsequent refinement pipeline are developed for blind image acquisition. By using this approach, we can suppress the correlation peak caused by the coded surface and recover the positional shifts with deep sub-pixel accuracy. In contrast with common positional refinement methods, the reported approach can be disentangled from the iterative phase retrieval process and is computationally efficient. It allows blind image acquisition without motion feedback from the scanning process. It also provides a robust and reliable solution for implementing ptychography with high imaging throughput. We validate this approach by performing high-resolution whole slide imaging of bio-specimens.

Deep distributed optimization for blind diffuser-modulation ptychography.

Blind diffuser-modulation ptychography has emerged as a low-cost technique for micro-nano holographic imaging, which enables breaking the resolution limit of optical systems. However, the existing reconstruction method requires thousands of measurements to recover object and diffuser profile simultaneously, which makes the data acquisition time-consuming and cumbersome. In this Letter, we report a novel, to the best of our knowledge, blind ptychography technique with deep distributed optimization, termed BPD2O. It decomposes the complicated optimization task into subproblems, then introduces extended ptychographical iterative engine and enhanced network solver to optimize each in a distributed strategy. In this way, BPD2O combines the advantages of both model-driven and data-driven strategies, realizing high-fidelity robust ptychography imaging. Extensive experiments validate that BPD2O can realize better resolution and lead to a reduction of more than one order of magnitude in the number of measurements.

Initial CT-based radiomics nomogram for predicting in-hospital mortality in patients with traumatic brain injury: a multicenter development and validation study.

OBJECTIVE: To develop and validate a radiomic prediction model using initial noncontrast computed tomography (CT) at admission to predict in-hospital mortality in patients with traumatic brain injury (TBI). METHODS: A total of 379 TBI patients from three cohorts were categorized into training, internal validation, and external validation sets. After filtering the unstable features with the minimum redundancy maximum relevance approach, the CT-based radiomics signature was selected by using the least absolute shrinkage and selection operator (LASSO) approach. A personalized predictive nomogram incorporating the radiomic signature and clinical features was developed using a multivariate logistic model to predict in-hospital mortality in patients with TBI. The calibration, discrimination, and clinical usefulness of the radiomics signature and nomogram were evaluated. RESULTS: The radiomic signature consisting of 12 features had areas under the curve (AUCs) of 0.734, 0.716, and 0.706 in the prediction of in-hospital mortality in the internal and two external validation cohorts. The personalized predictive nomogram integrating the radiomic and clinical features demonstrated significant calibration and discrimination with AUCs of 0.843, 0.811, and 0.834 in the internal and two external validation cohorts. Based on decision curve analysis (DCA), both the radiomic features and nomogram were found to be clinically significant and useful. CONCLUSION: This predictive nomogram incorporating the CT-based radiomic signature and clinical features had maximum accuracy and played an optimized role in the early prediction of in-hospital mortality. The results of this study provide vital insights for the early warning of death in TBI patients.

Ptychography-based high-throughput lensless on-chip microscopy via incremental proximal algorithms.

Ptychography-based lensless on-chip microscopy enables high-throughput imaging by retrieving the missing phase information from intensity measurements. Numerous reconstruction algorithms for ptychography have been proposed, yet only a few incremental algorithms can be extended to lensless on-chip microscopy because of large-scale datasets but limited computational efficiency. In this paper, we propose the use of accelerated proximal gradient methods for blind ptychographic phase retrieval in lensless on-chip microscopy. Incremental gradient approaches are adopted in the reconstruction routine. Our algorithms divide the phase retrieval problem into sub-problems involving the evaluation of proximal operator, stochastic gradient descent, and Wirtinger derivatives. We benchmark the performances of accelerated proximal gradient, extended ptychographic iterative engine, and alternating direction method of multipliers, and discuss their convergence and accuracy in both noisy and noiseless cases. We also validate our algorithms using experimental datasets, where full field of view measurements are captured to recover the high-resolution complex samples. Among these algorithms, accelerated proximal gradient presents the overall best performance regarding accuracy and convergence rate. The proposed methods may find applications in ptychographic reconstruction, especially for cases where a wide field of view and high resolution are desired at the same time.

Mask-modulated lensless imaging via translated structured illumination.

Lensless microscopy technique enables high-resolution image recovery over a large field of view. By integrating the concept of phase retrieval, it can also retrieve the lost phase information from intensity-only measurements. Here we report a mask-modulated lensless imaging platform based on translated structured illumination. In the reported platform, we sandwich the object in-between a coded mask and a naked image sensor for lensless data acquisition. An LED array is used to provide angle-varied illumination for projecting a translated structured pattern without involving mechanical scanning. For different LED elements, we acquire the lensless intensity data for recovering the complex-valued object. In the reconstruction process, we employ the regularized ptychographic iterative engine and implement an up-sampling process in the reciprocal space. As demonstrated by experimental results, the reported platform is able to recover complex-valued object images with higher resolution and better quality than previous implementations. Our approach may provide a cost-effective solution for high-resolution and wide field-of-view ptychographic imaging without involving mechanical scanning.

Deep learning-enabled whole slide imaging (DeepWSI): oil-immersion quality using dry objectives, longer depth of field, higher system throughput, and better functionality.

Whole slide imaging (WSI) has moved the traditional manual slide inspection process to the era of digital pathology. A typical WSI system translates the sample to different positions and captures images using a high numerical aperture (NA) objective lens. Performing oil-immersion microscopy is a major obstacle for WSI as it requires careful liquid handling during the scanning process. Switching between dry objective and oil-immersion lens is often impossible as it disrupts the acquisition process. For a high-NA objective lens, the sub-micron depth of field also poses a challenge to acquiring in-focus images of samples with uneven topography. Additionally, it implies a small field of view for each tile, thus limiting the system throughput and resulting in a long acquisition time. Here we report a deep learning-enabled WSI platform, termed DeepWSI, to substantially improve the system performance and imaging throughput. With this platform, we show that images captured with a regular dry objective lens can be transformed into images comparable to that of a 1.4-NA oil immersion lens. Blurred images with defocus distance from -5 µm to +5 µm can be virtually refocused to the in-focus plane post measurement. We demonstrate an equivalent data throughput of >2 gigapixels per second, the highest among existing WSI systems. Using the same deep neural network, we also report a high-resolution virtual staining strategy and demonstrate it for Fourier ptychographic WSI. The DeepWSI platform may provide a turnkey solution for developing high-performance diagnostic tools for digital pathology.

High-throughput lensless whole slide imaging via continuous height-varying modulation of a tilted sensor.

We report a new, to the best of our knowledge, lensless microscopy configuration by integrating the concepts of transverse translational ptychography and defocus multi-height phase retrieval. In this approach, we place a tilted image sensor under the specimen for introducing linearly increasing phase modulation along one lateral direction. Similar to the operation of ptychography, we laterally translate the specimen and acquire the diffraction images for reconstruction. Since the axial distance between the specimen and the sensor varies at different lateral positions, laterally translating the specimen effectively introduces defocus multi-height measurements while eliminating axial scanning. Lateral translation further introduces sub-pixel shift for pixel super-resolution imaging and naturally expands the field of view for rapid whole slide imaging. We show that the equivalent height variation can be precisely estimated from the lateral shift of the specimen, thereby addressing the challenge of precise axial positioning in conventional multi-height phase retrieval. Using a sensor with 1.67 µm pixel size, our low-cost and field-portable prototype can resolve the 690 nm linewidth on the resolution target. We show that a whole slide image of a blood smear with a 120mm2 field of view can be acquired in 18 s. We also demonstrate accurate automatic white blood cell counting from the recovered image. The reported approach may provide a turnkey solution for addressing point-of-care and telemedicine-related challenges.

Virtual brightfield and fluorescence staining for Fourier ptychography via unsupervised deep learning.

Fourier ptychographic microscopy (FPM) is a computational approach geared towards creating high-resolution and large field-of-view images without mechanical scanning. Acquiring color images of histology slides often requires sequential acquisitions with red, green, and blue illuminations. The color reconstructions often suffer from coherent artifacts that are not presented in regular incoherent microscopy images. As a result, it remains a challenge to employ FPM for digital pathology applications, where resolution and color accuracy are of critical importance. Here we report a deep learning approach for performing unsupervised image-to-image translation of FPM reconstructions. A cycle-consistent adversarial network with multiscale structure similarity loss is trained to perform virtual brightfield and fluorescence staining of the recovered FPM images. In the training stage, we feed the network with two sets of unpaired images: (1) monochromatic FPM recovery and (2) color or fluorescence images captured using a regular microscope. In the inference stage, the network takes the FPM input and outputs a virtually stained image with reduced coherent artifacts and improved image quality. We test the approach on various samples with different staining protocols. High-quality color and fluorescence reconstructions validate its effectiveness.

Super-resolved multispectral lensless microscopy via angle-tilted, wavelength-multiplexed ptychographic modulation.

We report an angle-tilted, wavelength-multiplexed ptychographic modulation approach for multispectral lensless on-chip microscopy. In this approach, we illuminate the specimen with lights at five wavelengths simultaneously. A prism is added at the illumination path for spectral dispersion. Thus, lightwaves at different wavelengths hit the specimen at slightly different incident angles, breaking the ambiguities in mixed-state ptychographic reconstruction. At the detection path, we place a thin diffuser between the specimen and the monochromatic image sensor for encoding the spectral information into 2D intensity measurements. By scanning the sample to different x-y positions, we acquire a sequence of monochromatic images for reconstructing the five complex object profiles at the five wavelengths. An up-sampling procedure is integrated into the recovery process to bypass the resolution limit imposed by the imager pixel size. We demonstrate a half-pitch resolution of 0.55 µm using an image sensor with 1.85 µm pixel size. We also demonstrate quantitative and high-quality multispectral reconstructions of stained tissue sections for digital pathology applications.

Near-field Fourier ptychography: super-resolution phase retrieval via speckle illumination.

High spatial resolution is the goal of many imaging systems. While designing a high-resolution lens with diffraction-limited performance over a large field of view remains a difficult task, creating a complex speckle pattern with wavelength-limited spatial features is easily accomplished with a simple random diffuser. With this observation and the concept of near-field ptychography, we report a new imaging modality, termed near-field Fourier ptychography, to address high-resolution imaging challenges in both microscopic and macroscopic imaging settings. 'Near-field' refers to placing the object at a short defocus distance with a large Fresnel number. We project a speckle pattern with fine spatial features on the object instead of directly resolving the spatial features via a high-resolution lens. We then translate the object (or speckle) to different positions and acquire the corresponding images by using a low-resolution lens. A ptychographic phase retrieval process is used to recover the complex object, the unknown speckle pattern, and the coherent transfer function at the same time. In a microscopic imaging setup, we use a 0.12 numerical aperture (NA) lens to achieve an NA of 0.85 in the reconstruction process. In a macroscale photographic imaging setup, we achieve ~7-fold resolution gain by using a photographic lens. The collection optics do not determine the final achievable resolution; rather, the speckle pattern's feature size does. This is similar to our recent demonstration in fluorescence imaging settings (Guo et al., Biomed. Opt. Express, 9(1), 2018). The reported imaging modality can be employed in light, coherent X-ray, and transmission electron imaging systems to increase resolution and provide quantitative absorption and object phase contrast.

Rapid and robust two-dimensional phase unwrapping via deep learning.

Two-dimensional phase unwrapping algorithms are widely used in optical metrology and measurements. The high noise from interference measurements, however, often leads to the failure of conventional phase unwrapping algorithms. In this paper, we propose a deep convolutional neural network (DCNN) based method to perform rapid and robust two-dimensional phase unwrapping. In our approach, we employ a DCNN architecture, DeepLabV3+, with noise suppression and strong feature representation capabilities. The employed DCNN is first used to perform semantic segmentation to obtain the segmentation result of the wrapped phase map. We then combine the wrapped phase map with the segmentation result to generate the unwrapped phase. We benchmarked our results by comparing them with well-established methods. The reported approach out-performed the conventional path-dependent and path-independent algorithms. We also tested the robustness of the reported approach using interference measurements from optical metrology setups. Our results, again, clearly out-performed the conventional phase unwrap algorithms. The reported approach may find applications in optical metrology and microscopy imaging.

Super-resolution microscopy via ptychographic structured modulation of a diffuser.

We report a new coherent imaging technique, termed ptychographic structured modulation (PSM), for quantitative super-resolution microscopy. In this technique, we place a thin diffuser (i.e., a scattering lens) in between the sample and the objective lens to modulate the complex light waves from the object. The otherwise inaccessible high-resolution object information can thus be encoded into the captured images. We then employ a ptychographic phase retrieval process to jointly recover the exit wavefront of the complex object and the unknown diffuser profile. Unlike the illumination-based super-resolution approach, the recovered image of our approach depends upon how the complex wavefront exits the sample-not enters it. Therefore, the sample thickness becomes irrelevant during reconstruction. After recovery, we can propagate the super-resolution complex wavefront to any position along the optical axis. We validate our approach using a resolution target, a quantitative phase target, a two-layer sample, and a thick polydimethylsiloxane sample. We demonstrate a 4.5-fold resolution gain over the diffraction limit. We also show that a four-fold resolution gain can be achieved with as few as ∼30 images. The reported approach may provide a quantitative super-resolution strategy for coherent light, x-ray, and electron imaging.

Axially shifted pattern illumination for macroscale turbidity suppression and virtual volumetric confocal imaging without axial scanning.

Structured illumination has been widely used for optical sectioning and 3D surface recovery. In a typical implementation, multiple images under non-uniform pattern illumination are used to recover a single object section. Axial scanning of the sample or the objective lens is needed for acquiring the 3D volumetric data. Here we demonstrate the use of axially shifted pattern illumination for virtual volumetric confocal imaging without axial scanning. In the reported approach, we project illumination patterns at a tilted angle with respect to the detection optics. As such, the illumination patterns shift laterally at different z sections, and the 3D sample information can be recovered based on the captured 2D images. We demonstrate the reported approach for virtual confocal imaging through a diffusing layer and underwater 3D imaging through diluted milk. We show that we can acquire the entire confocal volume in ∼1 s with a throughput of 420 megapixels per second. Our approach may provide new insights for developing confocal light ranging and detection systems in degraded visual environments.

Field-portable quantitative lensless microscopy based on translated speckle illumination and sub-sampled ptychographic phase retrieval.

We report a compact, cost-effective, and field-portable lensless imaging platform for quantitative microscopy. In this platform, the object is placed on top of an image sensor chip without using a lens. We use a low-cost galvo scanner to rapidly scan an unknown laser speckle pattern on the object. To address the positioning repeatability and accuracy issues, we directly recover the positional shifts of the speckle pattern based on the phase correlation of the captured images. To bypass the resolution limit set by the imager pixel size, we employ a sub-sampled ptychographic phase retrieval process to recover the complex object. We validate our approach using a resolution target, phase target, and biological sample. Our results show that accurate, high-quality complex images can be obtained from a lensless dataset with as few as ∼10 images. We also demonstrate the reported approach to achieve a 6.4-mm by 4.6-mm field of view and a half-pitch resolution of 1 µm. The reported approach may provide a quantitative lensless imaging strategy for addressing point-of-care-, global-health-, and telemedicine-related challenges.

Assembly of Metallacages into Soft Suprastructures with Dimensions of up to Micrometers and the Formation of Composite Materials.

This work provides a platform for the rapid generation of superstructure assemblies with a wide range of lengths that can be used to access a variety of metal-organic complex-based soft superstructures. Metallacage-based microneedles that are nanometers in diameter and millimeters in length were generated in dichloromethane and ethyl acetate; their size could be controlled by adjusting the ratio of the two solvents. Interestingly, microflower structures could be formed by further assembly of the microneedles during solvent evaporation. Our study establishes a feasible method designed to broaden the range of suprastructures with emissions from blue and green to red through the co-assembly of lysine-modified perylene. Similar to the co-assembly of lysine-modified perylene with microflowers, chlorophyll-a and vitamin B12 were introduced into the microflowers during the assembly process, which may be exploited in studies of energy capture and nerve repair in the future.

Recovering higher dimensional image data using multiplexed structured illumination.

Structured illumination (SI) using non-uniform intensity patterns is well-known for improving lateral resolution in microscopy. Here, we propose a multiplexed SI technique for recovering images with higher lateral resolution and with higher dimensional information at the same time. In this framework, we use unknown non-uniform intensity patterns for incoherent sample illumination and use the corresponding acquisitions for image recovery. In the first example, we use the reported framework to recover sample images with higher lateral resolution and separate different sections of the sample along the z-direction. In the second example, we recover the sample images with higher lateral resolution and separate the images at different spectral bands. The reported multiplexed-SI framework may find applications in general imaging settings where higher dimensional information is mixed in 2D image measurements. It can also be used in microscopy settings for computational sectioning and multispectral imaging.

DIRECT: Digital Microfluidics for Isolation-Free Shared Library Construction of Single-Cell DNA Methylome and Transcriptome.

Simultaneous profiling of DNA methylation and gene expression within single cells is a powerful technology to dissect complex gene regulatory network of cells. However, existing methods are based on picking a single-cell in a tube and split single-cell lysate into two parts for transcriptome and methylome library construction, respectively, which is costly and cumbersome. Here, DIRECT is proposed, a digital microfluidics-based method for high-efficiency single-cell isolation and simultaneous analysis of the methylome and transcriptome in a single library construction. The accuracy of DIRECT is demonstrated in comparison with bulk and single-omics data, and the high CpG site coverage of DIRECT allows for precise analysis of copy number variation information, enabling expansion of single cell analysis from two- to three-omics. By applying DIRECT to monitor the dynamics of mouse embryonic stem cell differentiation, the relationship between DNA methylation and changes in gene expression during differentiation is revealed. DIRECT enables accurate, robust, and reproducible single-cell DNA methylation and gene expression co-analysis in a more cost-effective, simpler library preparation and automated manner, broadening the application scenarios of single-cell multi-omics analysis and revealing a more comprehensive and fine-grained map of cellular regulatory landscapes.