Deep learning enables contrast-robust super-resolution reconstruction in structured illumination microscopy.

Structured illumination microscopy (SIM) is a powerful technique for super-resolution (SR) image reconstruction. However, conventional SIM methods require high-contrast illumination patterns, which necessitate precision optics and highly stable light sources. To overcome these challenges, we propose a new method called contrast-robust structured illumination microscopy (CR-SIM). CR-SIM employs a deep residual neural network to enhance the quality of SIM imaging, particularly in scenarios involving low-contrast illumination stripes. The key contribution of this study is the achievement of reliable SR image reconstruction even in suboptimal illumination contrast conditions. The results of our study will benefit various scientific disciplines.

Full field-of-view hexagonal lattice structured illumination microscopy based on the phase shift of electro-optic modulators.

High throughput has become an important research direction in the field of super-resolution (SR) microscopy, especially in improving the capability of dynamic observations. In this study, we present a hexagonal lattice structured illumination microscopy (hexSIM) system characterized by a large field of view (FOV), rapid imaging speed, and high power efficiency. Our approach employs spatial light interference to generate a two-dimensional hexagonal SIM pattern, and utilizes electro-optical modulators for high-speed phase shifting. This design enables the achievement of a 210-µm diameter SIM illumination FOV when using a 100×/1.49 objective lens, capturing 2048 × 2048 pixel images at an impressive 98 frames per second (fps) single frame rate. Notably, this method attains a near 100% full field-of-view and power efficiency, with the speed limited only by the camera's capabilities. Our hexSIM demonstrates a substantial 1.73-fold improvement in spatial resolution and necessitates only seven phase-shift images, thus enhancing the imaging speed compared to conventional 2D-SIM.

Untrained neural network enabling fast and universal structured-illumination microscopy.

Structured-illumination microscopy (SIM) offers a twofold resolution enhancement beyond the optical diffraction limit. At present, SIM requires several raw structured-illumination (SI) frames to reconstruct a super-resolution (SR) image, especially the time-consuming reconstruction of speckle SIM, which requires hundreds of SI frames. Considering this, we herein propose an untrained structured-illumination reconstruction neural network (USRNN) with known illumination patterns to reduce the amount of raw data that is required for speckle SIM reconstruction by 20 times and thus improve its temporal resolution. Benefiting from the unsupervised optimizing strategy and CNNs' structure priors, the high-frequency information is obtained from the network without the requirement of datasets; as a result, a high-fidelity SR image with approximately twofold resolution enhancement can be reconstructed using five frames or less. Experiments on reconstructing non-biological and biological samples demonstrate the high-speed and high-universality capabilities of our method.

Sub-diffraction optical beam lithography based on a center-non-zero depletion laser.

Photoinhibition (PI) mechanisms have been introduced in nanofabrication which allows breaking the diffraction limit by large factors. Donut-shaped laser is usually selected as a depletion beam to reduce linewidth, but the parasitic process has made the results of the experiment less than expected. As a result, the linewidth is difficult to achieve below 50 nm with 780 nm femtosecond and 532 nm continuous-wave lasers. Here, we propose a new, to the best of our knowledge, method based on a center-non-zero (CNZ) depletion laser to further reduce linewidth. By constructing a smaller zone of action under the condition of keeping the maximum depletion intensity constant, a minimum linewidth of 30 nm (λ / 26) was achieved. Two ways to construct CNZ spots were discussed and experimented, and the results show the advantages of our method to reduce the parasitic process to further improve the writing resolution.

Parallel two-photon lithography achieving uniform sub-200†nm features with thousands of individually controlled foci.

The limited throughput of nano-scale laser lithography has been the bottleneck for its industrial applications. Although using multiple laser foci to parallelize the lithography process is an effective and straightforward strategy to improve rate, most conventional multi-focus methods are plagued by non-uniform laser intensity distribution due to the lack of individual control for each focus, which greatly hinders the nano-scale precision. In this paper, we present a highly uniform parallel two-photon lithography method based on a digital mirror device (DMD) and microlens array (MLA), which allows the generation of thousands of femtosecond (fs) laser foci with individual on-off switching and intensity-tuning capability. In the experiments, we generated a 1,600-laser focus array for parallel fabrication. Notably, the intensity uniformity of the focus array reached 97.7%, where the intensity-tuning precision for each focus reached 0.83%. A uniform dot array structure was fabricated to demonstrate parallel fabrication of sub-diffraction limit features, i.e., below 1/4 λ or 200 nm. The multi-focus lithography method has the potential of realizing rapid fabrication of sub-diffraction, arbitrarily complex, and large-scale 3D structures with three orders of magnitude higher fabrication rate.

3D nanoprinting for fiber-integrated achromatic diffractive lens.

Achromatic performance is crucial for numerous multi-wavelength optical fiber applications, including endoscopic imaging and fiber sensing. This paper presents the design and nanoprinting of a fiber-integrated achromatic diffractive lens within the visible spectrum (450-650 nm). The 3D nanoprinting is achieved by a high-resolution direct laser writing technology, overcoming limitations in the optical performance caused by the lack of an arbitrary 3D structure writing capability and an insufficient feature resolution in the current manufacturing technology for visible light broadband achromatic diffractive lenses. A three-step optimization algorithm is proposed to effectively balance optical performance with writing difficulty. The characterization results demonstrate excellent achromatic focusing performance, paving the way towards the development of nanoprinted flat optical devices for applications such as optical fiber traps, miniature illumination systems, and integrated photonic chips.

High-resolution 3D nanoprinting based on two-step absorption via an integrated fiber-coupled laser diode.

Three-dimensional (3D) laser nanoprinting with high resolution and low cost is highly desirable for fabricating arbitrary 3D structures with fine feature size. In this work, we use a 405-nm integrated fiber-coupled continuous wave (cw) laser diode to establish an easy-to-build 3D nanoprinting system based on two-step absorption. Two-dimensional (2D) gratings with a sub-150-nm period and 3D woodpile nanostructures with a lateral period of 350 nm have been printed at a low speed. At a faster scan velocity of 1000 µm/s, 2D gratings with sub-200-nm resolution and sub-50-nm linewidth can still be fabricated with laser power less than 1 mW. The two-step absorption of the used benzil initiator enables us to use a second cw laser with 532-nm wavelength to enhance the polymerization with sub-100-nm feature size when starting with insufficient 405-nm laser power, which possess the potential to find applications in high-speed high-resolution parallel-writing and in situ manipulation.

High-accuracy differential autofocus system with an electrically tunable lens.

We propose a quasi-confocal microscopy autofocus system incorporating an electrically tunable lens (ETL) to achieve differential detection. The ETL changes its focal length to collect differential curves at speeds <300 Hz, allowing selective locking onto desired focal layers and high-speed differential operations close to the locked focal plane. By segmenting the system's pupil, the interference between the outgoing and incoming near-infrared beams is avoided, thereby greatly improving the signal-to-noise ratio. This ultra-sensitive system, with a focus drift accuracy better than 1/22 focal depth (∼20 nm @100× objective), provides a new, to the best of our knowledge, implementation pathway to meet the requirements of various microscopy techniques.

High-speed spatially re-modulated structured illumination microscopy.

Structured illumination microscopy (SIM) allows non-invasive visualization of nanoscale subcellular structures. However, image acquisition and reconstruction become the bottleneck to further improve the imaging speed. Here, we propose a method to accelerate SIM imaging by combining the spatial re-modulation principle with Fourier domain filtering and using measured illumination patterns. This approach enables high-speed, high-quality imaging of dense subcellular structures using a conventional nine-frame SIM modality without phase estimation of the patterns. In addition, seven-frame SIM reconstruction and additional hardware acceleration further improve the imaging speed using our method. Furthermore, our method is also applicable to other spatially uncorrelated illumination patterns, such as distorted sinusoidal, multifocal, and speckle patterns.

High-throughput multiplexed fluorescence lifetime microscopy.

Fluorescence lifetime microscopy has been widely used in quantifying cellular interaction or histopathological identification of different stained tissues. A novel, to the best of our knowledge, approach for high-throughput multiplexed fluorescence lifetime imaging is presented. To establish a high-throughput fluorescence lifetime acquisition system, a uniformed illumination optical focus array was generated by a novel computer-generated hologram algorithm based on matrix triple product. This, in conjunction with an array detector and multichannel time-correlated single-photon counting, enables the full use of the acquisition ability of each detector. By utilizing interval segmentation of photon time detection, a high-throughput multiplexed fluorescence lifetime imaging is achieved. Experimental results demonstrate that this method achieves a fivefold increase in the collection throughput of fluorescence lifetime and is capable of simultaneous dual-target fluorescence lifetime measurement.

Dichromatic "Breather Molecules" in a Mode-Locked Fiber Laser.

Bound states of solitons ("molecules") occur in various settings, playing an important role in the operation of fiber lasers, optical emulation, encoding, and communications. Soliton interactions are generally related to breathing dynamics in nonlinear dissipative systems, and maintain potential applications in spectroscopy. In the present work, dichromatic breather molecules (DBMs) are created in a synchronized mode-locked fiber laser. Real-time delay-shifting interference spectra are measured to display the temporal evolution of the DBMs, that cannot be observed by means of the usual real-time spectroscopy. As a result, robust out-of-phase vibrations are found as a typical intrinsic mode of DBMs. The same bound states are produced numerically in the framework of a model combining equations for the population inversion in the mode-locked laser and cross-phase-modulation-coupled complex Ginzburg-Landau equations for amplitudes of the optical fields in the fiber segments of the laser cavity. The results demonstrate that the Q-switching instability induces the onset of breathing oscillations. The findings offer new possibilities for the design of various regimes of the operation of ultrafast lasers.

High-precision detection method for an object edge based on a position-sensitive detector.

The paper proposed and verified a method of object edge detection based on the innovative defect spot working mode of the position-sensitive detector (PSD). With the output characteristics of the PSD in the defect spot mode and the size transformation properties of a focused beam, edge-detection sensitivity could be improved. Calibration experiments with the piezoelectric transducer (PZT) and object edge-detection experiments indicate that the object edge-detection sensitivity and accuracy of our method could reach 1 and 20 nm, respectively. Therefore, this method can be widely used in high-precision alignment, geometric parameters measurement, and other fields.

DMD-based compact SIM system with hexagonal-lattice-structured illumination.

In this study, we developed a novel, compact, and efficient structured illumination microscopy (SIM) system, to our best knowledge. A binary hexagonal lattice pattern was designed and implemented on a digital micromirror device (DMD), resulting in a projection-based structured-light generation. By leveraging the combination of the high-speed switching capability of the DMD with a high-speed CMOS camera, the system can capture 1024×1024 pixels images at a 200 fps frame rate when provided with sufficient illumination power. The loading of the hexagonal lattice pattern reduces the number of images required for reconstruction to seven, and by utilizing the DMD modulating characteristics on the illumination path, there is no need to use bulky mechanical structures for phase shifting. We designed a compact system with 110m m×150m m×170m m dimensions that displayed a 1.61 resolution enhancement for fluorescent particle and biological sample imaging.

A novel fusion method for X-ray phase contrast imaging based on fast adaptive bidimensional empirical mode decomposition.

BACKGROUNDS: X-ray phase contrast imaging (XPCI) can separate the attenuation, refraction, and scattering signals of the object. The application of image fusion enables the concentration of distinctive information into a single image. Some methods have been applied in XPCI field, but wavelet-based decomposition approaches often result in loss of original data. OBJECTIVE: To explore the application value of a novel image fusion method for XPCI system and computed tomography (CT) system. METHODS: The means of fast adaptive bidimensional empirical mode decomposition (FABEMD) is considered for image decomposition to avoid unnecessary information loss. A parameter δ is proposed to guide the fusion of bidimensional intrinsic mode functions which contain high-frequency information, using a pulse coupled neural network with morphological gradients (MGPCNN). The residual images are fused by the energy attribute fusion strategy. Image preprocessing and enhancement are performed on the result to ensure its quality. The effectiveness of other image fusion methods has been compared, such as discrete wavelet transforms and anisotropic diffusion fusion. RESULTS: The δ-guided FABEMD-MGPCNN method achieved either the first or second position in objective evaluation metrics with biological samples, as compared to other image fusion methods. Moreover, comparisons are made with other fusion methods used for XPCI. Finally, the proposed method applied in CT show expected results to retain the feature information. CONCLUSIONS: The proposed δ-guided FABEMD-MGPCNN method shows potential feasibility and superiority over traditional and recent image fusion methods for X-ray differential phase contrast imaging and computed tomography systems.

Real-time reconstruction using electro-optics modulator-based structured illumination microscopy.

Structured illumination microscopy (SIM), a super-resolution technology, has a wide range of applications in life sciences. In this study, we present an electro-optic high-speed phase-shift super-resolution microscopy imaging system including 2D SIM, total internal reflection fluorescence-SIM, and 3D SIM modes. This system uses galvanometers and an electro-optic modulator to flexibly and quickly control the phase and direction of structured illumination patterns. Moreover, its design consists of precise timing for improved acquisition speed and software architecture for real-time reconstruction. The highest acquisition rate achieved was 151 frames/s, while the highest real-time super-resolution reconstruction frame rate achieved was over 25 frames/s.

Simultaneous super-resolution estimation of single-molecule position and orientation with minimal photon fluxes.

The orientation of a single molecule provides valuable information on fundamental biological processes. We report a technique for the simultaneous estimation of single-molecule 2D position and 2D orientation with ultra-high localization precision (∼2-nm precision with ∼500 photons under a typical 100-nm diameter of excitation beam pattern), which is also compatible with tracking in living cells. In the proposed method, the theoretical precision limits are calculated, and the localization and orientation performance along with potential applications are explored using numerical simulations. Compared to other camera-based orientation measurement methods, it is confirmed that the proposed method can obtain reasonable estimates even under very weak signals (∼15 photons). Moreover, the maximum likelihood estimator (MLE) is found to converge to the theoretical limit when the total number of photons is less than 100.

Moiré artifacts reduction in Talbot-Lau X-ray phase contrast imaging using a three-step iterative approach.

Talbot-Lau X-ray phase contrast imaging is a promising technique in biological imaging since it can provide absorption, differential phase contrast, and dark-field images simultaneously. However, high accuracy motorized translation stages and high stability of the imaging system are needed to avoid moiré artifacts in the reconstructed images. In this work, the effects of the stepping errors and the dose fluctuations on the transmission, differential phase contrast, and dark-field images are theoretically derived and systematically summarized. A novel three-step iterative method is designed for image reconstruction in Talbot-Lau interferometry with phase-stepping errors and dose fluctuations. Phase distributions, phase-stepping errors, and dose fluctuation coefficients are iteratively updated via the least square method until the convergence criteria are met. Moiré artifacts are mostly reduced via the proposed method in both the numerical simulations and experiments. The reconstructed images are highly coincident with the ground truth, which verifies the high accuracy of this method. The proposed algorithm is also compared with other moiré artifacts reduction algorithms, which further demonstrates the high precision of this algorithm. This work is beneficial for reducing the strict requirements for the hardware system in the conventional Talbot-Lau interferometry, such as the high accuracy motorized stages and the X-ray tube with high stability, which is significant for advancing the X-ray phase contrast imaging towards the practical medical applications.

Effects of optical aberrations on localization of MINFLUX super-resolution microscopy.

A novel super-resolution imaging technique based on the minimum photon flux (MINFLUX), can achieve nanometer-scale localization precision and sub-5-nm imaging. However, aberrations can affect the localization performance and degrade the quality of reconstructed images. In this study, we analyze the effects of different low-order aberrations on the MINFLUX system through both theoretical limits and Monte Carlo methods. We report that 1) defocus and spherical aberration have little effect on 2D localization performance, whereas astigmatism and coma have significant negative effects; 2) system aberrations that can be measured in advance cause changes primarily in the magnitude and angular uniformity of localization precision, whereas sample-induced aberrations that cannot be a priori introduce large biases and reduce localization accuracy.

Total variation and spatial iteration-based 3D structured illumination microscopy.

Three-dimensional structured illumination microscopy (3D-SIM) plays an essential role in biological volumetric imaging with the capabilities of improving lateral and axial resolution. However, the traditional linear 3D algorithm is sensitive to noise and generates artifacts, while the low temporal resolution hinders live-cell imaging. In this paper, we propose a novel 3D-SIM algorithm based on total variation (TV) and fast iterative shrinkage threshold algorithm (FISTA), termed TV-FISTA-SIM. Compared to conventional algorithms, TV-FISTA-SIM achieves higher reconstruction fidelity with the least artifacts, even when the signal-to-noise ratio (SNR) is as low as 5 dB, and a faster reconstruction rate. Through simulation, we have verified that TV-FISTA-SIM can effectively reduce the amount of required data with less deterioration. Moreover, we demonstrate TV-FISTA-SIM for high-quality multi-color 3D super-resolution imaging, which can be potentially applied to live-cell imaging applications.

Modulated pattern scanning microscopy.

In confocal microscopy, the effective optical transfer function (OTFeff) with Gaussian plane wave illumination covers very few high-frequency components, which prohibits further improvement of the resolution. We propose modulated pattern scanning microscopy (MPSM) to achieve super-resolution imaging. In MPSM, the phase of the illumination beam is modulated to reassign the OTFeff in the Fourier domain. The phase mask is designed using an optimization algorithm to obtain the fluorescence emission pattern with rich high-frequency components. Then, the postprocessing algorithms are adapted to retrieve the super-resolved images from the modulated recordings. Simulation and experiment demonstrate that MPSM increases the resolution approximately 1.3 times better than confocal microscopy. Compared with conventional deconvolution, MPSM exhibits a higher signal-to-noise ratio.