Single image Fourier ring correlation.

We address resolution assessment for (light super-resolution) microscopy imaging. In modalities where imaging is not diffraction limited, correlation between two noise independent images is the standard way to infer the resolution. Here we take away the need for two noise independent images by computationally splitting one image acquisition into two noise independent realizations. This procedure generates two Poisson noise distributed images if the input is Poissonian distributed. As most modern cameras are shot-noise limited this procedure is directly applicable. However, also in the presence of readout noise we can compute the resolution faithfully via a correction factor. We evaluate our method on simulations and experimental data of widefield microscopy, STED microscopy, rescan confocal microscopy, image scanning microscopy, conventional confocal microscopy, and transmission electron microscopy. In all situations we find that using one image instead of two results in the same computed image resolution.

ZIMFLUX: Single molecule localization microscopy with patterned illumination in 3D.

Three dimensional modulation-enhanced single-molecule localization techniques, such as ModLoc, offer advancements in axial localization precision across the entire field of view and axial capture range, by applying phase shifting to the illumination pattern. However, this improvement is limited by the pitch of the illumination pattern that can be used and requires registration between separate regions of the camera. To overcome these limitations, we present ZIMFLUX, a method that combines astigmatic point-spread-function (PSF) engineering with a structured illumination pattern in all three spatial dimensions. In order to achieve this we address challenges such as optical aberrations, refractive index mismatch, supercritical angle fluorescence (SAF), and imaging at varying depths within a sample, by implementing a vectorial PSF model. In scenarios involving refractive index mismatch between the sample and immersion medium, the astigmatic PSF loses its ellipticity at greater imaging depths, leading to a deterioration in axial localization precision. In contrast, our simulations demonstrate that ZIMFLUX maintains high axial localization precision even when imaging deeper into the sample. Experimental results show unbiased localization of 3D 80 nm DNA-origami nanostructures in SAF conditions with a 1.5-fold improvement in axial localization precision when comparing ZIMFLUX to conventional SMLM methods that rely solely on astigmatic PSF engineering.

Detecting continuous structural heterogeneity in single-molecule localization microscopy data.

Fusion of multiple chemically identical complexes, so-called particles, in localization microscopy, can improve the signal-to-noise ratio and overcome under-labeling. To this end, structural homogeneity of the data must be assumed. Biological heterogeneity, however, could be present in the data originating from distinct conformational variations or (continuous) variations in particle shapes. We present a prior-knowledge-free method for detecting continuous structural variations with localization microscopy. Detecting this heterogeneity leads to more faithful fusions and reconstructions of the localization microscopy data as their heterogeneity is taken into account. In experimental datasets, we show the continuous variation of the height of DNA origami tetrahedrons imaged with 3D PAINT and of the radius of Nuclear Pore Complexes imaged in 2D with STORM. In simulation, we study the impact on the heterogeneity detection pipeline of Degree Of Labeling and of structural variations in the form of two independent modes.

Particle fusion of super-resolution data reveals the unit structure of Nup96 in Nuclear Pore Complex.

Single molecule localization microscopy offers resolution nearly down to the molecular level with specific molecular labelling, and is thereby a promising tool for structural biology. In practice, however, the actual value to this field is limited primarily by incomplete fluorescent labelling of the structure. This missing information can be completed by merging information from many structurally identical particles in a particle fusion approach similar to cryo-EM single-particle analysis. In this paper, we present a data analysis of particle fusion results of fluorescently labelled Nup96 nucleoporins in the Nuclear Pore Complex to show that Nup96 occurs in a spatial arrangement of two rings of 8 units with two Nup96 copies per unit giving a total of 32 Nup96 copies per pore. We use Artificial Intelligence assisted modeling in Alphafold to extend the existing cryo-EM model of Nup96 to accurately pinpoint the positions of the fluorescent labels and show the accuracy of the match between fluorescent and cryo-EM data to be better than 3 nm in-plane and 5 nm out-of-plane.

Erratum: Influence of edge enhancement applied in endoscopic systems on sharpness and noise (Erratum).

[This corrects the article DOI: 10.1117/1.JBO.27.10.106001.].

High-precision estimation of emitter positions using Bayesian grouping of localizations.

Single-molecule localization microscopy super-resolution methods rely on stochastic blinking/binding events, which often occur multiple times from each emitter over the course of data acquisition. Typically, the blinking/binding events from each emitter are treated as independent events, without an attempt to assign them to a particular emitter. Here, we describe a Bayesian method of inferring the positions of the tagged molecules by exploring the possible grouping and combination of localizations from multiple blinking/binding events. The results are position estimates of the tagged molecules that have improved localization precision and facilitate nanoscale structural insights. The Bayesian framework uses the localization precisions to learn the statistical distribution of the number of blinking/binding events per emitter and infer the number and position of emitters. We demonstrate the method on a range of synthetic data with various emitter densities, DNA origami constructs and biological structures using DNA-PAINT and dSTORM data. We show that under some experimental conditions it is possible to achieve sub-nanometer precision.

Influence of edge enhancement applied in endoscopic systems on sharpness and noise.

Significance: Flexible endoscopes are essential for medical internal examinations. Digital endoscopes are connected to a video processor that can apply various operations to enhance the image. One of those operations is edge enhancement, which has a major impact on the perceived image quality by medical professionals. However, the specific methods and parameters of this operation are undisclosed and the arbitrary units to express the level of edge enhancement differ per video processor. Aim: Objectively quantify the level of edge enhancement from the recorded images alone, and measure the effect on sharpness and noise Approach: Edge enhancement was studied in four types of flexible digital ear nose and throat endoscopes. Measurements were performed using slanted edges and gray patches. The level of edge enhancement was determined by subtracting the step response of an image without edge enhancement from images with selected settings of edge enhancement and measuring the resulting peak-to-peak differences. These values were then normalized by the step size. Sharpness was characterized by observing the normalized modulation transfer function (MTF) and computing the spatial frequency at 50% MTF. The noise was measured on the gray patches and computed as a weighted sum of variances from the luminance and two chrominance channels of the pixel values. Results: The measured levels were consistent with the level set via the user interface on the video processor and varied typically from 0 to 1.3. Both sharpness and noise increase with larger levels of edge enhancement with factors of 3 and 4 respectively. Conclusions: The presented method overcomes the issue of vendors expressing the level of edge enhancement each differently in arbitrary units. This allows us to compare the effects, and we can start exploring the relationship with the subjectively perceived image quality by medical professionals to find substantiated optimal settings.

Photon efficient orientation estimation using polarization modulation in single-molecule localization microscopy.

Combining orientation estimation with localization microscopy opens up the possibility to analyze the underlying orientation of biomolecules on the nanometer scale. Inspired by the recent improvement of the localization precision by shifting excitation patterns (MINFLUX, SIMFLUX), we have adapted the idea towards the modulation of excitation polarization to enhance the orientation precision. For this modality two modes are analyzed: i) normally incident excitation with three polarization steps to retrieve the in-plane angle of emitters and ii) obliquely incident excitation with p-polarization with five different azimuthal angles of incidence to retrieve the full orientation. Firstly, we present a theoretical study of the lower precision limit with a Cramér-Rao bound for these modes. For the oblique incidence mode we find a favorable isotropic orientation precision for all molecular orientations if the polar angle of incidence is equal to arccos ⁡ 2 / 3 ≈ 35 degrees. Secondly, a simulation study is performed to assess the performance for low signal-to-background ratios and how inaccurate illumination polarization angles affect the outcome. We show that a precision, at the Cramér-Rao bound (CRB) limit, of just 2.4 and 1.6 degrees in the azimuthal and polar angles can be achieved with only 1000 detected signal photons and 10 background photons per pixel (about twice better than reported earlier). Lastly, the alignment and calibration of an optical microscope with polarization control is described in detail. With this microscope a proof-of-principle experiment is carried out, demonstrating an experimental in-plane precision close to the CRB limit for signal photon counts ranging from 400 to 10,000.

Joint registration of multiple point clouds for fast particle fusion in localization microscopy.

SUMMARY: We present a fast particle fusion method for particles imaged with single-molecule localization microscopy. The state-of-the-art approach based on all-to-all registration has proven to work well but its computational cost scales unfavorably with the number of particles N, namely as N2. Our method overcomes this problem and achieves a linear scaling of computational cost with N by making use of the Joint Registration of Multiple Point Clouds (JRMPC) method. Straightforward application of JRMPC fails as mostly locally optimal solutions are found. These usually contain several overlapping clusters that each consist of well-aligned particles, but that have different poses. We solve this issue by repeated runs of JRMPC for different initial conditions, followed by a classification step to identify the clusters, and a connection step to link the different clusters obtained for different initializations. In this way a single well-aligned structure is obtained containing the majority of the particles. RESULTS: We achieve reconstructions of experimental DNA-origami datasets consisting of close to 400 particles within only 10 min on a CPU, with an image resolution of 3.2 nm. In addition, we show artifact-free reconstructions of symmetric structures without making any use of the symmetry. We also demonstrate that the method works well for poor data with a low density of labeling and for 3D data. AVAILABILITY AND IMPLEMENTATION: The code is available for download from https://github.com/wexw/Joint-Registration-of-Multiple-Point-Clouds-for-Fast-Particle-Fusion-in-Localization-Microscopy. SUPPLEMENTARY INFORMATION: Supplementary data are available at Bioinformatics online.

Waveguide-based total internal reflection fluorescence microscope enabling cellular imaging under cryogenic conditions.

Total internal reflection fluorescence (TIRF) microscopy is an important imaging tool for the investigation of biological structures, especially the study on cellular events near the plasma membrane. Imaging at cryogenic temperatures not only enables observing structures in a near-native and fixed state but also suppresses irreversible photo-bleaching rates, resulting in increased photo-stability of fluorophores. Traditional TIRF microscopes produce an evanescent field based on high numerical aperture immersion objective lenses with high magnification, which results in a limited field of view and is incompatible with cryogenic conditions. Here, we present a waveguide-based TIRF microscope, which is able to generate a uniform evanescent field using high refractive index waveguides on photonic chips and to obtain cellular observation at cryogenic temperatures. Our method provides an inexpensive way to achieve total-internal-reflection fluorescence imaging under cryogenic conditions.

Simultaneous orientation and 3D localization microscopy with a Vortex point spread function.

Estimating the orientation and 3D position of rotationally constrained emitters with localization microscopy typically requires polarization splitting or a large engineered Point Spread Function (PSF). Here we utilize a compact modified PSF for single molecule emitter imaging to estimate simultaneously the 3D position, dipole orientation, and degree of rotational constraint from a single 2D image. We use an affordable and commonly available phase plate, normally used for STED microscopy in the excitation light path, to alter the PSF in the emission light path. This resulting Vortex PSF does not require polarization splitting and has a compact PSF size, making it easy to implement and combine with localization microscopy techniques. In addition to a vectorial PSF fitting routine we calibrate for field-dependent aberrations which enables orientation and position estimation within 30% of the Cramér-Rao bound limit over a 66 µm field of view. We demonstrate this technique on reorienting single molecules adhered to the cover slip, λ-DNA with DNA intercalators using binding-activated localization microscopy, and we reveal periodicity on intertwined structures on supercoiled DNA.

Detecting structural heterogeneity in single-molecule localization microscopy data.

Particle fusion for single molecule localization microscopy improves signal-to-noise ratio and overcomes underlabeling, but ignores structural heterogeneity or conformational variability. We present a-priori knowledge-free unsupervised classification of structurally different particles employing the Bhattacharya cost function as dissimilarity metric. We achieve 96% classification accuracy on mixtures of up to four different DNA-origami structures, detect rare classes of origami occuring at 2% rate, and capture variation in ellipticity of nuclear pore complexes.

3D particle averaging and detection of macromolecular symmetry in localization microscopy.

Single molecule localization microscopy offers in principle resolution down to the molecular level, but in practice this is limited primarily by incomplete fluorescent labeling of the structure. This missing information can be completed by merging information from many structurally identical particles. In this work, we present an approach for 3D single particle analysis in localization microscopy which hugely increases signal-to-noise ratio and resolution and enables determining the symmetry groups of macromolecular complexes. Our method does not require a structural template, and handles anisotropic localization uncertainties. We demonstrate 3D reconstructions of DNA-origami tetrahedrons, Nup96 and Nup107 subcomplexes of the nuclear pore complex acquired using multiple single molecule localization microscopy techniques, with their structural symmetry deducted from the data.

High-speed multicolor structured illumination microscopy using a hexagonal single mode fiber array.

Structured illumination microscopy (SIM) is a widely used imaging technique that doubles the effective resolution of widefield microscopes. Most current implementations rely on diffractive elements, either gratings or programmable devices, to generate structured light patterns in the sample. These can be limited by spectral efficiency, speed, or both. Here we introduce the concept of fiber SIM that allows for camera frame rate limited pattern generation and manipulation over a broad wavelength range. Illumination patterns are generated by coupling laser beams into radially opposite pairs of fibers in a hexagonal single mode fiber array where the exit beams are relayed to the microscope objective's back focal plane. The phase stepping and rotation of the illumination patterns are controlled by fast electro-optic devices. We achieved a rate of 111 SIM frames per second and imaged with excitation patterns generated by both 488 nm and 532 nm lasers.

Localization microscopy at doubled precision with patterned illumination.

MINFLUX offers a breakthrough in single molecule localization precision, but is limited in field of view. Here we combine centroid estimation and illumination pattern induced photon count variations in a conventional widefield imaging setup to extract position information over a typical micrometer-sized field of view. We show a near two-fold improvement in precision over standard localization with the same photon count on DNA-origami nanostructures and tubulin in cells, using DNA-PAINT and STORM imaging.

Publisher Correction: Super-resolution fight club: assessment of 2D and 3D single-molecule localization microscopy software.

In the version of this paper originally published, Figure 4a contained errors that were introduced during typesetting. The bottom 11° ThunderSTORM image is an xz view but was incorrectly labeled as xy, and the low x-axis value in the four line profiles was incorrectly set as -60 instead of -50. These errors have been corrected in the PDF and HTML versions of the paper.

Super-resolution fight club: assessment of 2D and 3D single-molecule localization microscopy software.

With the widespread uptake of two-dimensional (2D) and three-dimensional (3D) single-molecule localization microscopy (SMLM), a large set of different data analysis packages have been developed to generate super-resolution images. In a large community effort, we designed a competition to extensively characterize and rank the performance of 2D and 3D SMLM software packages. We generated realistic simulated datasets for popular imaging modalities-2D, astigmatic 3D, biplane 3D and double-helix 3D-and evaluated 36 participant packages against these data. This provides the first broad assessment of 3D SMLM software and provides a holistic view of how the latest 2D and 3D SMLM packages perform in realistic conditions. This resource allows researchers to identify optimal analytical software for their experiments, allows 3D SMLM software developers to benchmark new software against the current state of the art, and provides insight into the current limits of the field.

A Multichannel Cross-Modal Fusion Framework for Electron Tomography.

In this paper, we present a multichannel cross-modal fusion algorithm to combine two complementary modalities in electron tomography: X-ray spectroscopy and scanning transmission electron microscopy (STEM). The former reveals compositions with high elemental specificity but low signal-to-noise ratio (SNR), while the latter characterizes the structure with high SNR but little chemical information. We use a multivariate regression to build a cross-modal fusion framework for these two modalities to simultaneously achieve high elemental specificity and high SNR for a target element chosen from the sample under study. Specifically, we first compute three-dimensional tomograms from tilt-series datasets of X-ray and STEM using different reconstruction algorithms. Then, we generate many feature images from each tomogram. Finally, we adopt partial least squares regression to assess the connection between these feature images and the reconstruction of the target element. Based on the simulated and experimental datasets of semiconductor devices, we demonstrate that our algorithm can not only produce continuous edges, homogeneous foreground, and clean background in its element-specific reconstructions but also can more accurately preserve fine structures than state-of-the-art tomography techniques. Moreover, we show that it can deliver results with high fidelity even for X-ray datasets with limited tilts or low counts. This property is highly desired in the semiconductor industry where acquisition time and sample damage are essential.