Single image Fourier ring correlation.

We address resolution assessment for (light super-resolution) microscopy imaging. In modalities where imaging is not diffraction limited, correlation between two noise independent images is the standard way to infer the resolution. Here we take away the need for two noise independent images by computationally splitting one image acquisition into two noise independent realizations. This procedure generates two Poisson noise distributed images if the input is Poissonian distributed. As most modern cameras are shot-noise limited this procedure is directly applicable. However, also in the presence of readout noise we can compute the resolution faithfully via a correction factor. We evaluate our method on simulations and experimental data of widefield microscopy, STED microscopy, rescan confocal microscopy, image scanning microscopy, conventional confocal microscopy, and transmission electron microscopy. In all situations we find that using one image instead of two results in the same computed image resolution.

Localization microscopy at doubled precision with patterned illumination.

MINFLUX offers a breakthrough in single molecule localization precision, but is limited in field of view. Here we combine centroid estimation and illumination pattern induced photon count variations in a conventional widefield imaging setup to extract position information over a typical micrometer-sized field of view. We show a near two-fold improvement in precision over standard localization with the same photon count on DNA-origami nanostructures and tubulin in cells, using DNA-PAINT and STORM imaging.

Joint registration of multiple point clouds for fast particle fusion in localization microscopy.

SUMMARY: We present a fast particle fusion method for particles imaged with single-molecule localization microscopy. The state-of-the-art approach based on all-to-all registration has proven to work well but its computational cost scales unfavorably with the number of particles N, namely as N2. Our method overcomes this problem and achieves a linear scaling of computational cost with N by making use of the Joint Registration of Multiple Point Clouds (JRMPC) method. Straightforward application of JRMPC fails as mostly locally optimal solutions are found. These usually contain several overlapping clusters that each consist of well-aligned particles, but that have different poses. We solve this issue by repeated runs of JRMPC for different initial conditions, followed by a classification step to identify the clusters, and a connection step to link the different clusters obtained for different initializations. In this way a single well-aligned structure is obtained containing the majority of the particles. RESULTS: We achieve reconstructions of experimental DNA-origami datasets consisting of close to 400 particles within only 10 min on a CPU, with an image resolution of 3.2 nm. In addition, we show artifact-free reconstructions of symmetric structures without making any use of the symmetry. We also demonstrate that the method works well for poor data with a low density of labeling and for 3D data. AVAILABILITY AND IMPLEMENTATION: The code is available for download from https://github.com/wexw/Joint-Registration-of-Multiple-Point-Clouds-for-Fast-Particle-Fusion-in-Localization-Microscopy. SUPPLEMENTARY INFORMATION: Supplementary data are available at Bioinformatics online.

ZIMFLUX: Single molecule localization microscopy with patterned illumination in 3D.

Three dimensional modulation-enhanced single-molecule localization techniques, such as ModLoc, offer advancements in axial localization precision across the entire field of view and axial capture range, by applying phase shifting to the illumination pattern. However, this improvement is limited by the pitch of the illumination pattern that can be used and requires registration between separate regions of the camera. To overcome these limitations, we present ZIMFLUX, a method that combines astigmatic point-spread-function (PSF) engineering with a structured illumination pattern in all three spatial dimensions. In order to achieve this we address challenges such as optical aberrations, refractive index mismatch, supercritical angle fluorescence (SAF), and imaging at varying depths within a sample, by implementing a vectorial PSF model. In scenarios involving refractive index mismatch between the sample and immersion medium, the astigmatic PSF loses its ellipticity at greater imaging depths, leading to a deterioration in axial localization precision. In contrast, our simulations demonstrate that ZIMFLUX maintains high axial localization precision even when imaging deeper into the sample. Experimental results show unbiased localization of 3D 80 nm DNA-origami nanostructures in SAF conditions with a 1.5-fold improvement in axial localization precision when comparing ZIMFLUX to conventional SMLM methods that rely solely on astigmatic PSF engineering.

Super-resolution fight club: assessment of 2D and 3D single-molecule localization microscopy software.

With the widespread uptake of two-dimensional (2D) and three-dimensional (3D) single-molecule localization microscopy (SMLM), a large set of different data analysis packages have been developed to generate super-resolution images. In a large community effort, we designed a competition to extensively characterize and rank the performance of 2D and 3D SMLM software packages. We generated realistic simulated datasets for popular imaging modalities-2D, astigmatic 3D, biplane 3D and double-helix 3D-and evaluated 36 participant packages against these data. This provides the first broad assessment of 3D SMLM software and provides a holistic view of how the latest 2D and 3D SMLM packages perform in realistic conditions. This resource allows researchers to identify optimal analytical software for their experiments, allows 3D SMLM software developers to benchmark new software against the current state of the art, and provides insight into the current limits of the field.

Publisher Correction: Super-resolution fight club: assessment of 2D and 3D single-molecule localization microscopy software.

In the version of this paper originally published, Figure 4a contained errors that were introduced during typesetting. The bottom 11° ThunderSTORM image is an xz view but was incorrectly labeled as xy, and the low x-axis value in the four line profiles was incorrectly set as -60 instead of -50. These errors have been corrected in the PDF and HTML versions of the paper.

Template-free 2D particle fusion in localization microscopy.

Methods that fuse multiple localization microscopy images of a single structure can improve signal-to-noise ratio and resolution, but they generally suffer from template bias or sensitivity to registration errors. We present a template-free particle-fusion approach based on an all-to-all registration that provides robustness against individual misregistrations and underlabeling. We achieved 3.3-nm Fourier ring correlation (FRC) image resolution by fusing 383 DNA origami nanostructures with 80% labeling density, and 5.0-nm resolution for structures with 30% labeling density.

Waveguide-based total internal reflection fluorescence microscope enabling cellular imaging under cryogenic conditions.

Total internal reflection fluorescence (TIRF) microscopy is an important imaging tool for the investigation of biological structures, especially the study on cellular events near the plasma membrane. Imaging at cryogenic temperatures not only enables observing structures in a near-native and fixed state but also suppresses irreversible photo-bleaching rates, resulting in increased photo-stability of fluorophores. Traditional TIRF microscopes produce an evanescent field based on high numerical aperture immersion objective lenses with high magnification, which results in a limited field of view and is incompatible with cryogenic conditions. Here, we present a waveguide-based TIRF microscope, which is able to generate a uniform evanescent field using high refractive index waveguides on photonic chips and to obtain cellular observation at cryogenic temperatures. Our method provides an inexpensive way to achieve total-internal-reflection fluorescence imaging under cryogenic conditions.

Photon Yield Enhancement of Red Fluorophores at Cryogenic Temperatures.

Single Molecule Localization Microscopy has become one of the most successful and widely applied methods of Super-resolution Fluorescence Microscopy. Its achievable resolution strongly depends on the number of detectable photons from a single molecule until photobleaching. By cooling a sample from room temperature down to liquid nitrogen temperatures, the photostability of dyes can be enhanced by more than 100 fold, which results in an improvement in localization precision greater than 10 times. Here, we investigate a variety of fluorescent dyes in the red spectral region, and we find an average photon yield between 3.5 ⋅ 106 to 11 ⋅ 106 photons before bleaching at liquid nitrogen temperatures, corresponding to a theoretical localization precision around 0.1 nm.

Phasor based single-molecule localization microscopy in 3D (pSMLM-3D): An algorithm for MHz localization rates using standard CPUs.

We present a fast and model-free 2D and 3D single-molecule localization algorithm that allows more than 3 × 106 localizations per second to be calculated on a standard multi-core central processing unit with localization accuracies in line with the most accurate algorithms currently available. Our algorithm converts the region of interest around a point spread function to two phase vectors (phasors) by calculating the first Fourier coefficients in both the x- and y-direction. The angles of these phasors are used to localize the center of the single fluorescent emitter, and the ratio of the magnitudes of the two phasors is a measure for astigmatism, which can be used to obtain depth information (z-direction). Our approach can be used both as a stand-alone algorithm for maximizing localization speed and as a first estimator for more time consuming iterative algorithms.

Measuring image resolution in optical nanoscopy.

Resolution in optical nanoscopy (or super-resolution microscopy) depends on the localization uncertainty and density of single fluorescent labels and on the sample's spatial structure. Currently there is no integral, practical resolution measure that accounts for all factors. We introduce a measure based on Fourier ring correlation (FRC) that can be computed directly from an image. We demonstrate its validity and benefits on two-dimensional (2D) and 3D localization microscopy images of tubulin and actin filaments. Our FRC resolution method makes it possible to compare achieved resolutions in images taken with different nanoscopy methods, to optimize and rank different emitter localization and labeling strategies, to define a stopping criterion for data acquisition, to describe image anisotropy and heterogeneity, and even to estimate the average number of localizations per emitter. Our findings challenge the current focus on obtaining the best localization precision, showing instead how the best image resolution can be achieved as fast as possible.

Deconvolution methods for structured illumination microscopy.

We compare two recently developed multiple-frame deconvolution approaches for the reconstruction of structured illumination microscopy (SIM) data: the pattern-illuminated Fourier ptychography algorithm (piFP) and the joint Richardson-Lucy deconvolution (jRL). The quality of the images reconstructed by these methods is compared in terms of the achieved resolution improvement, noise enhancement, and inherent artifacts. Furthermore, we study the issue of object-dependent resolution improvement by considering the modulation transfer functions derived from different types of objects. The performance of the considered methods is tested in experiments and benchmarked with a commercial SIM microscope. We find that the piFP method resolves periodic and isolated structures equally well, whereas the jRL method provides significantly higher resolution for isolated objects compared to periodic ones. Images reconstructed by the piFP and jRL algorithms are comparable to the images reconstructed using the generalized Wiener filter applied in most commercial SIM microscopes. An advantage of the discussed algorithms is that they allow the reconstruction of SIM images acquired under different types of illumination, such as multi-spot or random illumination.

Studying different illumination patterns for resolution improvement in fluorescence microscopy.

Various types of non-uniform illumination can be used for resolution improvement in fluorescence microscopy. Here we study the differences between several types of incoherent illumination patterns, such as multi-spot, line and pseudo-random patterns. This requires an imaging setup and an image reconstruction algorithm that are flexible enough to incorporate any type of illumination pattern. We employ fluorescence microscope with structured illumination generated by a Digital Micro-mirror Device (DMD) and the pattern-illuminated Fourier Ptychography reconstruction algorithm (piFP) to this end. The piFP method is modified and improved by identifying the algorithm as steepest descent optimization of a least squares function. We find that illumination patterns with regular structure are superior to those with irregular structure in terms of resolution enhancement and noise level in the reconstructed images.

Fully automatic evaluation of the corneal endothelium from in vivo confocal microscopy.

BACKGROUND: Manual and semi-automatic analyses of images, acquired in vivo by confocal microscopy, are often used to determine the quality of corneal endothelium in the human eye. These procedures are highly time consuming. Here, we present two fully automatic methods to analyze and quantify corneal endothelium imaged by in vivo white light slit-scanning confocal microscopy. METHODS: In the first approach, endothelial cell density is estimated with the help of spatial frequency analysis. We evaluate published methods, and propose a new, parameter-free method. In the second approach, based on the stochastic watershed, cells are automatically segmented and the result is used to estimate cell density, polymegathism (cell size variability) and pleomorphism (cell shape variation). We show how to determine optimal values for the three parameters of this algorithm, and compare its results to a semi-automatic delineation by a trained observer. RESULTS: The frequency analysis method proposed here is more precise than any published method. The segmentation method outperforms the fully automatic method in the NAVIS software (Nidek Technologies Srl, Padova, Italy), which significantly overestimates the number of cells for cell densities below approximately 1200 mm(-2), as well as previously published methods. CONCLUSIONS: The methods presented here provide a significant improvement over the state of the art, and make in vivo, automated assessment of corneal endothelium more accessible. The segmentation method proposed paves the way to many possible new morphometric parameters, which can quickly and precisely be determined from the segmented image.

Quantifying resolution limiting factors in subtomogram averaged cryo-electron tomography using simulations.

Cryo-electron tomography (CET) is the only available technique capable of characterizing the structure of biological macromolecules in conditions close to the native state. With the advent of subtomogram averaging, as a post-processing step to CET, resolutions in the (sub-) nanometer range have become within reach. In addition to advances in instrumentation and experiments, the reconstruction scheme has improved by inclusion of more accurate contrast transfer function (CTF) correction methods, better defocus estimation, and better alignments of the tilt-series and subtomograms. To quantify the importance of each contribution, we have split the full process from data collection to reconstruction into different steps. For the purpose of evaluation we have acquired tilt-series of ribosomes in such a way that we could precisely determine the defocus of each macromolecule. Then, we simulated tilt-series using the InSilicoTEM package and applied tomogram reconstruction and subtomogram averaging. Through large scale simulations under different conditions and parameter settings we find that tilt-series alignment is the resolution limiting factor for our experimental data. Using simulations, we find that when this alignment inaccuracy is alleviated, tilted CTF correction improves the final resolution, or equivalently, the same resolution can be achieved using less particles. Furthermore, we predict from which resolution onwards better CTF correction and defocus estimation methods are required. We obtain a final average using 3198 ribosomes with a resolution of 2.2nm on the experimental data. Our simulations suggest that with the same number of particles a resolution of 1.2nm could be achieved by improving the tilt-series alignment.

Super-resolution imaging visualizes the eightfold symmetry of gp210 proteins around the nuclear pore complex and resolves the central channel with nanometer resolution.

One of the most complex molecular machines of cells is the nuclear pore complex (NPC), which controls all trafficking of molecules in and out of the nucleus. Because of their importance for cellular processes such as gene expression and cytoskeleton organization, the structure of NPCs has been studied extensively during the last few decades, mainly by electron microscopy. We have used super-resolution imaging by direct stochastic optical reconstruction microscopy (dSTORM) to investigate the structure of NPCs in isolated Xenopus laevis oocyte nuclear envelopes, with a lateral resolution of ~15 nm. By generating accumulated super-resolved images of hundreds of NPCs we determined the diameter of the central NPC channel to be 41 ± 7 nm and demonstrate that the integral membrane protein gp210 is distributed in an eightfold radial symmetry. Two-color dSTORM experiments emphasize the highly symmetric NPCs as ideal model structures to control the quality of corrections to chromatic aberration and to test the capability and reliability of super-resolution imaging methods.

Simultaneous measurement of position and color of single fluorescent emitters using diffractive optics.

We propose a method for simultaneously measuring the position and emission color of single fluorescent emitters based on the use of a large pitch diffraction grating in the emission light path. The grating produces satellite spots adjacent to the main spot; the relative distance between the spots is a measure for the emission wavelength. We present proof-of-principle experiments on beads and mixtures of quantum dots using a spatial light modulator for making a programmable diffraction grating. A wavelength precision of around 10 nm can be achieved for 1000 signal photons and practical background levels, while maintaining a localization precision of around 10 nm.

The lateral and axial localization uncertainty in super-resolution light microscopy.

A study of the uncertainty of localizing single-molecule emitters for super-resolution light microscopy is presented. Maximum likelihood estimation (MLE) is found to be superior to least-squares fitting for low background levels, but the performance difference between the two methods decreases to a few percent for practical background levels. It is shown that the performance limit of MLE, the Cramér-Rao lower bound, is well described by a concise analytical formula with only spot width and signal and background photon count as input parameters. These predictions for the lateral localization uncertainty are compared with the localization error obtained from repeated localizations of the same single-molecule emitter. Agreement within a few percent is found, thus verifying the validity of the fitting model and the concise analytical approximation. The analysis is extended by novel analytical results for the dependence of the axial localization uncertainty on background level for the astigmatic, bifocal, and double-helix methods.

Image formation modeling in cryo-electron microscopy.

Accurate modeling of image formation in cryo-electron microscopy is an important requirement for quantitative image interpretation and optimization of the data acquisition strategy. Here we present a forward model that accounts for the specimen's scattering properties, microscope optics, and detector response. The specimen interaction potential is calculated with the isolated atom superposition approximation (IASA) and extended with the influences of solvent's dielectric and ionic properties as well as the molecular electrostatic distribution. We account for an effective charge redistribution via the Poisson-Boltzmann approach and find that the IASA-based potential forms the dominant part of the interaction potential, as the contribution of the redistribution is less than 10%. The electron wave is propagated through the specimen by a multislice approach and the influence of the optics is included via the contrast transfer function. We incorporate the detective quantum efficiency of the camera due to the difference between signal and noise transfer characteristics, instead of using only the modulation transfer function. The full model was validated against experimental images of 20S proteasome, hemoglobin, and GroEL. The simulations adequately predict the effects of phase contrast, changes due to the integrated electron flux, thickness, inelastic scattering, detective quantum efficiency and acceleration voltage. We suggest that beam-induced specimen movements are relevant in the experiments whereas the influence of the solvent amorphousness can be neglected. All simulation parameters are based on physical principles and, when necessary, experimentally determined.