Accurate and rapid background estimation in single-molecule localization microscopy using the deep neural network BGnet.

Background fluorescence, especially when it exhibits undesired spatial features, is a primary factor for reduced image quality in optical microscopy. Structured background is particularly detrimental when analyzing single-molecule images for 3-dimensional localization microscopy or single-molecule tracking. Here, we introduce BGnet, a deep neural network with a U-net-type architecture, as a general method to rapidly estimate the background underlying the image of a point source with excellent accuracy, even when point-spread function (PSF) engineering is in use to create complex PSF shapes. We trained BGnet to extract the background from images of various PSFs and show that the identification is accurate for a wide range of different interfering background structures constructed from many spatial frequencies. Furthermore, we demonstrate that the obtained background-corrected PSF images, for both simulated and experimental data, lead to a substantial improvement in localization precision. Finally, we verify that structured background estimation with BGnet results in higher quality of superresolution reconstructions of biological structures.

Perspective: Emerging strategies for determining atomic-resolution structures of macromolecular complexes within cells.

In principle, electron cryo-tomography (cryo-ET) of thin portions of cells provides high-resolution images of the three-dimensional spatial arrangement of all members of the proteome. In practice, however, radiation damage creates a tension between recording images at many different tilt angles, but at correspondingly reduced exposure levels, versus limiting the number of tilt angles in order to improve the signal-to-noise ratio (SNR). Either way, it is challenging to read the available information out at the level of atomic structure. Here, we first review work that explores the optimal strategy for data collection, which currently seems to favor the use of a limited angular range for tilting the sample or even the use of a single image to record the high-resolution information. Looking then to the future, we point to the alternative of so-called "deconvolution microscopy", which may be applied to tilt-series or optically-sectioned, focal series data. Recording data as a focal series has the advantage that little or no translational alignment of frames might be needed, and a three-dimensional reconstruction might require only 2/3 the number of images as does standard tomography. We also point to the unexploited potential of phase plates to increase the contrast, and thus to reduce the electron exposure levels while retaining the ability align and merge the data. In turn, using much lower exposures per image could have the advantage that high-resolution information is retained throughout the full data-set, whether recorded as a tilt series or a focal series of images.

Addressing systematic errors in axial distance measurements in single-emitter localization microscopy.

Nanoscale localization of point emitters is critical to several methods in optical fluorescence microscopy, including single-molecule super-resolution imaging and tracking. While the precision of the localization procedure has been the topic of extensive study, localization accuracy has been less emphasized, in part due to the challenge of producing an experimental sample containing unperturbed point emitters at known three-dimensional positions in a relevant geometry. We report a new experimental system which reproduces a widely-adopted geometry in high-numerical aperture localization microscopy, in which molecules are situated in an aqueous medium above a glass coverslip imaged with an oil-immersion objective. We demonstrate a calibration procedure that enables measurement of the depth-dependent point spread function (PSF) for open aperture imaging as well as imaging with engineered PSFs with index mismatch. We reveal the complicated, depth-varying behavior of the focal plane position in this system and discuss the axial localization biases incurred by common approximations of this behavior. We compare our results to theoretical calculations.

Light sheet approaches for improved precision in 3D localization-based super-resolution imaging in mammalian cells [Invited].

The development of imaging techniques beyond the diffraction limit has paved the way for detailed studies of nanostructures and molecular mechanisms in biological systems. Imaging thicker samples, such as mammalian cells and tissue, in all three dimensions, is challenging due to increased background and volumes to image. Light sheet illumination is a method that allows for selective irradiation of the image plane, and its inherent optical sectioning capability allows for imaging of biological samples with reduced background, photobleaching, and photodamage. In this review, we discuss the advantage of combining single-molecule imaging with light sheet illumination. We begin by describing the principles of single-molecule localization microscopy and of light sheet illumination. Finally, we present examples of designs that successfully have married single-molecule super-resolution imaging with light sheet illumination for improved precision in mammalian cells.

Measurement-based estimation of global pupil functions in 3D localization microscopy.

We report the use of a phase retrieval procedure based on maximum likelihood estimation (MLE) to produce an improved, experimentally calibrated model of a point spread function (PSF) for use in three-dimensional (3D) localization microscopy experiments. The method estimates a global pupil phase function (which includes both the PSF and system aberrations) over the full axial range from a simple calibration scan. The pupil function is used to refine the PSF model and hence enable superior localizations from experimental data. To demonstrate the utility of the procedure, we apply it to experimental data acquired with a microscope employing a tetrapod PSF with a 6 µm axial range. The phase-retrieved model demonstrates significant improvements in both accuracy and precision of 3D localizations relative to the model based on scalar diffraction theory. The localization precision of the phase-retrieved model is shown to be near the limits imposed by estimation theory, and the reproducibility of the procedure is characterized and discussed. Code which performs the phase retrieval algorithm is provided.

Modern approaches to improving phase contrast electron microscopy.

Although defocus can be used to generate partial phase contrast in transmission electron microscope images, cryo-electron microscopy (cryo-EM) can be further improved by the development of phase plates which increase contrast by applying a phase shift to the unscattered part of the electron beam. Many approaches have been investigated, including the ponderomotive interaction between light and electrons. We review the recent successes achieved with this method in high-resolution, single-particle cryo-EM. We also review the status of using pulsed or near-field enhanced laser light as alternatives, along with approaches that use scanning transmission electron microscopy (STEM) with a segmented detector rather than a phase plate.

Modern approaches to improving phase contrast electron microscopy.

Although defocus can be used to generate partial phase contrast in transmission electron microscope images, cryo-electron microscopy (cryo-EM) can be further improved by the development of phase plates which increase contrast by applying a phase shift to the unscattered part of the electron beam. Many approaches have been investigated, including the ponderomotive interaction between light and electrons. We review the recent successes achieved with this method in high-resolution, single-particle cryo-EM. We also review the status of using pulsed or near-field enhanced laser light as alternatives, along with approaches that use scanning transmission electron microscopy (STEM) with a segmented detector rather than a phase plate.

Overcoming resolution loss due to thermal magnetic field fluctuations from phase plates in transmission electron microscopy.

We identify thermal magnetic field fluctuations, caused by thermal electron motion ("Johnson noise") in electrically conductive materials, as a potential resolution limit in transmission electron microscopy with a phase plate. Specifically, resolution loss can occur if the electron diffraction pattern is magnified to extend phase contrast to lower spatial frequencies, and if conductive materials are placed too close to the electron beam. While our initial implementation of a laser phase plate (LPP) was significantly affected by these factors, a redesign eliminated the problem and brought the performance close to the expected level. The resolution now appears to be limited by residual Johnson noise arising from the electron beam liner tube in the region of the LPP, together with the chromatic aberration of the relay optics. These two factors can be addressed during future development of the LPP.

Overcoming resolution loss due to thermal magnetic field fluctuations from phase plates in transmission electron microscopy.

We identify thermal magnetic field fluctuations, caused by thermal electron motion ("Johnson noise") in electrically conductive materials, as a potential resolution limit in transmission electron microscopy with a phase plate. Specifically, resolution loss can occur if the electron diffraction pattern is magnified to extend phase contrast to lower spatial frequencies, and if conductive materials are placed too close to the electron beam. While our initial implementation of a laser phase plate (LPP) was significantly affected by these factors, a redesign eliminated the problem and brought the performance close to the expected level. The resolution now appears to be limited by residual Johnson noise arising from the electron beam liner tube in the region of the LPP, together with the chromatic aberration of the relay optics. These two factors can be addressed during future development of the LPP.

Fast and parallel nanoscale three-dimensional tracking of heterogeneous mammalian chromatin dynamics.

Chromatin organization and dynamics are critical for gene regulation. In this work we present a methodology for fast and parallel three-dimensional (3D) tracking of multiple chromosomal loci of choice over many thousands of frames on various timescales. We achieved this by developing and combining fluorogenic and replenishable nanobody arrays, engineered point spread functions, and light sheet illumination. The result is gentle live-cell 3D tracking with excellent spatiotemporal resolution throughout the mammalian cell nucleus. Correction for both sample drift and nuclear translation facilitated accurate long-term tracking of the chromatin dynamics. We demonstrate tracking both of fast dynamics (50 Hz) and over timescales extending to several hours, and we find both large heterogeneity between cells and apparent anisotropy in the dynamics in the axial direction. We further quantify the effect of inhibiting actin polymerization on the dynamics and find an overall increase in both the apparent diffusion coefficient D* and anomalous diffusion exponent α and a transition to more-isotropic dynamics in 3D after such treatment. We think that in the future our methodology will allow researchers to obtain a better fundamental understanding of chromatin dynamics and how it is altered during disease progression and after perturbations of cellular function.

High-power near-concentric Fabry-Perot cavity for phase contrast electron microscopy.

Transmission electron microscopy (TEM) of vitrified biological macromolecules (cryo-EM) is limited by the weak phase contrast signal that is available from such samples. Using a phase plate would thus substantially improve the signal-to-noise ratio. We have previously demonstrated the use of a high-power Fabry-Perot cavity as a phase plate for TEM. We now report improvements to our laser cavity that allow us to achieve record continuous wave intensities of over 450 GW/cm2, sufficient to produce the optimal 90° phase shift for 300 keV electrons. In addition, we have performed the first cryo-EM reconstruction using a laser phase plate, demonstrating that the stability of this laser phase plate is sufficient for use during standard cryo-EM data collection.

Novel fibrillar structure in the inversin compartment of primary cilia revealed by 3D single-molecule superresolution microscopy.

Primary cilia in many cell types contain a periaxonemal subcompartment called the inversin compartment. Four proteins have been found to assemble within the inversin compartment: INVS, ANKS6, NEK8, and NPHP3. The function of the inversin compartment is unknown, but it appears to be critical for normal development, including left-right asymmetry and renal tissue homeostasis. Here we combine superresolution imaging of human RPE1 cells, a classic model for studying primary cilia in vitro, with a genetic dissection of the protein-protein binding relationships that organize compartment assembly to develop a new structural model. We observe that INVS is the core structural determinant of a compartment composed of novel fibril-like substructures, which we identify here by three-dimensional single-molecule superresolution imaging. We find that NEK8 and ANKS6 depend on INVS for localization to these fibrillar assemblies and that ANKS6-NEK8 density within the compartment is regulated by NEK8. Together, NEK8 and ANKS6 are required downstream of INVS to localize and concentrate NPHP3 within the compartment. In the absence of these upstream components, NPHP3 is redistributed within cilia. These results provide a more detailed structure for the inversin compartment and introduce a new example of a membraneless compartment organized by protein-protein interactions.

Accurate phase retrieval of complex 3D point spread functions with deep residual neural networks.

Phase retrieval, i.e., the reconstruction of phase information from intensity information, is a central problem in many optical systems. Imaging the emission from a point source such as a single molecule is one example. Here, we demonstrate that a deep residual neural net is able to quickly and accurately extract the hidden phase for general point spread functions (PSFs) formed by Zernike-type phase modulations. Five slices of the 3D PSF at different focal positions within a two micrometer range around the focus are sufficient to retrieve the first six orders of Zernike coefficients.

3D single-molecule super-resolution microscopy with a tilted light sheet.

Tilted light sheet microscopy with 3D point spread functions (TILT3D) combines a novel, tilted light sheet illumination strategy with long axial range point spread functions (PSFs) for low-background, 3D super-localization of single molecules as well as 3D super-resolution imaging in thick cells. Because the axial positions of the single emitters are encoded in the shape of each single-molecule image rather than in the position or thickness of the light sheet, the light sheet need not be extremely thin. TILT3D is built upon a standard inverted microscope and has minimal custom parts. The result is simple and flexible 3D super-resolution imaging with tens of nm localization precision throughout thick mammalian cells. We validate TILT3D for 3D super-resolution imaging in mammalian cells by imaging mitochondria and the full nuclear lamina using the double-helix PSF for single-molecule detection and the recently developed tetrapod PSFs for fiducial bead tracking and live axial drift correction.

Tilted Light Sheet Microscopy with 3D Point Spread Functions for Single-Molecule Super-Resolution Imaging in Mammalian Cells.

To obtain a complete picture of subcellular nanostructures, cells must be imaged with high resolution in all three dimensions (3D). Here, we present tilted light sheet microscopy with 3D point spread functions (TILT3D), an imaging platform that combines a novel, tilted light sheet illumination strategy with engineered long axial range point spread functions (PSFs) for low-background, 3D super localization of single molecules as well as 3D super-resolution imaging in thick cells. TILT3D is built upon a standard inverted microscope and has minimal custom parts. The axial positions of the single molecules are encoded in the shape of the PSF rather than in the position or thickness of the light sheet, and the light sheet can therefore be formed using simple optics. The result is flexible and user-friendly 3D super-resolution imaging with tens of nm localization precision throughout thick mammalian cells. We validated TILT3D for 3D super-resolution imaging in mammalian cells by imaging mitochondria and the full nuclear lamina using the double-helix PSF for single-molecule detection and the recently developed Tetrapod PSF for fiducial bead tracking and live axial drift correction. We envision TILT3D to become an important tool not only for 3D super-resolution imaging, but also for live whole-cell single-particle and single-molecule tracking.

Observation of live chromatin dynamics in cells via 3D localization microscopy using Tetrapod point spread functions.

We report the observation of chromatin dynamics in living budding yeast (Saccharomyces cerevisiae) cells, in three-dimensions (3D). Using dual color localization microscopy and employing a Tetrapod point spread function, we analyze the spatio-temporal dynamics of two fluorescently labeled DNA loci surrounding the GAL locus. From the measured trajectories, we obtain different dynamical characteristics in terms of inter-loci distance and temporal variance; when the GAL locus is activated, the 3D inter-loci distance and temporal variance increase compared to the inactive state. These changes are visible in spite of the large thermally- and biologically-driven heterogeneity in the relative motion of the two loci. Our observations are consistent with current euchromatin vs. heterochromatin models.

Removing Orientation-Induced Localization Biases in Single-Molecule Microscopy Using a Broadband Metasurface Mask.

Nanoscale localization of single molecules is a crucial function in several advanced microscopy techniques, including single-molecule tracking and wide-field super-resolution imaging 1. To date, a central consideration of such techniques is how to optimize the precision of molecular localization. However, as these methods continue to push toward the nanometre size scale, an increasingly important concern is the localization accuracy. In particular, single fluorescent molecules emit with an anisotropic radiation pattern of an oscillating electric dipole, which can cause significant localization biases using common estimators 2-5. Here we present the theory and experimental demonstration of a solution to this problem based on azimuthal filtering in the Fourier plane of the microscope. We do so using a high efficiency dielectric metasurface polarization/phase device composed of nanoposts with sub-wavelength spacing 6. The method is demonstrated both on fluorophores embedded in a polymer matrix, and in dL5 protein complexes that bind Malachite green 7, 8.