Image scanning microscopy with a doughnut beam: signal strength and integrated intensity.

We discuss the effects of image scanning microscopy using doughnut beam illumination on the properties of signal strength and integrated intensity. Doughnut beam illumination can give better optical sectioning and background rejection than Airy disk illumination. The outer pixels of a detector array give a signal from defocused regions, so digital processing of these (e.g., by simple subtraction) can further improve optical sectioning and background rejection from a single in-focus scan.

Signal strength and integrated intensity in confocal and image scanning microscopy.

The properties of signal strength and integrated intensity in a scanned imaging system are reviewed. These properties are especially applied to confocal imaging systems, including image scanning microscopy. The integrated intensity, equal to the image of a uniform planar (sheet) object, rather than the peak of the point spread function, is a measure of the flux in an image. Analytic expressions are presented for the intensity in the detector plane for a uniform volume object, and for the resulting background. The variation in the integrated intensity with defocus for an offset point detector is presented. This axial fingerprint is independent of any pixel reassignment. The intensity in the detector plane is shown to contain the defocus information, and simple processing of the recorded data can improve optical sectioning and background rejection.

A robust and versatile platform for image scanning microscopy enabling super-resolution FLIM.

Image scanning microscopy (ISM) can improve the effective spatial resolution of confocal microscopy to its theoretical limit. However, current implementations are not robust or versatile, and are incompatible with fluorescence lifetime imaging (FLIM). We describe an implementation of ISM based on a single-photon detector array that enables super-resolution FLIM and improves multicolor, live-cell and in-depth imaging, thereby paving the way for a massive transition from confocal microscopy to ISM.

Pixel reassignment in image scanning microscopy with a doughnut beam: example of maximum likelihood restoration.

In image scanning microscopy, the pinhole of a confocal microscope is replaced by a detector array. The point spread function for each detector element can be interpreted as the probability density function of the signal, the peak giving the most likely origin. This thus allows a form of maximum likelihood restoration, and compensation for aberrations, with similarities to adaptive optics. As an example of an aberration, we investigate theoretically and experimentally illumination with a vortex doughnut beam. After reassignment and summation over the detector array, the point spread function is compact, and the resolution and signal level higher than in a conventional microscope.

Pixel reassignment in image scanning microscopy: a re-evaluation.

Image scanning microscopy is a technique based on confocal microscopy, in which the confocal pinhole is replaced by a detector array, and the resulting image is reconstructed, usually by the process of pixel reassignment. The detector array collects most of the fluorescent light, so the signal-to-noise ratio is much improved compared with confocal microscopy with a small pinhole, while the resolution is improved compared with conventional (wide-field) microscopy. In previous studies, it has usually been assumed that pixels should be reassigned by a constant factor, to a point midway between the illumination and detection spots. Here it is shown that the peak intensity of the effective point spread function (PSF) can be further increased by 4% by a new choice of the pixel reassignment factor. For an array of two Airy units, the peak of the effective PSF is 1.90 times that of a conventional microscope, and the transverse resolution is 1.53 times better. It is confirmed that image scanning microscopy gives optical sectioning strength identical to that of a confocal microscope with a pinhole equal to the size of the detector array. However, it is shown that image scanning microscopy exhibits axial resolution superior to a confocal microscope with a pinhole the same size as the detector array. For a two-Airy-unit array, the axial resolution is 1.34 times better than in a conventional microscope for the standard reassignment factor, and 1.28 times better for the new reassignment factor. The axial resolution of a confocal microscope with a two-Airy-unit pinhole is only 1.04 times better than conventional microscopy. We also examine the signal-to-noise ratio of a point object in a uniform background (called the detectability), and show that it is 1.6 times higher than in a confocal microscope.

Image scanning microscopy with multiphoton excitation or Bessel beam illumination.

Image scanning microscopy is a technique of confocal microscopy in which the confocal pinhole is replaced by a detector array, and the image is reconstructed most straightforwardly by pixel reassignment. In the fluorescence mode, the detector array collects most of the fluorescent light, so the signal-to-noise ratio is much improved compared with confocal microscopy with a small pinhole, while the resolution is improved compared with conventional fluorescence microscopy. Here we consider two cases in which the illumination and detection point spread functions are dissimilar: illumination with a Bessel beam and multiphoton microscopy. It has been shown previously that for Bessel beam illumination in image scanning microscopy with a large array, the imaging performance is degraded. On the other hand, it is also known that the resolution of confocal microscopy is improved by Bessel beam illumination. Here we analyze image scanning microscopy with Bessel beam illumination together with a small array and show that an improvement in transverse resolution (width of the point spread function) by a factor of 1.78 compared with a conventional fluorescence microscope can be obtained. We also examine the behavior of image scanning microscopy in two- or three-photon fluorescence and for two-photon excitation also with Bessel beam illumination. The combination of the optical sectioning effect of image scanning microscopy with multiphoton microscopy reduces background from the sample surface, which can increase penetration depth. For a detector array size of two Airy units, the resolution of two-photon image scanning microscopy is a factor 1.85 better and the peak of the point spread function 2.84 times higher than in nonconfocal two-photon fluorescence. The resolution of three-photon image scanning microscopy is a factor 2.10 better, and the peak of the point spread function is 3.77 times higher than in nonconfocal three-photon fluorescence. The resolution of two-photon image scanning microscopy with Bessel beam illumination is a factor 2.13 better than in standard two-photon fluorescence. Axial resolution and optical sectioning in two-photon or three-photon fluorescence are also improved by using the image scanning modality.

Expressions for parallel decomposition of the Mueller matrix: erratum.

An error in our paper [J. Opt. Soc. Am. A33, 741 (2016)JOAOD60740-323210.1364/JOSAA.33.000741] is pointed out.

Image formation in image scanning microscopy, including the case of two-photon excitation.

The effect of combining the image scanning microscopy (ISM) technique with two-photon fluorescence microscopy is analyzed. The effective spatial frequency cutoff can be doubled, as compared with conventional two-photon fluorescence microscopy, and the magnitude of the optical transfer function near the cutoff of conventional two-photon microscopy is increased by orders of magnitude. For the two-photon case, it is found that the optimum pixel reassignment factor in ISM is not equal to one half, as is often assumed in single-photon fluoresence image scanning microscopy, because the excitation and detection point spread functions are different. The optimum reassignment factor depends on the noise level, and in general the useful cutoff spatial frequency is about 1.8 times that for conventional two-photon microscopy. The effect of altering the reassignment factor in single-photon fluorescence ISM with a Stokes shift is also investigated. Illumination using pupil filters, such as by a Bessel beam, is considered. Using a ring detector array is found to result in good imaging behavior, exhibiting a sharpening of the point spread function by a factor of 1.7 compared with conventional fluorescence. Image formation in ISM can be considered in a four-dimensional spatial frequency space, giving new insight into the imaging properties. This approach is related to phase space representations such as the Wigner distribution function and the ambiguity function. A noniterative algorithm for image restoration is proposed.

Interpretation of the optical transfer function: Significance for image scanning microscopy.

The optical transfer function (OTF) is widely used to compare the performance of different optical systems. Conventionally, the OTF is normalized to unity for zero spatial frequency, but in some cases it is better to consider the unnormalized OTF, which gives the absolute value of the image signal. Examples are in confocal microscopy and image scanning microscopy, where the signal level increases with pinhole or array size. Comparison of the respective unnormalized OTFs gives useful insight into their relative performance. The significance of other properties of the general OTF is discussed.

Expressions for parallel decomposition of the Mueller matrix.

It is useful to convert between the Mueller matrix and two different Hermitian matrices, representing an optical material or system. We introduce forms for the matrices for transforming between the column vector forms of these different matrices. A review of matrix algebra is presented. We find that there is no great advantage, from the point of view of matrix manipulation, in using quantum mechanics ordering rather than the optical ordering of the Stokes parameters, as has been claimed elsewhere.

Three-dimensional polarization algebra.

If light is focused or collected with a high numerical aperture lens, as may occur in imaging and optical encryption applications, polarization should be considered in three dimensions (3D). The matrix algebra of polarization behavior in 3D is discussed. It is useful to convert between the Mueller matrix and two different Hermitian matrices, representing an optical material or system, which are in the literature. Explicit transformation matrices for converting the column vector form of these different matrices are extended to the 3D case, where they are large (81×81) but can be generated using simple rules. It is found that there is some advantage in using a generalization of the Chandrasekhar phase matrix treatment, rather than that based on Gell-Mann matrices, as the resultant matrices are of simpler form and reduce to the two-dimensional case more easily. Explicit expressions are given for 3D complex field components in terms of Chandrasekhar-Stokes parameters.

Image scanning microscopy with a quadrant detector.

Confocal scanning microscopy (CSM) is the most widely used modern optical microscopy technique. Theoretically, it allows the diffraction barrier to be surpassed by a factor of 2, but practically this improvement is sacrificed to obtain a good signal-to-noise ratio (SNR). Image scanning microscopy (ISM) solves this limitation but, in the current implementations, the system complexity is increased and the versatility of CSM is reduced. Here we show that ISM can be straightforwardly implemented by substituting the single point detector of a confocal microscope with a quadrant detector of the same size, thus using a small number of detector elements. This implementation offers resolution close to the CSM theoretical value and improves the SNR by a factor of 1.5 with respect to the CSM counterpart without losing the optical sectioning capability and the system versatility.

On the Advent of Super-Resolution Microscopy in the Realm of Polycomb Proteins.

The genomes of metazoans are organized at multiple spatial scales, ranging from the double helix of DNA to whole chromosomes. The intermediate genomic scale of kilobases to megabases, which corresponds to the 50-300 nm spatial scale, is particularly interesting, as the 3D arrangement of chromatin is implicated in multiple regulatory mechanisms. In this context, polycomb group (PcG) proteins stand as major epigenetic modulators of chromatin function, acting prevalently as repressors of gene transcription by combining chemical modifications of target histones with physical crosslinking of distal genomic regions and phase separation. The recent development of super-resolution microscopy (SRM) has strongly contributed to improving our comprehension of several aspects of nano-/mesoscale (10-200 nm) chromatin domains. Here, we review the current state-of-the-art SRM applied to PcG proteins, showing that the application of SRM to PcG activity and organization is still quite limited and mainly focused on the 3D assembly of PcG-controlled genomic loci. In this context, SRM approaches have mostly been applied to multilabel fluorescence in situ hybridization (FISH). However, SRM data have complemented the maps obtained from chromosome capture experiments and have opened a new window to observe how 3D chromatin topology is modulated by PcGs.

Focus image scanning microscopy for sharp and gentle super-resolved microscopy.

To date, the feasibility of super-resolution microscopy for imaging live and thick samples is still limited. Stimulated emission depletion (STED) microscopy requires high-intensity illumination to achieve sub-diffraction resolution, potentially introducing photodamage to live specimens. Moreover, the out-of-focus background may degrade the signal stemming from the focal plane. Here, we propose a new method to mitigate these limitations without drawbacks. First, we enhance a STED microscope with a detector array, enabling image scanning microscopy (ISM). Therefore, we implement STED-ISM, a method that exploits the working principle of ISM to reduce the depletion intensity and achieve a target resolution. Later, we develop Focus-ISM, a strategy to improve the optical sectioning and remove the background of any ISM-based imaging technique, with or without a STED beam. The proposed approach requires minimal architectural changes to a conventional microscope but provides substantial advantages for live and thick sample imaging.

Confocal-based fluorescence fluctuation spectroscopy with a SPAD array detector.

The combination of confocal laser-scanning microscopy (CLSM) and fluorescence fluctuation spectroscopy (FFS) is a powerful tool in studying fast, sub-resolution biomolecular processes in living cells. A detector array can further enhance CLSM-based FFS techniques, as it allows the simultaneous acquisition of several samples-essentially images-of the CLSM detection volume. However, the detector arrays that have previously been proposed for this purpose require tedious data corrections and preclude the combination of FFS with single-photon techniques, such as fluorescence lifetime imaging. Here, we solve these limitations by integrating a novel single-photon-avalanche-diode (SPAD) array detector in a CLSM system. We validate this new implementation on a series of FFS analyses: spot-variation fluorescence correlation spectroscopy, pair-correlation function analysis, and image-derived mean squared displacement analysis. We predict that the unique combination of spatial and temporal information provided by our detector will make the proposed architecture the method of choice for CLSM-based FFS.

Two-photon image-scanning microscopy with SPAD array and blind image reconstruction.

Two-photon excitation (2PE) laser scanning microscopy is the imaging modality of choice when one desires to work with thick biological samples. However, its spatial resolution is poor, below confocal laser scanning microscopy. Here, we propose a straightforward implementation of 2PE image scanning microscopy (2PE-ISM) that, by leveraging our recently introduced single-photon avalanche diode (SPAD) array detector and a novel blind image reconstruction method, is shown to enhance the effective resolution, as well as the overall image quality of 2PE microscopy. With our adaptive pixel reassignment procedure ∼1.6 times resolution increase is maintained deep into thick semi-transparent samples. The integration of Fourier ring correlation based semi-blind deconvolution is shown to further enhance the effective resolution by a factor of ∼2 - and automatic background correction is shown to boost the image quality especially in noisy images. Most importantly, our 2PE-ISM implementation requires no calibration measurements or other input from the user, which is an important aspect in terms of day-to-day usability of the technique.

Fourier ring correlation simplifies image restoration in fluorescence microscopy.

Fourier ring correlation (FRC) has recently gained popularity among fluorescence microscopists as a straightforward and objective method to measure the effective image resolution. While the knowledge of the numeric resolution value is helpful in e.g., interpreting imaging results, much more practical use can be made of FRC analysis-in this article we propose blind image restoration methods enabled by it. We apply FRC to perform image de-noising by frequency domain filtering. We propose novel blind linear and non-linear image deconvolution methods that use FRC to estimate the effective point-spread-function, directly from the images. We show how FRC can be used as a powerful metric to observe the progress of iterative deconvolution. We also address two important limitations in FRC that may be of more general interest: how to make FRC work with single images (within certain practical limits) and with three-dimensional images with highly anisotropic resolution.

Smart scanning for low-illumination and fast RESOLFT nanoscopy in vivo.

RESOLFT fluorescence nanoscopy can nowadays image details far beyond the diffraction limit. However, signal to noise ratio (SNR) and temporal resolution are still a concern, especially deep inside living cells and organisms. In this work, we developed a non-deterministic scanning approach based on a real-time feedback system which speeds up the acquisition up to 6-fold and decreases the light dose by 70-90% for in vivo imaging. Also, we extended the information content of the images by acquiring the complete temporal evolution of the fluorescence generated by reversible switchable fluorescent proteins. This generates a series of images with different spatial resolution and SNR, from conventional to RESOLFT images, which combined through a multi-image deconvolution algorithm further enhances the effective resolution. We reported nanoscale imaging of organelles up to 35 Hz and actin dynamics during an invasion process at a depth of 20-30 µm inside a living Caenorhabditis elegans worm.

Efficient two-photon excitation stimulated emission depletion nanoscope exploiting spatiotemporal information.

Stimulated emission depletion (STED) microscopy is a powerful bioimaging technique that theoretically provides molecular spatial resolution while preserving the most important assets of fluorescence microscopy. When combined with two-photon excitation (2PE) microscopy (2PE-STED), subdiffraction resolution may be achieved for thick biological samples. The most straightforward implementation of 2PE-STED microscopy entails introduction of an STED beam operating in continuous wave (CW) into a conventional Ti:sapphire-based 2PE microscope (2PE CW-STED). In this implementation, resolution enhancement is typically achieved using time-gated detection schemes, often resulting in drastic signal-to-noise/-background ratio (SNR/SBR) reductions. Herein, we employ a pixel-by-pixel phasor approach to discard fluorescence photons lacking super-resolution information to enhance image SNR/SBR in 2PE CW-STED microscopy. We compare this separation of photons by lifetime tuning approach against other postprocessing algorithms and combine it with image deconvolution to further optimize image quality.

Machine learning approach for single molecule localisation microscopy.

Single molecule localisation (SML) microscopy is a fundamental tool for biological discoveries; it provides sub-diffraction spatial resolution images by detecting and localizing "all" the fluorescent molecules labeling the structure of interest. For this reason, the effective resolution of SML microscopy strictly depends on the algorithm used to detect and localize the single molecules from the series of microscopy frames. To adapt to the different imaging conditions that can occur in a SML experiment, all current localisation algorithms request, from the microscopy users, the choice of different parameters. This choice is not always easy and their wrong selection can lead to poor performance. Here we overcome this weakness with the use of machine learning. We propose a parameter-free pipeline for SML learning based on support vector machine (SVM). This strategy requires a short supervised training that consists in selecting by the user few fluorescent molecules (∼ 10-20) from the frames under analysis. The algorithm has been extensively tested on both synthetic and real acquisitions. Results are qualitatively and quantitatively consistent with the state of the art in SML microscopy and demonstrate that the introduction of machine learning can lead to a new class of algorithms competitive and conceived from the user point of view.