Three-dimensional localization microscopy using deep learning.

Single molecule localization microscopy (SMLM) is one of the fastest evolving and most broadly used super-resolving imaging techniques in the biosciences. While image recordings could take up to hours only ten years ago, scientists are now reaching for real-time imaging in order to follow the dynamics of biology. To this end, it is crucial to have data processing strategies available that are capable of handling the vast amounts of data produced by the microscope. In this article, we report on the use of a deep convolutional neural network (CNN) for localizing particles in three dimensions on the basis of single images. In test experiments conducted on fluorescent microbeads, we show that the precision obtained with a CNN can be comparable to that of maximum likelihood estimation (MLE), which is the accepted gold standard. Regarding speed, the CNN performs with about 22k localizations per second more than three orders of magnitude faster than the MLE algorithm of ThunderSTORM. If only five parameters are estimated (3D position, signal and background), our CNN implementation is currently slower than the fastest, recently published GPU-based MLE algorithm. However, in this comparison the CNN catches up with every additional parameter, with only a few percent extra time required per additional dimension. Thus it may become feasible to estimate further variables such as molecule orientation, aberration functions or color. We experimentally demonstrate that jointly estimating Zernike mode magnitudes for aberration modeling can significantly improve the accuracy of the estimates.

Improved localization of cellular membrane receptors using combined fluorescence microscopy and simultaneous topography and recognition imaging.

The combination of fluorescence microscopy and atomic force microscopy has a great potential in single-molecule-detection applications, overcoming many of the limitations coming from each individual technique. Here we present a new platform of combined fluorescence and simultaneous topography and recognition imaging (TREC) for improved localization of cellular receptors. Green fluorescent protein (GFP) labeled human sodium-glucose cotransporter (hSGLT1) expressed Chinese Hamster Ovary (CHO) cells and endothelial cells (MyEnd) from mouse myocardium stained with phalloidin-rhodamine were used as cell systems to study AFM topography and fluorescence microscopy on the same surface area. Topographical AFM images revealed membrane features such as lamellipodia, cytoskeleton fibers, F-actin filaments and small globular structures with heights ranging from 20 to 30 nm. Combined fluorescence and TREC imaging was applied to detect density, distribution and localization of YFP-labeled CD1d molecules on alpha-galactosylceramide (alphaGalCer)-loaded THP1 cells. While the expression level, distribution and localization of CD1d molecules on THP1 cells were detected with fluorescence microscopy, the nanoscale distribution of binding sites was investigated with molecular recognition imaging by using a chemically modified AFM tip. Using TREC on the inverted light microscope, the recognition sites of cell receptors were detected in recognition images with domain sizes ranging from approximately 25 to approximately 160 nm, with the smaller domains corresponding to a single CD1d molecule.

Two-stage focus-hold system for rapid ultra-sensitive read-out of large-area biochips.

We report here the development of a method for holding the focal plane in a fluorescence-based biochip scanner. The fast read-out of large (multiple cm(2)) glass slides as used in modern chip technology imposes severe constraints on the focal system. The limited focal depth of high-NA objectives together with the demand for single-molecule sensitivity challenges traditional focus-hold systems. Various long- and short-term effects disturb the often multiple hour-long data-acquisitioning process and cause blurred or unusable image data. Traditional focus-hold systems were often limited in terms of range, reaction time, sensitivity or demanded a large number of additional components. Our system uses the back-reflected illumination beam always present in total internal reflection fluorescence microscopy to generate an error proportional electrical signal, which in turn drives an actuator correcting the objective-sample distance. The latter consists of a fast but range-limited piezo drive attached to the objective and a slower motor coupled to the microscope's z-drive. With this combination, fast reaction times and virtually unlimited correction distances are possible. We show the applicability by scanning DNA microarrays on 27 x 18-mm(2) glass slides with single-molecule sensitivity over the whole array. Single-fluorescence dyes are imaged as diffraction-limited spots.

Free Brownian motion of individual lipid molecules in biomembranes.

The mobility of phospholipids in free-standing and supported membranes was investigated on the level of individual molecules. For the analysis of trajectories a new statistical treatment was developed that permitted us to clearly distinguish different types of diffusional motion. A freely diffusing subfraction of lipids within supported membranes was identified. Its mobility was characterized by a mean lateral diffusion constant of D(supp) = 4.6 microm(2)/s. In comparison, the mobility of lipids embedded in "free-standing" planar membranes yielded an increase in the mean diffusion constant by a factor of 4.5, D(free) = 20.6 microm(2)/s. This increase is attributed to the ultrathin (

Single-molecule anisotropy imaging.

A novel method, single-molecule anisotropy imaging, has been employed to simultaneously study lateral and rotational diffusion of fluorescence-labeled lipids on supported phospholipid membranes. In a fluid membrane composed of 1-palmitoyl-2-oleoyl-sn-glycero-3-phosphocholine, in which the rotational diffusion time is on the order of the excited-state lifetime of the fluorophore rhodamine, a rotational diffusion constant, D(rot) = 7 x 10(7) rad(2)/s, was determined. The lateral diffusion constant, measured by direct analysis of single-molecule trajectories, was D(lat) = 3.5 x 10(-8) cm(2)/s. As predicted from the free-volume model for diffusion, the results exhibit a significantly enhanced mobility on the nanosecond time scale. For membranes of DPPC lipids in the L(beta) gel phase, the slow rotational mobility permitted the direct observation of the rotation of individual molecules characterized by D(rot) = 1.2 rad(2)/s. The latter data were evaluated by a mean square angular displacement analysis. The technique developed here should prove itself profitable for imaging of conformational motions of individual proteins on the time scale of milliseconds to seconds.

Single molecule fluorescence and force microscopy.

The investigation of biomolecules has entered a new age since the development of methodologies capable of studies at the level of single molecules. In biology, most molecules show a complex dynamical behavior, with individual motions and transitions between different states occurring highly correlated in space and time within an arrangement of various elements. Recent advances in the development of new microscopy techniques with sensitivity at the single molecule have gained access to essentially new types of information obtainable from imaging biomolecular samples. These methodologies are described here in terms of their applicability to the in vivo detection and visualization of molecular processes on surfaces, membranes, and cells. First examples of single molecule microscopy on cell membranes revealed new basic insight into the lateral organization of the plasma membrane, providing the captivating perspective of an ultra-sensitive methodology as a general tool to study local processes and heterogeneities in living cells.

Single-molecule microscopy on model membranes reveals anomalous diffusion.

The lateral mobility of lipids in phospholipid membranes has attracted numerous experimental and theoretical studies, inspired by the model of Singer and Nicholson (1972. Science, 175:720-731) and the theoretical description by Saffman and Delbrück (1975. Proc. Natl. Acad. Sci. USA. 72:3111-3113). Fluorescence recovery after photobleaching (FRAP) is used as the standard experimental technique for the study of lateral mobility, yielding an ensemble-averaged diffusion constant. Single-particle tracking (SPT) and the recently developed single-molecule imaging techniques now give access to data on individual displacements of molecules, which can be used for characterization of the mobility in a membrane. Here we present a new type of analysis for tracking data by making use of the probability distribution of square displacements. The potential of this new type of analysis is shown for single-molecule imaging, which was employed to follow the motion of individual fluorescence-labeled lipids in two systems: a fluid-supported phospholipid membrane and a solid polymerstabilized phospholipid monolayer. In the fluid membrane, a high-mobility component characterized by a diffusion constant of 4.4 microns2/s and a low-mobility component characterized by a diffusion constant of 0.07 micron2/s were identified. It is proposed that the latter characterizes the so-called immobile fraction often found in FRAP experiments. In the polymer-stabilized system, diffusion restricted to corrals of 140 nm was directly visualized. Both examples show the potentials of such detailed analysis in combination with single-molecule techniques: with minimal interference with the native structure, inhomogeneities of membrane mobility can be resolved with a spatial resolution of 100 nm, well below the diffraction limit.

Single molecule microscopy of biomembranes (review).

Recent advances in the development of new microscopy techniques with a sensitivity of a single molecule have gained access to essentially new types of information obtainable from imaging biomolecular samples. These methodologies are analysed here in terms of their applicability to the in vivo visualization of cellular processes on the molecular scale, in particular of processes in cell membranes. First examples of single molecule microscopy on cell membranes revealed new basic insight into the lateral organization of the plasma membrane, providing the captivating perspective of an ultrasensitive methodology as a general tool to study local processes and heterogeneities in living cells.

Direct observation of ligand colocalization on individual receptor molecules.

We have exploited the novel methodology of far-field fluorescence microscopy at the single molecule level to study colocalization of two different ligand molecules on an individual receptor. The use of dual-wavelength single molecule imaging allows discrimination between isolated and colocalized ligands with an accuracy of 40 nm. In the case of very close proximity of the two ligands, below 7 nm, single pair Forster energy-transfer was observed. The latter finding unequivocally demonstrates colocalization of two ligands on an individual receptor.

Statistical analysis of single-molecule colocalization assays.

In chemical assays, specific molecular recognition events result in close physical proximity of two molecular species, e.g., ligands and receptors. Microscopy techniques that are able to image individual molecules allow for achieving a positional accuracy far beyond the resolution limit Therefore, independent position determination, e.g., by dual-color microscopy, becomes possible, permitting determination of intermolecular distances beyond the resolution limit. Nonzero measured distances occur due to experimental inaccuracies in case of a recognition event or due to accidental close proximity between ligand-receptor pairs. Using general statistical considerations, finite measured distances between single ligand-receptor pairs are directly translated into probabilities for true molecular recognition or mere accidental proximity. This enables a quantitative statistical analysis of single recognition events. It is demonstrated that in a general assay, even in the presence of strong unspecific background, the probability for a certain diagnosis and a measure for its reliability can be extracted from the observation of a few binding events. The power of the method is demonstrated at the example of a single-molecule DNA hybridization assay. Our findings are of major importance for future assay miniaturization and assaying with minute amounts of analyte.

Imaging of single molecule diffusion.

In recent years observations at the level of individual atoms and molecules became possible by microscopy and spectroscopy. Imaging of single fluorescence molecules has been achieved but has so far been restricted to molecules in the immobile state. Here we provide methodology for visualization of the motion of individual fluorescent molecules. It is applied to imaging of the diffusional path of single molecules in a phospholipid membrane by using phospholipids carrying one rhodamine dye molecule. For this methodology, fluorescence microscopy was carried to a sensitivity so that single fluorescent molecules illuminated for only 5 ms were resolvable at a signal/noise ratio of 28. Repeated illuminations permitted direct observation of the diffusional motion of individual molecules with a positional accuracy of 30 nm. Such capability has fascinating potentials in bioscience--for example, to correlate biological functions of cell membranes with movements, spatial organization, and stoichiometries of individual components.

Properties of lipid microdomains in a muscle cell membrane visualized by single molecule microscopy.

The lateral motion of single fluorescence labeled lipid molecules was imaged in native cell membranes on a millisecond time scale and with positional accuracy of approximately 50 nm, using 'single dye tracing'. This first application of single molecule microscopy to living cells rendered possible the direct observation of lipid-specific membrane domains. These domains were sensed by a lipid probe with saturated acyl chains as small areas in a liquid-ordered phase: the probe showed confined but fast diffusion, with high partitioning (approximately 100-fold) and long residence time (approximately 13 s). The analogous probe with mono-unsaturated chains diffused predominantly unconfined within the membrane. With approximately 15 saturated probes per domain, the locations, sizes, shapes and motions of individual domains became clearly visible. Domains had a size of 0.7 micrometer (0.2-2 micrometer), covering approximately 13% of total membrane area. Both the liquid-ordered phase characteristics and the sizes of domains match properties of membrane fractions described as detergent-resistant membranes (DRMs), strongly suggesting that the domains seen are the in vivo correlate of DRMs and thus may be identified as lipid rafts.

Detection of individual oligonucleotide pairing by single-molecule microscopy.

Hybridization of 20 mer probe oligonucleotides to complementary, surface-immobilized target oligonucleotides was visualized on a single-molecule basis by fluorescence microscopy. Coincident determination of the positions of both the target and the probe oligonucleotides using dual-wavelength fluorescence labeling allowed for highly reliable discrimination of specifically bound probe molecules from those being physisorbed. The figures of merit of the assay are characterized by the low probability for false positive (10(-4)) events and the high speed for detection of up to hundreds of different DNA fragments per second. The probability for false negative events is limited by the biochemical binding probability of short oligonucleotides. The potentials and limitations of this methodology for single-cell single-DNA analysis are discussed.

Simultaneous optical and electrical recording of single gramicidin channels.

We report here an approach for simultaneous fluorescence imaging and electrical recording of single ion channels in planar bilayer membranes. As a test case, fluorescently labeled (Cy3 and Cy5) gramicidin derivatives were imaged at the single-molecule level using far-field illumination and cooled CCD camera detection. Gramicidin monomers were observed to diffuse in the plane of the membrane with a diffusion coefficient of 3.3 x 10(-8) cm(2)s(-1). Simultaneous electrical recording detected gramicidin homodimer (Cy3/Cy3, Cy5/Cy5) and heterodimer (Cy3/Cy5) channels. Heterodimer formation was observed optically by the appearance of a fluorescence resonance energy transfer (FRET) signal (irradiation of Cy3, detection of Cy5). The number of FRET signals was significantly smaller than the number of Cy3 signals (Cy3 monomers plus Cy3 homodimers) as expected. The number of FRET signals increased with increasing channel activity. In numerous cases the appearance of a FRET signal was observed to correlate with a channel opening event detected electrically. The heterodimers also diffused in the plane of the membrane with a diffusion coefficient of 3.0 x 10(-8) cm(2)s(-1). These experiments demonstrate the feasibility of simultaneous optical and electrical detection of structural changes in single ion channels as well as suggesting strategies for improving the reliability of such measurements.