Microscope-Cockpit: Python-based bespoke microscopy for bio-medical science.

We have developed "Microscope-Cockpit" (Cockpit), a highly adaptable open source user-friendly Python-based Graphical User Interface (GUI) environment for precision control of both simple and elaborate bespoke microscope systems. The user environment allows next-generation near instantaneous navigation of the entire slide landscape for efficient selection of specimens of interest and automated acquisition without the use of eyepieces. Cockpit uses "Python-Microscope" (Microscope) for high-performance coordinated control of a wide range of hardware devices using open source software. Microscope also controls complex hardware devices such as deformable mirrors for aberration correction and spatial light modulators for structured illumination via abstracted device models. We demonstrate the advantages of the Cockpit platform using several bespoke microscopes, including a simple widefield system and a complex system with adaptive optics and structured illumination. A key strength of Cockpit is its use of Python, which means that any microscope built with Cockpit is ready for future customisation by simply adding new libraries, for example machine learning algorithms to enable automated microscopy decision making while imaging.

Three-dimensional deconvolution processing for STEM cryotomography.

The complex environment of biological cells and tissues has motivated development of three-dimensional (3D) imaging in both light and electron microscopies. To this end, one of the primary tools in fluorescence microscopy is that of computational deconvolution. Wide-field fluorescence images are often corrupted by haze due to out-of-focus light, i.e., to cross-talk between different object planes as represented in the 3D image. Using prior understanding of the image formation mechanism, it is possible to suppress the cross-talk and reassign the unfocused light to its proper source post facto. Electron tomography based on tilted projections also exhibits a cross-talk between distant planes due to the discrete angular sampling and limited tilt range. By use of a suitably synthesized 3D point spread function, we show here that deconvolution leads to similar improvements in volume data reconstructed from cryoscanning transmission electron tomography (CSTET), namely a dramatic in-plane noise reduction and improved representation of features in the axial dimension. Contrast enhancement is demonstrated first with colloidal gold particles and then in representative cryotomograms of intact cells. Deconvolution of CSTET data collected from the periphery of an intact nucleus revealed partially condensed, extended structures in interphase chromatin.

CryoSIM: super-resolution 3D structured illumination cryogenic fluorescence microscopy for correlated ultrastructural imaging.

Rapid cryopreservation of biological specimens is the gold standard for visualizing cellular structures in their true structural context. However, current commercial cryo-fluorescence microscopes are limited to low resolutions. To fill this gap, we have developed cryoSIM, a microscope for 3D super-resolution fluorescence cryo-imaging for correlation with cryo-electron microscopy or cryo-soft X-ray tomography. We provide the full instructions for replicating the instrument mostly from off-the-shelf components and accessible, user-friendly, open-source Python control software. Therefore, cryoSIM democratizes the ability to detect molecules using super-resolution fluorescence imaging of cryopreserved specimens for correlation with their cellular ultrastructure.

High-resolution restoration of 3D structures from widefield images with extreme low signal-to-noise-ratio.

Four-dimensional fluorescence microscopy--which records 3D image information as a function of time--provides an unbiased way of tracking dynamic behavior of subcellular components in living samples and capturing key events in complex macromolecular processes. Unfortunately, the combination of phototoxicity and photobleaching can severely limit the density or duration of sampling, thereby limiting the biological information that can be obtained. Although widefield microscopy provides a very light-efficient way of imaging, obtaining high-quality reconstructions requires deconvolution to remove optical aberrations. Unfortunately, most deconvolution methods perform very poorly at low signal-to-noise ratios, thereby requiring moderate photon doses to obtain acceptable resolution. We present a unique deconvolution method that combines an entropy-based regularization function with kernels that can exploit general spatial characteristics of the fluorescence image to push the required dose to extreme low levels, resulting in an enabling technology for high-resolution in vivo biological imaging.

Effect of depth dependent spherical aberrations in 3D structured illumination microscopy.

We model the effect of depth dependent spherical aberration caused by a refractive index mismatch between the mounting and immersion mediums in a 3D structured illumination microscope (SIM). We first derive a forward model that takes into account the effect of the depth varying aberrations on both the illumination and the detection processes. From the model, we demonstrate that depth dependent spherical aberration leads to loss of signal only due to its effect on the detection response of the system, while its effect on illumination leads to phase shifts between orders that can be handled computationally in the reconstruction process. Further, by using the model, we provide guidelines for optical corrections of aberrations with different complexities, and explain how the proposed corrections simplify the forward model. Finally, we show that it is possible to correct both illumination and detection aberrations using a deformable mirror only on the detection path of the microscope.

A distributed multi-GPU system for high speed electron microscopic tomographic reconstruction.

Full resolution electron microscopic tomographic (EMT) reconstruction of large-scale tilt series requires significant computing power. The desire to perform multiple cycles of iterative reconstruction and realignment dramatically increases the pressing need to improve reconstruction performance. This has motivated us to develop a distributed multi-GPU (graphics processing unit) system to provide the required computing power for rapid constrained, iterative reconstructions of very large three-dimensional (3D) volumes. The participating GPUs reconstruct segments of the volume in parallel, and subsequently, the segments are assembled to form the complete 3D volume. Owing to its power and versatility, the CUDA (NVIDIA, USA) platform was selected for GPU implementation of the EMT reconstruction. For a system containing 10 GPUs provided by 5 GTX295 cards, 10 cycles of SIRT reconstruction for a tomogram of 4096(2) × 512 voxels from an input tilt series containing 122 projection images of 4096(2) pixels (single precision float) takes a total of 1845 s of which 1032 s are for computation with the remainder being the system overhead. The same system takes only 39 s total to reconstruct 1024(2) × 256 voxels from 122 1024(2) pixel projections. While the system overhead is non-trivial, performance analysis indicates that adding extra GPUs to the system would lead to steadily enhanced overall performance. Therefore, this system can be easily expanded to generate superior computing power for very large tomographic reconstructions and especially to empower iterative cycles of reconstruction and realignment.

Evidence of activity-specific, radial organization of mitotic chromosomes in Drosophila.

The organization and the mechanisms of condensation of mitotic chromosomes remain unsolved despite many decades of efforts. The lack of resolution, tight compaction, and the absence of function-specific chromatin labels have been the key technical obstacles. The correlation between DNA sequence composition and its contribution to the chromosome-scale structure has been suggested before; it is unclear though if all DNA sequences equally participate in intra- or inter-chromatin or DNA-protein interactions that lead to formation of mitotic chromosomes and if their mitotic positions are reproduced radially. Using high-resolution fluorescence microscopy of live or minimally perturbed, fixed chromosomes in Drosophila embryonic cultures or tissues expressing MSL3-GFP fusion protein, we studied positioning of specific MSL3-binding sites. Actively transcribed, dosage compensated Drosophila genes are distributed along the euchromatic arm of the male X chromosome. Several novel features of mitotic chromosomes have been observed. MSL3-GFP is always found at the periphery of mitotic chromosomes, suggesting that active, dosage compensated genes are also found at the periphery of mitotic chromosomes. Furthermore, radial distribution of chromatin loci on mitotic chromosomes was found to be correlated with their functional activity as judged by core histone modifications. Histone modifications specific to active chromatin were found peripheral with respect to silent chromatin. MSL3-GFP-labeled chromatin loci become peripheral starting in late prophase. In early prophase, dosage compensated chromatin regions traverse the entire width of chromosomes. These findings suggest large-scale internal rearrangements within chromosomes during the prophase condensation step, arguing against consecutive coiling models. Our results suggest that the organization of mitotic chromosomes is reproducible not only longitudinally, as demonstrated by chromosome-specific banding patterns, but also radially. Specific MSL3-binding sites, the majority of which have been demonstrated earlier to be dosage compensated DNA sequences, located on the X chromosomes, and actively transcribed in interphase, are positioned at the periphery of mitotic chromosomes. This potentially describes a connection between the DNA/protein content of chromatin loci and their contribution to mitotic chromosome structure. Live high-resolution observations of consecutive condensation states in MSL3-GFP expressing cells could provide additional details regarding the condensation mechanisms.

Condensed mitotic chromosome structure at nanometer resolution using PALM and EGFP- histones.

Photoactivated localization microscopy (PALM) and related fluorescent biological imaging methods are capable of providing very high spatial resolutions (up to 20 nm). Two major demands limit its widespread use on biological samples: requirements for photoactivatable/photoconvertible fluorescent molecules, which are sometimes difficult to incorporate, and high background signals from autofluorescence or fluorophores in adjacent focal planes in three-dimensional imaging which reduces PALM resolution significantly. We present here a high-resolution PALM method utilizing conventional EGFP as the photoconvertible fluorophore, improved algorithms to deal with high levels of biological background noise, and apply this to imaging higher order chromatin structure. We found that the emission wavelength of EGFP is efficiently converted from green to red when exposed to blue light in the presence of reduced riboflavin. The photon yield of red-converted EGFP using riboflavin is comparable to other bright photoconvertible fluorescent proteins that allow <20 nm resolution. We further found that image pre-processing using a combination of denoising and deconvolution of the raw PALM images substantially improved the spatial resolution of the reconstruction from noisy images. Performing PALM on Drosophila mitotic chromosomes labeled with H2AvD-EGFP, a histone H2A variant, revealed filamentous components of ∼70 nm. This is the first observation of fine chromatin filaments specific for one histone variant at a resolution approximating that of conventional electron microscope images (10-30 nm). As demonstrated by modeling and experiments on a challenging specimen, the techniques described here facilitate super-resolution fluorescent imaging with common biological samples.

Fast live simultaneous multiwavelength four-dimensional optical microscopy.

Live fluorescence microscopy has the unique capability to probe dynamic processes, linking molecular components and their localization with function. A key goal of microscopy is to increase spatial and temporal resolution while simultaneously permitting identification of multiple specific components. We demonstrate a new microscope platform, OMX, that enables subsecond, multicolor four-dimensional data acquisition and also provides access to subdiffraction structured illumination imaging. Using this platform to image chromosome movement during a complete yeast cell cycle at one 3D image stack per second reveals an unexpected degree of photosensitivity of fluorophore-containing cells. To avoid perturbation of cell division, excitation levels had to be attenuated between 100 and 10,000× below the level normally used for imaging. We show that an image denoising algorithm that exploits redundancy in the image sequence over space and time allows recovery of biological information from the low light level noisy images while maintaining full cell viability with no fading.

A Parallel Product-Convolution approach for representing the depth varying Point Spread Functions in 3D widefield microscopy based on principal component analysis.

We address the problem of computational representation of image formation in 3D widefield fluorescence microscopy with depth varying spherical aberrations. We first represent 3D depth-dependent point spread functions (PSFs) as a weighted sum of basis functions that are obtained by principal component analysis (PCA) of experimental data. This representation is then used to derive an approximating structure that compactly expresses the depth variant response as a sum of few depth invariant convolutions pre-multiplied by a set of 1D depth functions, where the convolving functions are the PCA-derived basis functions. The model offers an efficient and convenient trade-off between complexity and accuracy. For a given number of approximating PSFs, the proposed method results in a much better accuracy than the strata based approximation scheme that is currently used in the literature. In addition to yielding better accuracy, the proposed methods automatically eliminate the noise in the measured PSFs.

Closed loop adaptive optics for microscopy without a wavefront sensor.

A three-dimensional wide-field image of a small fluorescent bead contains more than enough information to accurately calculate the wavefront in the microscope objective back pupil plane using the phase retrieval technique. The phase-retrieved wavefront can then be used to set a deformable mirror to correct the point-spread function (PSF) of the microscope without the use of a wavefront sensor. This technique will be useful for aligning the deformable mirror in a widefield microscope with adaptive optics and could potentially be used to correct aberrations in samples where small fluorescent beads or other point sources are used as reference beacons. Another advantage is the high resolution of the retrieved wavefont as compared with current Shack-Hartmann wavefront sensors. Here we demonstrate effective correction of the PSF in 3 iterations. Starting from a severely aberrated system, we achieve a Strehl ratio of 0.78 and a greater than 10-fold increase in maximum intensity.

Dual-axis target mapping and automated sequential acquisition of dual-axis EM tomographic data.

Dual-axis electron microscopic tomography minimizes the missing wedge-induced resolution loss by taking two complementary tilt data sets of the same target along two orthogonal axes. The potential of this powerful approach has been hampered by the practical challenges inherent in finding the original targets that are dramatically displaced due to non-eucentric specimen rotation. Not only is the manual search for the original targets time consuming and tedious but the added dose during manual searching is uncontrollable. We have developed a hierarchical alignment scheme that allows tomographic data to be collected from an arbitrary number of target sites in one grid orientation and then to find and collect orthogonal data sets with little or no user intervention. Inspired by the successful multi-scale mapping in Leginon, our alignment is performed in three levels to gradually pinpoint the original targets. At the lowest level the grid lattice is used to determine the rotation angle and translational shift resulting from specimen rotation via auto- and cross-correlative analysis of a pair of atlas maps constructed before and after specimen rotation. The target locations are further refined at the next level using a pair of smaller atlas maps. The final refinement of target positions is done by aligning the target contained image tiles. Given the batch processing nature of this hierarchical alignment, multiple targets are initially selected in a group and then sequentially acquired. Upon completion of the data collection on all the targets along the first axis and after specimen rotation, the hierarchical alignment is performed to relocate the original targets. The data collection is then resumed on these targets for the second axis. Therefore, only one specimen rotation is needed for collecting multiple dual-axis tomographic data sets. The experiment of acquiring 20S Proteasomes dual-axis tomographic data sets in vitreous ice at 86,000x CCD magnification on our FEI Tecnai Polara TF30 electron microscope has suggested that the developed scheme is very robust. The extra doses for finding and centering the original targets are almost negligible. This scheme has been integrated into UCSF Tomography software suite that can be downloaded at www.msg.ucsf.edu/tomography free for academic use.

Rapid telomere motions in live human cells analyzed by highly time-resolved microscopy.

BACKGROUND: Telomeres cap chromosome ends and protect the genome. We studied individual telomeres in live human cancer cells. In capturing telomere motions using quantitative imaging to acquire complete high-resolution three-dimensional datasets every second for 200 seconds, telomere dynamics were systematically analyzed. RESULTS: The motility of individual telomeres within the same cancer cell nucleus was widely heterogeneous. One class of internal heterochromatic regions of chromosomes analyzed moved more uniformly and showed less motion and heterogeneity than telomeres. The single telomere analyses in cancer cells revealed that shorter telomeres showed more motion, and the more rapid telomere motions were energy dependent. Experimentally increasing bulk telomere length dampened telomere motion. In contrast, telomere uncapping, but not a DNA damaging agent, methyl methanesulfonate, significantly increased telomere motion. CONCLUSION: New methods for seconds-scale, four-dimensional, live cell microscopic imaging and data analysis, allowing systematic tracking of individual telomeres in live cells, have defined a previously undescribed form of telomere behavior in human cells, in which the degree of telomere motion was dependent upon telomere length and functionality.

Subdiffraction multicolor imaging of the nuclear periphery with 3D structured illumination microscopy.

Fluorescence light microscopy allows multicolor visualization of cellular components with high specificity, but its utility has until recently been constrained by the intrinsic limit of spatial resolution. We applied three-dimensional structured illumination microscopy (3D-SIM) to circumvent this limit and to study the mammalian nucleus. By simultaneously imaging chromatin, nuclear lamina, and the nuclear pore complex (NPC), we observed several features that escape detection by conventional microscopy. We could resolve single NPCs that colocalized with channels in the lamin network and peripheral heterochromatin. We could differentially localize distinct NPC components and detect double-layered invaginations of the nuclear envelope in prophase as previously seen only by electron microscopy. Multicolor 3D-SIM opens new and facile possibilities to analyze subcellular structures beyond the diffraction limit of the emitted light.

N-terminal processing of proteins exported by malaria parasites.

Malaria parasites utilize a short N-terminal amino acid motif termed the Plasmodium export element (PEXEL) to export an array of proteins to the host erythrocyte during blood stage infection. Using immunoaffinity chromatography and mass spectrometry, insight into this signal-mediated trafficking mechanism was gained by discovering that the PEXEL motif is cleaved and N-acetylated. PfHRPII and PfEMP2 are two soluble proteins exported by Plasmodium falciparum that were demonstrated to undergo PEXEL cleavage and N-acetylation, thus indicating that this N-terminal processing may be general to many exported soluble proteins. It was established that PEXEL processing occurs upstream of the brefeldin A-sensitive trafficking step in the P. falciparum secretory pathway, therefore cleavage and N-acetylation of the PEXEL motif occurs in the endoplasmic reticulum (ER) of the parasite. Furthermore, it was shown that the recognition of the processed N-terminus of exported proteins within the parasitophorous vacuole may be crucial for protein transport to the host erythrocyte. It appears that the PEXEL may be defined as a novel ER peptidase cleavage site and a classical N-acetyltransferase substrate sequence.

A presynaptic giant ankyrin stabilizes the NMJ through regulation of presynaptic microtubules and transsynaptic cell adhesion.

In a forward genetic screen for mutations that destabilize the neuromuscular junction, we identified a novel long isoform of Drosophila ankyrin2 (ank2-L). We demonstrate that loss of presynaptic Ank2-L not only causes synapse disassembly and retraction but also disrupts neuronal excitability and NMJ morphology. We provide genetic evidence that ank2-L is necessary to generate the membrane constrictions that normally separate individual synaptic boutons and is necessary to achieve the normal spacing of subsynaptic protein domains, including the normal organization of synaptic cell adhesion molecules. Mechanistically, synapse organization is correlated with a lattice-like organization of Ank2-L, visualized using extended high-resolution structured-illumination microscopy. The stabilizing functions of Ank2-L can be mapped to the extended C-terminal domain that we demonstrate can directly bind and organize synaptic microtubules. We propose that a presynaptic Ank2-L lattice links synaptic membrane proteins and spectrin to the underlying microtubule cytoskeleton to organize and stabilize the presynaptic terminal.

I5S: wide-field light microscopy with 100-nm-scale resolution in three dimensions.

A new type of wide-field fluorescence microscopy is described, which produces 100-nm-scale spatial resolution in all three dimensions, by using structured illumination in a microscope that has two opposing objective lenses. Illumination light is split by a grating and a beam splitter into six mutually coherent beams, three of which enter the specimen through each objective lens. The resulting illumination intensity pattern contains high spatial frequency components both axially and laterally. In addition, the emission is collected by both objective lenses coherently, and combined interferometrically on a single camera, resulting in a detection transfer function with axially extended support. These two effects combine to produce near-isotropic resolution. Experimental images of test samples and biological specimens confirm the theoretical predictions.

Three-dimensional resolution doubling in wide-field fluorescence microscopy by structured illumination.

Structured illumination microscopy is a method that can increase the spatial resolution of wide-field fluorescence microscopy beyond its classical limit by using spatially structured illumination light. Here we describe how this method can be applied in three dimensions to double the axial as well as the lateral resolution, with true optical sectioning. A grating is used to generate three mutually coherent light beams, which interfere in the specimen to form an illumination pattern that varies both laterally and axially. The spatially structured excitation intensity causes normally unreachable high-resolution information to become encoded into the observed images through spatial frequency mixing. This new information is computationally extracted and used to generate a three-dimensional reconstruction with twice as high resolution, in all three dimensions, as is possible in a conventional wide-field microscope. The method has been demonstrated on both test objects and biological specimens, and has produced the first light microscopy images of the synaptonemal complex in which the lateral elements are clearly resolved.

AIDA: an adaptive image deconvolution algorithm with application to multi-frame and three-dimensional data.

We describe an adaptive image deconvolution algorithm (AIDA) for myopic deconvolution of multi-frame and three-dimensional data acquired through astronomical and microscopic imaging. AIDA is a reimplementation and extension of the MISTRAL method developed by Mugnier and co-workers and shown to yield object reconstructions with excellent edge preservation and photometric precision [J. Opt. Soc. Am. A21, 1841 (2004)]. Written in Numerical Python with calls to a robust constrained conjugate gradient method, AIDA has significantly improved run times over the original MISTRAL implementation. Included in AIDA is a scheme to automatically balance maximum-likelihood estimation and object regularization, which significantly decreases the amount of time and effort needed to generate satisfactory reconstructions. We validated AIDA using synthetic data spanning a broad range of signal-to-noise ratios and image types and demonstrated the algorithm to be effective for experimental data from adaptive optics-equipped telescope systems and wide-field microscopy.