Multiview confocal super-resolution microscopy.

Confocal microscopy1 remains a major workhorse in biomedical optical microscopy owing to its reliability and flexibility in imaging various samples, but suffers from substantial point spread function anisotropy, diffraction-limited resolution, depth-dependent degradation in scattering samples and volumetric bleaching2. Here we address these problems, enhancing confocal microscopy performance from the sub-micrometre to millimetre spatial scale and the millisecond to hour temporal scale, improving both lateral and axial resolution more than twofold while simultaneously reducing phototoxicity. We achieve these gains using an integrated, four-pronged approach: (1) developing compact line scanners that enable sensitive, rapid, diffraction-limited imaging over large areas; (2) combining line-scanning with multiview imaging, developing reconstruction algorithms that improve resolution isotropy and recover signal otherwise lost to scattering; (3) adapting techniques from structured illumination microscopy, achieving super-resolution imaging in densely labelled, thick samples; (4) synergizing deep learning with these advances, further improving imaging speed, resolution and duration. We demonstrate these capabilities on more than 20 distinct fixed and live samples, including protein distributions in single cells; nuclei and developing neurons in Caenorhabditis elegans embryos, larvae and adults; myoblasts in imaginal disks of Drosophila wings; and mouse renal, oesophageal, cardiac and brain tissues.

Vascular Plugs Improve Pulmonary Arteriovenous Malformation Occlusion over Coil Embolization Alone: A Proof-of-Concept Study using Dual Energy Computed Tomography.

PURPOSE: To evaluate efficacy of pulmonary arteriovenous malformation (PAVM) embolization using dual-energy computed tomography (DECT) and spectral curve analysis by characterizing contrast enhancement and vascular perfusion as a surrogate of the degree of vascular occlusion after embolotherapy. METHODS: Nine consecutive adult patients underwent embolization for 21 PAVMs (size range 0.4-2.0cm; 15/21 simple angioarchitecture) and subsequent post-embolization chest DECT angiography. Twelve PAVMS were treated with vascular plugs ± coils, whereas nine PAVMs were treated with coils-only. Virtual spectral curves (VSC) were generated using dual-energy image post-processing in order to measure embolization effectiveness. RESULTS: Complete occlusion of target PAVM was achieved in all cases on digital subtraction angiography at the end of the embolization procedure. With a median follow-up of 12.7 months, the vascular plug group demonstrated significantly less vascular opacification compared to the coils-only group, as measured by opacification between upstream feeding artery and and different downstream vasculature locations (Δslope1: median 79.1 versus 28.6, p=0.0030; Δslope2: 76.4 versus 28.6, p=0.0197; Δslope3: 78.9 versus 28.6, p=0.0041). Persistence occurred in three PAVMs based on size criteria, which demonstrated higher vascular vascular opacification by DECT (Δslope1: 72 versus 28.6, p=0.253; Δslope2: 65.1 versus 32.7, p=0.326; Δslope3: 72.9 versus 53.5, p=0.733), although statistical significance was not reached. CONCLUSION: Similar to emerging literature, DECT showed improved occlusion in PAVMs treated with vascular plugs compared to coils alone.

A wide-field micro-computed tomography detector: micron resolution at half-centimetre scale.

Ideal three-dimensional imaging of complex samples made up of micron-scale structures extending over mm to cm, such as biological tissues, requires both wide field of view and high resolution. For existing optics and detectors used for micro-CT (computed tomography) imaging, sub-micron pixel resolution can only be achieved for fields of view of <2 mm. This article presents a unique detector system with a 6 mm field-of-view image circle and 0.5 µm pixel size that can be used in micro-CT units utilizing both synchrotron and commercial X-ray sources. A resolution-test pattern with linear microstructures and whole adult Daphnia magna were imaged at beamline 8.3.2 of the Berkeley Advanced Light Source. Volumes of 10000 × 10000 × 7096 isotropic 0.5 µm voxels were reconstructed over a 5.0 mm × 3.5 mm field of view. Measurements in the projection domain confirmed a 0.90 µm measured spatial resolution that is largely Nyquist-limited. This unprecedented combination of field of view and resolution dramatically reduces the need for sectional scans and computational stitching for large samples, ultimately offering the means to elucidate changes in tissue and cellular morphology in the context of larger, whole, intact model organisms and specimens. This system is also anticipated to benefit micro-CT imaging in materials science, microelectronics, agricultural science and biomedical engineering.

The role of spatial embedding in mouse brain networks constructed from diffusion tractography and tracer injections.

Diffusion MRI tractography is the only noninvasive method to measure the structural connectome in humans. However, recent validation studies have revealed limitations of modern tractography approaches, which lead to significant mistracking caused in part by local uncertainties in fiber orientations that accumulate to produce larger errors for longer streamlines. Characterizing the role of this length bias in tractography is complicated by the true underlying contribution of spatial embedding to brain topology. In this work, we compare graphs constructed with ex vivo tractography data in mice and neural tracer data from the Allen Mouse Brain Connectivity Atlas to random geometric surrogate graphs which preserve the low-order distance effects from each modality in order to quantify the role of geometry in various network properties. We find that geometry plays a substantially larger role in determining the topology of graphs produced by tractography than graphs produced by tracers. Tractography underestimates weights at long distances compared to neural tracers, which leads tractography to place network hubs close to the geometric center of the brain, as do corresponding tractography-derived random geometric surrogates, while tracer graphs place hubs further into peripheral areas of the cortex. We also explore the role of spatial embedding in modular structure, network efficiency and other topological measures in both modalities. Throughout, we compare the use of two different tractography streamline node assignment strategies and find that the overall differences between tractography approaches are small relative to the differences between tractography- and tracer-derived graphs. These analyses help quantify geometric biases inherent to tractography and promote the use of geometric benchmarking in future tractography validation efforts.

Multi-modal imaging of a single mouse brain over five orders of magnitude of resolution.

Mammalian neurons operate at length scales spanning six orders of magnitude; they project millimeters to centimeters across brain regions, are composed of micrometer-scale-diameter myelinated axons, and ultimately form nanometer scale synapses. Capturing these anatomical features across that breadth of scale has required imaging samples with multiple independent imaging modalities. Translating between the different modalities, however, requires imaging the same brain with each. Here, we imaged the same postmortem mouse brain over five orders of spatial resolution using MRI, whole brain micrometer-scale synchrotron x-ray tomography (µCT), and large volume automated serial electron microscopy. Using this pipeline, we can track individual myelinated axons previously relegated to axon bundles in diffusion tensor MRI or arbitrarily trace neurons and their processes brain-wide and identify individual synapses on them. This pipeline provides both an unprecedented look across a single brain's multi-scaled organization as well as a vehicle for studying the brain's multi-scale pathologies.

Synchrotron X-ray micro-CT as a validation dataset for diffusion MRI in whole mouse brain.

PURPOSE: To introduce synchrotron X-ray microcomputed tomography (microCT) and demonstrate its use as a natively isotropic, nondestructive, 3D validation modality for diffusion MRI in whole, fixed mouse brain. METHODS: Postmortem diffusion MRI and microCT data were acquired of the same whole mouse brain. Diffusion data were processed using constrained spherical deconvolution. Synchrotron data were acquired at an isotropic voxel size of 1.17 µm. Structure tensor analysis was used to calculate fiber orientation distribution functions from the microCT data. A pipeline was developed to spatially register the 2 datasets in order to perform qualitative comparisons of fiber geometries represented by fiber orientation distribution functions. Fiber orientations from both modalities were used to perform whole-brain deterministic tractography to demonstrate validation of long-range white matter pathways. RESULTS: Fiber orientation distribution functions were able to be extracted throughout the entire microCT dataset, with spatial registration to diffusion MRI simplified due to the whole brain extent of the microCT data. Fiber orientations and tract pathways showed good agreement between modalities. CONCLUSION: Synchrotron microCT is a potentially valuable new tool for future multi-scale diffusion MRI validation studies, providing comparable value to optical histology validation methods while addressing some key limitations in data acquisition and ease of processing.

Spatio-angular fluorescence microscopy III. Constrained angular diffusion, polarized excitation, and high-NA imaging.

We investigate rotational diffusion of fluorescent molecules in angular potential wells, the excitation and subsequent emissions from these diffusing molecules, and the imaging of these emissions with high-NA aplanatic optical microscopes. Although dipole emissions only transmit six low-frequency angular components, we show that angular structured illumination can alias higher-frequency angular components into the passband of the imaging system. We show that the number of measurable angular components is limited by the relationships between three time scales: the rotational diffusion time, the fluorescence decay time, and the acquisition time. We demonstrate our model by simulating a numerical phantom in the limits of fast angular diffusion, slow angular diffusion, and weak potentials.

Spatio-angular fluorescence microscopy I. Basic theory.

We introduce the basic elements of a spatio-angular theory of fluorescence microscopy, providing a unified framework for analyzing systems that image single fluorescent dipoles and ensembles of overlapping dipoles that label biological molecules. We model an aplanatic microscope imaging an ensemble of fluorescent dipoles as a linear Hilbert-space operator, and we show that the operator takes a particularly convenient form when expressed in a basis of complex exponentials and spherical harmonics-a form we call the dipole spatio-angular transfer function. We discuss the implications of our analysis for all quantitative fluorescence microscopy studies and lay out a path toward a complete theory.

Dissection of molecular assembly dynamics by tracking orientation and position of single molecules in live cells.

Regulation of order, such as orientation and conformation, drives the function of most molecular assemblies in living cells but remains difficult to measure accurately through space and time. We built an instantaneous fluorescence polarization microscope, which simultaneously images position and orientation of fluorophores in living cells with single-molecule sensitivity and a time resolution of 100 ms. We developed image acquisition and analysis methods to track single particles that interact with higher-order assemblies of molecules. We tracked the fluctuations in position and orientation of molecules from the level of an ensemble of fluorophores down to single fluorophores. We tested our system in vitro using fluorescently labeled DNA and F-actin, in which the ensemble orientation of polarized fluorescence is known. We then tracked the orientation of sparsely labeled F-actin network at the leading edge of migrating human keratinocytes, revealing the anisotropic distribution of actin filaments relative to the local retrograde flow of the F-actin network. Additionally, we analyzed the position and orientation of septin-GFP molecules incorporated in septin bundles in growing hyphae of a filamentous fungus. Our data indicate that septin-GFP molecules undergo positional fluctuations within ∼350 nm of the binding site and angular fluctuations within ∼30° of the central orientation of the bundle. By reporting position and orientation of molecules while they form dynamic higher-order structures, our approach can provide insights into how micrometer-scale ordered assemblies emerge from nanoscale molecules in living cells.

An early chondrichthyan and the evolutionary assembly of a shark body plan.

Although relationships among the major groups of living gnathostomes are well established, the relatedness of early jawed vertebrates to modern clades is intensely debated. Here, we provide a new description of Gladbachus, a Middle Devonian (Givetian approx. 385-million-year-old) stem chondrichthyan from Germany, and one of the very few early chondrichthyans in which substantial portions of the endoskeleton are preserved. Tomographic and histological techniques reveal new details of the gill skeleton, hyoid arch and jaws, neurocranium, cartilage, scales and teeth. Despite many features resembling placoderm or osteichthyan conditions, phylogenetic analysis confirms Gladbachus as a stem chondrichthyan and corroborates hypotheses that all acanthodians are stem chondrichthyans. The unfamiliar character combination displayed by Gladbachus, alongside conditions observed in acanthodians, implies that pre-Devonian stem chondrichthyans are severely under-sampled and strongly supports indications from isolated scales that the gnathostome crown group originated at the latest by the early Silurian (approx. 440 Ma). Moreover, phylogenetic results highlight the likely convergent evolution of conventional chondrichthyan conditions among earliest members of this primary gnathostome division, while skeletal morphology points towards the likely suspension feeding habits of Gladbachus, suggesting a functional origin of the gill slit condition characteristic of the vast majority of living and fossil chondrichthyans.

Single-fluorophore orientation determination with multiview polarized illumination: modeling and microscope design.

We investigate the use of polarized illumination in multiview microscopes for determining the orientation of single-molecule fluorescence transition dipoles. First, we relate the orientation of single dipoles to measurable intensities in multiview microscopes and develop an information-theoretic metric-the solid-angle uncertainty-to compare the ability of multiview microscopes to estimate the orientation of single dipoles. Next, we compare a broad class of microscopes using this metric-single- and dual-view microscopes with varying illumination polarization, illumination numerical aperture (NA), detection NA, obliquity, asymmetry, and exposure. We find that multi-view microscopes can measure all dipole orientations, while the orientations measurable with single-view microscopes is halved because of symmetries in the detection process. We also find that choosing a small illumination NA and a large detection NA are good design choices, that multiview microscopes can benefit from oblique illumination and detection, and that asymmetric NA microscopes can benefit from exposure asymmetry.

Simulation of Muon Tomography Projections to Image the Pyramids of Giza.

Purpose: A geometric simulation of a possible two-plane detector was developed to test the abilities of the detector to generate high-resolution images of the Great Pyramid using muon tomography. Methods and Materials: Trajectory range, angular resolution, and acceptance of the detector were calculated with a simulation. Trajectories and the corresponding sinogram space covered were simulated first with one detector in one location, and then two moving detectors on adjacent sides of the pyramid. The resolution at the center slice of the pyramid was calculated using the angular resolution of the detector. Results: The simulation returned trajectory range encompassing the pyramid and peak angular resolution of .0004sr. Sinogram space covered by one position was inadequate, however two moving detectors on adjacent sides of the pyramid cover a significant portion. Resolution at the center of the pyramid is roughly 3m. Conclusions: The simulation provides a way to calculate the detector positions needed to cover an adequate amount of sinogram space for high-resolution cosmic-ray tomographic reconstruction of the Great Pyramids.

Dual-energy computed tomography imaging with megavoltage and kilovoltage X-ray spectra.

Purpose: Single-energy computed tomography (CT) often suffers from poor contrast yet remains critical for effective radiotherapy treatment. Modern therapy systems are often equipped with both megavoltage (MV) and kilovoltage (kV) X-ray sources and thus already possess hardware for dual-energy (DE) CT. There is unexplored potential for enhanced image contrast using MV-kV DE-CT in radiotherapy contexts. Approach: A single-line integral toy model was designed for computing basis material signal-to-noise ratio (SNR) using estimation theory. Five dose-matched spectra (3 kV, 2 MV) and three variables were considered: spectral combination, spectral dose allocation, and object material composition. The single-line model was extended to a simulated CT acquisition of an anthropomorphic phantom with and without a metal implant. Basis material sinograms were computed and synthesized into virtual monoenergetic images (VMIs). MV-kV and kV-kV VMIs were compared with single-energy images. Results: The 80 kV-140 kV pair typically yielded the best SNRs, but for bone thicknesses >8 cm, the detunedMV-80 kV pair surpassed it. Peak MV-kV SNR was achieved with ∼90% dose allocated to the MV spectrum. In CT simulations of the pelvis with a steel implant, MV-kV VMIs yielded a higher contrast-to-noise ratio (CNR) than single-energy CT and kV-kV DE-CT. Without steel, the MV-kV VMIs produced higher contrast but lower CNR than single-energy CT. Conclusions: This work analyzes MV-kV DE-CT imaging and assesses its potential advantages. The technique may be used for metal artifact correction and generation of VMIs with higher native contrast than single-energy CT. Improved denoising is generally necessary for greater CNR without metal.

Phase diversity-based wavefront sensing for fluorescence microscopy.

Fluorescence microscopy is an invaluable tool in biology, yet its performance is compromised when the wavefront of light is distorted due to optical imperfections or the refractile nature of the sample. Such optical aberrations can dramatically lower the information content of images by degrading image contrast, resolution, and signal. Adaptive optics (AO) methods can sense and subsequently cancel the aberrated wavefront, but are too complex, inefficient, slow, or expensive for routine adoption by most labs. Here we introduce a rapid, sensitive, and robust wavefront sensing scheme based on phase diversity, a method successfully deployed in astronomy but underused in microscopy. Our method enables accurate wavefront sensing to less than λ/35 root mean square (RMS) error with few measurements, and AO with no additional hardware besides a corrective element. After validating the method with simulations, we demonstrate calibration of a deformable mirror > 100-fold faster than comparable methods (corresponding to wavefront sensing on the ~100 ms scale), and sensing and subsequent correction of severe aberrations (RMS wavefront distortion exceeding λ/2), restoring diffraction-limited imaging on extended biological samples.

Joint Estimation of Metal Density and Attenuation Maps with Pencil Beam XFET.

X-ray fluorescence emission tomography (XFET) is an emerging imaging modality that images the spatial distribution of metal without requiring biochemical modification or radioactivity. This work investigates the joint estimation of metal and attenuation maps with a pencil-beam XFET system that allows for direct metal measurement in the absence of attenuation. Using singular value decomposition on a simplified imaging model, we show that reconstructing metal and attenuation voxels far from the detector is an ill-conditioned problem. Using simulated data, we develop and compare two image reconstruction methods for joint estimation. The first method alternates between updating the attenuation map with a separable paraboloidal surrogates algorithm and updating the metal map with a closed-form solution. The second method performs simultaneous joint estimation with conjugate gradients based on a linearized imaging model. The alternating approach outperforms the linearized approach for iron and gold numerical phantom reconstructions. Reconstructing an (8 cm)3 object containing gold concentrations of 5 mg/cm3 and an unknown beam attenuation map using the alternating approach yields an accurate gold map (NRMSE = 0.19) and attenuation map (NRMSE = 0.14). This simulation demonstrates an accurate joint reconstruction of metal and attenuation maps, from emission data, without previous knowledge of any attenuation map.

Sinogram domain angular upsampling of sparse-view micro-CT with dense residual hierarchical transformer and attention-weighted loss.

Reduced angular sampling is a key strategy for increasing scanning efficiency of micron-scale computed tomography (micro-CT). Despite boosting throughput, this strategy introduces noise and extrapolation artifacts due to undersampling. In this work, we present a solution to this issue, by proposing a novel Dense Residual Hierarchical Transformer (DRHT) network to recover high-quality sinograms from 2×, 4× and 8× undersampled scans. DRHT is trained to utilize limited information available from sparsely angular sampled scans and once trained, it can be applied to recover higher-resolution sinograms from shorter scan sessions. Our proposed DRHT model aggregates the benefits of a hierarchical- multi-scale structure along with the combination of local and global feature extraction through dense residual convolutional blocks and non-overlapping window transformer blocks respectively. We also propose a novel noise-aware loss function named KL-L1 to improve sinogram restoration to full resolution. KL-L1, a weighted combination of pixel-level and distribution-level cost functions, leverages inconsistencies in noise distribution and uses learnable spatial weight maps to improve the training of the DRHT model. We present ablation studies and evaluations of our method against other state-of-the-art (SOTA) models over multiple datasets. Our proposed DRHT network achieves an average increase in peak signal to noise ratio (PSNR) of 17.73 dB and a structural similarity index (SSIM) of 0.161, for 8× upsampling, across the three diverse datasets, compared to their respective Bicubic interpolated versions. This novel approach can be utilized to decrease radiation exposure to patients and reduce imaging time for large-scale CT imaging projects.

Quantitative Geometric Modeling of Blood Cells from X-ray Histotomograms of Whole Zebrafish Larvae.

Tissue phenotyping is foundational to understanding and assessing the cellular aspects of disease in organismal context and an important adjunct to molecular studies in the dissection of gene function, chemical effects, and disease. As a first step toward computational tissue phenotyping, we explore the potential of cellular phenotyping from 3-Dimensional (3D), 0.74 µm isotropic voxel resolution, whole zebrafish larval images derived from X-ray histotomography, a form of micro-CT customized for histopathology. As proof of principle towards computational tissue phenotyping of cells, we created a semi-automated mechanism for the segmentation of blood cells in the vascular spaces of zebrafish larvae, followed by modeling and extraction of quantitative geometric parameters. Manually segmented cells were used to train a random forest classifier for blood cells, enabling the use of a generalized cellular segmentation algorithm for the accurate segmentation of blood cells. These models were used to create an automated data segmentation and analysis pipeline to guide the steps in a 3D workflow including blood cell region prediction, cell boundary extraction, and statistical characterization of 3D geometric and cytological features. We were able to distinguish blood cells at two stages in development (4- and 5-days-post-fertilization) and wild-type vs. polA2 huli hutu ( hht ) mutants. The application of geometric modeling across cell types to and across organisms and sample types may comprise a valuable foundation for computational phenotyping that is more open, informative, rapid, objective, and reproducible.

Deep learning-based aberration compensation improves contrast and resolution in fluorescence microscopy.

Optical aberrations hinder fluorescence microscopy of thick samples, reducing image signal, contrast, and resolution. Here we introduce a deep learning-based strategy for aberration compensation, improving image quality without slowing image acquisition, applying additional dose, or introducing more optics into the imaging path. Our method (i) introduces synthetic aberrations to images acquired on the shallow side of image stacks, making them resemble those acquired deeper into the volume and (ii) trains neural networks to reverse the effect of these aberrations. We use simulations to show that applying the trained 'de-aberration' networks outperforms alternative methods, and subsequently apply the networks to diverse datasets captured with confocal, light-sheet, multi-photon, and super-resolution microscopy. In all cases, the improved quality of the restored data facilitates qualitative image inspection and improves downstream image quantitation, including orientational analysis of blood vessels in mouse tissue and improved membrane and nuclear segmentation in C. elegans embryos.

Three-dimensional structured illumination microscopy with enhanced axial resolution.

The axial resolution of three-dimensional structured illumination microscopy (3D SIM) is limited to ∼300 nm. Here we present two distinct, complementary methods to improve axial resolution in 3D SIM with minimal or no modification to the optical system. We show that placing a mirror directly opposite the sample enables four-beam interference with higher spatial frequency content than 3D SIM illumination, offering near-isotropic imaging with ∼120-nm lateral and 160-nm axial resolution. We also developed a deep learning method achieving ∼120-nm isotropic resolution. This method can be combined with denoising to facilitate volumetric imaging spanning dozens of timepoints. We demonstrate the potential of these advances by imaging a variety of cellular samples, delineating the nanoscale distribution of vimentin and microtubule filaments, observing the relative positions of caveolar coat proteins and lysosomal markers and visualizing cytoskeletal dynamics within T cells in the early stages of immune synapse formation.