A proposed unified mitotic chromosome architecture.

A molecular architecture is proposed for a representative mitotic chromosome, human chromosome 10. This architecture is built on an interphase chromosome structure based on cryo-electron microscopy (cryo-EM) cellular tomography [J. Sedat et al., Proc. Natl. Acad. Sci. U.S.A., in press], thus unifying chromosome structure throughout the complete mitotic cycle. The basic organizational principle for mitotic chromosomes is specific coiling of the 11-nm nucleosome fiber into large scale, ∼200-nm interphase structures, a Slinky [https://en.wikipedia.org/wiki/Slinky; motif cited in S. Bowerman et al., eLife 10, e65587 (2021)], then further modified with subsequent additional coiling for the final mitotic chromosome structure. The final mitotic chromosome architecture accounts for the dimensional values as well as the well-known cytological configurations. In addition, proof is experimentally provided by digital PCR technology that G1 T cell nuclei are diploid with one DNA molecule per chromosome. Many nucleosome linker DNA sequences, the promotors and enhancers, are suggestive of optimal exposure on the surfaces of the large-scale coils.

A proposed unified interphase nucleus chromosome structure: Preliminary preponderance of evidence.

Cryoelectron tomography of the cell nucleus using scanning transmission electron microscopy and deconvolution processing technology has highlighted a large-scale, 100- to 300-nm interphase chromosome structure, which is present throughout the nucleus. This study further documents and analyzes these chromosome structures. The paper is divided into four parts: 1) evidence (preliminary) for a unified interphase chromosome structure; 2) a proposed unified interphase chromosome architecture; 3) organization as chromosome territories (e.g., fitting the 46 human chromosomes into a 10-µm-diameter nucleus); and 4) structure unification into a polytene chromosome architecture and lampbrush chromosomes. Finally, the paper concludes with a living light microscopy cell study showing that the G1 nucleus contains very similar structures throughout. The main finding is that this chromosome structure appears to coil the 11-nm nucleosome fiber into a defined hollow structure, analogous to a Slinky helical spring [https://en.wikipedia.org/wiki/Slinky; motif used in Bowerman et al., eLife 10, e65587 (2021)]. This Slinky architecture can be used to build chromosome territories, extended to the polytene chromosome structure, as well as to the structure of lampbrush chromosomes.

Entropy-regularized deconvolution of cellular cryotransmission electron tomograms.

Cryo-electron tomography (cryo-ET) allows for the high-resolution visualization of biological macromolecules. However, the technique is limited by a low signal-to-noise ratio (SNR) and variance in contrast at different frequencies, as well as reduced Z resolution. Here, we applied entropy-regularized deconvolution (ER-DC) to cryo-ET data generated from transmission electron microscopy (TEM) and reconstructed using weighted back projection (WBP). We applied deconvolution to several in situ cryo-ET datasets and assessed the results by Fourier analysis and subtomogram analysis (STA).

Multiresolution-based weighted regularization for denoised image interpolation from scattered samples with application to confocal microscopy.

The problem of reconstructing an image from nonuniformly spaced, spatial point measurements is frequently encountered in bioimaging and other scientific disciplines. The most successful class of methods in handling this problem uses the regularization approach involving the minimization of a derivative-based roughness functional. It has been well demonstrated, in the presence of noise, that nonquadratic roughness functionals such as ℓ1 measure yield better performance compared to the quadratic ones in inverse problems in general and in deconvolution in particular. However, for the present problem, all well-evaluated methods use quadratic roughness measures; indeed, ℓ1 performs worse than the quadratic roughness when the sampling density is low. This is due to the fact that the mutual incoherence between the measurement operator (dirac-delta) and the regularization operator (derivative) is low in the present problem. Here we develop a new multiresolution-based roughness functional that performs better than ℓ1 and quadratic functionals under a wide range of sampling densities. We also propose an efficient iterative method for minimizing the resulting cost function. We demonstrate the superiority of the proposed regularization functional in the context of reconstructing full images from nonuniformly undersampled data obtained from a confocal microscope.

High-resolution restoration of 3D structures from widefield images with extreme low signal-to-noise-ratio.

Four-dimensional fluorescence microscopy--which records 3D image information as a function of time--provides an unbiased way of tracking dynamic behavior of subcellular components in living samples and capturing key events in complex macromolecular processes. Unfortunately, the combination of phototoxicity and photobleaching can severely limit the density or duration of sampling, thereby limiting the biological information that can be obtained. Although widefield microscopy provides a very light-efficient way of imaging, obtaining high-quality reconstructions requires deconvolution to remove optical aberrations. Unfortunately, most deconvolution methods perform very poorly at low signal-to-noise ratios, thereby requiring moderate photon doses to obtain acceptable resolution. We present a unique deconvolution method that combines an entropy-based regularization function with kernels that can exploit general spatial characteristics of the fluorescence image to push the required dose to extreme low levels, resulting in an enabling technology for high-resolution in vivo biological imaging.

Effect of depth dependent spherical aberrations in 3D structured illumination microscopy.

We model the effect of depth dependent spherical aberration caused by a refractive index mismatch between the mounting and immersion mediums in a 3D structured illumination microscope (SIM). We first derive a forward model that takes into account the effect of the depth varying aberrations on both the illumination and the detection processes. From the model, we demonstrate that depth dependent spherical aberration leads to loss of signal only due to its effect on the detection response of the system, while its effect on illumination leads to phase shifts between orders that can be handled computationally in the reconstruction process. Further, by using the model, we provide guidelines for optical corrections of aberrations with different complexities, and explain how the proposed corrections simplify the forward model. Finally, we show that it is possible to correct both illumination and detection aberrations using a deformable mirror only on the detection path of the microscope.

Fast Roughness Minimizing Image Restoration Under Mixed Poisson-Gaussian Noise.

Image acquisition in many biomedical imaging modalities is corrupted by Poisson noise followed by additive Gaussian noise. While total variation and related regularization methods for solving biomedical inverse problems are known to yield high quality reconstructions in most situations, such methods mostly use log-likelihood of either Gaussian or Poisson noise models, and rarely use mixed Poisson-Gaussian (PG) noise model. There is a recent work which deals with exact PG likelihood and total variation regularization. This method adapts the primal-dual approach involving gradients steps on the PG log-likelihood, with step size limited by the inverse of its Lipschitz constant. This leads to limitations in the convergence speed. Although the alternating direction method of multipliers (ADMM) algorithm does not have such step size restrictions, ADMM has never been applied for this problem, for the possible reason that PG log-likelihood is quite complex. In this paper, we develop an ADMM based optimization for roughness minimizing image restoration under PG log-likelihood. We achieve this by first developing a novel iterative method for computing the proximal solution of PG log-likelihood, deriving the termination conditions for this iterative method, and then integrating into a provably convergent ADMM scheme. We experimentally demonstrate that the proposed method outperform primal-dual method in most of the cases.

A Parallel Product-Convolution approach for representing the depth varying Point Spread Functions in 3D widefield microscopy based on principal component analysis.

We address the problem of computational representation of image formation in 3D widefield fluorescence microscopy with depth varying spherical aberrations. We first represent 3D depth-dependent point spread functions (PSFs) as a weighted sum of basis functions that are obtained by principal component analysis (PCA) of experimental data. This representation is then used to derive an approximating structure that compactly expresses the depth variant response as a sum of few depth invariant convolutions pre-multiplied by a set of 1D depth functions, where the convolving functions are the PCA-derived basis functions. The model offers an efficient and convenient trade-off between complexity and accuracy. For a given number of approximating PSFs, the proposed method results in a much better accuracy than the strata based approximation scheme that is currently used in the literature. In addition to yielding better accuracy, the proposed methods automatically eliminate the noise in the measured PSFs.

Image Restoration by Combined Order Regularization with Optimal Spatial Adaptation.

Total Variation (TV) and related extensions have been popular in image restoration due to their robust performance and wide applicability. While the original formulation is still relevant after two decades of extensive research, its extensions that combine derivatives of first and second orders are now being explored for better performance, with examples being Combined Order TV (COTV) and Total Generalized Variation (TGV). As an improvement over such multi-order convex formulations, we propose a novel non-convex regularization functional which adaptively combines Hessian-Schatten (HS) norm and first order TV (TV1) functionals with spatially varying weight. This adaptive weight itself is controlled by another regularization term; the total cost becomes the sum of this adaptively weighted HS-TV1 term, the regularization term for the adaptive weight, and the data-fitting term. The reconstruction is obtained by jointly minimizing w.r.t. the required image and the adaptive weight. We construct a block coordinate descent method for this minimization with proof of convergence, which alternates between minimization w.r.t. the required image and the adaptive weights. We derive exact computational formula for minimization w.r.t. the adaptive weight, and construct an ADMM algorithm for minimization w.r.t. to the required image. We compare the proposed method with existing regularization methods, and a recently proposed Deep GAN method using image recovery examples including MRI reconstruction and microscopy deconvolution.

Full motion and flow field recovery from echo Doppler data.

We present a new computational method for reconstructing a vector velocity field from scattered, pulsed-wave ultrasound Doppler data. The main difficulty is that the Doppler measurements are incomplete, for they do only capture the velocity component along the beam direction. We thus propose to combine measurements from different beam directions. However, this is not yet sufficient to make the problem well posed because 1) the angle between the directions is typically small and 2) the data is noisy and nonuniformly sampled. We propose to solve this reconstruction problem in the continuous domain using regularization. The reconstruction is formulated as the minimizer of a cost that is a weighted sum of two terms: 1) the sum of squared difference between the Doppler data and the projected velocities 2) a quadratic regularization functional that imposes some smoothness on the velocity field. We express our solution for this minimization problem in a B-spline basis, obtaining a sparse system of equations that can be solved efficiently. Using synthetic phantom data, we demonstrate the significance of tuning the regularization according to the a priori knowledge about the physical property of the motion. Next, we validate our method using real phantom data for which the ground truth is known. We then present reconstruction results obtained from clinical data that originate from 1) blood flow in carotid bifurcation and 2) cardiac wall motion.

Nested Conjugate Gradient Algorithm With Nested Preconditioning for Non-Linear Image Restoration.

We develop a novel optimization algorithm, which we call nested non-linear conjugate gradient (CG) algorithm (NNCG), for image restoration based on quadratic data fitting and smooth non-quadratic regularization. The algorithm is constructed as a nesting of two conjugate gradient iterations. The outer iteration is constructed as a preconditioned non-linear CG algorithm; the preconditioning is performed by the inner CG iteration that is linear. The inner CG iteration, which performs preconditioning for outer CG iteration, itself is accelerated by an another FFT-based non-iterative preconditioner. We prove that the method converges to a stationary point for both convex and non-convex regularization functionals. We demonstrate experimentally that proposed method outperforms the well-known majorization-minimization method used for convex regularization, and a non-convex inertial-proximal method for non-convex regularization functional.

Multiscale motion mapping: a novel computer vision technique for quantitative, objective echocardiographic motion measurement independent of Doppler: first clinical description and validation.

BACKGROUND: Objective, quantitative, segmental noninvasive/bedside measurement of cardiac motion is highly desirable in cardiovascular medicine, but current technology suffers from significant drawbacks, such as subjectivity of conventional echocardiographic reading, angle dependence of tissue Doppler measurements, radiation exposure by computer tomography, and infrastructure requirements in MRI. We hypothesized that computer vision technology could represent a powerful new paradigm for quantification in echocardiography. METHODS AND RESULTS: We present multiscale motion mapping, a novel computer vision technology that is based on mathematical image processing and that exploits echocardiographic information in a fashion similar to the human visual system. It allows Doppler- and border-independent determination of motion and deformation in echocardiograms at arbitrary locations. Correctness of the measurements was documented in synthetic echocardiograms and phantom experiments. Exploratory case studies demonstrated its usefulness in a series of complex motion analyses that included abnormal septal motion and analysis of myocardial twisting. Clinical applicability was shown in a consecutive series of echocardiograms, in which good feasibility, good correlation with expert rating, and good intraobserver and interobserver concordance were documented. Separate assessment of 2D displacement and deformation at the same location was successfully applied to elucidate paradoxical septal motion, a common clinical problem. CONCLUSIONS: This is the first clinical report of multiscale motion mapping, a novel approach to echocardiographic motion quantification. For the first time, full 2D echocardiographic assessment of both motion and deformation is shown to be feasible. Overcoming current limitations, this computer vision-based technique opens a new door to objective analysis of complex heart motion.

Variational image reconstruction from arbitrarily spaced samples: a fast multiresolution spline solution.

We propose a novel method for image reconstruction from nonuniform samples with no constraints on their locations. We adopt a variational approach where the reconstruction is formulated as the minimizer of a cost that is a weighted sum of two terms: (1) the sum of squared errors at the specified points and (2) a quadratic functional that penalizes the lack of smoothness. We search for a solution that is a uniform spline and show how it can be determined by solving a large, sparse system of linear equations. We interpret the solution of our approach as an approximation of the analytical solution that involves radial basis functions and demonstrate the computational advantages of our approach. Using the two-scale relation for B-splines, we derive an algebraic relation that links together the linear systems of equations specifying reconstructions at different levels of resolution. We use this relation to develop a fast multigrid algorithm. We demonstrate the effectiveness of our approach on some image reconstruction examples.

Myocardial motion analysis from B-mode echocardiograms.

The quantitative assessment of cardiac motion is a fundamental concept to evaluate ventricular malfunction. We present a new optical-flow-based method for estimating heart motion from two-dimensional echocardiographic sequences. To account for typical heart motions, such as contraction/expansion and shear, we analyze the images locally by using a local-affine model for the velocity in space and a linear model in time. The regional motion parameters are estimated in the least-squares sense inside a sliding spatiotemporal B-spline window. Robustness and spatial adaptability is achieved by estimating the model parameters at multiple scales within a coarse-to-fine multiresoluion framework. We use a wavelet-like algorithm for computing B-spline-weighted inner products and moments at dyadic scales to increase computational efficiency. In order to characterize myocardial contractility and to simplify the detection of myocardial dysfunction, the radial component of the velocity with respect to a reference point is color coded and visualized inside a time-varying region of interest. The algorithm was first validated on synthetic data sets that simulate a beating heart with a speckle-like appearance of echocardiograms. The ability to estimate motion from real ultrasound sequences was demonstrated by a rotating phantom experiment. The method was also applied to a set of in vivo echocardiograms from an animal study. Motion estimation results were in good agreement with the expert echocardiographic reading.

Multiresolution moment filters: theory and applications.

We introduce local weighted geometric moments that are computed from an image within a sliding window at multiple scales. When the window function satisfies a two-scale relation, we prove that lower order moments can be computed efficiently at dyadic scales by using a multiresolution wavelet-like algorithm. We show that B-splines are well-suited window functions because, in addition to being refinable, they are positive, symmetric, separable, and very nearly isotropic (Gaussian shape). We present three applications of these multiscale local moments. The first is a feature-extraction method for detecting and characterizing elongated structures in images. The second is a noise-reduction method which can be viewed as a multiscale extension of Savitzky-Golay filtering. The third is a multiscale optical-flow algorithm that uses a local affine model for the motion field, extending the Lucas-Kanade optical-flow method. The results obtained in all cases are promising.