Deep Spatio-Temporal Network for Low SNR Cryo-EM Movie Frame Enhancement.

Cryo-EM in single particle analysis is known to have low SNR and requires to utilize several frames of the same particle sample to restore one high-quality image for visualizing that particle. However, the low SNR of cryo-EM movie and motion caused by beam striking make the task very challenging. Video enhancement algorithms in computer vision shed new light on tackling such tasks by utilizing deep neural networks. However, they are designed for natural images with high SNR. Meanwhile, the lack of ground truth in cryo-EM movie seems to be one major limiting factor of the progress. Hence, we present a synthetic cryo-EM movie generation pipeline, which can produce realistic diverse cryo-EM movie datasets with low-SNR movie frames and multiple ground truth values. Then we propose a deep spatio-temporal network (DST-Net) for cryo-EM movie frame enhancement trained on our synthetic data. Spatial and temporal features are first extracted from each frame. Spatio-temporal fusion and high-resolution re-constructor are designed to obtain the enhanced output. For evaluation, we train our model on seven synthetic cryo-EM movie datasets and infer on real cryo-EM data. The experimental results show that DST-Net can achieve better enhancement performance both quantitatively and qualitatively compared with others.

M-Denoiser: Unsupervised image denoising for real-world optical and electron microscopy data.

Real-world microscopy data have a large amount of noise due to the limited light/electron that can be used to capture images. The noise of microscopy data is composed of signal-dependent shot noise and signal-independent read noise, and the Poisson-Gaussian noise model is usually used to describe the noise distribution. Meanwhile, the noise is spatially correlated because of the data acquisition process. Due to the lack of clean ground truth, unsupervised and self-supervised denoising algorithms in computer vision shed new light on tackling such tasks by utilizing paired noisy images or one single noisy image. However, they usually make the assumption that the noise is signal-independent or pixel-wise independent, which contradicts with the actual case. Hence, we propose M-Denoiser for denoising real-world microscopy data in an unsupervised manner. Firstly, the shatter module is used to break the dependency and correlation before denoising. Secondly, a novelly designed unsupervised training loss based on a pair of noisy images is proposed for real-world microscopy data. For evaluation, we train our model on optical and electron microscopy datasets. The experimental results show that M-Denoiser achieves the best performance both quantitatively and qualitatively compared with all the baselines.