A Physics-Inspired Deep Learning Framework for an Efficient Fourier Ptychographic Microscopy Reconstruction under Low Overlap Conditions.

Two-dimensional observation of biological samples at hundreds of nanometers resolution or even below is of high interest for many sensitive medical applications. Recent advances have been obtained over the last ten years with computational imaging. Among them, Fourier Ptychographic Microscopy is of particular interest because of its important super-resolution factor. In complement to traditional intensity images, phase images are also produced. A large set of N raw images (with typically N = 225) is, however, required because of the reconstruction process that is involved. In this paper, we address the problem of FPM image reconstruction using a few raw images only (here, N = 37) as is highly desirable to increase microscope throughput. In contrast to previous approaches, we develop an algorithmic approach based on a physics-informed optimization deep neural network and statistical reconstruction learning. We demonstrate its efficiency with the help of simulations. The forward microscope image formation model is explicitly introduced in the deep neural network model to optimize its weights starting from an initialization that is based on statistical learning. The simulation results that are presented demonstrate the conceptual benefits of the approach. We show that high-quality images are effectively reconstructed without any appreciable resolution degradation. The learning step is also shown to be mandatory.

Investigating the Joint Amplitude and Phase Imaging of Stained Samples in Automatic Diagnosis.

The diagnosis of many diseases relies, at least on first intention, on an analysis of blood smears acquired with a microscope. However, image quality is often insufficient for the automation of such processing. A promising improvement concerns the acquisition of enriched information on samples. In particular, Quantitative Phase Imaging (QPI) techniques, which allow the digitization of the phase in complement to the intensity, are attracting growing interest. Such imaging allows the exploration of transparent objects not visible in the intensity image using the phase image only. Another direction proposes using stained images to reveal some characteristics of the cells in the intensity image; in this case, the phase information is not exploited. In this paper, we question the interest of using the bi-modal information brought by intensity and phase in a QPI acquisition when the samples are stained. We consider the problem of detecting parasitized red blood cells for diagnosing malaria from stained blood smears using a Deep Neural Network (DNN). Fourier Ptychographic Microscopy (FPM) is used as the computational microscopy framework to produce QPI images. We show that the bi-modal information enhances the detection performance by 4% compared to the intensity image only when the convolution in the DNN is implemented through a complex-based formalism. This proves that the DNN can benefit from the bi-modal enhanced information. We conjecture that these results should extend to other applications processed through QPI acquisition.

Angularly resolved polarization microscopy for birefringent materials with Fourier ptychography.

Polarization light microscopy is a very popular approach for structural imaging in optics. So far these methods mainly probe the sample at a fixed angle of illumination. They are consequently only sensitive to the polarization properties along the microscope optical axis. This paper presents a novel method to resolve angularly the polarization properties of birefringent materials, by retrieving quantitatively the spatial variation of their index ellipsoids. Since this method is based on Fourier ptychography microscopy the latter properties are retrieved with a spatial super-resolution factor. An adequate formalism for the Fourier ptychography forward model is introduced to cope with angularly resolved polarization properties. The inverse problem is solved using an unsupervised deep neural network approach that is proven efficient thanks to its performing regularization properties together with its automatic differentiation. Simulated results are reported showing the feasibility of the methods.

Fourier ptychographic microscopy image enhancement with bi-modal deep learning.

Digital pathology based on a whole slide imaging system is about to permit a major breakthrough in automated diagnosis for rapid and highly sensitive disease detection. High-resolution FPM (Fourier ptychographic microscopy) slide scanners delivering rich information on biological samples are becoming available. They allow new effective data exploitation for efficient automated diagnosis. However, when the sample thickness becomes comparable to or greater than the microscope depth of field, we report an observation of undesirable contrast change of sub-cellular compartments in phase images around the optimal focal plane, reducing their usability. In this article, a bi-modal U-Net artificial neural network (i.e., a two channels U-Net fed with intensity and phase images) is trained to reinforce specifically targeted sub-cellular compartments contrast for both intensity and phase images. The procedure used to construct a reference database is detailed. It is obtained by exploiting the FPM reconstruction algorithm to explore images around the optimal focal plane with virtual Z-stacking calculations and selecting those with adequate contrast and focus. By construction and once trained, the U-Net is able to simultaneously reinforce targeted cell compartment visibility and compensate for any focus imprecision. It is efficient over a large field of view at high resolution. The interest of the approach is illustrated considering the use-case of Plasmodium falciparum detection in blood smear where improvement in the detection sensitivity is demonstrated without degradation of the specificity. Post-reconstruction FPM image processing with such U-Net and its training procedure is general and applicable to demanding biological screening applications.