A Physics-Inspired Deep Learning Framework for an Efficient Fourier Ptychographic Microscopy Reconstruction under Low Overlap Conditions.

Two-dimensional observation of biological samples at hundreds of nanometers resolution or even below is of high interest for many sensitive medical applications. Recent advances have been obtained over the last ten years with computational imaging. Among them, Fourier Ptychographic Microscopy is of particular interest because of its important super-resolution factor. In complement to traditional intensity images, phase images are also produced. A large set of N raw images (with typically N = 225) is, however, required because of the reconstruction process that is involved. In this paper, we address the problem of FPM image reconstruction using a few raw images only (here, N = 37) as is highly desirable to increase microscope throughput. In contrast to previous approaches, we develop an algorithmic approach based on a physics-informed optimization deep neural network and statistical reconstruction learning. We demonstrate its efficiency with the help of simulations. The forward microscope image formation model is explicitly introduced in the deep neural network model to optimize its weights starting from an initialization that is based on statistical learning. The simulation results that are presented demonstrate the conceptual benefits of the approach. We show that high-quality images are effectively reconstructed without any appreciable resolution degradation. The learning step is also shown to be mandatory.

Investigating the Joint Amplitude and Phase Imaging of Stained Samples in Automatic Diagnosis.

The diagnosis of many diseases relies, at least on first intention, on an analysis of blood smears acquired with a microscope. However, image quality is often insufficient for the automation of such processing. A promising improvement concerns the acquisition of enriched information on samples. In particular, Quantitative Phase Imaging (QPI) techniques, which allow the digitization of the phase in complement to the intensity, are attracting growing interest. Such imaging allows the exploration of transparent objects not visible in the intensity image using the phase image only. Another direction proposes using stained images to reveal some characteristics of the cells in the intensity image; in this case, the phase information is not exploited. In this paper, we question the interest of using the bi-modal information brought by intensity and phase in a QPI acquisition when the samples are stained. We consider the problem of detecting parasitized red blood cells for diagnosing malaria from stained blood smears using a Deep Neural Network (DNN). Fourier Ptychographic Microscopy (FPM) is used as the computational microscopy framework to produce QPI images. We show that the bi-modal information enhances the detection performance by 4% compared to the intensity image only when the convolution in the DNN is implemented through a complex-based formalism. This proves that the DNN can benefit from the bi-modal enhanced information. We conjecture that these results should extend to other applications processed through QPI acquisition.

Angularly resolved polarization microscopy for birefringent materials with Fourier ptychography.

Polarization light microscopy is a very popular approach for structural imaging in optics. So far these methods mainly probe the sample at a fixed angle of illumination. They are consequently only sensitive to the polarization properties along the microscope optical axis. This paper presents a novel method to resolve angularly the polarization properties of birefringent materials, by retrieving quantitatively the spatial variation of their index ellipsoids. Since this method is based on Fourier ptychography microscopy the latter properties are retrieved with a spatial super-resolution factor. An adequate formalism for the Fourier ptychography forward model is introduced to cope with angularly resolved polarization properties. The inverse problem is solved using an unsupervised deep neural network approach that is proven efficient thanks to its performing regularization properties together with its automatic differentiation. Simulated results are reported showing the feasibility of the methods.

A Comparative Study of Functional Connectivity Measures for Brain Network Analysis in the Context of AD Detection with EEG.

This work addresses brain network analysis considering different clinical severity stages of cognitive dysfunction, based on resting-state electroencephalography (EEG). We use a cohort acquired in real-life clinical conditions, which contains EEG data of subjective cognitive impairment (SCI) patients, mild cognitive impairment (MCI) patients, and Alzheimer's disease (AD) patients. We propose to exploit an epoch-based entropy measure to quantify the connectivity links in the networks. This entropy measure relies on a refined statistical modeling of EEG signals with Hidden Markov Models, which allow a better estimation of the spatiotemporal characteristics of EEG signals. We also propose to conduct a comparative study by considering three other measures largely used in the literature: phase lag index, coherence, and mutual information. We calculated such measures at different frequency bands and computed different local graph parameters considering different proportional threshold values for a binary network analysis. After applying a feature selection procedure to determine the most relevant features for classification performance with a linear Support Vector Machine algorithm, our study demonstrates the effectiveness of the statistical entropy measure for analyzing the brain network in patients with different stages of cognitive dysfunction.

Fourier ptychographic microscopy image enhancement with bi-modal deep learning.

Digital pathology based on a whole slide imaging system is about to permit a major breakthrough in automated diagnosis for rapid and highly sensitive disease detection. High-resolution FPM (Fourier ptychographic microscopy) slide scanners delivering rich information on biological samples are becoming available. They allow new effective data exploitation for efficient automated diagnosis. However, when the sample thickness becomes comparable to or greater than the microscope depth of field, we report an observation of undesirable contrast change of sub-cellular compartments in phase images around the optimal focal plane, reducing their usability. In this article, a bi-modal U-Net artificial neural network (i.e., a two channels U-Net fed with intensity and phase images) is trained to reinforce specifically targeted sub-cellular compartments contrast for both intensity and phase images. The procedure used to construct a reference database is detailed. It is obtained by exploiting the FPM reconstruction algorithm to explore images around the optimal focal plane with virtual Z-stacking calculations and selecting those with adequate contrast and focus. By construction and once trained, the U-Net is able to simultaneously reinforce targeted cell compartment visibility and compensate for any focus imprecision. It is efficient over a large field of view at high resolution. The interest of the approach is illustrated considering the use-case of Plasmodium falciparum detection in blood smear where improvement in the detection sensitivity is demonstrated without degradation of the specificity. Post-reconstruction FPM image processing with such U-Net and its training procedure is general and applicable to demanding biological screening applications.

Weighted Brain Network Analysis on Different Stages of Clinical Cognitive Decline.

This study addresses brain network analysis over different clinical severity stages of cognitive dysfunction using electroencephalography (EEG). We exploit EEG data of subjective cognitive impairment (SCI) patients, mild cognitive impairment (MCI) patients and Alzheimer's disease (AD) patients. We propose a new framework to study the topological networks with a spatiotemporal entropy measure for estimating the connectivity. Our results show that functional connectivity and graph analysis are frequency-band dependent, and alterations start at the MCI stage. In delta, the SCI group exhibited a decrease of clustering coefficient and an increase of path length compared to MCI and AD. In alpha, the opposite behavior appeared, suggesting a rapid and high efficiency in information transmission across the SCI network. Modularity analysis showed that electrodes of the same brain region were distributed over several modules, and some obtained modules in SCI were extended from anterior to posterior regions. These results demonstrate that the SCI network was more resilient to neuronal damage compared to that of MCI and even more compared to that of AD. Finally, we confirm that MCI is a transitional stage between SCI and AD, with a predominance of high-strength intrinsic connectivity, which may reflect the compensatory response to the neuronal damage occurring early in the disease process.

Diabetic retinopathy detection using red lesion localization and convolutional neural networks.

Detecting the early signs of diabetic retinopathy (DR) is essential, as timely treatment might reduce or even prevent vision loss. Moreover, automatically localizing the regions of the retinal image that might contain lesions can favorably assist specialists in the task of detection. In this study, we designed a lesion localization model using a deep network patch-based approach. Our goal was to reduce the complexity of the model while improving its performance. For this purpose, we designed an efficient procedure (including two convolutional neural network models) for selecting the training patches, such that the challenging examples would be given special attention during the training process. Using the labeling of the region, a DR decision can be given to the initial image, without the need for special training. The model is trained on the Standard Diabetic Retinopathy Database, Calibration Level 1 (DIARETDB1) database and is tested on several databases (including Messidor) without any further adaptation. It reaches an area under the receiver operating characteristic curve of 0.912-95%CI(0.897-0.928) for DR screening, and a sensitivity of 0.940-95%CI(0.921-0.959). These values are competitive with other state-of-the-art approaches.

Incremental HMM training applied to ECG signal analysis.

This work discusses the implementation of incremental hidden Markov model (HMM) training methods for electrocardiogram (ECG) analysis. The HMMs are used to model the ECG signal as a sequence of connected elementary waveforms. Moreover, an adaptation process is implemented to adapt the HMMs to the ECG signal of a particular individual. The adaptation training strategy is based on incremental versions of the expectation-maximization, segmental k-means and Bayesian approaches. Performance of the training methods was assessed through experiments considering the QT and ST-T databases. The results obtained show that the incremental training improves beat segmentation and ischemia detection performance with the advantage of low computational effort.

Retinal image quality assessment using deep learning.

Poor-quality retinal images do not allow an accurate medical diagnosis, and it is inconvenient for a patient to return to a medical center to repeat the fundus photography exam. In this paper, a robust automatic system is proposed to assess the quality of retinal images at the moment of the acquisition, aiming at assisting health care professionals during a fundus photography exam. We propose a convolutional neural network (CNN) pretrained on non-medical images for extracting general image features. The weights of the CNN are further adjusted via a fine-tuning procedure, resulting in a performant classifier obtained only with a small quantity of labeled images. The CNN performance was evaluated on two publicly available databases (i.e., DRIMDB and ELSA-Brasil) using two different procedures: intra-database and inter-database cross-validation. The CNN achieved an area under the curve (AUC) of 99.98% on DRIMDB and an AUC of 98.56% on ELSA-Brasil in the inter-database experiment, where training and testing were not performed on the same database. These results show the robustness of the proposed model to various image acquisitions without requiring special adaptation, thus making it a good candidate for use in operational clinical scenarios.

On using the Viterbi path along with HMM likelihood information for online signature verification.

This paper describes a system using two complementary sorts of information issuing from a hidden Markov model (HMM) for online signature verification. At each point of the signature, 25 features are extracted. These features are normalized before training and testing in order to improve the performance of the system. This normalization is writer-dependent; it exploits only five genuine signatures used to train the writer HMM. A claimed identity is confirmed when the arithmetic mean of two similarity scores, obtained on an input signature, is higher than a threshold. The first score is related to the likelihood given by the HMM of the claimed identity; the second score is related to the segmentation given by such an HMM on the input signature. A personalized score normalization is also proposed before fusion. Our approach is evaluated on several online signature databases, such as BIOMET, PHILIPS, MCYT, and SVC2004, which were captured under different acquisition conditions. For the first time in signature verification, we show that the fusion of segmentation-based information generated by the HMM with likelihood-based information considerably improves the quality of the verification system. Finally, owing to our two-stage normalization (at the feature and score levels), we show that our system results in more stable client-score distributions across databases and in a better separation between the distributions of client and impostor scores.

Predicting postoperative gait in cerebral palsy.

In this work, postoperative lower limb kinematics are predicted with respect to preoperative kinematics, physical examination and surgery data. Data of 115 children with cerebral palsy that have undergone single-event multilevel surgery were considered. Preoperative data dimension was reduced utilizing principal component analysis. Then, multiple linear regressions with 80% confidence intervals were performed between postoperative kinematics and bilateral preoperative kinematics, 36 physical examination variables and combinations of 9 different surgical procedures. The mean prediction errors on test vary from 4° (pelvic obliquity and hip adduction) to 10° (hip rotation and foot progression), depending on the kinematic angle. The unilateral mean sizes of the confidence intervals vary from 5° to 15°. Frontal plane angles are predicted with the lowest errors, however the same performance is achieved when considering the postoperative average signals. Sagittal plane angles are better predicted than transverse plane angles, with statistical differences with respect to the average postoperative kinematics for both plane's angles except for ankle dorsiflexion. The mean prediction errors are smaller than the variability of gait parameters in cerebral palsy. The performance of the system is independent of the preoperative state severity of the patient. Even if the system is not yet accurate enough to define a surgery plan, it shows an unbiased estimation of the most likely outcome, which can be useful for both the clinician and the patient. More patients' data are necessary for improving the precision of the model in order to predict the kinematic outcome of a large number of possible surgeries and gait patterns.

ECG signal analysis through hidden Markov models.

This paper presents an original hidden Markov model (HMM) approach for online beat segmentation and classification of electrocardiograms. The HMM framework has been visited because of its ability of beat detection, segmentation and classification, highly suitable to the electrocardiogram (ECG) problem. Our approach addresses a large panel of topics some of them never studied before in other HMM related works: waveforms modeling, multichannel beat segmentation and classification, and unsupervised adaptation to the patient's ECG. The performance was evaluated on the two-channel QT database in terms of waveform segmentation precision, beat detection and classification. Our waveform segmentation results compare favorably to other systems in the literature. We also obtained high beat detection performance with sensitivity of 99.79% and a positive predictivity of 99.96%, using a test set of 59 recordings. Moreover, premature ventricular contraction beats were detected using an original classification strategy. The results obtained validate our approach for real world application.

The effect of sample size and MLP architecture on Bayesian learning for cancer prognosis--a case study.

In this paper we investigate the independent effects of training sample size and multilayer perceptron (MLP) architecture on Bayesian learning to build prognostic models for metastatic breast cancer. We trained two types of Bayesian neural networks on a data set of 1477 metastatic breast cancer patients followed at the Institut Curie using disjoint training sets of sizes k = 50, 100, 200, 300, and 450. The learning performance as measured by an expected loss appeared independent of the two architectures modelling the log hazard function under either proportional or non proportional hazard assumptions, thus indicating that no other sources of nonlinearity besides interactions are present. We found a performance breakdown at k = 50, and no sample size effect for k > or = 100.

A dynamic evidential network for fall detection.

This study is part of the development of a remote home healthcare monitoring application designed to detect distress situations through several types of sensors. The multisensor fusion can provide more accurate and reliable information compared to information provided by each sensor separately. Furthermore, data from multiple heterogeneous sensors present in the remote home healthcare monitoring systems have different degrees of imperfection and trust. Among the multisensor fusion methods, Dempster-Shafer theory (DST) is currently considered the most appropriate for representing and processing the imperfect information. Based on a graphical representation of the DST called evidential networks, a structure of heterogeneous data fusion from multiple sensors for fall detection has been proposed. The evidential networks, implemented on our remote medical monitoring platform, are also proposed in this paper to maximize the performance of automatic fall detection and thus make the system more reliable. However, the presence of noise, the variability of recorded signals by the sensors, and the failing or unreliable sensors may thwart the evidential networks performance. In addition, the sensors signals nonstationary nature may degrade the experimental conditions. To compensate the nonstationary effect, the time evolution is considered by introducing the dynamic evidential network which was evaluated by the simulated fall scenarios corresponding to various use cases.

The multiscenario multienvironment BioSecure Multimodal Database (BMDB).

A new multimodal biometric database designed and acquired within the framework of the European BioSecure Network of Excellence is presented. It is comprised of more than 600 individuals acquired simultaneously in three scenarios: 1) over the Internet, 2) in an office environment with desktop PC, and 3) in indoor/outdoor environments with mobile portable hardware. The three scenarios include a common part of audio/video data. Also, signature and fingerprint data have been acquired both with desktop PC and mobile portable hardware. Additionally, hand and iris data were acquired in the second scenario using desktop PC. Acquisition has been conducted by 11 European institutions. Additional features of the BioSecure Multimodal Database (BMDB) are: two acquisition sessions, several sensors in certain modalities, balanced gender and age distributions, multimodal realistic scenarios with simple and quick tasks per modality, cross-European diversity, availability of demographic data, and compatibility with other multimodal databases. The novel acquisition conditions of the BMDB allow us to perform new challenging research and evaluation of either monomodal or multimodal biometric systems, as in the recent BioSecure Multimodal Evaluation campaign. A description of this campaign including baseline results of individual modalities from the new database is also given. The database is expected to be available for research purposes through the BioSecure Association during 2008.

A new phase-correlation-based iris matching for degraded images.

In this paper, we present a new phase-correlation-based iris matching approach in order to deal with degradations in iris images due to unconstrained acquisition procedures. Our matching system is a fusion of global and local Gabor phase-correlation schemes. The main originality of our local approach is that we do not only consider the correlation peak amplitudes but also their locations in different regions of the images. Results on several degraded databases, namely, the CASIA-BIOSECURE and Iris Challenge Evaluation 2005 databases, show the improvement of our method compared to two available reference systems, Masek and Open Source for Iris (OSRIS), in verification mode.