Gold-viral particle identification by deep learning in wide-field photon scattering parametric images.

The ability to identify virus particles is important for research and clinical applications. Because of the optical diffraction limit, conventional optical microscopes are generally not suitable for virus particle detection, and higher resolution instruments such as transmission electron microscopy (TEM) and scanning electron microscopy (SEM) are required. In this paper, we propose a new method for identifying virus particles based on polarization parametric indirect microscopic imaging (PIMI) and deep learning techniques. By introducing an abrupt change of refractivity at the virus particle using antibody-conjugated gold nanoparticles (AuNPs), the strength of the photon scattering signal can be magnified. After acquiring the PIMI images, a deep learning method was applied to identify discriminating features and classify the virus particles, using electron microscopy (EM) images as the ground truth. Experimental results confirm that gold-virus particles can be identified in PIMI images with a high level of confidence.

Characterization of deep sub-wavelength nanowells by imaging the photon state scattering spectra.

Optical-matter interactions and photon scattering in a sub-wavelength space are of great interest in many applications, such as nanopore-based gene sequencing and molecule characterization. Previous studies show that spatial distribution features of the scattering photon states are highly sensitive to the dielectric and structural properties of the nanopore array and matter contained on or within them, as a result of the complex optical-matter interaction in a confined system. In this paper, we report a method for shape characterization of subwavelength nanowells using photon state spatial distribution spectra in the scattering near field. Far-field parametric images of the near-field optical scattering from sub-wavelength nanowell arrays on a SiN substrate were obtained experimentally. Finite-difference time-domain simulations were used to interpret the experimental results. The rich features of the parametric images originating from the interaction of the photons and the nanowells were analyzed to recover the size of the nanowells. Experiments on nanoholes modified with Shp2 proteins were also performed. Results show that the scattering distribution of modified nanoholes exhibits significant differences compared to empty nanoholes. This work highlights the potential of utilizing the photon status scattering of nanowells for molecular characterization or other virus detection applications.

Deep learning for detection and segmentation of artefact and disease instances in gastrointestinal endoscopy.

The Endoscopy Computer Vision Challenge (EndoCV) is a crowd-sourcing initiative to address eminent problems in developing reliable computer aided detection and diagnosis endoscopy systems and suggest a pathway for clinical translation of technologies. Whilst endoscopy is a widely used diagnostic and treatment tool for hollow-organs, there are several core challenges often faced by endoscopists, mainly: 1) presence of multi-class artefacts that hinder their visual interpretation, and 2) difficulty in identifying subtle precancerous precursors and cancer abnormalities. Artefacts often affect the robustness of deep learning methods applied to the gastrointestinal tract organs as they can be confused with tissue of interest. EndoCV2020 challenges are designed to address research questions in these remits. In this paper, we present a summary of methods developed by the top 17 teams and provide an objective comparison of state-of-the-art methods and methods designed by the participants for two sub-challenges: i) artefact detection and segmentation (EAD2020), and ii) disease detection and segmentation (EDD2020). Multi-center, multi-organ, multi-class, and multi-modal clinical endoscopy datasets were compiled for both EAD2020 and EDD2020 sub-challenges. The out-of-sample generalization ability of detection algorithms was also evaluated. Whilst most teams focused on accuracy improvements, only a few methods hold credibility for clinical usability. The best performing teams provided solutions to tackle class imbalance, and variabilities in size, origin, modality and occurrences by exploring data augmentation, data fusion, and optimal class thresholding techniques.

An objective comparison of detection and segmentation algorithms for artefacts in clinical endoscopy.

We present a comprehensive analysis of the submissions to the first edition of the Endoscopy Artefact Detection challenge (EAD). Using crowd-sourcing, this initiative is a step towards understanding the limitations of existing state-of-the-art computer vision methods applied to endoscopy and promoting the development of new approaches suitable for clinical translation. Endoscopy is a routine imaging technique for the detection, diagnosis and treatment of diseases in hollow-organs; the esophagus, stomach, colon, uterus and the bladder. However the nature of these organs prevent imaged tissues to be free of imaging artefacts such as bubbles, pixel saturation, organ specularity and debris, all of which pose substantial challenges for any quantitative analysis. Consequently, the potential for improved clinical outcomes through quantitative assessment of abnormal mucosal surface observed in endoscopy videos is presently not realized accurately. The EAD challenge promotes awareness of and addresses this key bottleneck problem by investigating methods that can accurately classify, localize and segment artefacts in endoscopy frames as critical prerequisite tasks. Using a diverse curated multi-institutional, multi-modality, multi-organ dataset of video frames, the accuracy and performance of 23 algorithms were objectively ranked for artefact detection and segmentation. The ability of methods to generalize to unseen datasets was also evaluated. The best performing methods (top 15%) propose deep learning strategies to reconcile variabilities in artefact appearance with respect to size, modality, occurrence and organ type. However, no single method outperformed across all tasks. Detailed analyses reveal the shortcomings of current training strategies and highlight the need for developing new optimal metrics to accurately quantify the clinical applicability of methods.

Prediction of Multidrug-Resistant TB from CT Pulmonary Images Based on Deep Learning Techniques.

While tuberculosis (TB) disease was discovered more than a century ago, it has not been eradicated yet. Quite contrary, at present, TB constitutes one of the top 10 causes of death and has shown signs of increasing. To complement the conventional diagnostic procedure of applying microbiological culture that takes several weeks and remains expensive, high resolution computer tomography (CT) of pulmonary images has been resorted to not only for aiding clinicians to expedite the process of diagnosis but also for monitoring prognosis when administering antibiotic drugs. This research undertakes the investigation of predicting multidrug-resistant (MDR) patients from drug-sensitive (DS) ones based on CT lung images to monitor the effectiveness of treatment. To contend with smaller data sets (i.e., hundreds) and the characteristics of CT TB images with limited regions capturing abnormities, patch-based deep convolutional neural network (CNN) allied to support vector machine (SVM) classifier is implemented on a collection of data sets from 230 patients obtained from the ImageCLEF 2017 competition. As a result, the proposed architecture of CNN + SVM + patch performs the best with classification accuracy rate at 91.11% (79.80% in terms of patches). In addition, a hand-crafted SIFT based approach accomplishes 88.88% in terms of subject and 83.56% with reference to patches, the highest in this study, which can be explained away by the fact that the data sets are in small numbers. Significantly, during the Tuberculosis Competition at ImageCLEF 2017, the authors took part in the task of classification of 5 types of TB disease and achieved the top one with regard to averaged classification accuracy (i.e., ACC = 0.4067), which is also premised on the approach of CNN + SVM + patch. On the other hand, when the whole slices of 3D TB data sets are applied to train a CNN network, the best result is achieved through the application of CNN coupled with orderless pooling and SVM at 64.71% accuracy rate.

Classification of CT brain images based on deep learning networks.

While computerised tomography (CT) may have been the first imaging tool to study human brain, it has not yet been implemented into clinical decision making process for diagnosis of Alzheimer's disease (AD). On the other hand, with the nature of being prevalent, inexpensive and non-invasive, CT does present diagnostic features of AD to a great extent. This study explores the significance and impact on the application of the burgeoning deep learning techniques to the task of classification of CT brain images, in particular utilising convolutional neural network (CNN), aiming at providing supplementary information for the early diagnosis of Alzheimer's disease. Towards this end, three categories of CT images (N = 285) are clustered into three groups, which are AD, lesion (e.g. tumour) and normal ageing. In addition, considering the characteristics of this collection with larger thickness along the direction of depth (z) (~3-5 mm), an advanced CNN architecture is established integrating both 2D and 3D CNN networks. The fusion of the two CNN networks is subsequently coordinated based on the average of Softmax scores obtained from both networks consolidating 2D images along spatial axial directions and 3D segmented blocks respectively. As a result, the classification accuracy rates rendered by this elaborated CNN architecture are 85.2%, 80% and 95.3% for classes of AD, lesion and normal respectively with an average of 87.6%. Additionally, this improved CNN network appears to outperform the others when in comparison with 2D version only of CNN network as well as a number of state of the art hand-crafted approaches. As a result, these approaches deliver accuracy rates in percentage of 86.3, 85.6 ± 1.10, 86.3 ± 1.04, 85.2 ± 1.60, 83.1 ± 0.35 for 2D CNN, 2D SIFT, 2D KAZE, 3D SIFT and 3D KAZE respectively. The two major contributions of the paper constitute a new 3-D approach while applying deep learning technique to extract signature information rooted in both 2D slices and 3D blocks of CT images and an elaborated hand-crated approach of 3D KAZE.

The state of the art of medical imaging technology: from creation to archive and back.

Medical imaging has learnt itself well into modern medicine and revolutionized medical industry in the last 30 years. Stemming from the discovery of X-ray by Nobel laureate Wilhelm Roentgen, radiology was born, leading to the creation of large quantities of digital images as opposed to film-based medium. While this rich supply of images provides immeasurable information that would otherwise not be possible to obtain, medical images pose great challenges in archiving them safe from corrupted, lost and misuse, retrievable from databases of huge sizes with varying forms of metadata, and reusable when new tools for data mining and new media for data storing become available. This paper provides a summative account on the creation of medical imaging tomography, the development of image archiving systems and the innovation from the existing acquired image data pools. The focus of this paper is on content-based image retrieval (CBIR), in particular, for 3D images, which is exemplified by our developed online e-learning system, MIRAGE, home to a repository of medical images with variety of domains and different dimensions. In terms of novelties, the facilities of CBIR for 3D images coupled with image annotation in a fully automatic fashion have been developed and implemented in the system, resonating with future versatile, flexible and sustainable medical image databases that can reap new innovations.