Multimodal deep learning model on interim [18F]FDG PET/CT for predicting primary treatment failure in diffuse large B-cell lymphoma.

OBJECTIVES: The prediction of primary treatment failure (PTF) is necessary for patients with diffuse large B-cell lymphoma (DLBCL) since it serves as a prominent means for improving front-line outcomes. Using interim 18F-fluoro-2-deoxyglucose ([18F]FDG) positron emission tomography/computed tomography (PET/CT) imaging data, we aimed to construct multimodal deep learning (MDL) models to predict possible PTF in low-risk DLBCL. METHODS: Initially, 205 DLBCL patients undergoing interim [18F]FDG PET/CT scans and the front-line standard of care were included in the primary dataset for model development. Then, 44 other patients were included in the external dataset for generalization evaluation. Based on the powerful backbone of the Conv-LSTM network, we incorporated five different multimodal fusion strategies (pixel intermixing, separate channel, separate branch, quantitative weighting, and hybrid learning) to make full use of PET/CT features and built five corresponding MDL models. Moreover, we found the best model, that is, the hybrid learning model, and optimized it by integrating the contrastive training objective to further improve its prediction performance. RESULTS: The final model with contrastive objective optimization, named the contrastive hybrid learning model, performed best, with an accuracy of 91.22% and an area under the receiver operating characteristic curve (AUC) of 0.926, in the primary dataset. In the external dataset, its accuracy and AUC remained at 88.64% and 0.925, respectively, indicating its good generalization ability. CONCLUSIONS: The proposed model achieved good performance, validated the predictive value of interim PET/CT, and holds promise for directing individualized clinical treatment. KEY POINTS: • The proposed multimodal models achieved accurate prediction of primary treatment failure in DLBCL patients. • Using an appropriate feature-level fusion strategy can make the same class close to each other regardless of the modal heterogeneity of the data source domain and positively impact the prediction performance. • Deep learning validated the predictive value of interim PET/CT in a way that exceeded human capabilities.

Prediction of orthognathic surgery plan from 3D cephalometric analysis via deep learning.

BACKGROUND: Preoperative planning of orthognathic surgery is indispensable for achieving ideal surgical outcome regarding the occlusion and jaws' position. However, orthognathic surgery planning is sophisticated and highly experience-dependent, which requires comprehensive consideration of facial morphology and occlusal function. This study aimed to investigate a robust and automatic method based on deep learning to predict reposition vectors of jawbones in orthognathic surgery plan. METHODS: A regression neural network named VSP transformer was developed based on Transformer architecture. Firstly, 3D cephalometric analysis was employed to quantify skeletal-facial morphology as input features. Next, input features were weighted using pretrained results to minimize bias resulted from multicollinearity. Through encoder-decoder blocks, ten landmark-based reposition vectors of jawbones were predicted. Permutation importance (PI) method was used to calculate contributions of each feature to final prediction to reveal interpretability of the proposed model. RESULTS: VSP transformer model was developed with 383 samples and clinically tested with 49 prospectively collected samples. Our proposed model outperformed other four classic regression models in prediction accuracy. Mean absolute errors (MAE) of prediction were 1.41 mm in validation set and 1.34 mm in clinical test set. The interpretability results of the model were highly consistent with clinical knowledge and experience. CONCLUSIONS: The developed model can predict reposition vectors of orthognathic surgery plan with high accuracy and good clinically practical-effectiveness. Moreover, the model was proved reliable because of its good interpretability.

Automatic craniomaxillofacial landmarks detection in CT images of individuals with dentomaxillofacial deformities by a two-stage deep learning model.

BACKGROUND: Accurate cephalometric analysis plays a vital role in the diagnosis and subsequent surgical planning in orthognathic and orthodontics treatment. However, manual digitization of anatomical landmarks in computed tomography (CT) is subject to limitations such as low accuracy, poor repeatability and excessive time consumption. Furthermore, the detection of landmarks has more difficulties on individuals with dentomaxillofacial deformities than normal individuals. Therefore, this study aims to develop a deep learning model to automatically detect landmarks in CT images of patients with dentomaxillofacial deformities. METHODS: Craniomaxillofacial (CMF) CT data of 80 patients with dentomaxillofacial deformities were collected for model development. 77 anatomical landmarks digitized by experienced CMF surgeons in each CT image were set as the ground truth. 3D UX-Net, the cutting-edge medical image segmentation network, was adopted as the backbone of model architecture. Moreover, a new region division pattern for CMF structures was designed as a training strategy to optimize the utilization of computational resources and image resolution. To evaluate the performance of this model, several experiments were conducted to make comparison between the model and manual digitization approach. RESULTS: The training set and the validation set included 58 and 22 samples respectively. The developed model can accurately detect 77 landmarks on bone, soft tissue and teeth with a mean error of 1.81 ± 0.89 mm. Removal of region division before training significantly increased the error of prediction (2.34 ± 1.01 mm). In terms of manual digitization, the inter-observer and intra-observer variations were 1.27 ± 0.70 mm and 1.01 ± 0.74 mm respectively. In all divided regions except Teeth Region (TR), our model demonstrated equivalent performance to experienced CMF surgeons in landmarks detection (p > 0.05). CONCLUSIONS: The developed model demonstrated excellent performance in detecting craniomaxillofacial landmarks when considering manual digitization work of expertise as benchmark. It is also verified that the region division pattern designed in this study remarkably improved the detection accuracy.

Structure-Aware Multimodal Deep Learning for Drug-Protein Interaction Prediction.

Identifying drug-protein interactions (DPIs) is crucial in drug discovery, and a number of machine learning methods have been developed to predict DPIs. Existing methods usually use unrealistic data sets with hidden bias, which will limit the accuracy of virtual screening methods. Meanwhile, most DPI prediction methods pay more attention to molecular representation but lack effective research on protein representation and high-level associations between different instances. To this end, we present the novel structure-aware multimodal deep DPI prediction model, STAMP-DPI, which was trained on a curated industry-scale benchmark data set. We built a high-quality benchmark data set named GalaxyDB for DPI prediction. This industry-scale data set along with an unbiased training procedure resulted in a more robust benchmark study. For informative protein representation, we constructed a structure-aware graph neural network method from the protein sequence by combining predicted contact maps and graph neural networks. Through further integration of structure-based representation and high-level pretrained embeddings for molecules and proteins, our model effectively captures the feature representation of the interactions between them. As a result, STAMP-DPI outperformed state-of-the-art DPI prediction methods by decreasing 7.00% mean square error (MSE) in the Davis data set and improving 8.89% area under the curve (AUC) in the GalaxyDB data set. Moreover, our model is an interpretable model with the transformer-based interaction mechanism, which can accurately reveal the binding sites between molecules and proteins.

SWnet: a deep learning model for drug response prediction from cancer genomic signatures and compound chemical structures.

BACKGROUND: One of the major challenges in precision medicine is accurate prediction of individual patient's response to drugs. A great number of computational methods have been developed to predict compounds activity using genomic profiles or chemical structures, but more exploration is yet to be done to combine genetic mutation, gene expression, and cheminformatics in one machine learning model. RESULTS: We presented here a novel deep-learning model that integrates gene expression, genetic mutation, and chemical structure of compounds in a multi-task convolutional architecture. We applied our model to the Genomics of Drug Sensitivity in Cancer (GDSC) and Cancer Cell Line Encyclopedia (CCLE) datasets. We selected relevant cancer-related genes based on oncology genetics database and L1000 landmark genes, and used their expression and mutations as genomic features in model training. We obtain the cheminformatics features for compounds from PubChem or ChEMBL. Our finding is that combining gene expression, genetic mutation, and cheminformatics features greatly enhances the predictive performance. CONCLUSION: We implemented an extended Graph Neural Network for molecular graphs and Convolutional Neural Network for gene features. With the employment of multi-tasking and self-attention functions to monitor the similarity between compounds, our model outperforms recently published methods using the same training and testing datasets.

Microfluidic integrated capacitive biosensor for C-reactive protein label-free and real-time detection.

A microfluidic chip has been integrated with a capacitive biosensor based on mass-producible three-dimensional (3D) interdigital electrode arrays. To achieve the monitoring of biosensor preparation and cardiac- and periodontitis-related biomarkers, all the processes were detected in a continuously on-site way. Fabrication steps for the microfluidic chip-bonded 3D interdigital capacitor biosensor include gold thiol modification, the activation of EDC/sulfo-NHS, and the bioconjugation of antibodies. Fluorescent characterization and X-ray photoelectron spectroscopy analysis were applied to assess the successful immobilization of the C-reactive protein (CRP) antibody. The experimental results indicate the good specificity and high sensitivity of the microfluidic integrated 3D capacitive biosensor. The limit of detection of the 3D capacitive biosensor for CRP label-free detection was about 1 pg mL-1. This 3D capacitive biosensor with integrated microfluidics is mass-producible and has achieved the on-site continuous detection of cardiac- and periodontitis-related biomarkers with high performance.

A Novel Mass-Producible Capacitive Sensor with Fully Symmetric 3D Structure and Microfluidics for Cells Detection.

Affinity biosensors of interdigitated electrodes have been widely used in cell detection. This research presents a mass-producible and disposable three-dimensional (3D) structure capacitive sensor based on the integrated circuit package lead frames for cell concentration detection. The fully symmetric 3D interdigital electrode structure makes the sensor more homogeneous and sensitive. (3-Aminopropyl) triethoxysilane (APTES) and glutaraldehyde are immobilized onto gold-plated electrodes. By overlaying the microfluidic channels on top, the volume of the solution is kept constant to obtain repeatable measured capacitance values. Moreover, using the upgraded reading and writing functions and circular measurement of the E4980A Data Transfer Program, an automatic multigroup test system is developed. It is shown that the cell concentration and capacitance are inversely correlated, and the cell concentration range of 10³â»106 CFU∙mL-1 is achieved. In addition, the rate of capacitance change matches that of state-of-the-art biosensors reported. A program is developed to find the optimal voltage and frequency for linear fitting between the capacitance change and cell concentration. Future work will employ machine learning-based data analysis to drug resistance sensitivity test of cell lines and cell survival status.

Telemedicine screening of retinal diseases with a handheld portable non-mydriatic fundus camera.

BACKGROUND: We modified and reconstructed a high image quality portable non-mydriatic fundus camera and compared it with the tabletop fundus camera to evaluate the efficacy of the new camera in detecting retinal diseases. METHODS: We designed and built a novel portable handheld fundus camera with telemedicine system. The image quality of fundus cameras was compared to that of existing commercial tabletop cameras by taking photographs of 364 eyes from the 254 patients. In all 800 fundus images taken by two camera types, 400 images per camera, were graded with the four image clarity classifications. RESULTS: Using the portable fundus camera, 63% (252/400) images were graded as excellent overall quality, 20.5% (82/400) were good, 11.75% (47/400) were fair, and 4.75% (19/400) were inadequate. Using the tabletop fundus camera, 70.75% (283/400) images were graded as excellent overall quality, 20.4% (51/400) were good, 13.25% (53/400) were fair, and 3.25% (13/400) were inadequate. Common retinal diseases were easily identified from fundus images obtained from the portable fundus camera. CONCLUSION: The new type of non-mydriatic portable fundus camera was qualified to have professional quality of fundus images. The revolutionary screening camera provides a foundational platform which can potentially improve the accessibility of retinal screening programmes.

[Measurement Error Analysis and Calibration Technique of NTC - Based Body Temperature Sensor].

A NTC thermistor-based wearable body temperature sensor was designed. This paper described the design principles and realization method of the NTC-based body temperature sensor. In this paper the temperature measurement error sources of the body temperature sensor were analyzed in detail. The automatic measurement and calibration method of ADC error was given. The results showed that the measurement accuracy of calibrated body temperature sensor is better than ± 0.04 degrees C. The temperature sensor has high accuracy, small size and low power consumption advantages.

A Simple Normalization Technique Using Window Statistics to Improve the Out-of-Distribution Generalization on Medical Images.

Since data scarcity and data heterogeneity are prevailing for medical images, well-trained Convolutional Neural Networks (CNNs) using previous normalization methods may perform poorly when deployed to a new site. However, a reliable model for real-world clinical applications should generalize well both on in-distribution (IND) and out-of-distribution (OOD) data (e.g., the new site data). In this study, we present a novel normalization technique called window normalization (WIN) to improve the model generalization on heterogeneous medical images, which offers a simple yet effective alternative to existing normalization methods. Specifically, WIN perturbs the normalizing statistics with the local statistics computed within a window. This feature-level augmentation technique regularizes the models well and improves their OOD generalization significantly. Leveraging its advantage, we propose a novel self-distillation method called WIN-WIN. WIN-WIN can be easily implemented with two forward passes and a consistency constraint, serving as a simple extension to existing methods. Extensive experimental results on various tasks (6 tasks) and datasets (24 datasets) demonstrate the generality and effectiveness of our methods.

Multi-step validation of a deep learning-based system with visual explanations for optical diagnosis of polyps with advanced features.

Artificial intelligence (AI) has been found to assist in optical differentiation of hyperplastic and adenomatous colorectal polyps. We investigated whether AI can improve the accuracy of endoscopists' optical diagnosis of polyps with advanced features. We introduced our AI system distinguishing polyps with advanced features with more than 0.870 of accuracy in the internal and external validation datasets. All 19 endoscopists with different levels showed significantly lower diagnostic accuracy (0.410-0.580) than the AI. Prospective randomized controlled study involving 120 endoscopists into optical diagnosis of polyps with advanced features with or without AI demonstration identified that AI improved endoscopists' proportion of polyps with advanced features correctly sent for histological examination (0.960 versus 0.840, p < 0.001), and the proportion of polyps without advanced features resected and discarded (0.490 versus 0.380, p = 0.007). We thus developed an AI technique that significantly increases the accuracy of colorectal polyps with advanced features.

Chromosome Detection in Metaphase Cell Images Using Morphological Priors.

Reliable chromosome detection in metaphase cell (MC) images can greatly alleviate the workload of cytogeneticists for karyotype analysis and the diagnosis of chromosomal disorders. However, it is still an extremely challenging task due to the complicated characteristics of chromosomes, e.g., dense distributions, arbitrary orientations, and various morphologies. In this article, we propose a novel rotated-anchor-based detection framework, named DeepCHM, for fast and accurate chromosome detection in MC images. Our framework has three main innovations: 1) A deep saliency map representing chromosomal morphological features is learned end-to-end with semantic features. This not only enhances the feature representations for anchor classification and regression but also guides the anchor setting to significantly reduce redundant anchors. This accelerates the detection and improves the performance; 2) A hardness-aware loss weights the contribution of positive anchors, which effectively reinforces the model to identify hard chromosomes; 3) A model-driven sampling strategy addresses the anchor imbalance issue by adaptively selecting hard negative anchors for model training. In addition, a large-scale benchmark dataset with a total of 624 images and 27,763 chromosome instances was built for chromosome detection and segmentation. Extensive experimental results demonstrate that our method outperforms most state-of-the-art (SOTA) approaches and successfully handles chromosome detection, with an AP score of 93.53%.

Further predictive value of lymphovascular invasion explored via supervised deep learning for lymph node metastases in breast cancer.

Lymphovascular invasion, specifically lymph-blood vessel invasion (LBVI), is a risk factor for metastases in breast invasive ductal carcinoma (IDC) and is routinely screened using hematoxylin-eosin histopathological images. However, routine reports only describe whether LBVI is present and does not provide other potential prognostic information of LBVI. This study aims to evaluate the clinical significance of LBVI in 685 IDC cases and explore the added predictive value of LBVI on lymph node metastases (LNM) via supervised deep learning (DL), an expert-experience embedded knowledge transfer learning (EEKT) model in 40 LBVI-positive cases signed by the routine report. Multivariate logistic regression and propensity score matching analysis demonstrated that LBVI (OR 4.203, 95% CI 2.809-6.290, P < 0.001) was a significant risk factor for LNM. Then, the EEKT model trained on 5780 image patches automatically segmented LBVI with a patch-wise Dice similarity coefficient of 0.930 in the test set and output counts, location, and morphometric features of the LBVIs. Some morphometric features were beneficial for further stratification within the 40 LBVI-positive cases. The results showed that LBVI in cases with LNM had a higher short-to-long side ratio of the minimum rectangle (MR) (0.686 vs. 0.480, P = 0.001), LBVI-to-MR area ratio (0.774 vs. 0.702, P = 0.002), and solidity (0.983 vs. 0.934, P = 0.029) compared to LBVI in cases without LNM. The results highlight the potential of DL to assist pathologists in quantifying LBVI and, more importantly, in exploring added prognostic information from LBVI.

Deep learning-based PET/MR radiomics for the classification of annualized relapse rate in multiple sclerosis.

Background Annualized Relapse Rate (ARR) is one of the most important indicators of disease progression in patients with Multiple Sclerosis (MS). However, imaging markers that can effectively predict ARR are currently unavailable. In this study, we developed a deep learning-based method for the automated extraction of radiomics features from Positron Emission Computed Tomography (PET) and Magnetic Resonance (MR) images to predict ARR in patients with MS. Methods Twenty-five patients with a definite diagnosis of Relapsing-Remitting MS (RRMS) were enrolled in this study. We designed a multi-branch fully convolutional neural network to segment lesions from PET/MR images. After that, radiomics features were extracted from the obtained lesion volume of interest. Three feature selection methods were used to retain features highly correlated with ARR. We combined four classifiers with different feature selection methods to form twelve models for ARR classification. Finally, the model with the best performance was chosen. Results Our network achieved precise automatic lesion segmentation with a Dice Similarity Coefficient (DSC) of 0.81 and a precision of 0.86. Radiomics features from lesions filtered by Recursive Feature Elimination (RFE) achieved the best performance in the Support Vector Machines (SVM) classifier. The classification model performance was best when radiomics from both PET and MR were combined to predict ARR, with high accuracy at 0.88 and Area Under the ROC curves (AUC) at 0.96, which outperformed MR or PET-based model and clinical indicators-based model. Conclusion Our automatic segmentation masks can replace manual ones with excellent performance. Furthermore, the deep learning and PET/MR radiomics-based model in our research is an effective tool in assisting ARR classification of MS patients.

Deep multi-instance transfer learning for pneumothorax classification in chest X-ray images.

PURPOSE: Pneumothorax is a life-threatening emergency that requires immediate treatment. Frontal-view chest X-ray images are typically used for pneumothorax detection in clinical practice. However, manual review of radiographs is time-consuming, labor-intensive, and highly dependent on the experience of radiologists, which may lead to misdiagnosis. Here, we aim to develop a reliable automatic classification method to assist radiologists in rapidly and accurately diagnosing pneumothorax in frontal chest radiographs. METHODS: A novel residual neural network (ResNet)-based two-stage deep-learning strategy is proposed for pneumothorax identification: local feature learning (LFL) followed by global multi-instance learning (GMIL). Most of the nonlesion regions in the images are removed for learning discriminative features. Two datasets are used for large-scale validation: a private dataset (27 955 frontal-view chest X-ray images) and a public dataset (the National Institutes of Health [NIH] ChestX-ray14; 112 120 frontal-view X-ray images). The model performance of the identification was evaluated using the accuracy, precision, recall, specificity, F1-score, receiver operating characteristic (ROC), and area under ROC curve (AUC). Fivefold cross-validation is conducted on the datasets, and then the mean and standard deviation of the above-mentioned metrics are calculated to assess the overall performance of the model. RESULTS: The experimental results demonstrate that the proposed learning strategy can achieve state-of-the-art performance on the NIH dataset with an accuracy, AUC, precision, recall, specificity, and F1-score of 94.4% ± 0.7%, 97.3% ± 0.5%, 94.2% ± 0.3%, 94.6% ± 1.5%, 94.2% ± 0.4%, and 94.4% ± 0.7%, respectively. CONCLUSIONS: The experimental results demonstrate that our proposed CAD system is an efficient assistive tool in the identification of pneumothorax.

Porous Inorganic Materials for Bioanalysis and Diagnostic Applications.

Porous inorganic materials play an important role in adsorbing targeted analytes and supporting efficient reactions in analytical science. The detection performance relies on the structural properties of porous materials, considering the tunable pore size, shape, connectivity, etc. Herein, we first clarify the enhancement mechanisms of porous materials for bioanalysis, concerning the detection sensitivity and selectivity. The diagnostic applications of porous material-assisted platforms by coupling with various analytical techniques, including electrochemical sensing, optical spectrometry, and mass spectrometry, etc., are then reviewed. We foresee that advanced porous materials will bring far-reaching implications in bioanalysis toward real-case applications, especially as diagnostic assays in clinical settings.

3D Graph-Connectivity Constrained Network for Hepatic Vessel Segmentation.

Segmentation of hepatic vessels from 3D CT images is necessary for accurate diagnosis and preoperative planning for liver cancer. However, due to the low contrast and high noises of CT images, automatic hepatic vessel segmentation is a challenging task. Hepatic vessels are connected branches containing thick and thin blood vessels, showing an important structural characteristic or a prior: the connectivity of blood vessels. However, this is rarely applied in existing methods. In this paper, we segment hepatic vessels from 3D CT images by utilizing the connectivity prior. To this end, a graph neural network (GNN) used to describe the connectivity prior of hepatic vessels is integrated into a general convolutional neural network (CNN). Specifically, a graph attention network (GAT) is first used to model the graphical connectivity information of hepatic vessels, which can be trained with the vascular connectivity graph constructed directly from the ground truths. Second, the GAT is integrated with a lightweight 3D U-Net by an efficient mechanism called the plug-in mode, in which the GAT is incorporated into the U-Net as a multi-task branch and is only used to supervise the training procedure of the U-Net with the connectivity prior. The GAT will not be used in the inference stage, and thus will not increase the hardware and time costs of the inference stage compared with the U-Net. Therefore, hepatic vessel segmentation can be well improved in an efficient mode. Extensive experiments on two public datasets show that the proposed method is superior to related works in accuracy and connectivity of hepatic vessel segmentation.

Size-adaptive mediastinal multilesion detection in chest CT images via deep learning and a benchmark dataset.

PURPOSE: Many deep learning methods have been developed for pulmonary lesion detection in chest computed tomography (CT) images. However, these methods generally target one particular lesion type, that is, pulmonary nodules. In this work, we intend to develop and evaluate a novel deep learning method for a more challenging task, detecting various benign and malignant mediastinal lesions with wide variations in sizes, shapes, intensities, and locations in chest CT images. METHODS: Our method for mediastinal lesion detection contains two main stages: (a) size-adaptive lesion candidate detection followed by (b) false-positive (FP) reduction and benign-malignant classification. For candidate detection, an anchor-free and one-stage detector, namely 3D-CenterNet is designed to locate suspicious regions (i.e., candidates with various sizes) within the mediastinum. Then, a 3D-SEResNet-based classifier is used to differentiate FPs, benign lesions, and malignant lesions from the candidates. RESULTS: We evaluate the proposed method by conducting five-fold cross-validation on a relatively large-scale dataset, which consists of data collected on 1136 patients from a grade A tertiary hospital. The method can achieve sensitivity scores of 84.3% ± 1.9%, 90.2% ± 1.4%, 93.2% ± 0.8%, and 93.9% ± 1.1%, respectively, in finding all benign and malignant lesions at 1/8, 1/4, ½, and 1 FPs per scan, and the accuracy of benign-malignant classification can reach up to 78.7% ± 2.5%. CONCLUSIONS: The proposed method can effectively detect mediastinal lesions with various sizes, shapes, and locations in chest CT images. It can be integrated into most existing pulmonary lesion detection systems to promote their clinical applications. The method can also be readily extended to other similar 3D lesion detection tasks.

Improving the generalization of glaucoma detection on fundus images via feature alignment between augmented views.

Convolutional neural networks (CNNs) are commonly used in glaucoma detection. Due to the various data distribution shift, however, a well-behaved model may be plummeting in performance when deployed in a new environment. On the other hand, the most straightforward method, data collection, is costly and even unrealistic in practice. To address these challenges, we propose a new method named data augmentation-based (DA) feature alignment (DAFA) to improve the out-of-distribution (OOD) generalization with a single dataset, which is based on the principle of feature alignment to learn the invariant features and eliminate the effect of data distribution shifts. DAFA creates two views of a sample by data augmentation and performs the feature alignment between that augmented views through latent feature recalibration and semantic representation alignment. Latent feature recalibration is normalizing the middle features to the same distribution by instance normalization (IN) layers. Semantic representation alignment is conducted by minimizing the Topk NT-Xent loss and the maximum mean discrepancy (MMD), which maximize the semantic agreement across augmented views from individual and population levels. Furthermore, a benchmark is established with seven glaucoma detection datasets and a new metric named mean of clean area under curve (mcAUC) for a comprehensive evaluation of the model performance. Experimental results of five-fold cross-validation demonstrate that DAFA can consistently and significantly improve the out-of-distribution generalization (up to +16.3% mcAUC) regardless of the training data, network architectures, and augmentation policies and outperform lots of state-of-the-art methods.

Deep Learning for nasopharyngeal Carcinoma Identification Using Both White Light and Narrow-Band Imaging Endoscopy.

OBJECTIVES/HYPOTHESIS: To develop a deep-learning-based automatic diagnosis system for identifying nasopharyngeal carcinoma (NPC) from noncancer (inflammation and hyperplasia), using both white light imaging (WLI) and narrow-band imaging (NBI) nasopharyngoscopy images. STUDY DESIGN: Retrospective study. METHODS: A total of 4,783 nasopharyngoscopy images (2,898 WLI and 1,885 NBI) of 671 patients were collected and a novel deep convolutional neural network (DCNN) framework was developed named Siamese deep convolutional neural network (S-DCNN), which can simultaneously utilize WLI and NBI images to improve the classification performance. To verify the effectiveness of combining the above-mentioned two modal images for prediction, we compared the proposed S-DCNN with two baseline models, namely DCNN-1 (only considering WLI images) and DCNN-2 (only considering NBI images). RESULTS: In the threefold cross-validation, an overall accuracy and area under the curve of the three DCNNs achieved 94.9% (95% confidence interval [CI] 93.3%-96.5%) and 0.986 (95% CI 0.982-0.992), 87.0% (95% CI 84.2%-89.7%) and 0.930 (95% CI 0.906-0.961), and 92.8% (95% CI 90.4%-95.3%) and 0.971 (95% CI 0.953-0.992), respectively. The accuracy of S-DCNN is significantly improved compared with DCNN-1 (P-value <.001) and DCNN-2 (P-value = .008). CONCLUSION: Using the deep-learning technology to automatically diagnose NPC under nasopharyngoscopy can provide valuable reference for NPC screening. Superior performance can be obtained by simultaneously utilizing the multimodal features of NBI image and WLI image of the same patient. LEVEL OF EVIDENCE: 3 Laryngoscope, 132:999-1007, 2022.