Transformer-based multi-task learning for classification and segmentation of gastrointestinal tract endoscopic images.

Despite being widely utilized to help endoscopists identify gastrointestinal (GI) tract diseases using classification and segmentation, models based on convolutional neural network (CNN) have difficulties in distinguishing the similarities among some ambiguous types of lesions presented in endoscopic images, and in the training when lacking labeled datasets. Those will prevent CNN from further improving the accuracy of diagnosis. To address these challenges, we first proposed a Multi-task Network (TransMT-Net) capable of simultaneously learning two tasks (classification and segmentation), which has the transformer designed to learn global features and can combine the advantages of CNN in learning local features so that to achieve a more accurate prediction in identifying the lesion types and regions in GI tract endoscopic images. We further adopted the active learning in TransMT-Net to tackle the labeled image-hungry problem. A dataset was created from the CVC-ClinicDB dataset, Macau Kiang Wu Hospital, and Zhongshan Hospital to evaluate the model performance. Then, the experimental results show that our model not only achieved 96.94% accuracy in the classification task and 77.76% Dice Similarity Coefficient in the segmentation task but also outperformed those of other models on our test set. Meanwhile, active learning also produced positive results for the performance of our model with a small-scale initial training set, and even its performance with 30% of the initial training set was comparable to that of most comparable models with the full training set. Consequently, the proposed TransMT-Net has demonstrated its potential performance in GI tract endoscopic images and it through active learning can alleviate the shortage of labeled images.

CLELNet: A continual learning network for esophageal lesion analysis on endoscopic images.

BACKGROUND AND OBJECTIVE: A deep learning-based intelligent diagnosis system can significantly reduce the burden of endoscopists in the daily analysis of esophageal lesions. Considering the need to add new tasks in the diagnosis system, a deep learning model that can train a series of tasks incrementally using endoscopic images is essential for identifying the types and regions of esophageal lesions. METHOD: In this paper, we proposed a continual learning-based esophageal lesion network (CLELNet), in which a convolutional autoencoder was designed to extract representation features of endoscopic images among different esophageal lesions. The proposed CLELNet consists of shared layers and task-specific layers. Shared layers are used to extract common features among different lesions while task-specific layers can complete different tasks. The first two tasks trained by the CLELNet are the classification (task 1) and the segmentation (task 2). We collected a dataset of esophageal endoscopic images from Macau Kiang Wu Hospital for training and testing the CLELNet. RESULTS: The experimental results showed that the classification accuracy of task 1 was 95.96%, and the Intersection Over Union and the Dice Similarity Coefficient of task 2 were 65.66% and 78.08%, respectively. CONCLUSIONS: The proposed CLELNet can realize task-incremental learning without forgetting the previous tasks and thus become a useful computer-aided diagnosis system in esophageal lesions analysis.

Diagnosis of Esophageal Lesions by Multi-Classification and Segmentation Using an Improved Multi-Task Deep Learning Model.

It is challenging for endoscopists to accurately detect esophageal lesions during gastrointestinal endoscopic screening due to visual similarities among different lesions in terms of shape, size, and texture among patients. Additionally, endoscopists are busy fighting esophageal lesions every day, hence the need to develop a computer-aided diagnostic tool to classify and segment the lesions at endoscopic images to reduce their burden. Therefore, we propose a multi-task classification and segmentation (MTCS) model, including the Esophageal Lesions Classification Network (ELCNet) and Esophageal Lesions Segmentation Network (ELSNet). The ELCNet was used to classify types of esophageal lesions, and the ELSNet was used to identify lesion regions. We created a dataset by collecting 805 esophageal images from 255 patients and 198 images from 64 patients to train and evaluate the MTCS model. Compared with other methods, the proposed not only achieved a high accuracy (93.43%) in classification but achieved a dice similarity coefficient (77.84%) in segmentation. In conclusion, the MTCS model can boost the performance of endoscopists in the detection of esophageal lesions as it can accurately multi-classify and segment the lesions and is a potential assistant for endoscopists to reduce the risk of oversight.

Hepatoprotective effect of Qushihuayu formula on non-alcoholic steatohepatitis induced by MCD diet in rat.

BACKGROUND: Non-alcoholic steatohepatitis (NASH) is an advanced form of non-alcoholic fatty liver disease (NAFLD) for which there is yet any standard pharmacotherapy. Traditional Chinese medicine formula such as Qushihuayu (QSHY) composing of multiple bioactive compounds has been used to treat NAFLD and NASH and shows beneficial effects over single compound treatment. This study aimed to investigate the mechanism of hepatoprotective effect of QSHY formula using a rat model. METHODS: Six-weeks old male Wistar rats were given methionine/choline supplemented (MCS) diet for 8 weeks and used as the blank control. Another 7 rats, which received methionine/choline deficient (MCD) diet in the first 6 weeks and a MCS&MCD (1:1) mixture diet in the last 2 weeks, were used as the model group. The groups of QSHY pre-treatment, low dosage, medium dosage and high dosage were given the same diet as the model group. Except for pre-treatment group (1 week in advanced of other groups), all QSHY treatment groups received QSHY formula by gavage every day since the MCD diet started. RESULTS: In the MCD diet group, the QSHY formula decreased the serum ALT and AST levels, lipid droplets, inflammation foci, FAS and α-SMA protein expression than MCD diet group. MAPK pathways phospharylation were markedly depressed by the QSHY formula. Moreover, QSHY formula enhanced PPAR-γ and p-p65 translocating into nucleus. The administration of QSHY increased hepatic mRNA levels of Transcription Factor 1 alpha (HNF1A), Hepatocyte Nuclear Factor 4 alpha (HNF4A) and Forkhead box protein A3 (FOXA3) which play a pivotal role in Hepatic stellate cell (HSCs) reprogramming. CONCLUSION: These findings suggest that QSHY formula exerts a hepatoprotective effect against steatosis and fibrosis presumably via depressed MAPK pathways phosphorylation, reinforcement of PPAR-γ and p-p65 translocating into nucleus and enhanced HSCs reprogramming.

Multi-Task Model for Esophageal Lesion Analysis Using Endoscopic Images: Classification with Image Retrieval and Segmentation with Attention.

The automatic analysis of endoscopic images to assist endoscopists in accurately identifying the types and locations of esophageal lesions remains a challenge. In this paper, we propose a novel multi-task deep learning model for automatic diagnosis, which does not simply replace the role of endoscopists in decision making, because endoscopists are expected to correct the false results predicted by the diagnosis system if more supporting information is provided. In order to help endoscopists improve the diagnosis accuracy in identifying the types of lesions, an image retrieval module is added in the classification task to provide an additional confidence level of the predicted types of esophageal lesions. In addition, a mutual attention module is added in the segmentation task to improve its performance in determining the locations of esophageal lesions. The proposed model is evaluated and compared with other deep learning models using a dataset of 1003 endoscopic images, including 290 esophageal cancer, 473 esophagitis, and 240 normal. The experimental results show the promising performance of our model with a high accuracy of 96.76% for the classification and a Dice coefficient of 82.47% for the segmentation. Consequently, the proposed multi-task deep learning model can be an effective tool to help endoscopists in judging esophageal lesions.

Intelligent diagnosis of gastric intestinal metaplasia based on convolutional neural network and limited number of endoscopic images.

BACKGROUND: Gastric intestinal metaplasia (GIM) is a precancerous lesion of gastric cancer. Currently, diagnosis of GIM is based on the experience of a physician, which is liable to interobserver variability. Thus, an intelligent diagnostic (ID) system, based on narrow-band and magnifying narrow-band images, was constructed to provide objective assistance in the diagnosis of GIM. METHOD: We retrospectively collected 1880 endoscopic images (1048 GIM and 832 non-GIM) via biopsy from 336 patients confirmed histologically as GIM or non-GIM, from the Kiang Wu Hospital, Macau. We developed an ID system with these images using a modified convolutional neural network algorithm. A separate test dataset containing 477 pathologically confirmed images (242 GIM and 235 non-GIM) from 80 patients was used to test the performance of the ID system. Experienced endoscopists also examined the same test dataset, for comparison with the ID system. One of the challenges faced in this study was that it was difficult to obtain a large number of training images. Thus, data augmentation and transfer learning were applied together. RESULTS: The area under the receiver operating characteristic curve was 0.928 for the pre-patient analysis of the ID system, while the sensitivities, specificities, and accuracies of the ID system against those of the human experts were (91.9% vs. 86.5%, p-value = 1.000) (86.0% vs. 81.4%, p-value = 0.754), and (88.8% vs. 83.8%, p-value = 0.424), respectively. Even though the three indices of the ID system were slightly higher than those of the human experts, there were no significant differences. CONCLUSIONS: In this pilot study, a novel ID system was developed to diagnose GIM. This system exhibits promising diagnostic performance. It is believed that the proposed system has the potential for clinical application in the future.

Hepatoprotective Effect of Jianpi Huoxue Formula on Nonalcoholic Fatty Liver Disease Induced by Methionine-Choline-Deficient Diet in Rat.

In parallel with the prevalence metabolic syndrome, nonalcoholic fatty liver disease (NAFLD) has become the most common chronic liver disease in most countries. It features a constellation of simple steatosis, nonalcoholic steatohepatitis (NASH), fibrosis, cirrhosis, and even hepatocellular carcinoma. There are no approved drugs for effective management of NAFLD and NASH. Jianpi Huoxue formula (JPHX) mainly consists of Atractylodes macrocephal (Baizhu), Salvia miltiorrhiza (Danshen), Rasux Paeonia Alba (Baishao), Rhizoma Alismatis (Zexie), and Fructus Schisandrae Chinensis (Wuweizi), which may have beneficial effects on NAFLD. The aim of the study was to identify the effect of JPHX on NAFLD. A NAFLD model was induced by methionine-choline-deficient food (MCD) in Wistar rats and orally administered with simultaneous JPHX, once a day for 8 weeks. Hepatocellular injury, lipid profile, inflammation, fibrosis, and apoptosis were evaluated. The results showed that JPHX significantly decreased the abnormal serum alanine aminotransferase (ALT) and aspartate aminotransferase (AST) levels compared with the MCD model (P<0.05). Furthermore, JPHX protected MCD diet-fed rats from accumulation of hepatic triglycerides (TG) and total cholesterol (TC). Histological examination demonstrated that JPHX noticeably normalized the NAFLD activity score (NAS). Moreover, JPHX ameliorated liver inflammation by decreasing TNF-α levels and reduced collagen and matrix metalloproteinases in MCD diet-fed rats. In addition, JPHX prevented rats from MCD-induced cellular apoptosis, as suggested by TUNEL staining, and suppressed the activation of caspase 3 and 7 proteins. JPHX also inhibited the phosphorylation of JNK. In conclusion, JPHX exhibited a hepatoprotective effect against NAFLD in an MCD experimental model.