Joint segmentation and classification of hepatic lesions in ultrasound images using deep learning.

OBJECTIVES: To develop a convolutional neural network system to jointly segment and classify a hepatic lesion selected by user clicks in ultrasound images. METHODS: In total, 4309 anonymized ultrasound images of 3873 patients with hepatic cyst (n = 1214), hemangioma (n = 1220), metastasis (n = 1001), or hepatocellular carcinoma (HCC) (n = 874) were collected and annotated. The images were divided into 3909 training and 400 test images. Our network is composed of one shared encoder and two inference branches used for segmentation and classification and takes the concatenation of an input image and two Euclidean distance maps of foreground and background clicks provided by a user as input. The performance of hepatic lesion segmentation was evaluated based on the Jaccard index (JI), and the performance of classification was based on accuracy, sensitivity, specificity, and the area under the receiver operating characteristic curve (AUROC). RESULTS: We achieved performance improvements by jointly conducting segmentation and classification. In the segmentation only system, the mean JI was 68.5%. In the classification only system, the accuracy of classifying four types of hepatic lesions was 79.8%. The mean JI and classification accuracy were 68.5% and 82.2%, respectively, for the proposed joint system. The optimal sensitivity and specificity and the AUROC of classifying benign and malignant hepatic lesions of the joint system were 95.0%, 86.0%, and 0.970, respectively. The respective sensitivity, specificity, and the AUROC for classifying four hepatic lesions of the joint system were 86.7%, 89.7%, and 0.947. CONCLUSIONS: The proposed joint system exhibited fair performance compared to segmentation only and classification only systems. KEY POINTS: • The joint segmentation and classification system using deep learning accurately segmented and classified hepatic lesions selected by user clicks in US examination. • The joint segmentation and classification system for hepatic lesions in US images exhibited higher performance than segmentation only and classification only systems. • The joint segmentation and classification system could assist radiologists with minimal experience in US imaging by characterizing hepatic lesions.

Deep learning with ultrasonography: automated classification of liver fibrosis using a deep convolutional neural network.

OBJECTIVES: The aim of this study was to develop a deep convolutional neural network (DCNN) for the prediction of the METAVIR score using B-mode ultrasonography images. METHODS: Datasets from two tertiary academic referral centers were used. A total of 13,608 ultrasonography images from 3446 patients who underwent surgical resection, biopsy, or transient elastography were used for training a DCNN for the prediction of the METAVIR score. Pathological specimens or estimated METAVIR scores derived from transient elastography were used as a reference standard. A four-class model (F0 vs. F1 vs. F23 vs. F4) was developed. Diagnostic performance of the algorithm was validated on a separate internal test set of 266 patients with 300 images and external test set of 572 patients with 1232 images. Performance in classification of cirrhosis was compared between the DCNN and five radiologists. RESULTS: The accuracy of the four-class model was 83.5% and 76.4% on the internal and external test set, respectively. The area under the receiver operating characteristic curve (AUC) for classification of cirrhosis (F4) was 0.901 (95% confidence interval [CI], 0.865-0.937) on the internal test set and 0.857 (95% CI, 0.825-0.889) on the external test set, respectively. The AUC of the DCNN for classification of cirrhosis (0.857) was significantly higher than that of all five radiologists (AUC range, 0.656-0.816; p value < 0.05) using the external test set. CONCLUSIONS: The DCNN showed high accuracy for determining METAVIR score using ultrasonography images and achieved better performance than that of radiologists in the diagnosis of cirrhosis. KEY POINTS: • DCNN accurately classified the ultrasonography images according to the METAVIR score. • The AUROC of this algorithm for cirrhosis assessment was significantly higher than that of radiologists. • DCNN using US images may offer an alternative tool for monitoring liver fibrosis.

Deep learning-based automated quantification of the hepatorenal index for evaluation of fatty liver by ultrasonography.

PURPOSE: The aim of this study was to develop and validate a fully-automatic quantification of the hepatorenal index (HRI) calculated by a deep convolutional neural network (DCNN) comparable to the interpretations of radiologists experienced in ultrasound (US) imaging. METHODS: In this retrospective analysis, DCNN-based organ segmentation with Gaussian mixture modeling for automated quantification of the HRI was developed using abdominal US images from a previous study. For validation, 294 patients who underwent abdominal US examination before living-donor liver transplantation were selected. Interobserver agreement for the measured brightness of the liver and kidney and the calculated HRI were analyzed between two board-certified radiologists and DCNN using intraclass correlation coefficients (ICCs). RESULTS: Most patients had normal (n=95) or mild (n=198) fatty liver. The ICCs of hepatic and renal brightness measurements and the calculated HRI between the two radiologists were 0.892 (95% confidence interval [CI], 0.866 to 0.913), 0.898 (95% CI, 0.873 to 0.918), and 0.681 (95% CI, 0.615 to 0.738) for the first session and 0.920 (95% CI, 0.901 to 0.936), 0.874 (95% CI, 0.844 to 0.898), and 0.579 (95% CI, 0.497 to 0.650) for the second session, respectively; the results ranged from moderate to excellent agreement. Using the same task, the ICCs of the hepatic and renal measurements and the calculated HRI between the average values of the two radiologists and DCNN were 0.919 (95% CI, 0.899 to 0.935), 0.916 (95% CI, 0.895 to 0.932), and 0.734 (95% CI, 0.676 to 0.782), respectively, showing high to excellent agreement. CONCLUSION: Automated quantification of HRI using DCNN can yield HRI measurements similar to those obtained by experienced radiologists in patients with normal or mild fatty liver.

Technology trends and applications of deep learning in ultrasonography: image quality enhancement, diagnostic support, and improving workflow efficiency.

In this review of the most recent applications of deep learning to ultrasound imaging, the architectures of deep learning networks are briefly explained for the medical imaging applications of classification, detection, segmentation, and generation. Ultrasonography applications for image processing and diagnosis are then reviewed and summarized, along with some representative imaging studies of the breast, thyroid, heart, kidney, liver, and fetal head. Efforts towards workflow enhancement are also reviewed, with an emphasis on view recognition, scanning guide, image quality assessment, and quantification and measurement. Finally some future prospects are presented regarding image quality enhancement, diagnostic support, and improvements in workflow efficiency, along with remarks on hurdles, benefits, and necessary collaborations.

Automated detection of bone metastatic changes using serial CT scans.

Bone metastases resulting from a primary tumor invasion to the bone are common and cause significant morbidity in advanced cancer patients. Although the detection of bone metastases is often straightforward, it is difficult to identify their spread and track their changes, particularly in early stages. This paper presents a novel method that automatically finds the changes in appearance and the progress of bone metastases using longitudinal CT images. In contrast to previous methods based on nodule detection within a specific bone site in an individual CT scan, the approach in the present study is based on the subtraction between two registered CT volumes. The volumes registered using the proposed weighted-Demons registration and symmetric warping were subtracted with minimizing noise, and the Jacobian and false positive suppressions were performed to reduce false alarms. The proposed method detects the changes in bone metastases within 3min for entire chest bone structures covering the spine, ribs, and sternum. The method was validated based on 3-fold cross validation using the radiologists' markings of 459 lesions in 24 subjects and was performed with a sensitivity of 92.59%, a false positive volume of 2.58%, and 9.71 false positives per patient. Note that 113 lesions (24%) missed by the radiologists were identified by the present system and confirmed to be true metastases. Indeed, three patients diagnosed initially as normal, having no metastatic difference, by radiologists were found to be abnormal using the proposed system. Automatic detection method of bone metastatic changes in the entire chest bone was developed. Weighted Demons, symmetric warping, following false positive suppressions, and their parallel computing implementation enabled precise and fast computation of delicate changes in serial CT scans. The cross validation proved that this method can be quite useful for assisting radiologists in sensing minute metastatic changes from early stage.