Computational models can distinguish the contribution from different mechanisms to familiarity recognition.

Familiarity is the strange feeling of knowing that something has already been seen in our past. Over the past decades, several attempts have been made to model familiarity using artificial neural networks. Recently, two learning algorithms successfully reproduced the functioning of the perirhinal cortex, a key structure involved during familiarity: Hebbian and anti-Hebbian learning. However, performance of these learning rules is very different from one to another thus raising the question of their complementarity. In this work, we designed two distinct computational models that combined Deep Learning and a Hebbian learning rule to reproduce familiarity on natural images, the Hebbian model and the anti-Hebbian model, respectively. We compared the performance of both models during different simulations to highlight the inner functioning of both learning rules. We showed that the anti-Hebbian model fits human behavioral data whereas the Hebbian model fails to fit the data under large training set sizes. Besides, we observed that only our Hebbian model is highly sensitive to homogeneity between images. Taken together, we interpreted these results considering the distinction between absolute and relative familiarity. With our framework, we proposed a novel way to distinguish the contribution of these familiarity mechanisms to the overall feeling of familiarity. By viewing them as complementary, our two models allow us to make new testable predictions that could be of interest to shed light on the familiarity phenomenon.

Multitask deep learning on mammography to predict extensive intraductal component in invasive breast cancer.

OBJECTIVES: To develop a multitask deep learning (DL) algorithm to automatically classify mammography imaging findings and predict the existence of extensive intraductal component (EIC) in invasive breast cancer. METHODS: Mammograms with invasive breast cancers from 2010 to 2019 were downloaded for two radiologists performing image segmentation and imaging findings annotation. Images were randomly split into training, validation, and test datasets. A multitask approach was performed on the EfficientNet-B0 neural network mainly to predict EIC and classify imaging findings. Three more models were trained for comparison, including a single-task model (predicting EIC), a two-task model (predicting EIC and cell receptor status), and a three-task model (combining the abovementioned tasks). Additionally, these models were trained in a subgroup of invasive ductal carcinoma. The DeLong test was used to examine the difference in model performance. RESULTS: This study enrolled 1459 breast cancers on 3076 images. The EIC-positive rate was 29.0%. The three-task model was the best DL model with an area under the curve (AUC) of EIC prediction of 0.758 and 0.775 at the image and breast (patient) levels, respectively. Mass was the most accurately classified imaging finding (AUC = 0.915), followed by calcifications and mass with calcifications (AUC = 0.878 and 0.824, respectively). Cell receptor status prediction was less accurate (AUC = 0.625-0.653). The multitask approach improves the model training compared to the single-task model, but without significant effects. CONCLUSIONS: A mammography-based multitask DL model can perform simultaneous imaging finding classification and EIC prediction. CLINICAL RELEVANCE STATEMENT: The study results demonstrated the potential of deep learning to extract more information from mammography for clinical decision-making. KEY POINTS: • Extensive intraductal component (EIC) is an independent risk factor of local tumor recurrence after breast-conserving surgery. • A mammography-based deep learning model was trained to predict extensive intraductal component close to radiologists' reading. • The developed multitask deep learning model could perform simultaneous imaging finding classification and extensive intraductal component prediction.

Smart forecasting of artifacts in contrast-enhanced breast MRI before contrast agent administration.

OBJECTIVES: To evaluate whether artifacts on contrast-enhanced (CE) breast MRI maximum intensity projections (MIPs) might already be forecast before gadolinium-based contrast agent (GBCA) administration during an ongoing examination by analyzing the unenhanced T1-weighted images acquired before the GBCA injection. MATERIALS AND METHODS: This IRB-approved retrospective analysis consisted of n = 2884 breast CE MRI examinations after intravenous administration of GBCA, acquired with n = 4 different MRI devices at different field strengths (1.5 T/3 T) during clinical routine. CE-derived subtraction MIPs were used to conduct a multi-class multi-reader evaluation of the presence and severity of artifacts with three independent readers. An ensemble classifier (EC) of five DenseNet models was used to predict artifacts for the post-contrast subtraction MIPs, giving as the input source only the pre-contrast T1-weighted sequence. Thus, the acquisition directly preceded the GBCA injection. The area under ROC (AuROC) and diagnostics accuracy scores were used to assess the performance of the neural network in an independent holdout test set (n = 285). RESULTS: After majority voting, potentially significant artifacts were detected in 53.6% (n = 1521) of all breast MRI examinations (age 49.6 ± 12.6 years). In the holdout test set (mean age 49.7 ± 11.8 years), at a specificity level of 89%, the EC could forecast around one-third of artifacts (sensitivity 31%) before GBCA administration, with an AuROC = 0.66. CONCLUSION: This study demonstrates the capability of a neural network to forecast the occurrence of artifacts on CE subtraction data before the GBCA administration. If confirmed in larger studies, this might enable a workflow-blended approach to prevent breast MRI artifacts by implementing in-scan personalized predictive algorithms. CLINICAL RELEVANCE STATEMENT: Some artifacts in contrast-enhanced breast MRI maximum intensity projections might be predictable before gadolinium-based contrast agent injection using a neural network. KEY POINTS: • Potentially significant artifacts can be observed in a relevant proportion of breast MRI subtraction sequences after gadolinium-based contrast agent administration (GBCA). • Forecasting the occurrence of such artifacts in subtraction maximum intensity projections before GBCA administration for individual patients was feasible at 89% specificity, which allowed correctly predicting one in three future artifacts. • Further research is necessary to investigate the clinical value of such smart personalized imaging approaches.

Automated inversion time selection for late gadolinium-enhanced cardiac magnetic resonance imaging.

OBJECTIVES: To develop and share a deep learning method that can accurately identify optimal inversion time (TI) from multi-vendor, multi-institutional and multi-field strength inversion scout (TI scout) sequences for late gadolinium enhancement cardiac MRI. MATERIALS AND METHODS: Retrospective multicentre study conducted on 1136 1.5-T and 3-T cardiac MRI examinations from four centres and three scanner vendors. Deep learning models, comprising a convolutional neural network (CNN) that provides input to a long short-term memory (LSTM) network, were trained on TI scout pixel data from centres 1 to 3 to identify optimal TI, using ground truth annotations by two readers. Accuracy within 50 ms, mean absolute error (MAE), Lin's concordance coefficient (LCCC) and reduced major axis regression (RMAR) were used to select the best model from validation results, and applied to holdout test data. Robustness of the best-performing model was also tested on imaging data from centre 4. RESULTS: The best model (SE-ResNet18-LSTM) produced accuracy of 96.1%, MAE 22.9 ms and LCCC 0.47 compared to ground truth on the holdout test set and accuracy of 97.3%, MAE 15.2 ms and LCCC 0.64 when tested on unseen external (centre 4) data. Differences in vendor performance were observed, with greatest accuracy for the most commonly represented vendor in the training data. CONCLUSION: A deep learning model was developed that can identify optimal inversion time from TI scout images on multi-vendor data with high accuracy, including on previously unseen external data. We make this model available to the scientific community for further assessment or development. CLINICAL RELEVANCE STATEMENT: A robust automated inversion time selection tool for late gadolinium-enhanced imaging allows for reproducible and efficient cross-vendor inversion time selection. KEY POINTS: • A model comprising convolutional and recurrent neural networks was developed to extract optimal TI from TI scout images. • Model accuracy within 50 ms of ground truth on multi-vendor holdout and external data of 96.1% and 97.3% respectively was achieved. • This model could improve workflow efficiency and standardise optimal TI selection for consistent LGE imaging.

Early prediction of ventricular peritoneal shunt dependency in aneurysmal subarachnoid haemorrhage patients by recurrent neural network-based machine learning using routine intensive care unit data.

Aneurysmal subarachnoid haemorrhage (aSAH) can lead to complications such as acute hydrocephalic congestion. Treatment of this acute condition often includes establishing an external ventricular drainage (EVD). However, chronic hydrocephalus develops in some patients, who then require placement of a permanent ventriculoperitoneal (VP) shunt. The aim of this study was to employ recurrent neural network (RNN)-based machine learning techniques to identify patients who require VP shunt placement at an early stage. This retrospective single-centre study included all patients who were diagnosed with aSAH and treated in the intensive care unit (ICU) between November 2010 and May 2020 (n = 602). More than 120 parameters were analysed, including routine neurocritical care data, vital signs and blood gas analyses. Various machine learning techniques, including RNNs and gradient boosting machines, were evaluated for their ability to predict VP shunt dependency. VP-shunt dependency could be predicted using an RNN after just one day of ICU stay, with an AUC-ROC of 0.77 (CI: 0.75-0.79). The accuracy of the prediction improved after four days of observation (Day 4: AUC-ROC 0.81, CI: 0.79-0.84). At that point, the accuracy of the prediction was 76% (CI: 75.98-83.09%), with a sensitivity of 85% (CI: 83-88%) and a specificity of 74% (CI: 71-78%). RNN-based machine learning has the potential to predict VP shunt dependency on Day 4 after ictus in aSAH patients using routine data collected in the ICU. The use of machine learning may allow early identification of patients with specific therapeutic needs and accelerate the execution of required procedures.

Convolutional neural network for automated tooth segmentation on intraoral scans.

BACKGROUND: Tooth segmentation on intraoral scanned (IOS) data is a prerequisite for clinical applications in digital workflows. Current state-of-the-art methods lack the robustness to handle variability in dental conditions. This study aims to propose and evaluate the performance of a convolutional neural network (CNN) model for automatic tooth segmentation on IOS images. METHODS: A dataset of 761 IOS images (380 upper jaws, 381 lower jaws) was acquired using an intraoral scanner. The inclusion criteria included a full set of permanent teeth, teeth with orthodontic brackets, and partially edentulous dentition. A multi-step 3D U-Net pipeline was designed for automated tooth segmentation on IOS images. The model's performance was assessed in terms of time and accuracy. Additionally, the model was deployed on an online cloud-based platform, where a separate subsample of 18 IOS images was used to test the clinical applicability of the model by comparing three modes of segmentation: automated artificial intelligence-driven (A-AI), refined (R-AI), and semi-automatic (SA) segmentation. RESULTS: The average time for automated segmentation was 31.7 ± 8.1 s per jaw. The CNN model achieved an Intersection over Union (IoU) score of 91%, with the full set of teeth achieving the highest performance and the partially edentulous group scoring the lowest. In terms of clinical applicability, SA took an average of 860.4 s per case, whereas R-AI showed a 2.6-fold decrease in time (328.5 s). Furthermore, R-AI offered higher performance and reliability compared to SA, regardless of the dentition group. CONCLUSIONS: The 3D U-Net pipeline was accurate, efficient, and consistent for automatic tooth segmentation on IOS images. The online cloud-based platform could serve as a viable alternative for IOS segmentation.

Improving CBCT image quality to the CT level using RegGAN in esophageal cancer adaptive radiotherapy.

OBJECTIVE: This study aimed to improve the image quality and CT Hounsfield unit accuracy of daily cone-beam computed tomography (CBCT) using registration generative adversarial networks (RegGAN) and apply synthetic CT (sCT) images to dose calculations in radiotherapy. METHODS: The CBCT/planning CT images of 150 esophageal cancer patients undergoing radiotherapy were used for training (120 patients) and testing (30 patients). An unsupervised deep-learning method, the 2.5D RegGAN model with an adaptively trained registration network, was proposed, through which sCT images were generated. The quality of deep-learning-generated sCT images was quantitatively compared to the reference deformed CT (dCT) image using mean absolute error (MAE), root mean square error (RMSE) of Hounsfield units (HU), and peak signal-to-noise ratio (PSNR). The dose calculation accuracy was further evaluated for esophageal cancer radiotherapy plans, and the same plans were calculated on dCT, CBCT, and sCT images. RESULTS: The quality of sCT images produced by RegGAN was significantly improved compared to the original CBCT images. ReGAN achieved image quality in the testing patients with MAE sCT vs. CBCT: 43.7 ± 4.8 vs. 80.1 ± 9.1; RMSE sCT vs. CBCT: 67.2 ± 12.4 vs. 124.2 ± 21.8; and PSNR sCT vs. CBCT: 27.9 ± 5.6 vs. 21.3 ± 4.2. The sCT images generated by the RegGAN model showed superior accuracy on dose calculation, with higher gamma passing rates (93.3 ± 4.4, 90.4 ± 5.2, and 84.3 ± 6.6) compared to original CBCT images (89.6 ± 5.7, 85.7 ± 6.9, and 72.5 ± 12.5) under the criteria of 3 mm/3%, 2 mm/2%, and 1 mm/1%, respectively. CONCLUSION: The proposed deep-learning RegGAN model seems promising for generation of high-quality sCT images from stand-alone thoracic CBCT images in an efficient way and thus has the potential to support CBCT-based esophageal cancer adaptive radiotherapy.

Performance of a deep convolutional neural network for MRI-based vertebral body measurements and insufficiency fracture detection.

OBJECTIVES: The aim is to validate the performance of a deep convolutional neural network (DCNN) for vertebral body measurements and insufficiency fracture detection on lumbar spine MRI. METHODS: This retrospective analysis included 1000 vertebral bodies in 200 patients (age 75.2 ± 9.8 years) who underwent lumbar spine MRI at multiple institutions. 160/200 patients had ≥ one vertebral body insufficiency fracture, 40/200 had no fracture. The performance of the DCNN and that of two fellowship-trained musculoskeletal radiologists in vertebral body measurements (anterior/posterior height, extent of endplate concavity, vertebral angle) and evaluation for insufficiency fractures were compared. Statistics included (a) interobserver reliability metrics using intraclass correlation coefficient (ICC), kappa statistics, and Bland-Altman analysis, and (b) diagnostic performance metrics (sensitivity, specificity, accuracy). A statistically significant difference was accepted if the 95% confidence intervals did not overlap. RESULTS: The inter-reader agreement between radiologists and the DCNN was excellent for vertebral body measurements, with ICC values of > 0.94 for anterior and posterior vertebral height and vertebral angle, and good to excellent for superior and inferior endplate concavity with ICC values of 0.79-0.85. The performance of the DCNN in fracture detection yielded a sensitivity of 0.941 (0.903-0.968), specificity of 0.969 (0.954-0.980), and accuracy of 0.962 (0.948-0.973). The diagnostic performance of the DCNN was independent of the radiological institution (accuracy 0.964 vs. 0.960), type of MRI scanner (accuracy 0.957 vs. 0.964), and magnetic field strength (accuracy 0.966 vs. 0.957). CONCLUSIONS: A DCNN can achieve high diagnostic performance in vertebral body measurements and insufficiency fracture detection on heterogeneous lumbar spine MRI. KEY POINTS: • A DCNN has the potential for high diagnostic performance in measuring vertebral bodies and detecting insufficiency fractures of the lumbar spine.

The role of input imaging combination and ADC threshold on segmentation of acute ischemic stroke lesion using U-Net.

BACKGROUND: To evaluate the effect of the weighting of input imaging combo and ADC threshold on the performance of the U-Net and to find an optimized input imaging combo and ADC threshold in segmenting acute ischemic stroke (AIS) lesion. METHODS: This study retrospectively enrolled a total of 212 patients having AIS. Four combos, including ADC-ADC-ADC (AAA), DWI-ADC-ADC (DAA), DWI-DWI-ADC (DDA), and DWI-DWI-DWI (DDD), were used as input images, respectively. Three ADC thresholds including 0.6, 0.8 and 1.8 × 10-3 mm2/s were applied. Dice similarity coefficient (DSC) was used to evaluate the segmentation performance of U-Nets. Nonparametric Kruskal-Wallis test with Tukey-Kramer post-hoc tests were used for comparison. A p < .05 was considered statistically significant. RESULTS: The DSC significantly varied among different combos of images and different ADC thresholds. Hybrid U-Nets outperformed uniform U-Nets at ADC thresholds of 0.6 × 10-3 mm2/s and 0.8 × 10-3 mm2/s (p < .001). The U-Net with imaging combo of DDD had segmentation performance similar to hybrid U-Nets at an ADC threshold of 1.8 × 10-3 mm2/s (p = .062 to 1). The U-Net using the imaging combo of DAA at the ADC threshold of 0.6 × 10-3 mm2/s achieved the highest DSC in the segmentation of AIS lesion. CONCLUSIONS: The segmentation performance of U-Net for AIS varies among the input imaging combos and ADC thresholds. The U-Net is optimized by choosing the imaging combo of DAA at an ADC threshold of 0.6 × 10-3 mm2/s in segmentating AIS lesion with highest DSC. KEY POINTS: • Segmentation performance of U-Net for AIS differs among input imaging combos. • Segmentation performance of U-Net for AIS differs among ADC thresholds. • U-Net is optimized using DAA with ADC = 0.6 × 10-3 mm2/s.

Semiautomated segmentation of hepatocellular carcinoma tumors with MRI using convolutional neural networks.

OBJECTIVE: To assess the performance of convolutional neural networks (CNNs) for semiautomated segmentation of hepatocellular carcinoma (HCC) tumors on MRI. METHODS: This retrospective single-center study included 292 patients (237 M/55F, mean age 61 years) with pathologically confirmed HCC between 08/2015 and 06/2019 and who underwent MRI before surgery. The dataset was randomly divided into training (n = 195), validation (n = 66), and test sets (n = 31). Volumes of interest (VOIs) were manually placed on index lesions by 3 independent radiologists on different sequences (T2-weighted imaging [WI], T1WI pre-and post-contrast on arterial [AP], portal venous [PVP], delayed [DP, 3 min post-contrast] and hepatobiliary phases [HBP, when using gadoxetate], and diffusion-weighted imaging [DWI]). Manual segmentation was used as ground truth to train and validate a CNN-based pipeline. For semiautomated segmentation of tumors, we selected a random pixel inside the VOI, and the CNN provided two outputs: single slice and volumetric outputs. Segmentation performance and inter-observer agreement were analyzed using the 3D Dice similarity coefficient (DSC). RESULTS: A total of 261 HCCs were segmented on the training/validation sets, and 31 on the test set. The median lesion size was 3.0 cm (IQR 2.0-5.2 cm). Mean DSC (test set) varied depending on the MRI sequence with a range between 0.442 (ADC) and 0.778 (high b-value DWI) for single-slice segmentation; and between 0.305 (ADC) and 0.667 (T1WI pre) for volumetric-segmentation. Comparison between the two models showed better performance in single-slice segmentation, with statistical significance on T2WI, T1WI-PVP, DWI, and ADC. Inter-observer reproducibility of segmentation analysis showed a mean DSC of 0.71 in lesions between 1 and 2 cm, 0.85 in lesions between 2 and 5 cm, and 0.82 in lesions > 5 cm. CONCLUSION: CNN models have fair to good performance for semiautomated HCC segmentation, depending on the sequence and tumor size, with better performance for the single-slice approach. Refinement of volumetric approaches is needed in future studies. KEY POINTS: • Semiautomated single-slice and volumetric segmentation using convolutional neural networks (CNNs) models provided fair to good performance for hepatocellular carcinoma segmentation on MRI. • CNN models' performance for HCC segmentation accuracy depends on the MRI sequence and tumor size, with the best results on diffusion-weighted imaging and T1-weighted imaging pre-contrast, and for larger lesions.

Voxel-based morphometry in single subjects without a scanner-specific normal database using a convolutional neural network.

OBJECTIVES: Reliable detection of disease-specific atrophy in individual T1w-MRI by voxel-based morphometry (VBM) requires scanner-specific normal databases (NDB), which often are not available. The aim of this retrospective study was to design, train, and test a deep convolutional neural network (CNN) for single-subject VBM without the need for a NDB (CNN-VBM). MATERIALS AND METHODS: The training dataset comprised 8945 T1w scans from 65 different scanners. The gold standard VBM maps were obtained by conventional VBM with a scanner-specific NDB for each of the 65 scanners. CNN-VBM was tested in an independent dataset comprising healthy controls (n = 37) and subjects with Alzheimer's disease (AD, n = 51) or frontotemporal lobar degeneration (FTLD, n = 30). A scanner-specific NDB for the generation of the gold standard VBM maps was available also for the test set. The technical performance of CNN-VBM was characterized by the Dice coefficient of CNN-VBM maps relative to VBM maps from scanner-specific VBM. For clinical testing, VBM maps were categorized visually according to the clinical diagnoses in the test set by two independent readers, separately for both VBM methods. RESULTS: The VBM maps from CNN-VBM were similar to the scanner-specific VBM maps (median Dice coefficient 0.85, interquartile range [0.81, 0.90]). Overall accuracy of the visual categorization of the VBM maps for the detection of AD or FTLD was 89.8% for CNN-VBM and 89.0% for scanner-specific VBM. CONCLUSION: CNN-VBM without NDB provides a similar performance in the detection of AD- and FTLD-specific atrophy as conventional VBM. CLINICAL RELEVANCE STATEMENT: A deep convolutional neural network for voxel-based morphometry eliminates the need of scanner-specific normal databases without relevant performance loss and, therefore, could pave the way for the widespread clinical use of voxel-based morphometry to support the diagnosis of neurodegenerative diseases. KEY POINTS: • The need of normal databases is a barrier for widespread use of voxel-based brain morphometry. • A convolutional neural network achieved a similar performance for detection of atrophy than conventional voxel-based morphometry. • Convolutional neural networks can pave the way for widespread clinical use of voxel-based morphometry.

Deep learning HASTE sequence compared with T2-weighted BLADE sequence for liver MRI at 3 Tesla: a qualitative and quantitative prospective study.

OBJECTIVES: To qualitatively and quantitatively compare a single breath-hold fast half-Fourier single-shot turbo spin echo sequence with deep learning reconstruction (DL HASTE) with T2-weighted BLADE sequence for liver MRI at 3 T. METHODS: From December 2020 to January 2021, patients with liver MRI were prospectively included. For qualitative analysis, sequence quality, presence of artifacts, conspicuity, and presumed nature of the smallest lesion were assessed using the chi-squared and McNemar tests. For quantitative analysis, number of liver lesions, size of the smallest lesion, signal-to-noise ratio (SNR), and contrast-to-noise ratio (CNR) in both sequences were assessed using the paired Wilcoxon signed-rank test. Intraclass correlation coefficients (ICCs) and kappa coefficients were used to assess agreement between the two readers. RESULTS: One hundred and twelve patients were evaluated. Overall image quality (p = .006), artifacts (p < .001), and conspicuity of the smallest lesion (p = .001) were significantly better for the DL HASTE sequence than for the T2-weighted BLADE sequence. Significantly more liver lesions were detected with the DL HASTE sequence (356 lesions) than with the T2-weighted BLADE sequence (320 lesions; p < .001). CNR was significantly higher for the DL HASTE sequence (p < .001). SNR was higher for the T2-weighted BLADE sequence (p < .001). Interreader agreement was moderate to excellent depending on the sequence. Of the 41 supernumerary lesions visible only on the DL HASTE sequence, 38 (93%) were true-positives. CONCLUSION: The DL HASTE sequence can be used to improve image quality and contrast and reduces artifacts, allowing the detection of more liver lesions than with the T2-weighted BLADE sequence. CLINICAL RELEVANCE STATEMENT: The DL HASTE sequence is superior to the T2-weighted BLADE sequence for the detection of focal liver lesions and can be used in daily practice as a standard sequence. KEY POINTS: • The half-Fourier acquisition single-shot turbo spin echo sequence with deep learning reconstruction (DL HASTE sequence) has better overall image quality, reduced artifacts (particularly motion artifacts), and improved contrast, allowing the detection of more liver lesions than with the T2-weighted BLADE sequence. • The acquisition time of the DL HASTE sequence is at least eight times faster (21 s) than that of the T2-weighted BLADE sequence (3-5 min). • The DL HASTE sequence could replace the conventional T2-weighted BLADE sequence to meet the growing indication for hepatic MRI in clinical practice, given its diagnostic and time-saving performance.

Construction and evaluation of a gated high-resolution neural network for automatic brain metastasis detection and segmentation.

OBJECTIVES: To construct and evaluate a gated high-resolution convolutional neural network for detecting and segmenting brain metastasis (BM). METHODS: This retrospective study included craniocerebral MRI scans of 1392 patients with 14,542 BMs and 200 patients with no BM between January 2012 and April 2022. A primary dataset including 1000 cases with 11,686 BMs was employed to construct the model, while an independent dataset including 100 cases with 1069 BMs from other hospitals was used to examine the generalizability. The potential of the model for clinical use was also evaluated by comparing its performance in BM detection and segmentation to that of radiologists, and comparing radiologists' lesion detecting performances with and without model assistance. RESULTS: Our model yielded a recall of 0.88, a dice similarity coefficient (DSC) of 0.90, a positive predictive value (PPV) of 0.93 and a false positives per patient (FP) of 1.01 in the test set, and a recall of 0.85, a DSC of 0.89, a PPV of 0.93, and a FP of 1.07 in dataset from other hospitals. With the model's assistance, the BM detection rates of 4 radiologists improved significantly, ranging from 5.2 to 15.1% (all p < 0.001), and also for detecting small BMs with diameter ≤ 5 mm (ranging from 7.2 to 27.0%, all p < 0.001). CONCLUSIONS: The proposed model enables accurate BM detection and segmentation with higher sensitivity and less time consumption, showing the potential to augment radiologists' performance in detecting BM. CLINICAL RELEVANCE STATEMENT: This study offers a promising computer-aided tool to assist the brain metastasis detection and segmentation in routine clinical practice for cancer patients. KEY POINTS: • The GHR-CNN could accurately detect and segment BM on contrast-enhanced 3D-T1W images. • The GHR-CNN improved the BM detection rate of radiologists, including the detection of small lesions. • The GHR-CNN enabled automated segmentation of BM in a very short time.

Visual ensemble selection of deep convolutional neural networks for 3D segmentation of breast tumors on dynamic contrast enhanced MRI.

OBJECTIVES: To develop a visual ensemble selection of deep convolutional neural networks (CNN) for 3D segmentation of breast tumors using T1-weighted dynamic contrast-enhanced (T1-DCE) MRI. METHODS: Multi-center 3D T1-DCE MRI (n = 141) were acquired for a cohort of patients diagnosed with locally advanced or aggressive breast cancer. Tumor lesions of 111 scans were equally divided between two radiologists and segmented for training. The additional 30 scans were segmented independently by both radiologists for testing. Three 3D U-Net models were trained using either post-contrast images or a combination of post-contrast and subtraction images fused at either the image or the feature level. Segmentation accuracy was evaluated quantitatively using the Dice similarity coefficient (DSC) and the Hausdorff distance (HD95) and scored qualitatively by a radiologist as excellent, useful, helpful, or unacceptable. Based on this score, a visual ensemble approach selecting the best segmentation among these three models was proposed. RESULTS: The mean and standard deviation of DSC and HD95 between the two radiologists were equal to 77.8 ± 10.0% and 5.2 ± 5.9 mm. Using the visual ensemble selection, a DSC and HD95 equal to 78.1 ± 16.2% and 14.1 ± 40.8 mm was reached. The qualitative assessment was excellent (resp. excellent or useful) in 50% (resp. 77%). CONCLUSION: Using subtraction images in addition to post-contrast images provided complementary information for 3D segmentation of breast lesions by CNN. A visual ensemble selection allowing the radiologist to select the most optimal segmentation obtained by the three 3D U-Net models achieved comparable results to inter-radiologist agreement, yielding 77% segmented volumes considered excellent or useful. KEY POINTS: • Deep convolutional neural networks were developed using T1-weighted post-contrast and subtraction MRI to perform automated 3D segmentation of breast tumors. • A visual ensemble selection allowing the radiologist to choose the best segmentation among the three 3D U-Net models outperformed each of the three models. • The visual ensemble selection provided clinically useful segmentations in 77% of cases, potentially allowing for a valuable reduction of the manual 3D segmentation workload for the radiologist and greatly facilitating quantitative studies on non-invasive biomarker in breast MRI.

Super-resolution application of generative adversarial network on brain time-of-flight MR angiography: image quality and diagnostic utility evaluation.

OBJECTIVES: To develop a generative adversarial network (GAN) model to improve image resolution of brain time-of-flight MR angiography (TOF-MRA) and to evaluate the image quality and diagnostic utility of the reconstructed images. METHODS: We included 180 patients who underwent 1-min low-resolution (LR) and 4-min high-resolution (routine) brain TOF-MRA scans. We used 50 patients' datasets for training, 12 for quantitative image quality evaluation, and the rest for diagnostic validation. We modified a pix2pix GAN to suit TOF-MRA datasets and fine-tuned GAN-related parameters, including loss functions. Maximum intensity projection images were generated and compared using multi-scale structural similarity (MS-SSIM) and information theoretic-based statistic similarity measure (ISSM) index. Two radiologists scored vessels' visibilities using a 5-point Likert scale. Finally, we evaluated sensitivities and specificities of GAN-MRA in depicting aneurysms, stenoses, and occlusions. RESULTS: The optimal model was achieved with a lambda of 1e5 and L1 + MS-SSIM loss. Image quality metrics for GAN-MRA were higher than those for LR-MRA (MS-SSIM, 0.87 vs. 0.73; ISSM, 0.60 vs. 0.35; p.adjusted < 0.001). Vessels' visibility of GAN-MRA was superior to LR-MRA (rater A, 4.18 vs. 2.53; rater B, 4.61 vs. 2.65; p.adjusted < 0.001). In depicting vascular abnormalities, GAN-MRA showed comparable sensitivities and specificities, with greater sensitivity for aneurysm detection by one rater (93% vs. 84%, p < 0.05). CONCLUSIONS: An optimized GAN could significantly improve the image quality and vessel visibility of low-resolution brain TOF-MRA with equivalent sensitivity and specificity in detecting aneurysms, stenoses, and occlusions. KEY POINTS: • GAN could significantly improve the image quality and vessel visualization of low-resolution brain MR angiography (MRA). • With optimally adjusted training parameters, the GAN model did not degrade diagnostic performance by generating substantial false positives or false negatives. • GAN could be a promising approach for obtaining higher resolution TOF-MRA from images scanned in a fraction of time.

An automatic and accurate deep learning-based neuroimaging pipeline for the neonatal brain.

BACKGROUND: Accurate segmentation of neonatal brain tissues and structures is crucial for studying normal development and diagnosing early neurodevelopmental disorders. However, there is a lack of an end-to-end pipeline for automated segmentation and imaging analysis of the normal and abnormal neonatal brain. OBJECTIVE: To develop and validate a deep learning-based pipeline for neonatal brain segmentation and analysis of structural magnetic resonance images (MRI). MATERIALS AND METHODS: Two cohorts were enrolled in the study, including cohort 1 (582 neonates from the developing Human Connectome Project) and cohort 2 (37 neonates imaged using a 3.0-tesla MRI scanner in our hospital).We developed a deep leaning-based architecture capable of brain segmentation into 9 tissues and 87 structures. Then, extensive validations were performed for accuracy, effectiveness, robustness and generality of the pipeline. Furthermore, regional volume and cortical surface estimation were measured through in-house bash script implemented in FSL (Oxford Centre for Functional MRI of the Brain Software Library) to ensure reliability of the pipeline. Dice similarity score (DSC), the 95th percentile Hausdorff distance (H95) and intraclass correlation coefficient (ICC) were calculated to assess the quality of our pipeline. Finally, we finetuned and validated our pipeline on 2-dimensional thick-slice MRI in cohorts 1 and 2. RESULTS: The deep learning-based model showed excellent performance for neonatal brain tissue and structural segmentation, with the best DSC and the 95th percentile Hausdorff distance (H95) of 0.96 and 0.99 mm, respectively. In terms of regional volume and cortical surface analysis, our model showed good agreement with ground truth. The ICC values for the regional volume were all above 0.80. Considering the thick-slice image pipeline, the same trend was observed for brain segmentation and analysis. The best DSC and H95 were 0.92 and 3.00 mm, respectively. The regional volumes and surface curvature had ICC values just below 0.80. CONCLUSIONS: We propose an automatic, accurate, stable and reliable pipeline for neonatal brain segmentation and analysis from thin and thick structural MRI. The external validation showed very good reproducibility of the pipeline.

Predictive model for urosepsis in patients with Upper Urinary Tract Calculi based on ultrasonography and urinalysis using artificial intelligence learning.

PURPOSE: To construct a predicting model for urosepsis risk for patients with upper urinary tract calculi based on ultrasound and urinalysis. MATERIALS AND METHODS: A retrospective study was conducted in patients with upper urinary tract calculi admitted between January 2016 and January 2020. The patients were randomly grouped into the training and validation sets. The training set was used to identify the urosepsis risk factors and construct a risk prediction model based on ultrasound and urinalysis. The validation set was used to test the performance of the artificial neural network (ANN). RESULTS: Ultimately, 1716 patients (10.8% cases and 89.2% control) were included. Eight variables were selected for the model: sex, age, body temperature, diabetes history, urine leukocytes, urine nitrite, urine glucose, and degree of hydronephrosis. The area under the receiver operating curve in the validation and training sets was 0.945 (95% CI: 0.903-0.988) and 0.992 (95% CI: 0.988-0.997), respectively. Sensitivity, specificity, and Yuden index of the validation set (training set) were 80.4% (85.9%), 98.2% (99.0%), and 0.786 (0.849), respectively. CONCLUSIONS: A preliminary screening model for urosepsis based on ultrasound and urinalysis was constructed using ANN. The model could provide risk assessments for urosepsis in patients with upper urinary tract calculi.

Deep convolutional neural network for preoperative prediction of microvascular invasion and clinical outcomes in patients with HCCs.

OBJECTIVES: We aimed to develop and validate a deep convolutional neural network (DCNN) model for preoperative prediction of microvascular invasion (MVI) in hepatocellular carcinoma (HCC) and its clinical outcomes using contrast-enhanced computed tomography (CECT) in a large population of candidates for surgery. METHODS: This retrospective study included 1116 patients with HCC who had undergone preoperative CECT and curative hepatectomy. Radiological (R), DCNN, and combined nomograms were constructed in a training cohort (n = 892) respectively based on clinicoradiological factors, DCNN probabilities, and all factors; the performance of each model was confirmed in a validation cohort (n = 244). Accuracy and the AUC to predict MVI were calculated. Disease-free survival (DFS) and overall survival (OS) after surgery were recorded. RESULTS: The proportion of MVI-positive patients was respectively 38.8% (346/892) and 35.7 % (87/244) in the training and validation cohorts. The AUCs of the R, DCNN, and combined nomograms were respectively 0.809, 0.929, and 0.940 in the training cohorts and 0.837, 0.865, and 0.897 in the validation cohort. The combined nomogram outperformed the R nomogram in the training (p < 0.001) and validation (p = 0.009) cohorts. There was a significant difference in DFS and OS between the R, DCNN, and combined nomogram-predicted groups with and without MVI (p < 0.001). CONCLUSIONS: The combined nomogram based on preoperative CECT performs well for preoperative prediction of MVI and outcome. KEY POINTS: • A combined nomogram based on clinical information, preoperative CECT, and DCNN can predict MVI and clinical outcomes of patients with HCC. • DCNN provides added diagnostic ability to predict MVI. • The AUCs of the combined nomogram are 0.940 and 0.897 in the training and validation cohorts, respectively.

Highly accelerated 3D MPRAGE using deep neural network-based reconstruction for brain imaging in children and young adults.

OBJECTIVES: This study aimed to accelerate the 3D magnetization-prepared rapid gradient-echo (MPRAGE) sequence for brain imaging through the deep neural network (DNN). METHODS: This retrospective study used the k-space data of 240 scans (160 for the training set, mean ± standard deviation age, 93 ± 80 months, 94 males; 80 for the test set, 106 ± 83 months, 44 males) of conventional MPRAGE (C-MPRAGE) and 102 scans (77 ± 74 months, 52 males) of both C-MPRAGE and accelerated MPRAGE. All scans were acquired with 3T scanners. DNN was developed with simulated-acceleration data generated by under-sampling. Quantitative error metrics were compared between images reconstructed with DNN, GRAPPA, and E-SPIRIT using the paired t-test. Qualitative image quality was compared between C-MPRAGE and accelerated MPRAGE reconstructed with DNN (DNN-MPRAGE) by two readers. Lesions were segmented and the agreement between C-MPRAGE and DNN-MPRAGE was assessed using linear regression. RESULTS: Accelerated MPRAGE reduced scan times by 38% compared to C-MPRAGE (142 s vs. 320 s). For quantitative error metrics, DNN showed better performance than GRAPPA and E-SPIRIT (p < 0.001). For qualitative evaluation, overall image quality of DNN-MPRAGE was comparable (p > 0.999) or better (p = 0.025) than C-MPRAGE, depending on the reader. Pixelation was reduced in DNN-MPRAGE (p < 0.001). Other qualitative parameters were comparable (p > 0.05). Lesions in C-MPRAGE and DNN-MPRAGE showed good agreement for the dice similarity coefficient (= 0.68) and linear regression (R2 = 0.97; p < 0.001). CONCLUSIONS: DNN-MPRAGE reduced acquisition time by 38% and revealed comparable image quality to C-MPRAGE. KEY POINTS: • DNN-MPRAGE reduced acquisition times by 38%. • DNN-MPRAGE outperformed conventional reconstruction on accelerated scans (SSIM of DNN-MPRAGE = 0.96, GRAPPA = 0.43, E-SPIRIT = 0.88; p < 0.001). • Compared to C-MPRAGE scans, DNN-MPRAGE showed improved mean scores for overall image quality (2.46 vs. 2.52; p < 0.001) or comparable perceived SNR (2.56 vs. 2.58; p = 0.08).

Developing an understanding of artificial intelligence lung nodule risk prediction using insights from the Brock model.

OBJECTIVES: To determine if predictions of the Lung Cancer Prediction convolutional neural network (LCP-CNN) artificial intelligence (AI) model are analogous to the Brock model. METHODS: In total, 10,485 lung nodules in 4660 participants from the National Lung Screening Trial (NLST) were analysed. Both manual and automated nodule measurements were inputted into the Brock model, and this was compared to LCP-CNN. The performance of an experimental AI model was tested after ablating imaging features in a manner analogous to removing predictors from the Brock model. First, the nodule was ablated leaving lung parenchyma only. Second, a sphere of the same size as the nodule was implanted in the parenchyma. Third, internal texture of both nodule and parenchyma was ablated. RESULTS: Automated axial diameter (AUC 0.883) and automated equivalent spherical diameter (AUC 0.896) significantly improved the accuracy of Brock when compared to manual measurement (AUC 0.873), although not to the level of the LCP-CNN (AUC 0.936). Ablating nodule and parenchyma texture (AUC 0.915) led to a small drop in AI predictive accuracy, as did implanting a sphere of the same size as the nodule (AUC 0.889). Ablating the nodule leaving parenchyma only led to a large drop in AI performance (AUC 0.717). CONCLUSIONS: Feature ablation is a feasible technique for understanding AI model predictions. Nodule size and morphology play the largest role in AI prediction, with nodule internal texture and background parenchyma playing a limited role. This is broadly analogous to the relative importance of morphological factors over clinical factors within the Brock model. KEY POINTS: • Brock lung cancer risk prediction accuracy was significantly improved using automated axial or equivalent spherical measurements of lung nodule diameter, when compared to manual measurements. • Predictive accuracy was further improved by using the Lung Cancer Prediction convolutional neural network, an artificial intelligence-based model which obviates the requirement for nodule measurement. • Nodule size and morphology are important factors in artificial intelligence lung cancer risk prediction, with nodule texture and background parenchyma contributing a small, but measurable, role.