Mine Your Own Anatomy: Revisiting Medical Image Segmentation With Extremely Limited Labels.

Recent studies on contrastive learning have achieved remarkable performance solely by leveraging few labels in the context of medical image segmentation. Existing methods mainly focus on instance discrimination and invariant mapping (i.e., pulling positive samples closer and negative samples apart in the feature space). However, they face three common pitfalls: (1) tailness: medical image data usually follows an implicit long-tail class distribution. Blindly leveraging all pixels in training hence can lead to the data imbalance issues, and cause deteriorated performance; (2) consistency: it remains unclear whether a segmentation model has learned meaningful and yet consistent anatomical features due to the intra-class variations between different anatomical features; and (3) diversity: the intra-slice correlations within the entire dataset have received significantly less attention. This motivates us to seek a principled approach for strategically making use of the dataset itself to discover similar yet distinct samples from different anatomical views. In this paper, we introduce a novel semi-supervised 2D medical image segmentation framework termed Mine yOur owNAnatomy (MONA), and make three contributions. First, prior work argues that every pixel equally matters to the model training; we observe empirically that this alone is unlikely to define meaningful anatomical features, mainly due to lacking the supervision signal. We show two simple solutions towards learning invariances-through the use of stronger data augmentations and nearest neighbors. Second, we construct a set of objectives that encourage the model to be capable of decomposing medical images into a collection of anatomical features in an unsupervised manner. Lastly, we both empirically and theoretically, demonstrate the efficacy of our MONA on three benchmark datasets with different labeled settings, achieving new state-of-the-art under different labeled semi-supervised settings. MONA makes minimal assumptions on domain expertise, and hence constitutes a practical and versatile solution in medical image analysis. We provide the PyTorch-like pseudo-code in supplementary.

Breast Cancers Detected during a Decade of Screening with Digital Breast Tomosynthesis: Comparison with Digital Mammography.

Background Digital breast tomosynthesis (DBT) has been shown to help increase cancer detection compared with two-dimensional digital mammography (DM). However, it is unclear whether additional tumor detection will improve outcomes or lead to overdiagnosis of breast cancer. Purpose This study aimed to compare cancer types and stages over 3 years of DM screening and 10 years of DBT screening to determine the effect of DBT. Materials and Methods A retrospective search identified breast cancers detected by using screening mammography from August 2008 through July 2021. Data collected included demographic, imaging, and pathologic information. Invasive cancers 2 cm or larger, human epidermal growth factor 2-positive or triple-negative tumors greater than 10 mm, axillary nodes positive for cancer, and distant organ spread were considered advanced cancers. The DBT and DM cohorts were compared and further analyzed by prevalent versus incident examinations. False-negative findings were also assessed. Results A total of 1407 breast cancers were analyzed (142 with DM, 1265 with DBT). DBT showed a higher rate of cancer depiction than DM (5.3 vs four cancers per 1000, respectively; P = .001), with a similar ratio of invasive cancers to ductal carcinomas in situ (76.5%:23.5% [968 and 297 of 1265, respectively] vs 71.1%:28.9% [101 and 41 of 142, respectively]). Mean invasive cancer size did not differ between DM and DBT (1.44 cm ± 0.93 [SD] vs 1.36 cm ± 1.14, respectively; P = .49), but incident DBT cases were smaller than prevalent cases (1.2 cm ± 1.0 vs 1.6 cm ± 1.4, respectively; P < .001). DBT and DM had similar rates of invasive cancer subtypes: low grade (26.5% [243 of 912] vs 29% [28 of 96], respectively), moderate grade (57.2% [522 of 912] vs 51% [49 of 96], respectively), and high grade (16.1% [147 of 912] vs 20% [19 of 96], respectively) (P = .65). The proportion of advanced cancers was lower with DBT than DM (32.6% [316 of 968] vs 43.6% [44 of 101], respectively; P = .04) and between DBT prevalent and incident screening (39.1% [133 of 340] vs 29.1% [183 of 628], respectively; P = .003). There was no difference in interval cancer rates (0.14 per 1000 with DM and 0.2 per 1000 with DBT; P = .42) for both groups. Conclusion DBT helped to increase breast cancer detection rate and depicted invasive cancers with a lower rate of advanced cancers compared with DM, with further improvement observed at incident rounds of screening. © RSNA, 2024 See also the editorial by Kim and Woo in this issue.

Coordinate-Independent 3-D Ultrasound Principal Stretch and Direction Imaging.

OBJECTIVE: In clinical ultrasound, current 2-D strain imaging faces challenges in quantifying three orthogonal normal strain components. This requires separate image acquisitions based on the pixel-dependent cardiac coordinate system, leading to additional computations and estimation discrepancies due to probe orientation. Most systems lack shear strain information, as displaying all components is challenging to interpret. METHODS: This paper presents a 3-D high-spatial-resolution, coordinate-independent strain imaging approach based on principal stretch and axis estimation. All strain components are transformed into three principal stretches along three normal principal axes, enabling direct visualization of the primary deformation. We devised an efficient 3-D speckle tracking method with tilt filtering, incorporating randomized searching in a two-pass tracking framework and rotating the phase of the 3-D correlation function for robust filtering. The proposed speckle tracking approach significantly improves estimates of displacement gradients related to the axial displacement component. Non-axial displacement gradient estimates are enhanced using a correlation-weighted least-squares method constrained by tissue incompressibility. RESULTS: Simulated and in vivo canine cardiac datasets were evaluated to estimate Lagrangian strains from end-diastole to end-systole. The proposed speckle tracking method improves displacement estimation by a factor of 4.3 to 10.5 over conventional 1-pass processing. Maximum principal axis/direction imaging enables better detection of local disease regions than conventional strain imaging. CONCLUSION: Coordinate-independent tracking can identify myocardial abnormalities with high accuracy. SIGNIFICANCE: This study offers enhanced accuracy and robustness in strain imaging compared to current methods.

TAI-GAN: A Temporally and Anatomically Informed Generative Adversarial Network for early-to-late frame conversion in dynamic cardiac PET inter-frame motion correction.

Inter-frame motion in dynamic cardiac positron emission tomography (PET) using rubidium-82 (82Rb) myocardial perfusion imaging impacts myocardial blood flow (MBF) quantification and the diagnosis accuracy of coronary artery diseases. However, the high cross-frame distribution variation due to rapid tracer kinetics poses a considerable challenge for inter-frame motion correction, especially for early frames where intensity-based image registration techniques often fail. To address this issue, we propose a novel method called Temporally and Anatomically Informed Generative Adversarial Network (TAI-GAN) that utilizes an all-to-one mapping to convert early frames into those with tracer distribution similar to the last reference frame. The TAI-GAN consists of a feature-wise linear modulation layer that encodes channel-wise parameters generated from temporal information and rough cardiac segmentation masks with local shifts that serve as anatomical information. Our proposed method was evaluated on a clinical 82Rb PET dataset, and the results show that our TAI-GAN can produce converted early frames with high image quality, comparable to the real reference frames. After TAI-GAN conversion, the motion estimation accuracy and subsequent myocardial blood flow (MBF) quantification with both conventional and deep learning-based motion correction methods were improved compared to using the original frames. The code is available at https://github.com/gxq1998/TAI-GAN.

In vivo neuropil density from anatomical MRI and machine learning.

Brain energy budgets specify metabolic costs emerging from underlying mechanisms of cellular and synaptic activities. While current bottom-up energy budgets use prototypical values of cellular density and synaptic density, predicting metabolism from a person's individualized neuropil density would be ideal. We hypothesize that in vivo neuropil density can be derived from magnetic resonance imaging (MRI) data, consisting of longitudinal relaxation (T1) MRI for gray/white matter distinction and diffusion MRI for tissue cellularity (apparent diffusion coefficient, ADC) and axon directionality (fractional anisotropy, FA). We present a machine learning algorithm that predicts neuropil density from in vivo MRI scans, where ex vivo Merker staining and in vivo synaptic vesicle glycoprotein 2A Positron Emission Tomography (SV2A-PET) images were reference standards for cellular and synaptic density, respectively. We used Gaussian-smoothed T1/ADC/FA data from 10 healthy subjects to train an artificial neural network, subsequently used to predict cellular and synaptic density for 54 test subjects. While excellent histogram overlaps were observed both for synaptic density (0.93) and cellular density (0.85) maps across all subjects, the lower spatial correlations both for synaptic density (0.89) and cellular density (0.58) maps are suggestive of individualized predictions. This proof-of-concept artificial neural network may pave the way for individualized energy atlas prediction, enabling microscopic interpretations of functional neuroimaging data.

Multicenter Quantification of Radiation Exposure and Associated Risks for Prostatic Artery Embolization in 1476 Patients.

Background Prostatic artery embolization (PAE) is a safe, minimally invasive angiographic procedure that effectively treats benign prostatic hyperplasia; however, PAE-related patient radiation exposure and associated risks are not completely understood. Purpose To quantify radiation dose and assess radiation-related adverse events in patients who underwent PAE at multiple centers. Materials and Methods This retrospective study included patients undergoing PAE for any indication performed by experienced operators at 10 high-volume international centers from January 2014 to May 2021. Patient characteristics, procedural and radiation dose data, and radiation-related adverse events were collected. Procedural radiation effective doses were calculated by multiplying kerma-area product values by an established conversion factor for abdominopelvic fluoroscopy-guided procedures. Relationships between cumulative air kerma (CAK) or effective dose and patient body mass index (BMI), fluoroscopy time, or radiation field area were assessed with linear regression. Differences in radiation dose stemming from radiopaque prostheses or fluoroscopy unit type were assessed using two-sample t tests and Wilcoxon rank sum tests. Results A total of 1476 patients (mean age, 69.9 years ± 9.0 [SD]) were included, of whom 1345 (91.1%) and 131 (8.9%) underwent the procedure with fixed interventional or mobile fluoroscopy units, respectively. Median procedure effective dose was 17.8 mSv for fixed interventional units and 12.3 mSv for mobile units. CAK and effective dose both correlated positively with BMI (R2 = 0.15 and 0.17; P < .001) and fluoroscopy time (R2 = 0.16 and 0.08; P < .001). No radiation-related 90-day adverse events were reported. Patients with radiopaque implants versus those without implants had higher median CAK (1452 mGy [range, 900-2685 mGy] vs 1177 mGy [range, 700-1959 mGy], respectively; P = .01). Median effective dose was lower for mobile than for fixed interventional systems (12.3 mSv [range, 8.5-22.0 mSv] vs 20.4 mSv [range, 13.8-30.6 mSv], respectively; P < .001). Conclusion Patients who underwent PAE performed with fixed interventional or mobile fluoroscopy units were exposed to a median effective radiation dose of 17.8 mSv or 12.3 mSv, respectively. No radiation-related adverse events at 90 days were reported. © RSNA, 2024 See also the editorial by Mahesh in this issue.

Multi-Task Learning for Motion Analysis and Segmentation in 3D Echocardiography.

Characterizing left ventricular deformation and strain using 3D+time echocardiography provides useful insights into cardiac function and can be used to detect and localize myocardial injury. To achieve this, it is imperative to obtain accurate motion estimates of the left ventricle. In many strain analysis pipelines, this step is often accompanied by a separate segmentation step; however, recent works have shown both tasks to be highly related and can be complementary when optimized jointly. In this work, we present a multi-task learning network that can simultaneously segment the left ventricle and track its motion between multiple time frames. Two task-specific networks are trained using a composite loss function. Cross-stitch units combine the activations of these networks by learning shared representations between the tasks at different levels. We also propose a novel shape-consistency unit that encourages motion propagated segmentations to match directly predicted segmentations. Using a combined synthetic and in-vivo 3D echocardiography dataset, we demonstrate that our proposed model can achieve excellent estimates of left ventricular motion displacement and myocardial segmentation. Additionally, we observe strong correlation of our image-based strain measurements with crystal-based strain measurements as well as good correspondence with SPECT perfusion mappings. Finally, we demonstrate the clinical utility of the segmentation masks in estimating ejection fraction and sphericity indices that correspond well with benchmark measurements.

Reliable Prostate Cancer Risk Mapping From MRI Using Targeted and Systematic Core Needle Biopsy Histopathology.

OBJECTIVE: To compute a dense prostate cancer risk map for the individual patient post-biopsy from magnetic resonance imaging (MRI) and to provide a more reliable evaluation of its fitness in prostate regions that were not identified as suspicious for cancer by a human-reader in pre- and intra-biopsy imaging analysis. METHODS: Low-level pre-biopsy MRI biomarkers from targeted and non-targeted biopsy locations were extracted and statistically tested for representativeness against biomarkers from non-biopsied prostate regions. A probabilistic machine learning classifier was optimized to map biomarkers to their core-level pathology, followed by extrapolation of pathology scores to non-biopsied prostate regions. Goodness-of-fit was assessed at targeted and non-targeted biopsy locations for the post-biopsy individual patient. RESULTS: Our experiments showed high predictability of imaging biomarkers in differentiating histopathology scores in thousands of non-targeted core-biopsy locations (ROC-AUCs: 0.85-0.88), but also high variability between patients (Median ROC-AUC [IQR]: 0.81-0.89 [0.29-0.40]). CONCLUSION: The sparseness of prostate biopsy data makes the validation of a whole gland risk mapping a non-trivial task. Previous studies i) focused on targeted-biopsy locations although biopsy-specimens drawn from systematically scattered locations across the prostate constitute a more representative sample to non-biopsied regions, and ii) estimated prediction-power across predicted instances (e.g., biopsy specimens) with no patient distinction, which may lead to unreliable estimation of model fitness to the individual patient due to variation between patients in instance count, imaging characteristics, and pathologies. SIGNIFICANCE: This study proposes a personalized whole-gland prostate cancer risk mapping post-biopsy to allow clinicians to better stage and personalize focal therapy treatment plans.

Deferral of Estimated Glomerular Filtration Rate Testing Before Contrast-Enhanced CT in Low-Risk Emergency Department Patients: Assessment of Safety and Impact on Throughput.

OBJECTIVE: To assess the safety and utility of deferring estimated glomerular filtration rate (eGFR) testing before contrast-enhanced CT (CECT) in low-risk emergency department (ED) patients. METHODS: A new question was added to CECT order screens, allowing ordering ED providers to defer eGFR testing in patients deemed low risk for contrast-induced acute kidney injury (AKI). Low risk was defined as no known chronic kidney disease (CKD) or risk factors for AKI or CKD. Patients on chronic dialysis were deemed low risk. The project included three phases: baseline, pilot (optional order question), and full implementation (required order question). Outcomes were operational throughput metrics of CECT order to protocol (O to P) and order to begin (O to B) times. As a balancing safety measure, the proportion of patients deemed to be "low risk" and subsequently found to have eGFR value less than 30 mL/min/1.73 m2 was reported. RESULTS: A total of 16,446 CECT studies were included from four EDs. In the pilot phase, provider engagement rates with the question were low (5%-14%). After full implementation, median O to P time improved from 23.93 min at baseline to 13.02 (P < .0001) and median O to B time improved from 80.34 min to 76.48 (P = .0002). In 0.3% (2 of 646) studies, CECT was completed in patients categorized as low risk by the ED provider with subsequently resulted eGFR <30 mL/min/1.73 m2. DISCUSSION: Upfront clinical risk assessment for AKI and CKD by ED providers can be used to safely defer eGFR testing and improve operational performance for patients requiring CECT.

Performance and reliability assessment of a lower dose, task-based scoliosis radiography protocol in pediatric patients.

BACKGROUND: Follow-up scoliosis radiographs are performed to assess the degree of spinal curvature and skeletal maturity, which can be done at lower radiation exposures than those in standard-dose radiography. OBJECTIVE: Describe and evaluate a protocol that reduced the radiation in follow-up frontal-view scoliosis radiographs. MATERIALS AND METHODS: We implemented a postero-anterior lower dose modified-technique for scoliosis radiography with task-based definition of adequate image quality and use of technique charts based on target exposure index and patient's height and weight. We subsequently retrospectively evaluated 40 consecutive patients who underwent a follow-up radiograph using the modified-technique after an initial standard-technique radiograph. We evaluated comparisons of proportions for subjective assessment with chi-squared tests, and agreements of reader's scores with intraclass correlation coefficients and Bland-Altman plots. We determined incident air kerma, exposure index, deviation index/standard deviation, dose-area product (DAP), and effective dose for each radiograph. We set statistical significance at P<0.05. RESULTS: Forty patients (65% female), aged 4-17 years. Median effective dose was reduced from 39 to 10 µSv (P<0.001), incident air kerma from 139 to 29 µSv (P<0.001), and DAP from 266 to 55 mGy*cm2 (P<0.001). All modified-technique parameters were rated with a mean score of acceptable or above. All modified-technique measurements obtained inter- and intra-observer correlation coefficient agreements of 0.86 ("Good") or greater. CONCLUSION: Substantial dose reduction on follow-up scoliosis imaging with existing radiography units is achievable through task-based definition of adequate image quality and tailoring of radiation to each patient's height and weight, while still allowing for reliable assessment and reproducible measurements.

INTEGRATING PROSTATE SPECIFIC ANTIGEN DENSITY BIOMARKER INTO DEEP LEARNING PROSTATE MRI LESION SEGMENTATION MODELS.

Prostate cancer lesion segmentation in multi-parametric magnetic resonance imaging (mpMRI) is crucial for pre-biopsy diagnosis and targeted biopsy guidance. Deep convolution neural networks have been widely utilized for lesion segmentation. However, these methods fail to achieve a high Dice coefficient because of the large variations in lesion size and location within the gland. To address this problem, we integrate the clinically-meaningful prostate specific antigen density (PSAD) biomarker into the deep learning model using feature-wise transformations to condition the features in latent space, and thus control the size of lesion prediction. We tested our models on a public dataset with 214 annotated mpMRI scans and compared the segmentation performance to a baseline 3D U-Net model. Results demonstrate that integrating the PSAD biomarker significantly improves segmentation performance in both Dice coefficient and centroid distance metric.

Bootstrapping Semi-supervised Medical Image Segmentation with Anatomical-Aware Contrastive Distillation.

Contrastive learning has shown great promise over annotation scarcity problems in the context of medical image segmentation. Existing approaches typically assume a balanced class distribution for both labeled and unlabeled medical images. However, medical image data in reality is commonly imbalanced (i.e., multi-class label imbalance), which naturally yields blurry contours and usually incorrectly labels rare objects. Moreover, it remains unclear whether all negative samples are equally negative. In this work, we present ACTION, an Anatomical-aware ConTrastive dIstillatiON framework, for semi-supervised medical image segmentation. Specifically, we first develop an iterative contrastive distillation algorithm by softly labeling the negatives rather than binary supervision between positive and negative pairs. We also capture more semantically similar features from the randomly chosen negative set compared to the positives to enforce the diversity of the sampled data. Second, we raise a more important question: Can we really handle imbalanced samples to yield better performance? Hence, the key innovation in ACTION is to learn global semantic relationship across the entire dataset and local anatomical features among the neighbouring pixels with minimal additional memory footprint. During the training, we introduce anatomical contrast by actively sampling a sparse set of hard negative pixels, which can generate smoother segmentation boundaries and more accurate predictions. Extensive experiments across two benchmark datasets and different unlabeled settings show that ACTION significantly outperforms the current state-of-the-art semi-supervised methods.

A Multiclass Radiomics Method-Based WHO Severity Scale for Improving COVID-19 Patient Assessment and Disease Characterization From CT Scans.

OBJECTIVES: The aim of this study was to evaluate the severity of COVID-19 patients' disease by comparing a multiclass lung lesion model to a single-class lung lesion model and radiologists' assessments in chest computed tomography scans. MATERIALS AND METHODS: The proposed method, AssessNet-19, was developed in 2 stages in this retrospective study. Four COVID-19-induced tissue lesions were manually segmented to train a 2D-U-Net network for a multiclass segmentation task followed by extensive extraction of radiomic features from the lung lesions. LASSO regression was used to reduce the feature set, and the XGBoost algorithm was trained to classify disease severity based on the World Health Organization Clinical Progression Scale. The model was evaluated using 2 multicenter cohorts: a development cohort of 145 COVID-19-positive patients from 3 centers to train and test the severity prediction model using manually segmented lung lesions. In addition, an evaluation set of 90 COVID-19-positive patients was collected from 2 centers to evaluate AssessNet-19 in a fully automated fashion. RESULTS: AssessNet-19 achieved an F1-score of 0.76 ± 0.02 for severity classification in the evaluation set, which was superior to the 3 expert thoracic radiologists (F1 = 0.63 ± 0.02) and the single-class lesion segmentation model (F1 = 0.64 ± 0.02). In addition, AssessNet-19 automated multiclass lesion segmentation obtained a mean Dice score of 0.70 for ground-glass opacity, 0.68 for consolidation, 0.65 for pleural effusion, and 0.30 for band-like structures compared with ground truth. Moreover, it achieved a high agreement with radiologists for quantifying disease extent with Cohen κ of 0.94, 0.92, and 0.95. CONCLUSIONS: A novel artificial intelligence multiclass radiomics model including 4 lung lesions to assess disease severity based on the World Health Organization Clinical Progression Scale more accurately determines the severity of COVID-19 patients than a single-class model and radiologists' assessment.

Predicting tumor recurrence on baseline MR imaging in patients with early-stage hepatocellular carcinoma using deep machine learning.

Tumor recurrence affects up to 70% of early-stage hepatocellular carcinoma (HCC) patients, depending on treatment option. Deep learning algorithms allow in-depth exploration of imaging data to discover imaging features that may be predictive of recurrence. This study explored the use of convolutional neural networks (CNN) to predict HCC recurrence in patients with early-stage HCC from pre-treatment magnetic resonance (MR) images. This retrospective study included 120 patients with early-stage HCC. Pre-treatment MR images were fed into a machine learning pipeline (VGG16 and XGBoost) to predict recurrence within six different time frames (range 1-6 years). Model performance was evaluated with the area under the receiver operating characteristic curves (AUC-ROC). After prediction, the model's clinical relevance was evaluated using Kaplan-Meier analysis with recurrence-free survival (RFS) as the endpoint. Of 120 patients, 44 had disease recurrence after therapy. Six different models performed with AUC values between 0.71 to 0.85. In Kaplan-Meier analysis, five of six models obtained statistical significance when predicting RFS (log-rank p < 0.05). Our proof-of-concept study indicates that deep learning algorithms can be utilized to predict early-stage HCC recurrence. Successful identification of high-risk recurrence candidates may help optimize follow-up imaging and improve long-term outcomes post-treatment.

Operator Learning Curve for Prostatic Artery Embolization and Its Impact on Outcomes in 296 Patients.

PURPOSE: To define operator learning curve inflection points for prostatic artery embolization (PAE) and their impact on technical efficiency, clinical outcomes, and adverse events. MATERIALS AND METHODS: Between May 2013 and May 2021, 296 consecutive patients with moderate-to-severe lower urinary tract symptoms, urinary retention, or gross hematuria from benign prostatic hyperplasia underwent PAE by an interventional radiologist without prior PAE-specific experience. Operator learning curves plotted procedure time, fluoroscopy time, contrast volume, and embolic endpoint data against sequential procedure number. Multiple regression analysis evaluated for improvements in these parameters, with segmented linear regression to detect learning curve inflection points. Linear and logistic regression evaluated for learning curve impacts on 6-month clinical outcomes and 90-day adverse events. RESULTS: No baseline patient characteristic varied over the series apart from decreasing pre-procedural gland volume (P < 0.01). Multiple regression analysis demonstrated experience-dependent improvements in procedure time, fluoroscopy time, and contrast volume (P < 0.01), with corresponding learning curve inflection points at 76 (P < 0.01), 78 (P < 0.01), and 73 (P = 0.10) procedures. Embolic endpoints did not vary with experience (P > 0.05). Post-procedure reductions in International Prostate Symptom Score (21.5 ± 6.2 to 6.7 ± 4.7), Quality of Life score (4.5 ± 1.2 to 1.3 ± 1.2), post-void residual (190 ± 203 to 97 ± 148 mL), and gland volume (142 ± 97 to 76 ± 47 mL) were substantial (P < 0.01) but did not vary with experience (P > 0.05), nor did adverse event frequency/severity (P > 0.05). CONCLUSION: Operator technical efficiency plateaued after 73-78 PAE procedures. Clinical improvements were substantial and adverse event frequency/severity low, and neither varied with experience. Operators without prior PAE-specific experience may perform PAE safely and effectively from the outset. LEVEL OF EVIDENCE: Level 2b, Cohort Study.

Machine Learning Models for Prediction of Posttreatment Recurrence in Early-Stage Hepatocellular Carcinoma Using Pretreatment Clinical and MRI Features: A Proof-of-Concept Study.

BACKGROUND. Posttreatment recurrence is an unpredictable complication after liver transplant for hepatocellular carcinoma (HCC) that is associated with poor survival. Biomarkers are needed to estimate recurrence risk before organ allocation. OBJECTIVE. This proof-of-concept study evaluated the use of machine learning (ML) to predict recurrence from pretreatment laboratory, clinical, and MRI data in patients with early-stage HCC initially eligible for liver transplant. METHODS. This retrospective study included 120 patients (88 men, 32 women; median age, 60.0 years) with early-stage HCC diagnosed who were initially eligible for liver transplant and underwent treatment by transplant, resection, or thermal ablation between June 2005 and March 2018. Patients underwent pretreatment MRI and posttreatment imaging surveillance. Imaging features were extracted from postcontrast phases of pretreatment MRI examinations using a pretrained convolutional neural network. Pretreatment clinical characteristics (including laboratory data) and extracted imaging features were integrated to develop three ML models (clinical model, imaging model, combined model) for predicting recurrence within six time frames ranging from 1 through 6 years after treatment. Kaplan-Meier analysis with time to recurrence as the endpoint was used to assess the clinical relevance of model predictions. RESULTS. Tumor recurred in 44 of 120 (36.7%) patients during follow-up. The three models predicted recurrence with AUCs across the six time frames of 0.60-0.78 (clinical model), 0.71-0.85 (imaging model), and 0.62-0.86 (combined model). The mean AUC was higher for the imaging model than the clinical model (0.76 vs 0.68, respectively; p = .03), but the mean AUC was not significantly different between the clinical and combined models or between the imaging and combined models (p > .05). Kaplan-Meier curves were significantly different between patients predicted to be at low risk and those predicted to be at high risk by all three models for the 2-, 3-, 4-, 5-, and 6-year time frames (p < .05). CONCLUSION. The findings suggest that ML-based models can predict recurrence before therapy allocation in patients with early-stage HCC initially eligible for liver transplant. Adding MRI data as model input improved predictive performance over clinical parameters alone. The combined model did not surpass the imaging model's performance. CLINICAL IMPACT. ML-based models applied to currently underutilized imaging features may help design more reliable criteria for organ allocation and liver transplant eligibility.

ACTION++: Improving Semi-supervised Medical Image Segmentation with Adaptive Anatomical Contrast.

Medical data often exhibits long-tail distributions with heavy class imbalance, which naturally leads to difficulty in classifying the minority classes (i.e., boundary regions or rare objects). Recent work has significantly improved semi-supervised medical image segmentation in long-tailed scenarios by equipping them with unsupervised contrastive criteria. However, it remains unclear how well they will perform in the labeled portion of data where class distribution is also highly imbalanced. In this work, we present ACTION++, an improved contrastive learning framework with adaptive anatomical contrast for semi-supervised medical segmentation. Specifically, we propose an adaptive supervised contrastive loss, where we first compute the optimal locations of class centers uniformly distributed on the embedding space (i.e., off-line), and then perform online contrastive matching training by encouraging different class features to adaptively match these distinct and uniformly distributed class centers. Moreover, we argue that blindly adopting a constant temperature τ in the contrastive loss on long-tailed medical data is not optimal, and propose to use a dynamic τ via a simple cosine schedule to yield better separation between majority and minority classes. Empirically, we evaluate ACTION++ on ACDC and LA benchmarks and show that it achieves state-of-the-art across two semi-supervised settings. Theoretically, we analyze the performance of adaptive anatomical contrast and confirm its superiority in label efficiency.

Implicit Anatomical Rendering for Medical Image Segmentation with Stochastic Experts.

Integrating high-level semantically correlated contents and low-level anatomical features is of central importance in medical image segmentation. Towards this end, recent deep learning-based medical segmentation methods have shown great promise in better modeling such information. However, convolution operators for medical segmentation typically operate on regular grids, which inherently blur the high-frequency regions, i.e., boundary regions. In this work, we propose MORSE, a generic implicit neural rendering framework designed at an anatomical level to assist learning in medical image segmentation. Our method is motivated by the fact that implicit neural representation has been shown to be more effective in fitting complex signals and solving computer graphics problems than discrete grid-based representation. The core of our approach is to formulate medical image segmentation as a rendering problem in an end-to-end manner. Specifically, we continuously align the coarse segmentation prediction with the ambiguous coordinate-based point representations and aggregate these features to adaptively refine the boundary region. To parallelly optimize multi-scale pixel-level features, we leverage the idea from Mixture-of-Expert (MoE) to design and train our MORSE with a stochastic gating mechanism. Our experiments demonstrate that MORSE can work well with different medical segmentation backbones, consistently achieving competitive performance improvements in both 2D and 3D supervised medical segmentation methods. We also theoretically analyze the superiority of MORSE.

Rethinking Semi-Supervised Medical Image Segmentation: A Variance-Reduction Perspective.

For medical image segmentation, contrastive learning is the dominant practice to improve the quality of visual representations by contrasting semantically similar and dissimilar pairs of samples. This is enabled by the observation that without accessing ground truth labels, negative examples with truly dissimilar anatomical features, if sampled, can significantly improve the performance. In reality, however, these samples may come from similar anatomical regions and the models may struggle to distinguish the minority tail-class samples, making the tail classes more prone to misclassification, both of which typically lead to model collapse. In this paper, we propose ARCO, a semi-supervised contrastive learning (CL) framework with stratified group theory for medical image segmentation. In particular, we first propose building ARCO through the concept of variance-reduced estimation and show that certain variance-reduction techniques are particularly beneficial in pixel/voxel-level segmentation tasks with extremely limited labels. Furthermore, we theoretically prove these sampling techniques are universal in variance reduction. Finally, we experimentally validate our approaches on eight benchmarks, i.e., five 2D/3D medical and three semantic segmentation datasets, with different label settings, and our methods consistently outperform state-of-the-art semi-supervised methods. Additionally, we augment the CL frameworks with these sampling techniques and demonstrate significant gains over previous methods. We believe our work is an important step towards semi-supervised medical image segmentation by quantifying the limitation of current self-supervision objectives for accomplishing such challenging safety-critical tasks.

Learning Sequential Information in Task-based fMRI for Synthetic Data Augmentation.

Insufficiency of training data is a persistent issue in medical image analysis, especially for task-based functional magnetic resonance images (fMRI) with spatio-temporal imaging data acquired using specific cognitive tasks. In this paper, we propose an approach for generating synthetic fMRI sequences that can then be used to create augmented training datasets in downstream learning tasks. To synthesize high-resolution task-specific fMRI, we adapt the α-GAN structure, leveraging advantages of both GAN and variational autoencoder models, and propose different alternatives in aggregating temporal information. The synthetic images are evaluated from multiple perspectives including visualizations and an autism spectrum disorder (ASD) classification task. The results show that the synthetic task-based fMRI can provide effective data augmentation in learning the ASD classification task.