Automated full body tumor segmentation in DOTATATE PET/CT for neuroendocrine cancer patients.

PURPOSE: Neuroendocrine tumors (NETs) are a rare form of cancer that can occur anywhere in the body and commonly metastasizes. The large variance in location and aggressiveness of the tumors makes it a difficult cancer to treat. Assessments of the whole-body tumor burden in a patient image allow for better tracking of disease progression and inform better treatment decisions. Currently, radiologists rely on qualitative assessments of this metric since manual segmentation is unfeasible within a typical busy clinical workflow. METHODS: We address these challenges by extending the application of the nnU-net pipeline to produce automatic NET segmentation models. We utilize the ideal imaging type of 68Ga-DOTATATE PET/CT to produce segmentation masks from which to calculate total tumor burden metrics. We provide a human-level baseline for the task and perform ablation experiments of model inputs, architectures, and loss functions. RESULTS: Our dataset is comprised of 915 PET/CT scans and is divided into a held-out test set (87 cases) and 5 training subsets to perform cross-validation. The proposed models achieve test Dice scores of 0.644, on par with our inter-annotator Dice score on a subset 6 patients of 0.682. If we apply our modified Dice score to the predictions, the test performance reaches a score of 0.80. CONCLUSION: In this paper, we demonstrate the ability to automatically generate accurate NET segmentation masks given PET images through supervised learning. We publish the model for extended use and to support the treatment planning of this rare cancer.

Fast, light, and scalable: harnessing data-mined line annotations for automated tumor segmentation on brain MRI.

OBJECTIVES: While fully supervised learning can yield high-performing segmentation models, the effort required to manually segment large training sets limits practical utility. We investigate whether data mined line annotations can facilitate brain MRI tumor segmentation model development without requiring manually segmented training data. METHODS: In this retrospective study, a tumor detection model trained using clinical line annotations mined from PACS was leveraged with unsupervised segmentation to generate pseudo-masks of enhancing tumors on T1-weighted post-contrast images (9911 image slices; 3449 adult patients). Baseline segmentation models were trained and employed within a semi-supervised learning (SSL) framework to refine the pseudo-masks. Following each self-refinement cycle, a new model was trained and tested on a held-out set of 319 manually segmented image slices (93 adult patients), with the SSL cycles continuing until Dice score coefficient (DSC) peaked. DSCs were compared using bootstrap resampling. Utilizing the best-performing models, two inference methods were compared: (1) conventional full-image segmentation, and (2) a hybrid method augmenting full-image segmentation with detection plus image patch segmentation. RESULTS: Baseline segmentation models achieved DSC of 0.768 (U-Net), 0.831 (Mask R-CNN), and 0.838 (HRNet), improving with self-refinement to 0.798, 0.871, and 0.873 (each p < 0.001), respectively. Hybrid inference outperformed full image segmentation alone: DSC 0.884 (Mask R-CNN) vs. 0.873 (HRNet), p < 0.001. CONCLUSIONS: Line annotations mined from PACS can be harnessed within an automated pipeline to produce accurate brain MRI tumor segmentation models without manually segmented training data, providing a mechanism to rapidly establish tumor segmentation capabilities across radiology modalities. KEY POINTS: • A brain MRI tumor detection model trained using clinical line measurement annotations mined from PACS was leveraged to automatically generate tumor segmentation pseudo-masks. • An iterative self-refinement process automatically improved pseudo-mask quality, with the best-performing segmentation pipeline achieving a Dice score of 0.884 on a held-out test set. • Tumor line measurement annotations generated in routine clinical radiology practice can be harnessed to develop high-performing segmentation models without manually segmented training data, providing a mechanism to rapidly establish tumor segmentation capabilities across radiology modalities.

Uncertainty estimation for margin detection in cancer surgery using mass spectrometry.

PURPOSE: Rapid evaporative ionization mass spectrometry (REIMS) is an emerging technology for clinical margin detection. Deployment of REIMS depends on construction of reliable deep learning models that can categorize tissue according to its metabolomic signature. Challenges associated with developing these models include the presence of noise during data acquisition and the variance in tissue signatures between patients. In this study, we propose integration of uncertainty estimation in deep models to factor predictive confidence into margin detection in cancer surgery. METHODS: iKnife is used to collect 693 spectra of cancer and healthy samples acquired from 91 patients during basal cell carcinoma resection. A Bayesian neural network and two baseline models are trained on these data to perform classification as well as uncertainty estimation. The samples with high estimated uncertainty are then removed, and new models are trained using the clean data. The performance of proposed and baseline models, with different ratios of filtered data, is then compared. RESULTS: The data filtering does not improve the performance of the baseline models as they cannot provide reliable estimations of uncertainty. In comparison, the proposed model demonstrates a statistically significant improvement in average balanced accuracy (75.2%), sensitivity (74.1%) and AUC (82.1%) after removing uncertain training samples. We also demonstrate that if highly uncertain samples are predicted and removed from the test data, sensitivity further improves to 88.2%. CONCLUSIONS: This is the first study that applies uncertainty estimation to inform model training and deployment for tissue recognition in cancer surgery. Uncertainty estimation is leveraged in two ways: by factoring a measure of input noise in training the models and by including predictive confidence in reporting the outputs. We empirically show that considering uncertainty for model development can help improve the overall accuracy of a margin detection system using REIMS.

Application of Intraoperative Mass Spectrometry and Data Analytics for Oncological Margin Detection, A Review.

OBJECTIVE: A common phase of early-stage oncological treatment is the surgical resection of cancerous tissue. The presence of cancer cells on the resection margin, referred to as positive margin, is correlated with the recurrence of cancer and may require re-operation, negatively impacting many facets of patient outcomes. There exists a significant gap in the surgeon's ability to intraoperatively delineate between tissues. Mass spectrometry methods have shown considerable promise as intraoperative tissue profiling tools that can assist with the complete resection of cancer. To do so, the vastness of the information collected through these modalities must be digested, relying on robust and efficient extraction of insights through data analysis pipelines. METHODS: We review clinical mass spectrometry literature and prioritize intraoperatively applied modalities. We also survey the data analysis methods employed in these studies. RESULTS: Our review outlines the advantages and shortcomings of mass spectrometry imaging and point-based tissue probing methods. For each modality, we identify statistical, linear transformation and machine learning techniques that demonstrate high performance in classifying cancerous tissues across several organ systems. A limited number of studies presented results captured intraoperatively. CONCLUSION: Through continued research of data centric techniques, like mass spectrometry, and the development of robust analysis approaches, intraoperative margin assessment is becoming feasible. SIGNIFICANCE: By establishing the relatively short history of mass spectrometry techniques applied to surgical studies, we hope to inform future applications and aid in the selection of suitable data analysis frameworks for the development of intraoperative margin detection technologies.

Domain adaptation and self-supervised learning for surgical margin detection.

PURPOSE: One in five women who undergo breast conserving surgery will need a second revision surgery due to remaining tumor. The iKnife is a mass spectrometry modality that produces real-time margin information based on the metabolite signatures in surgical smoke. Using this modality and real-time tissue classification, surgeons could remove all cancerous tissue during the initial surgery, improving many facets of patient outcomes. An obstacle in developing a iKnife breast cancer recognition model is the destructive, time-consuming and sensitive nature of the data collection that limits the size of the datasets. METHODS: We address these challenges by first, building a self-supervised learning model from limited, weakly labeled data. By doing so, the model can learn to contextualize the general features of iKnife data from a more accessible cancer type. Second, the trained model can then be applied to a cancer classification task on breast data. This domain adaptation allows for the transfer of learnt weights from models of one tissue type to another. RESULTS: Our datasets contained 320 skin burns (129 tumor burns, 191 normal burns) from 51 patients and 144 breast tissue burns (41 tumor and 103 normal) from 11 patients. We investigate the effect of different hyper-parameters on the performance of the final classifier. The proposed two-step method performed statistically significantly better than a baseline model (p-value < 0.0001), by achieving an accuracy, sensitivity and specificity of 92%, 88% and 92%, respectively. CONCLUSION: This is the first application of domain transfer for iKnife REIMS data. We showed that having a limited number of breast data samples for training a classifier can be compensated by self-supervised learning and domain adaption on a set of unlabeled skin data. We plan to confirm this performance by collecting new breast samples and extending it to incorporate other cancer tissues.

Navigated tissue characterization during skin cancer surgery.

PURPOSE: Basal cell carcinoma (BCC) is the most commonly diagnosed skin cancer and is treated by surgical resection. Incomplete tumor removal requires surgical revision, leading to significant healthcare costs and impaired cosmesis. We investigated the clinical feasibility of a surgical navigation system for BCC surgery, based on molecular tissue characterization using rapid evaporative ionization mass spectrometry (REIMS). METHODS: REIMS enables direct tissue characterization by analysis of cell-specific molecules present within surgical smoke, produced during electrocautery tissue resection. A tissue characterization model was built by acquiring REIMS spectra of BCC, healthy skin and fat from ex vivo skin cancer specimens. This model was used for tissue characterization during navigated skin cancer surgery. Navigation was enabled by optical tracking and real-time visualization of the cautery relative to a contoured resection volume. The surgical smoke was aspirated into a mass spectrometer and directly analyzed with REIMS. Classified BCC was annotated at the real-time position of the cautery. Feasibility of the navigation system, and tissue classification accuracy for ex vivo and intraoperative surgery were evaluated. RESULTS: Fifty-four fresh excision specimens were used to build the ex vivo model of BCC, normal skin and fat, with 92% accuracy. While 3 surgeries were successfully navigated without breach of sterility, the intraoperative performance of the ex vivo model was low (< 50%). Hypotheses are: (1) the model was trained on heterogeneous mass spectra that did not originate from a single tissue type, (2) during surgery mixed tissue types were resected and thus presented to the model, and (3) the mass spectra were not validated by pathology. CONCLUSION: REIMS-navigated skin cancer surgery has the potential to detect and localize remaining tumor intraoperatively. Future work will be focused on improving our model by using a precise pencil cautery tip for burning localized tissue types, and having pathology-validated mass spectra.

Perioperative margin detection in basal cell carcinoma using a deep learning framework: a feasibility study.

PURPOSE: Basal cell carcinoma (BCC) is the most commonly diagnosed cancer and the number of diagnosis is growing worldwide due to increased exposure to solar radiation and the aging population. Reduction of positive margin rates when removing BCC leads to fewer revision surgeries and consequently lower health care costs, improved cosmetic outcomes and better patient care. In this study, we propose the first use of a perioperative mass spectrometry technology (iKnife) along with a deep learning framework for detection of BCC signatures from tissue burns. METHODS: Resected surgical specimen were collected and inspected by a pathologist. With their guidance, data were collected by burning regions of the specimen labeled as BCC or normal, with the iKnife. Data included 190 scans of which 127 were normal and 63 were BCC. A data augmentation approach was proposed by modifying the location and intensity of the peaks of the original spectra, through noise addition in the time and frequency domains. A symmetric autoencoder was built by simultaneously optimizing the spectral reconstruction error and the classification accuracy. Using t-SNE, the latent space was visualized. RESULTS: The autoencoder achieved an accuracy (standard deviation) of 96.62 (1.35%) when classifying BCC and normal scans, a statistically significant improvement over the baseline state-of-the-art approach used in the literature. The t-SNE plot of the latent space distinctly showed the separability between BCC and normal data, not visible with the original data. Augmented data resulted in significant improvements to the classification accuracy of the baseline model. CONCLUSION: We demonstrate the utility of a deep learning framework applied to mass spectrometry data for surgical margin detection. We apply the proposed framework to an application with light surgical overhead and high incidence, the removal of BCC. The learnt models can accurately separate BCC from normal tissue.