Toward confident prostate cancer detection using ultrasound: a multi-center study.

PURPOSE: Deep learning-based analysis of micro-ultrasound images to detect cancerous lesions is a promising tool for improving prostate cancer (PCa) diagnosis. An ideal model should confidently identify cancer while responding with appropriate uncertainty when presented with out-of-distribution inputs that arise during deployment due to imaging artifacts and the biological heterogeneity of patients and prostatic tissue. METHODS: Using micro-ultrasound data from 693 patients across 5 clinical centers who underwent micro-ultrasound guided prostate biopsy, we train and evaluate convolutional neural network models for PCa detection. To improve robustness to out-of-distribution inputs, we employ and comprehensively benchmark several state-of-the-art uncertainty estimation methods. RESULTS: PCa detection models achieve performance scores up to 76 % average AUROC with a 10-fold cross validation setup. Models with uncertainty estimation obtain expected calibration error scores as low as 2 % , indicating that confident predictions are very likely to be correct. Visualizations of the model output demonstrate that the model correctly identifies healthy versus malignant tissue. CONCLUSION: Deep learning models have been developed to confidently detect PCa lesions from micro-ultrasound. The performance of these models, determined from a large and diverse dataset, is competitive with visual analysis of magnetic resonance imaging, the clinical benchmark to identify PCa lesions for targeted biopsy. Deep learning with micro-ultrasound should be further studied as an avenue for targeted prostate biopsy.

From quantitative metrics to clinical success: assessing the utility of deep learning for tumor segmentation in breast surgery.

PURPOSE: Preventing positive margins is essential for ensuring favorable patient outcomes following breast-conserving surgery (BCS). Deep learning has the potential to enable this by automatically contouring the tumor and guiding resection in real time. However, evaluation of such models with respect to pathology outcomes is necessary for their successful translation into clinical practice. METHODS: Sixteen deep learning models based on established architectures in the literature are trained on 7318 ultrasound images from 33 patients. Models are ranked by an expert based on their contours generated from images in our test set. Generated contours from each model are also analyzed using recorded cautery trajectories of five navigated BCS cases to predict margin status. Predicted margins are compared with pathology reports. RESULTS: The best-performing model using both quantitative evaluation and our visual ranking framework achieved a mean Dice score of 0.959. Quantitative metrics are positively associated with expert visual rankings. However, the predictive value of generated contours was limited with a sensitivity of 0.750 and a specificity of 0.433 when tested against pathology reports. CONCLUSION: We present a clinical evaluation of deep learning models trained for intraoperative tumor segmentation in breast-conserving surgery. We demonstrate that automatic contouring is limited in predicting pathology margins despite achieving high performance on quantitative metrics.

ImSpect: Image-driven self-supervised learning for surgical margin evaluation with mass spectrometry.

PURPOSE: Real-time assessment of surgical margins is critical for favorable outcomes in cancer patients. The iKnife is a mass spectrometry device that has demonstrated potential for margin detection in cancer surgery. Previous studies have shown that using deep learning on iKnife data can facilitate real-time tissue characterization. However, none of the existing literature on the iKnife facilitate the use of publicly available, state-of-the-art pretrained networks or datasets that have been used in computer vision and other domains. METHODS: In a new framework we call ImSpect, we convert 1D iKnife data, captured during basal cell carcinoma (BCC) surgery, into 2D images in order to capitalize on state-of-the-art image classification networks. We also use self-supervision to leverage large amounts of unlabeled, intraoperative data to accommodate the data requirements of these networks. RESULTS: Through extensive ablation studies, we show that we can surpass previous benchmarks of margin evaluation in BCC surgery using iKnife data, achieving an area under the receiver operating characteristic curve (AUC) of 81%. We also depict the attention maps of the developed DL models to evaluate the biological relevance of the embedding space CONCLUSIONS: We propose a new method for characterizing tissue at the surgical margins, using mass spectrometry data from cancer surgery.

LensePro: label noise-tolerant prototype-based network for improving cancer detection in prostate ultrasound with limited annotations.

PURPOSE: The standard of care for prostate cancer (PCa) diagnosis is the histopathological analysis of tissue samples obtained via transrectal ultrasound (TRUS) guided biopsy. Models built with deep neural networks (DNNs) hold the potential for direct PCa detection from TRUS, which allows targeted biopsy and subsequently enhances outcomes. Yet, there are ongoing challenges with training robust models, stemming from issues such as noisy labels, out-of-distribution (OOD) data, and limited labeled data. METHODS: This study presents LensePro, a unified method that not only excels in label efficiency but also demonstrates robustness against label noise and OOD data. LensePro comprises two key stages: first, self-supervised learning to extract high-quality feature representations from abundant unlabeled TRUS data and, second, label noise-tolerant prototype-based learning to classify the extracted features. RESULTS: Using data from 124 patients who underwent systematic prostate biopsy, LensePro achieves an AUROC, sensitivity, and specificity of 77.9%, 85.9%, and 57.5%, respectively, for detecting PCa in ultrasound. Our model shows it is effective for detecting OOD data in test time, critical for clinical deployment. Ablation studies demonstrate that each component of our method improves PCa detection by addressing one of the three challenges, reinforcing the benefits of a unified approach. CONCLUSION: Through comprehensive experiments, LensePro demonstrates its state-of-the-art performance for TRUS-based PCa detection. Although further research is necessary to confirm its clinical applicability, LensePro marks a notable advancement in enhancing automated computer-aided systems for detecting prostate cancer in ultrasound.

Self-Supervised Learning With Limited Labeled Data for Prostate Cancer Detection in High-Frequency Ultrasound.

Deep learning-based analysis of high-frequency, high-resolution micro-ultrasound data shows great promise for prostate cancer (PCa) detection. Previous approaches to analysis of ultrasound data largely follow a supervised learning (SL) paradigm. Ground truth labels for ultrasound images used for training deep networks often include coarse annotations generated from the histopathological analysis of tissue samples obtained via biopsy. This creates inherent limitations on the availability and quality of labeled data, posing major challenges to the success of SL methods. However, unlabeled prostate ultrasound data are more abundant. In this work, we successfully apply self-supervised representation learning to micro-ultrasound data. Using ultrasound data from 1028 biopsy cores of 391 subjects obtained in two clinical centers, we demonstrate that feature representations learned with this method can be used to classify cancer from noncancer tissue, obtaining an AUROC score of 91% on an independent test set. To the best of our knowledge, this is the first successful end-to-end self-SL (SSL) approach for PCa detection using ultrasound data. Our method outperforms baseline SL approaches, generalizes well between different data centers, and scales well in performance as more unlabeled data are added, making it a promising approach for future research using large volumes of unlabeled data. Our code is publicly available at https://www.github.com/MahdiGilany/SSL_micro_ultrasound.

Molecular characterization of human peripheral nerves using desorption electrospray ionization mass spectrometry imaging.

Desorption electrospray ionization mass spectrometry imaging (DESI-MSI) is a molecular imaging method that can be used to elucidate the small-molecule composition of tissues and map their spatial information using two-dimensional ion images. This technique has been used to investigate the molecular profiles of variety of tissues, including within the central nervous system, specifically the brain and spinal cord. To our knowledge, this technique has yet to be applied to tissues of the peripheral nervous system (PNS). Data generated from such analyses are expected to advance the characterization of these structures. The study aimed to: (i) establish whether DESI-MSI can discriminate the molecular characteristics of peripheral nerves and distinguish them from surrounding tissues and (ii) assess whether different peripheral nerve subtypes are characterized by unique molecular profiles. Four different nerves for which are known to carry various nerve fiber types were harvested from a fresh cadaveric donor: mixed, motor and sensory (sciatic and femoral); cutaneous, sensory (sural); and autonomic (vagus). Tissue samples were harvested to include the nerve bundles in addition to surrounding connective tissue. Samples were flash-frozen, embedded in optimal cutting temperature compound in cross-section, and sectioned at 14 µm. Following DESI-MSI analysis, identical tissue sections were stained with hematoxylin and eosin. In this proof-of-concept study, a combination of multivariate and univariate statistical methods was used to evaluate molecular differences between the nerve and adjacent tissue and between nerve subtypes. The acquired mass spectral profiles of the peripheral nerve samples presented trends in ion abundances that seemed to be characteristic of nerve tissue and spatially corresponded to the associated histology of the tissue sections. Principal component analysis (PCA) supported the separation of the samples into distinct nerve and adjacent tissue classes. This classification was further supported by the K-means clustering analysis, which showed separation of the nerve and background ions. Differences in ion expression were confirmed using ANOVA which identified statistically significant differences in ion expression between the nerve subtypes. The PCA plot suggested some separation of the nerve subtypes into four classes which corresponded with the nerve types. This was supported by the K-means clustering. Some overlap in classes was noted in these two clustering analyses. This study provides emerging evidence that DESI-MSI is an effective tool for metabolomic profiling of peripheral nerves. Our results suggest that peripheral nerves have molecular profiles that are distinct from the surrounding connective tissues and that DESI-MSI may be able to discriminate between nerve subtypes. DESI-MSI of peripheral nerves may be a valuable technique that could be used to improve our understanding of peripheral nerve anatomy and physiology. The ability to utilize ambient mass spectrometry techniques in real time could also provide an unprecedented advantage for surgical decision making, including in nerve-sparing procedures in the future.

TRUSformer: improving prostate cancer detection from micro-ultrasound using attention and self-supervision.

PURPOSE: A large body of previous machine learning methods for ultrasound-based prostate cancer detection classify small regions of interest (ROIs) of ultrasound signals that lie within a larger needle trace corresponding to a prostate tissue biopsy (called biopsy core). These ROI-scale models suffer from weak labeling as histopathology results available for biopsy cores only approximate the distribution of cancer in the ROIs. ROI-scale models do not take advantage of contextual information that are normally considered by pathologists, i.e., they do not consider information about surrounding tissue and larger-scale trends when identifying cancer. We aim to improve cancer detection by taking a multi-scale, i.e., ROI-scale and biopsy core-scale, approach. METHODS: Our multi-scale approach combines (i) an "ROI-scale" model trained using self-supervised learning to extract features from small ROIs and (ii) a "core-scale" transformer model that processes a collection of extracted features from multiple ROIs in the needle trace region to predict the tissue type of the corresponding core. Attention maps, as a by-product, allow us to localize cancer at the ROI scale. RESULTS: We analyze this method using a dataset of micro-ultrasound acquired from 578 patients who underwent prostate biopsy, and compare our model to baseline models and other large-scale studies in the literature. Our model shows consistent and substantial performance improvements compared to ROI-scale-only models. It achieves [Formula: see text] AUROC, a statistically significant improvement over ROI-scale classification. We also compare our method to large studies on prostate cancer detection, using other imaging modalities. CONCLUSIONS: Taking a multi-scale approach that leverages contextual information improves prostate cancer detection compared to ROI-scale-only models. The proposed model achieves a statistically significant improvement in performance and outperforms other large-scale studies in the literature. Our code is publicly available at www.github.com/med-i-lab/TRUSFormer .

Metabolically Active Zones Involving Fatty Acid Elongation Delineated by DESI-MSI Correlate with Pathological and Prognostic Features of Colorectal Cancer.

Colorectal cancer (CRC) is the second leading cause of cancer deaths. Despite recent advances, five-year survival rates remain largely unchanged. Desorption electrospray ionization mass spectrometry imaging (DESI) is an emerging nondestructive metabolomics-based method that retains the spatial orientation of small-molecule profiles on tissue sections, which may be validated by 'gold standard' histopathology. In this study, CRC samples were analyzed by DESI from 10 patients undergoing surgery at Kingston Health Sciences Center. The spatial correlation of the mass spectral profiles was compared with histopathological annotations and prognostic biomarkers. Fresh frozen sections of representative colorectal cross sections and simulated endoscopic biopsy samples containing tumour and non-neoplastic mucosa for each patient were generated and analyzed by DESI in a blinded fashion. Sections were then hematoxylin and eosin (H and E) stained, annotated by two independent pathologists, and analyzed. Using PCA/LDA-based models, DESI profiles of the cross sections and biopsies achieved 97% and 75% accuracies in identifying the presence of adenocarcinoma, using leave-one-patient-out cross validation. Among the m/z ratios exhibiting the greatest differential abundance in adenocarcinoma were a series of eight long-chain or very-long-chain fatty acids, consistent with molecular and targeted metabolomics indicators of de novo lipogenesis in CRC tissue. Sample stratification based on the presence of lympovascular invasion (LVI), a poor CRC prognostic indicator, revealed the abundance of oxidized phospholipids, suggestive of pro-apoptotic mechanisms, was increased in LVI-negative compared to LVI-positive patients. This study provides evidence of the potential clinical utility of spatially-resolved DESI profiles to enhance the information available to clinicians for CRC diagnosis and prognosis.

Automated Cell Phenotyping for Imaging Mass Cytometry.

Imaging mass cytometry (IMC) is a new advancement in tissue imaging that is quickly gaining wider usage since its recent launch. It improves upon current tissue imaging methods by allowing for a significantly higher number of proteins to be imaged at once on a single tissue slide. For most analyses of IMC data, determining the phenotype of each cell is a crucial step. Current methods of phenotyping require sufficient biological knowledge regarding the protein expression profile of the various cell types. Here, we develop a deep convolutional autoencoder-classifier to automate the cell phenotyping process into four basic cell types. Biopsy tissue from bladder cancer patients is used to evaluate the efficacy of the classification. The model is evaluated and validated through feature importance, confirming that the significant features are biologically relevant. Our results demonstrate the potential of deep learning to automate the task of cell phenotyping for high-dimensional IMC data.

Uncertainty estimation for margin detection in cancer surgery using mass spectrometry.

PURPOSE: Rapid evaporative ionization mass spectrometry (REIMS) is an emerging technology for clinical margin detection. Deployment of REIMS depends on construction of reliable deep learning models that can categorize tissue according to its metabolomic signature. Challenges associated with developing these models include the presence of noise during data acquisition and the variance in tissue signatures between patients. In this study, we propose integration of uncertainty estimation in deep models to factor predictive confidence into margin detection in cancer surgery. METHODS: iKnife is used to collect 693 spectra of cancer and healthy samples acquired from 91 patients during basal cell carcinoma resection. A Bayesian neural network and two baseline models are trained on these data to perform classification as well as uncertainty estimation. The samples with high estimated uncertainty are then removed, and new models are trained using the clean data. The performance of proposed and baseline models, with different ratios of filtered data, is then compared. RESULTS: The data filtering does not improve the performance of the baseline models as they cannot provide reliable estimations of uncertainty. In comparison, the proposed model demonstrates a statistically significant improvement in average balanced accuracy (75.2%), sensitivity (74.1%) and AUC (82.1%) after removing uncertain training samples. We also demonstrate that if highly uncertain samples are predicted and removed from the test data, sensitivity further improves to 88.2%. CONCLUSIONS: This is the first study that applies uncertainty estimation to inform model training and deployment for tissue recognition in cancer surgery. Uncertainty estimation is leveraged in two ways: by factoring a measure of input noise in training the models and by including predictive confidence in reporting the outputs. We empirically show that considering uncertainty for model development can help improve the overall accuracy of a margin detection system using REIMS.

Sensor-Based Automated Detection of Electrosurgical Cautery States.

In computer-assisted surgery, it is typically required to detect when the tool comes into contact with the patient. In activated electrosurgery, this is known as the energy event. By continuously tracking the electrosurgical tools' location using a navigation system, energy events can help determine locations of sensor-classified tissues. Our objective was to detect the energy event and determine the settings of electrosurgical cautery-robustly and automatically based on sensor data. This study aims to demonstrate the feasibility of using the cautery state to detect surgical incisions, without disrupting the surgical workflow. We detected current changes in the wires of the cautery device and grounding pad using non-invasive current sensors and an oscilloscope. An open-source software was implemented to apply machine learning on sensor data to detect energy events and cautery settings. Our methods classified each cautery state at an average accuracy of 95.56% across different tissue types and energy level parameters altered by surgeons during an operation. Our results demonstrate the feasibility of automatically identifying energy events during surgical incisions, which could be an important safety feature in robotic and computer-integrated surgery. This study provides a key step towards locating tissue classifications during breast cancer operations and reducing the rate of positive margins.

TITAN: An end-to-end data analysis environment for the Hyperion™ imaging system.

Imaging Mass Cytometry (IMC) is a powerful high-throughput technique enabling resolution of up to 37 markers in a single fixed tissue section while also preserving in situ spatial relationships. Currently, IMC processing and analysis necessitates the use of multiple different software, labour-intensive pipeline development, different operating systems and knowledge of bioinformatics, all of which are a barrier to many potential users. Here we present TITAN - an open-source, single environment, end-to-end pipeline that can be utilized for image visualization, segmentation, analysis and export of IMC data. TITAN is implemented as an extension within the publicly available 3D Slicer software. We demonstrate the utility, application, reliability and comparability of TITAN using publicly available IMC data from recently-published breast cancer and COVID-19 lung injury studies. Compared with current IMC analysis methods, TITAN provides a user-friendly, efficient single environment to accurately visualize, segment, and analyze IMC data for all users.

Multi-Omic Graph Transformers for Cancer Classification and Interpretation.

Next-generation sequencing has provided rapid collection and quantification of 'big' biological data. In particular, multi-omics and integration of different molecular data such as miRNA and mRNA can provide important insights to disease classification and processes. There is a need for computational methods that can correctly model and interpret these relationships, and handle the difficulties of large-scale data. In this study, we develop a novel method of representing miRNA-mRNA interactions to classify cancer. Specifically, graphs are designed to account for the interactions and biological communication between miRNAs and mRNAs, using message-passing and attention mechanisms. Patient-matched miRNA and mRNA expression data is obtained from The Cancer Genome Atlas for 12 cancers, and targeting information is incorporated from TargetScan. A Graph Transformer Network (GTN) is selected to provide high interpretability of classification through self-attention mechanisms. The GTN is able to classify the 12 different cancers with an accuracy of 93.56% and is compared to a Graph Convolutional Network, Random Forest, Support Vector Machine, and Multilayer Perceptron. While the GTN does not outperform all of the other classifiers in terms of accuracy, it allows high interpretation of results. Multi-omics models are compared and generally outperform their respective single-omics performance. Extensive analysis of attention identifies important targeting pathways and molecular biomarkers based on integrated miRNA and mRNA expression.

Prediction of patient survival following postanoxic coma using EEG data and clinical features.

Electroencephalography (EEG) is an effective and non-invasive technique commonly used to monitor brain activity and assist in outcome prediction for comatose patients post cardiac arrest. EEG data may demonstrate patterns associated with poor neurological outcome for patients with hypoxic injury. Thus, both quantitative EEG (qEEG) and clinical data contain prognostic information for patient outcome. In this study we use machine learning (ML) techniques, random forest (RF) and support vector machine (SVM) to classify patient outcome post cardiac arrest using qEEG and clinical feature sets, individually and combined. Our ML experiments show RF and SVM perform better using the joint feature set. In addition, we extend our work by implementing a convolutional neural network (CNN) based on time-frequency images derived from EEG to compare with our qEEG ML models. The results demonstrate significant performance improvement in outcome prediction using non-feature based CNN compared to our feature based ML models. Implementation of ML and DL methods in clinical practice have the potential to improve reliability of traditional qualitative assessments for postanoxic coma patients.

Combined Mass Spectrometry and Histopathology Imaging for Perioperative Tissue Assessment in Cancer Surgery.

Mass spectrometry is an effective imaging tool for evaluating biological tissue to detect cancer. With the assistance of deep learning, this technology can be used as a perioperative tissue assessment tool that will facilitate informed surgical decisions. To achieve such a system requires the development of a database of mass spectrometry signals and their corresponding pathology labels. Assigning correct labels, in turn, necessitates precise spatial registration of histopathology and mass spectrometry data. This is a challenging task due to the domain differences and noisy nature of images. In this study, we create a registration framework for mass spectrometry and pathology images as a contribution to the development of perioperative tissue assessment. In doing so, we explore two opportunities in deep learning for medical image registration, namely, unsupervised, multi-modal deformable image registration and evaluation of the registration. We test this system on prostate needle biopsy cores that were imaged with desorption electrospray ionization mass spectrometry (DESI) and show that we can successfully register DESI and histology images to achieve accurate alignment and, consequently, labelling for future training. This automation is expected to improve the efficiency and development of a deep learning architecture that will benefit the use of mass spectrometry imaging for cancer diagnosis.

Domain adaptation and self-supervised learning for surgical margin detection.

PURPOSE: One in five women who undergo breast conserving surgery will need a second revision surgery due to remaining tumor. The iKnife is a mass spectrometry modality that produces real-time margin information based on the metabolite signatures in surgical smoke. Using this modality and real-time tissue classification, surgeons could remove all cancerous tissue during the initial surgery, improving many facets of patient outcomes. An obstacle in developing a iKnife breast cancer recognition model is the destructive, time-consuming and sensitive nature of the data collection that limits the size of the datasets. METHODS: We address these challenges by first, building a self-supervised learning model from limited, weakly labeled data. By doing so, the model can learn to contextualize the general features of iKnife data from a more accessible cancer type. Second, the trained model can then be applied to a cancer classification task on breast data. This domain adaptation allows for the transfer of learnt weights from models of one tissue type to another. RESULTS: Our datasets contained 320 skin burns (129 tumor burns, 191 normal burns) from 51 patients and 144 breast tissue burns (41 tumor and 103 normal) from 11 patients. We investigate the effect of different hyper-parameters on the performance of the final classifier. The proposed two-step method performed statistically significantly better than a baseline model (p-value < 0.0001), by achieving an accuracy, sensitivity and specificity of 92%, 88% and 92%, respectively. CONCLUSION: This is the first application of domain transfer for iKnife REIMS data. We showed that having a limited number of breast data samples for training a classifier can be compensated by self-supervised learning and domain adaption on a set of unlabeled skin data. We plan to confirm this performance by collecting new breast samples and extending it to incorporate other cancer tissues.

Navigated tissue characterization during skin cancer surgery.

PURPOSE: Basal cell carcinoma (BCC) is the most commonly diagnosed skin cancer and is treated by surgical resection. Incomplete tumor removal requires surgical revision, leading to significant healthcare costs and impaired cosmesis. We investigated the clinical feasibility of a surgical navigation system for BCC surgery, based on molecular tissue characterization using rapid evaporative ionization mass spectrometry (REIMS). METHODS: REIMS enables direct tissue characterization by analysis of cell-specific molecules present within surgical smoke, produced during electrocautery tissue resection. A tissue characterization model was built by acquiring REIMS spectra of BCC, healthy skin and fat from ex vivo skin cancer specimens. This model was used for tissue characterization during navigated skin cancer surgery. Navigation was enabled by optical tracking and real-time visualization of the cautery relative to a contoured resection volume. The surgical smoke was aspirated into a mass spectrometer and directly analyzed with REIMS. Classified BCC was annotated at the real-time position of the cautery. Feasibility of the navigation system, and tissue classification accuracy for ex vivo and intraoperative surgery were evaluated. RESULTS: Fifty-four fresh excision specimens were used to build the ex vivo model of BCC, normal skin and fat, with 92% accuracy. While 3 surgeries were successfully navigated without breach of sterility, the intraoperative performance of the ex vivo model was low (< 50%). Hypotheses are: (1) the model was trained on heterogeneous mass spectra that did not originate from a single tissue type, (2) during surgery mixed tissue types were resected and thus presented to the model, and (3) the mass spectra were not validated by pathology. CONCLUSION: REIMS-navigated skin cancer surgery has the potential to detect and localize remaining tumor intraoperatively. Future work will be focused on improving our model by using a precise pencil cautery tip for burning localized tissue types, and having pathology-validated mass spectra.

Improving detection of prostate cancer foci via information fusion of MRI and temporal enhanced ultrasound.

PURPOSE: The detection of clinically significant prostate cancer (PCa) is shown to greatly benefit from MRI-ultrasound fusion biopsy, which involves overlaying pre-biopsy MRI volumes (or targets) with real-time ultrasound images. In previous literature, machine learning models trained on either MRI or ultrasound data have been proposed to improve biopsy guidance and PCa detection. However, quantitative fusion of information from MRI and ultrasound has not been explored in depth in a large study. This paper investigates information fusion approaches between MRI and ultrasound to improve targeting of PCa foci in biopsies. METHODS: We build models of fully convolutional networks (FCN) using data from a newly proposed ultrasound modality, temporal enhanced ultrasound (TeUS), and apparent diffusion coefficient (ADC) from 107 patients with 145 biopsy cores. The architecture of our models is based on U-Net and U-Net with attention gates. Models are built using joint training through intermediate and late fusion of the data. We also build models with data from each modality, separately, to use as baseline. The performance is evaluated based on the area under the curve (AUC) for predicting clinically significant PCa. RESULTS: Using our proposed deep learning framework and intermediate fusion, integration of TeUS and ADC outperforms the individual modalities for cancer detection. We achieve an AUC of 0.76 for detection of all PCa foci, and 0.89 for PCa with larger foci. Results indicate a shared representation between multiple modalities outperforms the average unimodal predictions. CONCLUSION: We demonstrate the significant potential of multimodality integration of information from MRI and TeUS to improve PCa detection, which is essential for accurate targeting of cancer foci during biopsy. By using FCNs as the architecture of choice, we are able to predict the presence of clinically significant PCa in entire imaging planes immediately, without the need for region-based analysis. This reduces the overall computational time and enables future intra-operative deployment of this technology.

Perioperative margin detection in basal cell carcinoma using a deep learning framework: a feasibility study.

PURPOSE: Basal cell carcinoma (BCC) is the most commonly diagnosed cancer and the number of diagnosis is growing worldwide due to increased exposure to solar radiation and the aging population. Reduction of positive margin rates when removing BCC leads to fewer revision surgeries and consequently lower health care costs, improved cosmetic outcomes and better patient care. In this study, we propose the first use of a perioperative mass spectrometry technology (iKnife) along with a deep learning framework for detection of BCC signatures from tissue burns. METHODS: Resected surgical specimen were collected and inspected by a pathologist. With their guidance, data were collected by burning regions of the specimen labeled as BCC or normal, with the iKnife. Data included 190 scans of which 127 were normal and 63 were BCC. A data augmentation approach was proposed by modifying the location and intensity of the peaks of the original spectra, through noise addition in the time and frequency domains. A symmetric autoencoder was built by simultaneously optimizing the spectral reconstruction error and the classification accuracy. Using t-SNE, the latent space was visualized. RESULTS: The autoencoder achieved an accuracy (standard deviation) of 96.62 (1.35%) when classifying BCC and normal scans, a statistically significant improvement over the baseline state-of-the-art approach used in the literature. The t-SNE plot of the latent space distinctly showed the separability between BCC and normal data, not visible with the original data. Augmented data resulted in significant improvements to the classification accuracy of the baseline model. CONCLUSION: We demonstrate the utility of a deep learning framework applied to mass spectrometry data for surgical margin detection. We apply the proposed framework to an application with light surgical overhead and high incidence, the removal of BCC. The learnt models can accurately separate BCC from normal tissue.

Noninvasive Prediction of Renal Stone Surface Irregularities by Numerical Analysis of the Color Doppler Twinkling Artifact: An Ex Vivo Study.

OBJECTIVES: The physical structures of renal stones are highly correlated with their breakability. Noninvasive estimation of stone roughness will be beneficial for management. The intensity of the twinkling artifact appearing at the site of renal stones on Doppler ultrasound imaging is also influenced by the stone's roughness level. This article proposes a quantitative method for roughness prediction of ex vivo renal stones based on a twinkling analysis of their color Doppler images. METHODS: Twenty surgically removed renal stones were first spatially modeled by an optical method, and 12 standard roughness measures were extracted from them. Stones were then embedded in an agar-based phantom and Doppler imaged with a calibrated ultrasound system. The images were preprocessed, and 11 twinkling intensities were measured numerically. The twinkling data along with the roughness labels were then analyzed by multiple linear regressions, and finally, a linear roughness predictor was trained for renal stones. RESULTS: The core height measure of roughness had the best linear fit to the twinkling data among other roughness parameters. The results of the multiple linear regression analysis indicated a strong linear relationship between twinkling data and stones' roughness, with an R2 value of 83.29% and high statistical significance of F(11,868) = 393.36 and P < .001. CONCLUSIONS: It was possible to predict the core roughness of renal stones using the proposed method and the twinkling artifact data acquired from the color Doppler images ex vivo.