RESUMO
PURPOSE: Disease progression after definitive stereotactic body radiation therapy (SBRT) for early-stage non-small cell lung cancer (NSCLC) occurs in 20-40% of patients. Here, we explored published and novel pre-treatment CT and PET radiomics features to identify patients at risk of progression. MATERIALS/METHODS: Published CT and PET features were identified and explored along with 15 other CT and PET features in 408 consecutively treated early-stage NSCLC patients having CT and PET < 3 months pre-SBRT (training/set-aside validation subsets: n = 286/122). Features were associated with progression-free survival (PFS) using bootstrapped Cox regression (Bonferroni-corrected univariate predictor: p ≤ 0.002) and only non-strongly correlated predictors were retained (|Rs|<0.70) in forward-stepwise multivariate analysis. RESULTS: Tumor diameter and SUVmax were the two most frequently reported features associated with progression/survival (in 6/20 and 10/20 identified studies). These two features and 12 of the 15 additional features (CT: 6; PET: 6) were candidate PFS predictors. A re-fitted model including diameter and SUVmax presented with the best performance (c-index: 0.78; log-rank p-value < 0.0001). A model built with the two best additional features (CTspiculation1 and SUVentropy) had a c-index of 0.75 (log-rank p-value < 0.0001). CONCLUSIONS: A re-fitted pre-treatment model using the two most frequently published features - tumor diameter and SUVmax - successfully stratified early-stage NSCLC patients by PFS after receiving SBRT.
Assuntos
Carcinoma Pulmonar de Células não Pequenas , Neoplasias Pulmonares , Radiocirurgia , Carcinoma de Pequenas Células do Pulmão , Humanos , Carcinoma Pulmonar de Células não Pequenas/diagnóstico por imagem , Carcinoma Pulmonar de Células não Pequenas/radioterapia , Neoplasias Pulmonares/diagnóstico por imagem , Neoplasias Pulmonares/radioterapia , Radiômica , Fluordesoxiglucose F18 , Tomografia por Emissão de Pósitrons , Tomografia Computadorizada por Raios X , Tomografia por Emissão de Pósitrons combinada à Tomografia Computadorizada , Estudos Retrospectivos , PrognósticoRESUMO
We introduce a new AI-ready computational pathology dataset containing restained and co-registered digitized images from eight head-and-neck squamous cell carcinoma patients. Specifically, the same tumor sections were stained with the expensive multiplex immunofluorescence (mIF) assay first and then restained with cheaper multiplex immunohistochemistry (mIHC). This is a first public dataset that demonstrates the equivalence of these two staining methods which in turn allows several use cases; due to the equivalence, our cheaper mIHC staining protocol can offset the need for expensive mIF staining/scanning which requires highly-skilled lab technicians. As opposed to subjective and error-prone immune cell annotations from individual pathologists (disagreement > 50%) to drive SOTA deep learning approaches, this dataset provides objective immune and tumor cell annotations via mIF/mIHC restaining for more reproducible and accurate characterization of tumor immune microenvironment (e.g. for immunotherapy). We demonstrate the effectiveness of this dataset in three use cases: (1) IHC quantification of CD3/CD8 tumor-infiltrating lymphocytes via style transfer, (2) virtual translation of cheap mIHC stains to more expensive mIF stains, and (3) virtual tumor/immune cellular phenotyping on standard hematoxylin images. The dataset is available at https://github.com/nadeemlab/DeepLIIF.
RESUMO
AIMS: The International Medullary Thyroid Carcinoma Grading System, introduced in 2022, mandates evaluation of the Ki67 proliferation index to assign a histological grade for medullary thyroid carcinoma. However, manual counting remains a tedious and time-consuming task. METHODS AND RESULTS: We aimed to evaluate the performance of three other counting techniques for the Ki67 index, eyeballing by a trained experienced investigator, a machine learning-based deep learning algorithm (DeepLIIF) and an image analysis software with internal thresholding compared to the gold standard manual counting in a large cohort of 260 primarily resected medullary thyroid carcinoma. The Ki67 proliferation index generated by all three methods correlate near-perfectly with the manual Ki67 index, with kappa values ranging from 0.884 to 0.979 and interclass correlation coefficients ranging from 0.969 to 0.983. Discrepant Ki67 results were only observed in cases with borderline manual Ki67 readings, ranging from 3 to 7%. Medullary thyroid carcinomas with a high Ki67 index (≥ 5%) determined using any of the four methods were associated with significantly decreased disease-specific survival and distant metastasis-free survival. CONCLUSIONS: We herein validate a machine learning-based deep-learning platform and an image analysis software with internal thresholding to generate accurate automatic Ki67 proliferation indices in medullary thyroid carcinoma. Manual Ki67 count remains useful when facing a tumour with a borderline Ki67 proliferation index of 3-7%. In daily practice, validation of alternative evaluation methods for the Ki67 index in MTC is required prior to implementation.
Assuntos
Aprendizado Profundo , Neoplasias da Glândula Tireoide , Humanos , Antígeno Ki-67 , Proliferação de CélulasRESUMO
As predictive biomarkers of response to immune checkpoint inhibitors (ICIs) remain a major unmet clinical need in patients with urothelial carcinoma (UC), we sought to identify tissue-based immune biomarkers of clinical benefit to ICIs using multiplex immunofluorescence and to integrate these findings with previously identified peripheral blood biomarkers of response. Fifty-five pretreatment and 12 paired on-treatment UC specimens were identified from patients treated with nivolumab with or without ipilimumab. Whole tissue sections were stained with a 12-plex mIF panel, including CD8, PD-1/CD279, PD-L1/CD274, CD68, CD3, CD4, FoxP3, TCF1/7, Ki67, LAG-3, MHC-II/HLA-DR, and pancytokeratin+SOX10 to identify over three million cells. Immune tissue densities were compared to progression-free survival (PFS) and best overall response (BOR) by RECIST version 1.1. Correlation coefficients were calculated between tissue-based and circulating immune populations. The frequency of intratumoral CD3+ LAG-3+ cells was higher in responders compared to nonresponders (p = 0.0001). LAG-3+ cellular aggregates were associated with response, including CD3+ LAG-3+ in proximity to CD3+ (p = 0.01). Exploratory multivariate modeling showed an association between intratumoral CD3+ LAG-3+ cells and improved PFS independent of prognostic clinical factors (log HR -7.0; 95% confidence interval [CI] -12.7 to -1.4), as well as established biomarkers predictive of ICI response (log HR -5.0; 95% CI -9.8 to -0.2). Intratumoral LAG-3+ immune cell populations warrant further study as a predictive biomarker of clinical benefit to ICIs. Differences in LAG-3+ lymphocyte populations across the intratumoral and peripheral compartments may provide complementary information that could inform the future development of multimodal composite biomarkers of ICI response. © 2023 The Authors. The Journal of Pathology published by John Wiley & Sons Ltd on behalf of The Pathological Society of Great Britain and Ireland.
RESUMO
We introduce a new AI-ready computational pathology dataset containing restained and co-registered digitized images from eight head-and-neck squamous cell carcinoma patients. Specifically, the same tumor sections were stained with the expensive multiplex immunofluorescence (mIF) assay first and then restained with cheaper multiplex immunohistochemistry (mIHC). This is a first public dataset that demonstrates the equivalence of these two staining methods which in turn allows several use cases; due to the equivalence, our cheaper mIHC staining protocol can offset the need for expensive mIF staining/scanning which requires highly-skilled lab technicians. As opposed to subjective and error-prone immune cell annotations from individual pathologists (disagreement > 50%) to drive SOTA deep learning approaches, this dataset provides objective immune and tumor cell annotations via mIF/mIHC restaining for more reproducible and accurate characterization of tumor immune microenvironment (e.g. for immunotherapy). We demonstrate the effectiveness of this dataset in three use cases: (1) IHC quantification of CD3/CD8 tumor-infiltrating lymphocytes via style transfer, (2) virtual translation of cheap mIHC stains to more expensive mIF stains, and (3) virtual tumor/immune cellular phenotyping on standard hematoxylin images. The dataset is available at \url{https://github.com/nadeemlab/DeepLIIF}.
RESUMO
Objective.This work aims to generate realistic anatomical deformations from static patient scans. Specifically, we present a method to generate these deformations/augmentations via deep learning driven respiratory motion simulation that provides the ground truth for validating deformable image registration (DIR) algorithms and driving more accurate deep learning based DIR.Approach.We present a novel 3D Seq2Seq deep learning respiratory motion simulator (RMSim) that learns from 4D-CT images and predicts future breathing phases given a static CT image. The predicted respiratory patterns, represented by time-varying displacement vector fields (DVFs) at different breathing phases, are modulated through auxiliary inputs of 1D breathing traces so that a larger amplitude in the trace results in more significant predicted deformation. Stacked 3D-ConvLSTMs are used to capture the spatial-temporal respiration patterns. Training loss includes a smoothness loss in the DVF and mean-squared error between the predicted and ground truth phase images. A spatial transformer deforms the static CT with the predicted DVF to generate the predicted phase image. 10-phase 4D-CTs of 140 internal patients were used to train and test RMSim. The trained RMSim was then used to augment a public DIR challenge dataset for training VoxelMorph to show the effectiveness of RMSim-generated deformation augmentation.Main results.We validated our RMSim output with both private and public benchmark datasets (healthy and cancer patients). The structure similarity index measure (SSIM) for predicted breathing phases and ground truth 4D CT images was 0.92 ± 0.04, demonstrating RMSim's potential to generate realistic respiratory motion. Moreover, the landmark registration error in a public DIR dataset was improved from 8.12 ± 5.78 mm to 6.58mm ± 6.38 mm using RMSim-augmented training data.Significance.The proposed approach can be used for validating DIR algorithms as well as for patient-specific augmentations to improve deep learning DIR algorithms. The code, pretrained models, and augmented DIR validation datasets will be released athttps://github.com/nadeemlab/SeqX2Y.
Assuntos
Tomografia Computadorizada Quadridimensional , Processamento de Imagem Assistida por Computador , Humanos , Processamento de Imagem Assistida por Computador/métodos , Simulação por Computador , Tomografia Computadorizada Quadridimensional/métodos , Algoritmos , Movimento (Física)RESUMO
We present a novel approach for volume exploration that is versatile yet effective in isolating semantic structures in both noisy and clean data. Specifically, we describe a hierarchical active contours approach based on Bhattacharyya gradient flow which is easier to control, robust to noise, and can incorporate various types of statistical information to drive an edge-agnostic exploration process. To facilitate a time-bound user-driven volume exploration process that is applicable to a wide variety of data sources, we present an efficient multi-GPU implementation that (1) is approximately 400 times faster than a single thread CPU implementation, (2) allows hierarchical exploration of 2D and 3D images, (3) supports customization through multidimensional attribute spaces, and (4) is applicable to a variety of data sources and semantic structures. The exploration system follows a 2-step process. It first applies active contours to isolate semantically meaningful subsets of the volume. It then applies transfer functions to the isolated regions locally to produce clear and clutter-free visualizations. We show the effectiveness of our approach in isolating and visualizing structures-of-interest without needing any specialized segmentation methods on a variety of data sources, including 3D optical microscopy, multi-channel optical volumes, abdominal and chest CT, micro-CT, MRI, simulation, and synthetic data. We also gathered feedback from a medical trainee regarding the usefulness of our approach and discussion on potential applications in clinical workflows.
RESUMO
We propose an interactive visual analytics tool, Vis-SPLIT, for partitioning a population of individuals into groups with similar gene signatures. Vis-SPLIT allows users to interactively explore a dataset and exploit visual separations to build a classification model for specific cancers. The visualization components reveal gene expression and correlation to assist specific partitioning decisions, while also providing overviews for the decision model and clustered genetic signatures. We demonstrate the effectiveness of our framework through a case study and evaluate its usability with domain experts. Our results show that Vis-SPLIT can classify patients based on their genetic signatures to effectively gain insights into RNA sequencing data, as compared to an existing classification system.
RESUMO
Lifestyle behavior modification is an essential component to prevention and treatment of non-communicable diseases worldwide. For the last 40 years, studies have recognized that there is suboptimal training of physicians in lifestyle medicine and its implementation in clinical settings. The lack of nutrition and exercise counseling occurring in the medical office does not reflect the high level of evidence supporting its use. Lifestyle behavior counseling is complex; as are the individualized needs of patients. Therefore, we suspect that the lack of knowledge in nutrition and exercise prescriptions are not the only barriers to providing optimal care. Reframing lifestyle medicine interventions like nutrition and exercise from adjunctive to central to treatment and reframing the role of the physician therein may be necessary to address important barriers to overall lifestyle behavioral counseling.
RESUMO
Spiculations/lobulations, sharp/curved spikes on the surface of lung nodules, are good predictors of lung cancer malignancy and hence, are routinely assessed and reported by radiologists as part of the standardized Lung-RADS clinical scoring criteria. Given the 3D geometry of the nodule and 2D slice-by-slice assessment by radiologists, manual spiculation/lobulation annotation is a tedious task and thus no public datasets exist to date for probing the importance of these clinically-reported features in the SOTA malignancy prediction algorithms. As part of this paper, we release a large-scale Clinically-Interpretable Radiomics Dataset, CIRDataset, containing 956 radiologist QA/QC'ed spiculation/lobulation annotations on segmented lung nodules from two public datasets, LIDC-IDRI (N=883) and LUNGx (N=73). We also present an end-to-end deep learning model based on multi-class Voxel2Mesh extension to segment nodules (while preserving spikes), classify spikes (sharp/spiculation and curved/lobulation), and perform malignancy prediction. Previous methods have performed malignancy prediction for LIDC and LUNGx datasets but without robust attribution to any clinically reported/actionable features (due to known hyperparameter sensitivity issues with general attribution schemes). With the release of this comprehensively-annotated CIRDataset and end-to-end deep learning baseline, we hope that malignancy prediction methods can validate their explanations, benchmark against our baseline, and provide clinically-actionable insights. Dataset, code, pretrained models, and docker containers are available at https://github.com/nadeemlab/CIR.
RESUMO
In the clinic, resected tissue samples are stained with Hematoxylin-and-Eosin (H&E) and/or Immunhistochemistry (IHC) stains and presented to the pathologists on glass slides or as digital scans for diagnosis and assessment of disease progression. Cell-level quantification, e.g. in IHC protein expression scoring, can be extremely inefficient and subjective. We present DeepLIIF (https://deepliif.org), a first free online platform for efficient and reproducible IHC scoring. DeepLIIF outperforms current state-of-the-art approaches (relying on manual error-prone annotations) by virtually restaining clinical IHC slides with more informative multiplex immunofluorescence staining. Our DeepLIIF cloud-native platform supports (1) more than 150 proprietary/non-proprietary input formats via the Bio-Formats standard, (2) interactive adjustment, visualization, and downloading of the IHC quantification results and the accompanying restained images, (3) consumption of an exposed workflow API programmatically or through interactive plugins for open source whole slide image viewers such as QuPath/ImageJ, and (4) auto scaling to efficiently scale GPU resources based on user demand.
RESUMO
Reporting biomarkers assessed by routine immunohistochemical (IHC) staining of tissue is broadly used in diagnostic pathology laboratories for patient care. To date, clinical reporting is predominantly qualitative or semi-quantitative. By creating a multitask deep learning framework referred to as DeepLIIF, we present a single-step solution to stain deconvolution/separation, cell segmentation, and quantitative single-cell IHC scoring. Leveraging a unique de novo dataset of co-registered IHC and multiplex immunofluorescence (mpIF) staining of the same slides, we segment and translate low-cost and prevalent IHC slides to more expensive-yet-informative mpIF images, while simultaneously providing the essential ground truth for the superimposed brightfield IHC channels. Moreover, a new nuclear-envelop stain, LAP2beta, with high (>95%) cell coverage is introduced to improve cell delineation/segmentation and protein expression quantification on IHC slides. By simultaneously translating input IHC images to clean/separated mpIF channels and performing cell segmentation/classification, we show that our model trained on clean IHC Ki67 data can generalize to more noisy and artifact-ridden images as well as other nuclear and non-nuclear markers such as CD3, CD8, BCL2, BCL6, MYC, MUM1, CD10, and TP53. We thoroughly evaluate our method on publicly available benchmark datasets as well as against pathologists' semi-quantitative scoring. The code, the pre-trained models, along with easy-to-run containerized docker files as well as Google CoLab project are available at https://github.com/nadeemlab/deepliif.
RESUMO
Automated analysis of optical colonoscopy (OC) video frames (to assist endoscopists during OC) is challenging due to variations in color, lighting, texture, and specular reflections. Previous methods either remove some of these variations via preprocessing (making pipelines cumbersome) or add diverse training data with annotations (but expensive and time-consuming). We present CLTS-GAN, a new deep learning model that gives fine control over color, lighting, texture, and specular reflection synthesis for OC video frames. We show that adding these colonoscopy-specific augmentations to the training data can improve state-of-the-art polyp detection/segmentation methods as well as drive next generation of OC simulators for training medical students. The code and pre-trained models for CLTS-GAN are available on Computational Endoscopy Platform GitHub (https://github.com/nadeemlab/CEP).
RESUMO
Objective.To propose a novel moment-based loss function for predicting 3D dose distribution for the challenging conventional lung intensity modulated radiation therapy plans. The moment-based loss function is convex and differentiable and can easily incorporate clinical dose volume histogram (DVH) domain knowledge in any deep learning (DL) framework without computational overhead.Approach.We used a large dataset of 360 (240 for training, 50 for validation and 70 for testing) conventional lung patients with 2 Gy × 30 fractions to train the DL model using clinically treated plans at our institution. We trained a UNet like convolutional neural network architecture using computed tomography, planning target volume and organ-at-risk contours as input to infer corresponding voxel-wise 3D dose distribution. We evaluated three different loss functions: (1) the popular mean absolute error (MAE) loss, (2) the recently developed MAE + DVH loss, and (3) the proposed MAE + moments loss. The quality of the predictions was compared using different DVH metrics as well as dose-score and DVH-score, recently introduced by theAAPM knowledge-based planning grand challenge. Main results.Model with (MAE + moment) loss function outperformed the model with MAE loss by significantly improving the DVH-score (11%,p< 0.01) while having similar computational cost. It also outperformed the model trained with (MAE + DVH) by significantly improving the computational cost (48%) and the DVH-score (8%,p< 0.01).Significance.DVH metrics are widely accepted evaluation criteria in the clinic. However, incorporating them into the 3D dose prediction model is challenging due to their non-convexity and non-differentiability. Moments provide a mathematically rigorous and computationally efficient way to incorporate DVH information in any DL architecture. The code, pretrained models, docker container, and Google Colab project along with a sample dataset are available on our DoseRTX GitHub (https://github.com/nadeemlab/DoseRTX).
Assuntos
Órgãos em Risco , Radioterapia de Intensidade Modulada , Humanos , Redes Neurais de Computação , Dosagem Radioterapêutica , Planejamento da Radioterapia Assistida por Computador/métodos , Radioterapia de Intensidade Modulada/métodosRESUMO
Despite considerable evidence that plant-based diets can significantly improve health, medical professionals seldom discuss this with their patients. This issue might occur due to minimal training received in medical education, lack of time, and low self-efficacy for counseling patients about diet. Nutrition and lifestyle change should be considered a core competency for all physicians and health professionals looking for cost-effective ways to improve patient health outcomes and reduce nutrition-related chronic diseases. Strategies for health professionals to acquire nutrition counseling skills in medical training and clinical practices are discussed.
RESUMO
BACKGROUND: Pathologic response at the time of surgery after neoadjuvant therapy for HER2 positive early breast cancer impacts both prognosis and subsequent adjuvant therapy. Comprehensive descriptions of the tumor microenvironment (TME) in patients with HER2 positive early breast cancer is not well described. We utilized standard stromal pathologist-assessed tumor infiltrating lymphocyte (TIL) quantification, quantitative multiplex immunofluorescence, and RNA-based gene pathway signatures to assess pretreatment TME characteristics associated pathologic complete response in patients with hormone receptor positive, HER2 positive early breast cancer treated in the neoadjuvant setting. METHODS: We utilized standard stromal pathologist-assessed TIL quantification, quantitative multiplex immunofluorescence, and RNA-based gene pathway signatures to assess pretreatment TME characteristics associated pathologic complete response in 28 patients with hormone receptor positive, HER2 positive early breast cancer treated in the neoadjuvant setting. RESULTS: Pathologist-assessed stromal TILs were significantly associated with pathologic complete response (pCR). By quantitative multiplex immunofluorescence, univariate analysis revealed significant increases in CD3+, CD3+CD8-FOXP3-, CD8+ and FOXP3+ T-cell densities as well as increased immune cell aggregates in pCR patients. In subsets of paired pre/post-treatment samples, we observed significant changes in gene expression signatures in non-pCR patients and significant decreases in CD8+ densities after treatment in pCR patients. No RNA based pathway signature was associated with pCR. CONCLUSION: TME characterization HER2 positive breast cancer patients revealed several stromal T-cell densities and immune cell aggregates associated with pCR. These results demonstrate the feasibility of these novel methods in TME evaluation and contribute to ongoing investigations of the TME in HER2+ early breast cancer to identify robust biomarkers to best identify patients eligible for systemic de-escalation strategies.
Assuntos
Neoplasias da Mama , Terapia Neoadjuvante , Protocolos de Quimioterapia Combinada Antineoplásica/uso terapêutico , Biomarcadores Tumorais/genética , Biomarcadores Tumorais/metabolismo , Neoplasias da Mama/tratamento farmacológico , Neoplasias da Mama/genética , Neoplasias da Mama/patologia , Feminino , Fatores de Transcrição Forkhead/metabolismo , Fatores de Transcrição Forkhead/uso terapêutico , Hormônios/metabolismo , Humanos , Linfócitos do Interstício Tumoral , Terapia Neoadjuvante/métodos , Prognóstico , Receptor ErbB-2/metabolismo , Microambiente TumoralRESUMO
Cancer immunotherapy can result in lasting tumor regression, but predictive biomarkers of treatment response remain ill-defined. Here, we performed single-cell proteomics, transcriptomics, and genomics on matched untreated and IL2 injected metastases from patients with melanoma. Lesions that completely regressed following intralesional IL2 harbored increased fractions and densities of nonproliferating CD8+ T cells lacking expression of PD-1, LAG-3, and TIM-3 (PD-1-LAG-3-TIM-3-). Untreated lesions from patients who subsequently responded with complete eradication of all tumor cells in all injected lesions (individuals referred to herein as "extreme responders") were characterized by proliferating CD8+ T cells with an exhausted phenotype (PD-1+LAG-3+TIM-3+), stromal B-cell aggregates, and expression of IFNγ and IL2 response genes. Loss of membranous MHC class I expression in tumor cells of untreated lesions was associated with resistance to IL2 therapy. We validated this finding in an independent cohort of metastatic melanoma patients treated with intralesional or systemic IL2. Our study suggests that intact tumor-cell antigen presentation is required for melanoma response to IL2 and describes a multidimensional and spatial approach to develop immuno-oncology biomarker hypotheses using routinely collected clinical biospecimens.
Assuntos
Interleucina-2 , Melanoma , Receptor Celular 2 do Vírus da Hepatite A , Humanos , Imunoterapia/métodos , Interleucina-2/uso terapêutico , Melanoma/tratamento farmacológico , Melanoma/genética , Melanoma/patologia , Receptor de Morte Celular Programada 1/metabolismoRESUMO
Optical colonoscopy (OC), the most prevalent colon cancer screening tool, has a high miss rate due to a number of factors, including the geometry of the colon (haustral fold and sharp bends occlusions), endoscopist inexperience or fatigue, endoscope field of view. We present a framework to visualize the missed regions per-frame during OC, and provides a workable clinical solution. Specifically, we make use of 3D reconstructed virtual colonoscopy (VC) data and the insight that VC and OC share the same underlying geometry but differ in color, texture and specular reflections, embedded in the OC. A lossy unpaired image-to-image translation model is introduced with enforced shared latent space for OC and VC. This shared space captures the geometric information while deferring the color, texture, and specular information creation to additional Gaussian noise input. The latter can be utilized to generate one-to-many mappings from VC to OC and OC to OC. The code, data and trained models will be released via our Computational Endoscopy Platform at https://github.com/nadeemlab/CEP.
RESUMO
PURPOSE: In current clinical practice, noisy and artifact-ridden weekly cone beam computed tomography (CBCT) images are only used for patient setup during radiotherapy. Treatment planning is performed once at the beginning of the treatment using high-quality planning CT (pCT) images and manual contours for organs-at-risk (OARs) structures. If the quality of the weekly CBCT images can be improved while simultaneously segmenting OAR structures, this can provide critical information for adapting radiotherapy mid-treatment as well as for deriving biomarkers for treatment response. METHODS: Using a novel physics-based data augmentation strategy, we synthesize a large dataset of perfectly/inherently registered pCT and synthetic-CBCT pairs for locally advanced lung cancer patient cohort, which are then used in a multitask three-dimensional (3D) deep learning framework to simultaneously segment and translate real weekly CBCT images to high-quality pCT-like images. RESULTS: We compared the synthetic CT and OAR segmentations generated by the model to real pCT and manual OAR segmentations and showed promising results. The real week 1 (baseline) CBCT images which had an average mean absolute error (MAE) of 162.77 HU compared to pCT images are translated to synthetic CT images that exhibit a drastically improved average MAE of 29.31 HU and average structural similarity of 92% with the pCT images. The average DICE scores of the 3D OARs segmentations are: lungs 0.96, heart 0.88, spinal cord 0.83, and esophagus 0.66. CONCLUSIONS: We demonstrate an approach to translate artifact-ridden CBCT images to high-quality synthetic CT images, while simultaneously generating good quality segmentation masks for different OARs. This approach could allow clinicians to adjust treatment plans using only the routine low-quality CBCT images, potentially improving patient outcomes. Our code, data, and pre-trained models will be made available via our physics-based data augmentation library, Physics-ArX, at https://github.com/nadeemlab/Physics-ArX.
Assuntos
Tomografia Computadorizada de Feixe Cônico Espiral , Tomografia Computadorizada de Feixe Cônico , Humanos , Processamento de Imagem Assistida por Computador , Órgãos em Risco , Física , Planejamento da Radioterapia Assistida por ComputadorRESUMO
PURPOSE: Radiotherapy presents unique challenges and clinical requirements for longitudinal tumor and organ-at-risk (OAR) prediction during treatment. The challenges include tumor inflammation/edema and radiation-induced changes in organ geometry, whereas the clinical requirements demand flexibility in input/output sequence timepoints to update the predictions on rolling basis and the grounding of all predictions in relationship to the pre-treatment imaging information for response and toxicity assessment in adaptive radiotherapy. METHODS: To deal with the aforementioned challenges and to comply with the clinical requirements, we present a novel 3D sequence-to-sequence model based on Convolution Long Short-Term Memory (ConvLSTM) that makes use of series of deformation vector fields (DVFs) between individual timepoints and reference pre-treatment/planning CTs to predict future anatomical deformations and changes in gross tumor volume as well as critical OARs. High-quality DVF training data are created by employing hyper-parameter optimization on the subset of the training data with DICE coefficient and mutual information metric. We validated our model on two radiotherapy datasets: a publicly available head-and-neck dataset (28 patients with manually contoured pre-, mid-, and post-treatment CTs), and an internal non-small cell lung cancer dataset (63 patients with manually contoured planning CT and 6 weekly CBCTs). RESULTS: The use of DVF representation and skip connections overcomes the blurring issue of ConvLSTM prediction with the traditional image representation. The mean and standard deviation of DICE for predictions of lung GTV at weeks 4, 5, and 6 were 0.83 ± 0.09, 0.82 ± 0.08, and 0.81 ± 0.10, respectively, and for post-treatment ipsilateral and contralateral parotids, were 0.81 ± 0.06 and 0.85 ± 0.02. CONCLUSION: We presented a novel DVF-based Seq2Seq model for medical images, leveraging the complete 3D imaging information of a relatively large longitudinal clinical dataset, to carry out longitudinal GTV/OAR predictions for anatomical changes in HN and lung radiotherapy patients, which has potential to improve RT outcomes.