Revolutionizing Digital Pathology With the Power of Generative Artificial Intelligence and Foundation Models.

Digital pathology has transformed the traditional pathology practice of analyzing tissue under a microscope into a computer vision workflow. Whole-slide imaging allows pathologists to view and analyze microscopic images on a computer monitor, enabling computational pathology. By leveraging artificial intelligence (AI) and machine learning (ML), computational pathology has emerged as a promising field in recent years. Recently, task-specific AI/ML (eg, convolutional neural networks) has risen to the forefront, achieving above-human performance in many image-processing and computer vision tasks. The performance of task-specific AI/ML models depends on the availability of many annotated training datasets, which presents a rate-limiting factor for AI/ML development in pathology. Task-specific AI/ML models cannot benefit from multimodal data and lack generalization, eg, the AI models often struggle to generalize to new datasets or unseen variations in image acquisition, staining techniques, or tissue types. The 2020s are witnessing the rise of foundation models and generative AI. A foundation model is a large AI model trained using sizable data, which is later adapted (or fine-tuned) to perform different tasks using a modest amount of task-specific annotated data. These AI models provide in-context learning, can self-correct mistakes, and promptly adjust to user feedback. In this review, we provide a brief overview of recent advances in computational pathology enabled by task-specific AI, their challenges and limitations, and then introduce various foundation models. We propose to create a pathology-specific generative AI based on multimodal foundation models and present its potentially transformative role in digital pathology. We describe different use cases, delineating how it could serve as an expert companion of pathologists and help them efficiently and objectively perform routine laboratory tasks, including quantifying image analysis, generating pathology reports, diagnosis, and prognosis. We also outline the potential role that foundation models and generative AI can play in standardizing the pathology laboratory workflow, education, and training.

The predictive value of pretherapy [68Ga]Ga-DOTA-TATE PET and biomarkers in [177Lu]Lu-PRRT tumor dosimetry.

PURPOSE: Metastatic neuroendocrine tumors (NETs) overexpressing type 2 somatostatin receptors are the target for peptide receptor radionuclide therapy (PRRT) through the theragnostic pair of 68Ga/177Lu-DOTATATE. The main purpose of this study was to develop machine learning models to predict therapeutic tumor dose using pre therapy 68Ga -PET and clinicopathological biomarkers. METHODS: We retrospectively analyzed 90 segmented metastatic NETs from 25 patients (M14/F11, age 63.7 ± 9.5, range 38-76) treated by 177Lu-DOTATATE at our institute. Patients underwent both pretherapy [68Ga]Ga-DOTA-TATE PET/CT and four timepoints SPECT/CT at ~ 4, 24, 96, and 168 h post-177Lu-DOTATATE infusion. Tumors were segmented by a radiologist on baseline CT or MRI and transferred to co-registered PET/CT and SPECT/CT, and normal organs were segmented by deep learning-based method on CT of the PET and SPECT. The SUV metrics and tumor-to-normal tissue SUV ratios (SUV_TNRs) were calculated from 68Ga -PET at the contour-level. Posttherapy dosimetry was performed based on the co-registration of SPECT/CTs to generate time-integrated-activity, followed by an in-house Monte Carlo-based absorbed dose estimation. The correlation between delivered 177Lu Tumor absorbed dose and PET-derived metrics along with baseline clinicopathological biomarkers (such as Creatinine, Chromogranin A and prior therapies) were evaluated. Multiple interpretable machine-learning algorithms were developed to predict tumor dose using these pretherapy information. Model performance on a nested tenfold cross-validation was evaluated in terms of coefficient of determination (R2), mean-absolute-error (MAE), and mean-relative-absolute-error (MRAE). RESULTS: SUVmean showed a significant correlation (q-value < 0.05) with absorbed dose (Spearman ρ = 0.64), followed by TLSUVmean (SUVmean of total-lesion-burden) and SUVpeak (ρ = 0.45 and 0.41, respectively). The predictive value of PET-SUVmean in estimation of posttherapy absorbed dose was stronger compared to PET-SUVpeak, and SUV_TNRs in terms of univariate analysis (R2 = 0.28 vs. R2 ≤ 0.12). An optimal trivariate random forest model composed of SUVmean, TLSUVmean, and total liver SUVmean (normal and tumoral liver) provided the best performance in tumor dose prediction with R2 = 0.64, MAE = 0.73 Gy/GBq, and MRAE = 0.2. CONCLUSION: Our preliminary results demonstrate the feasibility of using baseline PET images for prediction of absorbed dose prior to 177Lu-PRRT. Machine learning models combining multiple PET-based metrics performed better than using a single SUV value and using other investigated clinicopathological biomarkers. Developing such quantitative models forms the groundwork for the role of 68Ga -PET not only for the implementation of personalized treatment planning but also for patient stratification in the era of precision medicine.

Enhanced immune activation within the tumor microenvironment and circulation of female high-risk melanoma patients and improved survival with adjuvant CTLA4 blockade compared to males.

BACKGROUND: We hypothesized that a gender difference in clinical response may exist to adjuvant CTLA4 blockade with ipilimumab versus high-dose IFNα (HDI). We investigated differences in candidate immune biomarkers in the circulation and tumor microenvironment (TME). PATIENTS AND METHODS: This gender-based analysis was nested within the E1609 trial that tested adjuvant therapy with ipilimumab 3 mg/kg (ipi3) and 10 mg/kg (ipi10) versus HDI in high risk resected melanoma. We investigated gender differences in treatment efficacy with ipi3 and ipi10 versus HDI while adjusting for age, stage, ECOG performance (PS), ulceration, primary tumor status and lymph node number. Forest plots were created to compare overall survival (OS) and relapse free survival (RFS) between ipi and HDI. Gene expression profiling (GEP) was performed on tumors of 718 (454 male, 264 female) patients. Similarly, serum and peripheral blood mononuclear cells (PBMC) samples were tested for soluble and cellular biomarkers (N = 321 patients; 109 female and 212 male). RESULTS: The subgroups of female, stage IIIC, PS = 1, ulcerated primary, in-transit metastasis demonstrated significant improvement in RFS and/or OS with ipi3 versus HDI. Female gender was significant for both OS and RFS and was further explored. In the RFS comparison, a multivariate Cox regression model including significant variables indicated a significant interaction between gender and treatment (P = 0.024). In peripheral blood, percentages of CD3+ T cells (P = 0.024) and CD3+ CD4+ helper T cells (P = 0.0001) were higher in females compared to males. Trends toward higher circulating levels of IL1ß (P = 0.07) and IL6 (P = 0.06) were also found in females. Males had higher percentages of monocytes (P = 0.03) with trends toward higher percentages of regulatory T cells (T-reg). Tumor GEP analysis supported enhanced infiltration with immune cells including gammadelta T cells (P = 0.005), NK cells (P = 0.01), dendritic cells (P = 0.01), CD4+ T cells (P = 0.03), CD8+ T cells (P = 0.03) and T-reg (P = 0.008) in the tumors of females compared to males and a higher T-effector and IFNγ gene signature score (P = 0.0244). CONCLUSION: Female gender was associated with adjuvant CTLA4 blockade clinical benefits and female patients were more likely to have evidence of type1 immune activation within the TME and the circulation. Trial registration ClinicalTrials.gov NCT01274338. Registered 11 January 2011, https://www. CLINICALTRIALS: gov/ct2/show/NCT01274338.

Quantitative Molecular Positron Emission Tomography Imaging Using Advanced Deep Learning Techniques.

The widespread availability of high-performance computing and the popularity of artificial intelligence (AI) with machine learning and deep learning (ML/DL) algorithms at the helm have stimulated the development of many applications involving the use of AI-based techniques in molecular imaging research. Applications reported in the literature encompass various areas, including innovative design concepts in positron emission tomography (PET) instrumentation, quantitative image reconstruction and analysis techniques, computer-aided detection and diagnosis, as well as modeling and prediction of outcomes. This review reflects the tremendous interest in quantitative molecular imaging using ML/DL techniques during the past decade, ranging from the basic principles of ML/DL techniques to the various steps required for obtaining quantitatively accurate PET data, including algorithms used to denoise or correct for physical degrading factors as well as to quantify tracer uptake and metabolic tumor volume for treatment monitoring or radiation therapy treatment planning and response prediction.This review also addresses future opportunities and current challenges facing the adoption of ML/DL approaches and their role in multimodality imaging.

Cluster model incorporating heterogeneous dose distribution of partial parotid irradiation for radiotherapy induced xerostomia prediction with machine learning methods.

PURPOSE: A cluster model incorporating heterogeneous dose distribution within the parotid gland was developed and validated retrospectively for radiotherapy (RT) induced xerostomia prediction with machine learning (ML) techniques. METHODS: Sixty clusters were obtained at 1 Gy step size with threshold doses ranging from 1 to 60 Gy, for each of the enrolled 155 patients with HNC from three institutions. Feature clusters were selected with the neighborhood component analysis (NCA) and subsequently fed into four supervised ML models for xerostomia prediction comparison: support vector machines (SVM), k-nearest neighbor (kNN), naïve Bayes (NB), and random forest (RF). The predictive performance of each model was evaluated using cross validation resampling with the area-under-the-curves (AUC) of the receiver-operating-characteristic (ROC). The xerostomia predicting capacity using testing data was assessed with accuracy, sensitivity, and specificity for these models and three cluster connectivity choices. Mean dose based logistic regression served as the benchmark for evaluation. RESULTS: Feature clusters identified by NCA fell in three threshold dose ranges: 5-15Gy, 25-35Gy, and 45-50Gy. Mean dose predictive power was 15% lower than that of the cluster model using the logistic regression classifier. Model validation demonstrated that kNN model outperformed slightly other three models but no substantial difference was observed. Applying the fine-tuned models to testing data yielded that the mean accuracy from SVM, kNN and NB models were between 0.68 and 0.7 while that of RF was ∼0.6. SVM model yielded the best sensitivity (0.76) and kNN model delivered consistent sensitivity and specificity. This is consistent with cross validation. Clusters calculated with three connectivity choices exhibited minimally different predictions. CONCLUSION: Compared to mean dose, the proposed cluster model has shown its improvement as the xerostomia predictor. When combining with ML techniques, it could provide a clinically useful tool for xerostomia prediction and facilitate decision making during radiotherapy planning for patients with HNC.

The Image Biomarker Standardization Initiative: Standardized Quantitative Radiomics for High-Throughput Image-based Phenotyping.

Background Radiomic features may quantify characteristics present in medical imaging. However, the lack of standardized definitions and validated reference values have hampered clinical use. Purpose To standardize a set of 174 radiomic features. Materials and Methods Radiomic features were assessed in three phases. In phase I, 487 features were derived from the basic set of 174 features. Twenty-five research teams with unique radiomics software implementations computed feature values directly from a digital phantom, without any additional image processing. In phase II, 15 teams computed values for 1347 derived features using a CT image of a patient with lung cancer and predefined image processing configurations. In both phases, consensus among the teams on the validity of tentative reference values was measured through the frequency of the modal value and classified as follows: less than three matches, weak; three to five matches, moderate; six to nine matches, strong; 10 or more matches, very strong. In the final phase (phase III), a public data set of multimodality images (CT, fluorine 18 fluorodeoxyglucose PET, and T1-weighted MRI) from 51 patients with soft-tissue sarcoma was used to prospectively assess reproducibility of standardized features. Results Consensus on reference values was initially weak for 232 of 302 features (76.8%) at phase I and 703 of 1075 features (65.4%) at phase II. At the final iteration, weak consensus remained for only two of 487 features (0.4%) at phase I and 19 of 1347 features (1.4%) at phase II. Strong or better consensus was achieved for 463 of 487 features (95.1%) at phase I and 1220 of 1347 features (90.6%) at phase II. Overall, 169 of 174 features were standardized in the first two phases. In the final validation phase (phase III), most of the 169 standardized features could be excellently reproduced (166 with CT; 164 with PET; and 164 with MRI). Conclusion A set of 169 radiomics features was standardized, which enabled verification and calibration of different radiomics software. © RSNA, 2020 Online supplemental material is available for this article. See also the editorial by Kuhl and Truhn in this issue.

Machine Learning and Imaging Informatics in Oncology.

In the era of personalized and precision medicine, informatics technologies utilizing machine learning (ML) and quantitative imaging are witnessing a rapidly increasing role in medicine in general and in oncology in particular. This expanding role ranges from computer-aided diagnosis to decision support of treatments with the potential to transform the current landscape of cancer management. In this review, we aim to provide an overview of ML methodologies and imaging informatics techniques and their recent application in modern oncology. We will review example applications of ML in oncology from the literature, identify current challenges and highlight future potentials.

Oncology Informatics: Status Quo and Outlook.

Oncology has undergone rapid progress, with emerging developments in areas including cancer stem cells, molecularly targeted therapies, genomic analyses, and individually tailored immunotherapy. These advances have expanded the tools available in the fight against cancer. Some of these have seen broad media coverage resulting in justified public attention. However, these achievements have only been possible due to rapid developments in the expanding field of biomedical informatics and information technology (IT). Artificial intelligence, radiomics, electronic health records, and electronic patient-reported outcome measures (ePROMS) are only a few of the developments enabling further progress in oncology. The promising impact of IT in oncology will only become reality through a multidisciplinary approach to the complex challenges ahead.

Machine learning for radiomics-based multimodality and multiparametric modeling.

Due to the recent developments of both hardware and software technologies, multimodality medical imaging techniques have been increasingly applied in clinical practice and research studies. Previously, the application of multimodality imaging in oncology has been mainly related to combining anatomical and functional imaging to improve diagnostic specificity and/or target definition, such as positron emission tomography/computed tomography (PET/CT) and single-photon emission CT (SPECT)/CT. More recently, the fusion of various images, such as multiparametric magnetic resonance imaging (MRI) sequences, different PET tracer images, PET/MRI, has become more prevalent, which has enabled more comprehensive characterization of the tumor phenotype. In order to take advantage of these valuable multimodal data for clinical decision making using radiomics, we present two ways to implement the multimodal image analysis, namely radiomic (handcrafted feature) based and deep learning (machine learned feature) based methods. Applying advanced machine (deep) learning algorithms across multimodality images have shown better results compared with single modality modeling for prognostic and/or prediction of clinical outcomes. This holds great potentials for providing more personalized treatment for patients and achieve better outcomes.

Integrating radiomics into clinical trial design.

In radiomics, quantitative features that describe phenotypic tumor characteristics are derived from radiographic images. Because radiomics generates information from routine medical images, it is a powerful way to non-invasively examine the spatial and temporal heterogeneity of disease, and thus has potential to significantly impact clinical trial design, execution, and ultimately patient care. The aim of this review article is to discuss how radiomics may address some of the current challenges in clinical randomized control trials, and the difficulties of integrating robust and repeatable radiomics analysis into trial design. Each step of the radiomics process, including image acquisition and reconstruction, image segmentation, feature extraction, and computational analysis, requires extensive standardization in order to be successfully incorporated into clinical trials and inform clinical decision making. By addressing these challenges, the potential of radiomics may be realized.

A prediction model for early death in non-small cell lung cancer patients following curative-intent chemoradiotherapy.

BACKGROUND: Early death after a treatment can be seen as a therapeutic failure. Accurate prediction of patients at risk for early mortality is crucial to avoid unnecessary harm and reducing costs. The goal of our work is two-fold: first, to evaluate the performance of a previously published model for early death in our cohorts. Second, to develop a prognostic model for early death prediction following radiotherapy. MATERIAL AND METHODS: Patients with NSCLC treated with chemoradiotherapy or radiotherapy alone were included in this study. Four different cohorts from different countries were available for this work (N = 1540). The previous model used age, gender, performance status, tumor stage, income deprivation, no previous treatment given (yes/no) and body mass index to make predictions. A random forest model was developed by learning on the Maastro cohort (N = 698). The new model used performance status, age, gender, T and N stage, total tumor volume (cc), total tumor dose (Gy) and chemotherapy timing (none, sequential, concurrent) to make predictions. Death within 4 months of receiving the first radiotherapy fraction was used as the outcome. RESULTS: Early death rates ranged from 6 to 11% within the four cohorts. The previous model performed with AUC values ranging from 0.54 to 0.64 on the validation cohorts. Our newly developed model had improved AUC values ranging from 0.62 to 0.71 on the validation cohorts. CONCLUSIONS: Using advanced machine learning methods and informative variables, prognostic models for early mortality can be developed. Development of accurate prognostic tools for early mortality is important to inform patients about treatment options and optimize care.

Perspectives on making big data analytics work for oncology.

Oncology, with its unique combination of clinical, physical, technological, and biological data provides an ideal case study for applying big data analytics to improve cancer treatment safety and outcomes. An oncology treatment course such as chemoradiotherapy can generate a large pool of information carrying the 5Vs hallmarks of big data. This data is comprised of a heterogeneous mixture of patient demographics, radiation/chemo dosimetry, multimodality imaging features, and biological markers generated over a treatment period that can span few days to several weeks. Efforts using commercial and in-house tools are underway to facilitate data aggregation, ontology creation, sharing, visualization and varying analytics in a secure environment. However, open questions related to proper data structure representation and effective analytics tools to support oncology decision-making need to be addressed. It is recognized that oncology data constitutes a mix of structured (tabulated) and unstructured (electronic documents) that need to be processed to facilitate searching and subsequent knowledge discovery from relational or NoSQL databases. In this context, methods based on advanced analytics and image feature extraction for oncology applications will be discussed. On the other hand, the classical p (variables)≫n (samples) inference problem of statistical learning is challenged in the Big data realm and this is particularly true for oncology applications where p-omics is witnessing exponential growth while the number of cancer incidences has generally plateaued over the past 5-years leading to a quasi-linear growth in samples per patient. Within the Big data paradigm, this kind of phenomenon may yield undesirable effects such as echo chamber anomalies, Yule-Simpson reversal paradox, or misleading ghost analytics. In this work, we will present these effects as they pertain to oncology and engage small thinking methodologies to counter these effects ranging from incorporating prior knowledge, using information-theoretic techniques to modern ensemble machine learning approaches or combination of these. We will particularly discuss the pros and cons of different approaches to improve mining of big data in oncology.

Modeling non-genetic information dynamics in cells using reservoir computing.

Virtually all cells use energy-driven, ion-specific membrane pumps to maintain large transmembrane gradients of Na+, K+, Cl-, Mg++, and Ca++, but the corresponding evolutionary benefit remains unclear. We propose that these gradients enable a dynamic and versatile biological system that acquires, analyzes, and responds to environmental information. We hypothesize that environmental signals are transmitted into the cell by ion fluxes along pre-existing gradients through gated ion-specific membrane channels. The consequent changes in cytoplasmic ion concentration can generate a local response or orchestrate global/regional cellular dynamics through wire-like ion fluxes along pre-existing and self-assembling cytoskeleton to engage the endoplasmic reticulum, mitochondria, and nucleus.

Presurgical Upgrade Prediction of DCIS to Invasive Ductal Carcinoma Using Time-dependent Deep Learning Models with DCE MRI.

"Just Accepted" papers have undergone full peer review and have been accepted for publication in Radiology: Artificial Intelligence. This article will undergo copyediting, layout, and proof review before it is published in its final version. Please note that during production of the final copyedited article, errors may be discovered which could affect the content. Purpose To determine whether time-dependent deep learning models can outperform single timepoint models in predicting preoperative upgrade of ductal carcinoma in situ (DCIS) to invasive malignancy on dynamic contrastenhanced (DCE) breast MRI without lesion segmentation prerequisite. Materials and Methods In this exploratory study, 154 cases of biopsy-proven DCIS (25 upgraded at surgery and 129 not upgraded) were selected consecutively from a retrospective cohort of preoperative DCE MRI in women with an average age of 58.6 years at time of diagnosis from 2012 to 2022. Binary classification was implemented with convolutional neural network-long short-term memory (CNN-LSTM) architectures benchmarked against traditional CNNs without manual segmentation of the lesions. Combinatorial performance analysis of ResNet50 versus VGG16-based models was performed with each contrast phase. Binary classification area under the receiver operating characteristic curve (AUC) was reported. Results VGG16-based models consistently provided better hold-out test AUCs than ResNet50 in CNN and CNNLSTM studies (multiphase test AUC: 0.67 versus 0.59, respectively, for CNN models; P = .04 and 0.73 versus 0.62 for CNN-LSTM models; P = .008). The time-dependent model (CNN-LSTM) provided a better multiphase test AUC over single-timepoint (CNN) models (0.73 versus 0.67, P = .04). Conclusion Compared with single-timepoint architectures, sequential deep learning algorithms using preoperative DCE MRI improved prediction of DCIS lesions upgraded to invasive malignancy without the need for lesion segmentation. ©RSNA, 2024.

A joint ESTRO and AAPM guideline for development, clinical validation and reporting of artificial intelligence models in radiation therapy.

BACKGROUND AND PURPOSE: Artificial Intelligence (AI) models in radiation therapy are being developed with increasing pace. Despite this, the radiation therapy community has not widely adopted these models in clinical practice. A cohesive guideline on how to develop, report and clinically validate AI algorithms might help bridge this gap. METHODS AND MATERIALS: A Delphi process with all co-authors was followed to determine which topics should be addressed in this comprehensive guideline. Separate sections of the guideline, including Statements, were written by subgroups of the authors and discussed with the whole group at several meetings. Statements were formulated and scored as highly recommended or recommended. RESULTS: The following topics were found most relevant: Decision making, image analysis, volume segmentation, treatment planning, patient specific quality assurance of treatment delivery, adaptive treatment, outcome prediction, training, validation and testing of AI model parameters, model availability for others to verify, model quality assurance/updates and upgrades, ethics. Key references were given together with an outlook on current hurdles and possibilities to overcome these. 19 Statements were formulated. CONCLUSION: A cohesive guideline has been written which addresses main topics regarding AI in radiation therapy. It will help to guide development, as well as transparent and consistent reporting and validation of new AI tools and facilitate adoption.

Pancreatic Cyst Size Measurement on Magnetic Resonance Imaging Compared to Pathology.

BACKGROUND: While multiple cyst features are evaluated for stratifying pancreatic intraductal papillary mucinous neoplasms (IPMN), cyst size is an important factor that can influence treatment strategies. When magnetic resonance imaging (MRI) is used to evaluate IPMNs, no universally accepted sequence provides optimal size measurements. T2-weighted coronal/axial have been suggested as primary measurement sequences; however, it remains unknown how well these and maximum all-sequence diameter measurements correlate with pathology size. This study aims to compare agreement and bias between IPMN long-axis measurements on seven commonly obtained MRI sequences with pathologic size measurements. METHODS: This retrospective cohort included surgically resected IPMN cases with preoperative MRI exams. Long-axis diameter tumor measurements and the presence of worrisome features and/orhigh-risk stigmata were noted on all seven MRI sequences. MRI size and pathology agreement and MRI inter-observer agreement involved concordance correlation coefficient (CCC) and intraclass correlation coefficient (ICC), respectively. The presence of worrisome features and high-risk stigmata were compared to the tumor grade using kappa analysis. The Bland-Altman analysis assessed the systematic bias between MRI-size and pathology. RESULTS: In 52 patients (age 68 ± 13 years, 22 males), MRI sequences produced mean long-axis tumor measurements from 2.45-2.65 cm. The maximum MRI lesion size had a strong agreement with pathology (CCC = 0.82 (95% CI: 0.71-0.89)). The maximum IPMN size was typically observed on the axial T1 arterial post-contrast and MRCP coronal series and overestimated size versus pathology with bias +0.34 cm. The radiologist interobserver agreement reached ICCs 0.74 to 0.91 on the MRI sequences. CONCLUSION: The maximum MRI IPMN size strongly correlated with but tended to overestimate the length compared to the pathology, potentially related to formalin tissue shrinkage during tissue processing.

Intricacies of Human-AI Interaction in Dynamic Decision-Making for Precision Oncology: A Case Study in Response-Adaptive Radiotherapy.

Background: Adaptive treatment strategies that can dynamically react to individual cancer progression can provide effective personalized care. Longitudinal multi-omics information, paired with an artificially intelligent clinical decision support system (AI-CDSS) can assist clinicians in determining optimal therapeutic options and treatment adaptations. However, AI-CDSS is not perfectly accurate, as such, clinicians' over/under reliance on AI may lead to unintended consequences, ultimately failing to develop optimal strategies. To investigate such collaborative decision-making process, we conducted a Human-AI interaction case study on response-adaptive radiotherapy (RT). Methods: We designed and conducted a two-phase study for two disease sites and two treatment modalities-adaptive RT for non-small cell lung cancer (NSCLC) and adaptive stereotactic body RT for hepatocellular carcinoma (HCC)-in which clinicians were asked to consider mid-treatment modification of the dose per fraction for a number of retrospective cancer patients without AI-support (Unassisted Phase) and with AI-assistance (AI-assisted Phase). The AI-CDSS graphically presented trade-offs in tumor control and the likelihood of toxicity to organs at risk, provided an optimal recommendation, and associated model uncertainties. In addition, we asked for clinicians' decision confidence level and trust level in individual AI recommendations and encouraged them to provide written remarks. We enrolled 13 evaluators (radiation oncology physicians and residents) from two medical institutions located in two different states, out of which, 4 evaluators volunteered in both NSCLC and HCC studies, resulting in a total of 17 completed evaluations (9 NSCLC, and 8 HCC). To limit the evaluation time to under an hour, we selected 8 treated patients for NSCLC and 9 for HCC, resulting in a total of 144 sets of evaluations (72 from NSCLC and 72 from HCC). Evaluation for each patient consisted of 8 required inputs and 2 optional remarks, resulting in up to a total of 1440 data points. Results: AI-assistance did not homogeneously influence all experts and clinical decisions. From NSCLC cohort, 41 (57%) decisions and from HCC cohort, 34 (47%) decisions were adjusted after AI assistance. Two evaluations (12%) from the NSCLC cohort had zero decision adjustments, while the remaining 15 (88%) evaluations resulted in at least two decision adjustments. Decision adjustment level positively correlated with dissimilarity in decision-making with AI [NSCLC: ρ = 0.53 ( p < 0.001); HCC: ρ = 0.60 ( p < 0.001)] indicating that evaluators adjusted their decision closer towards AI recommendation. Agreement with AI-recommendation positively correlated with AI Trust Level [NSCLC: ρ = 0.59 ( p < 0.001); HCC: ρ = 0.7 ( p < 0.001)] indicating that evaluators followed AI's recommendation if they agreed with that recommendation. The correlation between decision confidence changes and decision adjustment level showed an opposite trend [NSCLC: ρ = -0.24 ( p = 0.045), HCC: ρ = 0.28 ( p = 0.017)] reflecting the difference in behavior due to underlying differences in disease type and treatment modality. Decision confidence positively correlated with the closeness of decisions to the standard of care (NSCLC: 2 Gy/fx; HCC: 10 Gy/fx) indicating that evaluators were generally more confident in prescribing dose fractionations more similar to those used in standard clinical practice. Inter-evaluator agreement increased with AI-assistance indicating that AI-assistance can decrease inter-physician variability. The majority of decisions were adjusted to achieve higher tumor control in NSCLC and lower normal tissue complications in HCC. Analysis of evaluators' remarks indicated concerns for organs at risk and RT outcome estimates as important decision-making factors. Conclusions: Human-AI interaction depends on the complex interrelationship between expert's prior knowledge and preferences, patient's state, disease site, treatment modality, model transparency, and AI's learned behavior and biases. The collaborative decision-making process can be summarized as follows: (i) some clinicians may not believe in an AI system, completely disregarding its recommendation, (ii) some clinicians may believe in the AI system but will critically analyze its recommendations on a case-by-case basis; (iii) when a clinician finds that the AI recommendation indicates the possibility for better outcomes they will adjust their decisions accordingly; and (iv) When a clinician finds that the AI recommendation indicate a worse possible outcome they will disregard it and seek their own alternative approach.