18F-fluciclovine PET/CT to distinguish radiation necrosis from tumor progression for brain metastases treated with radiosurgery: results of a prospective pilot study.

PURPOSE: Distinguishing radiation necrosis from tumor progression among patients with brain metastases previously treated with stereotactic radiosurgery represents a common diagnostic challenge. We performed a prospective pilot study to determine whether PET/CT with 18F-fluciclovine, a widely available amino acid PET radiotracer, repurposed intracranially, can accurately diagnose equivocal lesions. METHODS: Adults with brain metastases previously treated with radiosurgery presenting with a follow-up tumor-protocol MRI brain equivocal for radiation necrosis versus tumor progression underwent an 18F-fluciclovine PET/CT of the brain within 30 days. The reference standard for final diagnosis consisted of clinical follow-up until multidisciplinary consensus or tissue confirmation. RESULTS: Of 16 patients imaged from 7/2019 to 11/2020, 15 subjects were evaluable with 20 lesions (radiation necrosis, n = 16; tumor progression, n = 4). Higher SUVmax statistically significantly predicted tumor progression (AUC = 0.875; p = 0.011). Lesion SUVmean (AUC = 0.875; p = 0.018), SUVpeak (AUC = 0.813; p = 0.007), and SUVpeak-to-normal-brain (AUC = 0.859; p = 0.002) also predicted tumor progression, whereas SUVmax-to-normal-brain (p = 0.1) and SUVmean-to-normal-brain (p = 0.5) did not. Qualitative visual scores were significant predictors for readers 1 (AUC = 0.750; p < 0.001) and 3 (AUC = 0.781; p = 0.045), but not for reader 2 (p = 0.3). Visual interpretations were significant predictors for reader 1 (AUC = 0.898; p = 0.012) but not for reader 2 (p = 0.3) or 3 (p = 0.2). CONCLUSIONS: In this prospective pilot study of patients with brain metastases previously treated with radiosurgery presenting with a contemporary MRI brain with a lesion equivocal for radiation necrosis versus tumor progression, 18F-fluciclovine PET/CT repurposed intracranially demonstrated encouraging diagnostic accuracy, supporting the pursuit of larger clinical trials which will be necessary to establish diagnostic criteria and performance.

Test-retest repeatability of ADC in prostate using the multi b-Value VERDICT acquisition.

PURPOSE: VERDICT (Vascular, Extracellular, Restricted Diffusion for Cytometry in Tumours) MRI is a multi b-value, variable diffusion time DWI sequence that allows generation of ADC maps from different b-value and diffusion time combinations. The aim was to assess precision of prostate ADC measurements from varying b-value combinations using VERDICT and determine which protocol provides the most repeatable ADC. MATERIALS AND METHODS: Forty-one men (median age: 67.7 years) from a prior prospective VERDICT study (April 2016-October 2017) were analysed retrospectively. Men who were suspected of prostate cancer and scanned twice using VERDICT were included. ADC maps were formed using 5b-value combinations and the within-subject standard deviations (wSD) were calculated per ADC map. Three anatomical locations were analysed per subject: normal TZ (transition zone), normal PZ (peripheral zone), and index lesions. Repeated measures ANOVAs showed which b-value range had the lowest wSD, Spearman correlation and generalized linear model regression analysis determined whether wSD was related to ADC magnitude and ROI size. RESULTS: The mean lesion ADC for b0b1500 had the lowest wSD in most zones (0.18-0.58x10-4 mm2/s). The wSD was unaffected by ADC magnitude (Lesion: p = 0.064, TZ: p = 0.368, PZ: p = 0.072) and lesion Likert score (p = 0.95). wSD showed a decrease with ROI size pooled over zones (p = 0.019, adjusted regression coefficient = -1.6x10-3, larger ROIs for TZ versus PZ versus lesions). ADC maps formed with a maximum b-value of 500 s/mm2 had the largest wSDs (1.90-10.24x10-4 mm2/s). CONCLUSION: ADC maps generated from b0b1500 have better repeatability in normal TZ, normal PZ, and index lesions.

Multiparametric Data-driven Imaging Markers: Guidelines for Development, Application and Reporting of Model Outputs in Radiomics.

This paper is the fifth in a five-part series on statistical methodology for performance assessment of multi-parametric quantitative imaging biomarkers (mpQIBs) for radiomic analysis. Radiomics is the process of extracting visually imperceptible features from radiographic medical images using data-driven algorithms. We refer to the radiomic features as data-driven imaging markers (DIMs), which are quantitative measures discovered under a data-driven framework from images beyond visual recognition but evident as patterns of disease processes irrespective of whether or not ground truth exists for the true value of the DIM. This paper aims to set guidelines on how to build machine learning models using DIMs in radiomics and to apply and report them appropriately. We provide a list of recommendations, named RANDAM (an abbreviation of "Radiomic ANalysis and DAta Modeling"), for analysis, modeling, and reporting in a radiomic study to make machine learning analyses in radiomics more reproducible. RANDAM contains five main components to use in reporting radiomics studies: design, data preparation, data analysis and modeling, reporting, and material availability. Real case studies in lung cancer research are presented along with simulation studies to compare different feature selection methods and several validation strategies.

Comparing the Accuracy of Diagnostic Tests When Disease Is Characterized by an Ordinal Scale.

In diagnostic medicine, the true disease status of a patient is often represented on an ordinal scale-for example, cancer stage (0, I, II, III, or IV) or coronary artery disease severity measured using the Coronary Artery Disease Reporting and Data System (CAD-RADS) scale (none, minimal, mild, moderate, severe, or occluded). With advances in quantitation of diagnostic images and in artificial intelligence (AI), both supervised and unsupervised algorithms are being developed to help physicians correctly grade disease. Most of the diagnostic accuracy literature deals with binary disease status (disease present or absent); however, tests diagnosing ordinal-scaled diseases should not be reduced to a binary status just to simplify diagnostic accuracy testing. In this paper, we propose different characterizations of ordinal-scale accuracy for different clinical use scenarios, along with methods for comparing tests. In the simplest scenario, just the proportion of correct grades is considered; other scenarios address the magnitude and direction of misgrading; and at the other extreme, a weighted accuracy measure with weights based on the relative costs of different types of misgrading is presented. The various scenarios are illustrated using a coronary artery disease example where the accuracy of AI algorithms in providing patients with the correct CAD-RADS grade is assessed.

Multiparametric Quantitative Imaging in Risk Prediction: Recommendations for Data Acquisition, Technical Performance Assessment, and Model Development and Validation.

Combinations of multiple quantitative imaging biomarkers (QIBs) are often able to predict the likelihood of an event of interest such as death or disease recurrence more effectively than single imaging measurements can alone. The development of such multiparametric quantitative imaging and evaluation of its fitness of use differs from the analogous processes for individual QIBs in several key aspects. A computational procedure to combine the QIB values into a model output must be specified. The output must also be reproducible and be shown to have reasonably strong ability to predict the risk of an event of interest. Attention must be paid to statistical issues not often encountered in the single QIB scenario, including overfitting and bias in the estimates of model performance. This is the fourth in a five-part series on statistical methodology for assessing the technical performance of multiparametric quantitative imaging. Considerations for data acquisition are discussed and recommendations from the literature on methodology to construct and evaluate QIB-based models for risk prediction are summarized. The findings in the literature upon which these recommendations are based are demonstrated through simulation studies. The concepts in this manuscript are applied to a real-life example involving prediction of major adverse cardiac events using automated plaque analysis.

Imaging Service Navigators: An Approach Toward More Efficient and Effective Communications.

PURPOSE: Many practices have implemented support services to assist radiologists with noninterpretive tasks; however, little research has been performed to assess the overall effect of these services. The purpose of this study was to evaluate the effect of a team of imaging service navigators (ISNs) incorporated into a practice on (1) number of communications, (2) time saved by radiologists, and (3) radiologist satisfaction with the service. METHODS: The numbers and types of reports dictated by radiologists were captured for 6-month periods before and after ISN implementation. Communication rates before and after implementation were then calculated. The amount of perceived time savings using the ISN team and satisfaction with the service were assessed through pre- and postimplementation surveys of participating radiologists. Mean and median time savings and satisfaction rates were calculated. RESULTS: The overall communication rate increased from 2.196% before ISNs to 3.278% after ISNs (49% increase; 95% confidence interval, 47%-52%). Communication rates increased among all communication subtypes (critical, urgent, routine, and actionable), with the highest increases in urgent (94%) and actionable (75%) findings. Before implementation, radiologists reported spending 39 min on average per day on communications tasks, with only 33% of radiologists indicating that the communication process was efficient. After implementation, radiologists reported mean time savings of 28 min (95% confidence interval, 19.9-35.1), and 82% of radiologists indicated a positive or highly positive view of the ISN service. CONCLUSIONS: After ISN implementation, communication rates increased and radiologists reported spending less time performing communications. Most radiologists were satisfied with the service.

A Framework for Evaluating the Technical Performance of Multiparameter Quantitative Imaging Biomarkers (mp-QIBs).

Multiparameter quantitative imaging incorporates anatomical, functional, and/or behavioral biomarkers to characterize tissue, detect disease, identify phenotypes, define longitudinal change, or predict outcome. Multiple imaging parameters are sometimes considered separately but ideally are evaluated collectively. Often, they are transformed as Likert interpretations, ignoring the correlations of quantitative properties that may result in better reproducibility or outcome prediction. In this paper we present three use cases of multiparameter quantitative imaging: i) multidimensional descriptor, ii) phenotype classification, and iii) risk prediction. A fourth application based on data-driven markers from radiomics is also presented. We describe the technical performance characteristics and their metrics common to all use cases, and provide a structure for the development, estimation, and testing of multiparameter quantitative imaging. This paper serves as an overview for a series of individual articles on the four applications, providing the statistical framework for multiparameter imaging applications in medicine.

The RSNA QIBA Profile for Amyloid PET as an Imaging Biomarker for Cerebral Amyloid Quantification.

A standardized approach to acquiring amyloid PET images increases their value as disease and drug response biomarkers. Most 18F PET amyloid brain scans often are assessed only visually (per regulatory labels), with a binary decision indicating the presence or absence of Alzheimer disease amyloid pathology. Minimizing technical variance allows precise, quantitative SUV ratios (SUVRs) for early detection of ß-amyloid plaques and allows the effectiveness of antiamyloid treatments to be assessed with serial studies. Methods: The Quantitative Imaging Biomarkers Alliance amyloid PET biomarker committee developed and validated a profile to characterize and reduce the variability of SUVRs, increasing statistical power for these assessments. Results: On achieving conformance, sites can justify a claim that brain amyloid burden reflected by the SUVR is measurable to a within-subject coefficient of variation of no more than 1.94% when the same radiopharmaceutical, scanner, acquisition, and analysis protocols are used. Conclusion: This overview explains the claim, requirements, barriers, and potential future developments of the profile to achieve precision in clinical and research amyloid PET imaging.

Meniscal Treatment as a Predictor of Worse Articular Cartilage Damage on MRI at 2 Years After ACL Reconstruction: The MOON Nested Cohort.

BACKGROUND: Patients undergoing anterior cruciate ligament reconstruction (ACLR) are at an increased risk for posttraumatic osteoarthritis (PTOA). While we have previously shown that meniscal treatment with ACLR predicts more radiographic PTOA at 2 to 3 years postoperatively, there are a limited number of similar studies that have assessed cartilage directly with magnetic resonance imaging (MRI). HYPOTHESIS: Meniscal repair or partial meniscectomy at the time of ACLR independently predicts more articular cartilage damage on 2- to 3-year postoperative MRI compared with a healthy meniscus or a stable untreated tear. STUDY DESIGN: Cohort study; Level of evidence, 2. METHODS: A consecutive series of patients undergoing ACLR from 1 site within the prospective, nested Multicenter Orthopaedic Outcomes Network (MOON) cohort underwent bilateral knee MRI at 2 to 3 years postoperatively. Patients were aged <36 years without previous knee injuries, were injured while playing sports, and had no history of concomitant ligament surgery or contralateral knee surgery. MRI scans were graded by a board-certified musculoskeletal radiologist using the modified MRI Osteoarthritis Knee Score (MOAKS). A proportional odds logistic regression model was built to predict a MOAKS-based cartilage damage score (CDS) relative to the contralateral control knee for each compartment as well as for the whole knee, pooled by meniscal treatment, while controlling for sex, age, body mass index, baseline Marx activity score, and baseline operative cartilage grade. For analysis, meniscal injuries surgically treated with partial meniscectomy or meniscal repair were grouped together. RESULTS: The cohort included 60 patients (32 female; median age, 18.7 years). Concomitant meniscal treatment at the time of index ACLR was performed in 17 medial menisci (13 meniscal repair and 4 partial meniscectomy) and 27 lateral menisci (3 meniscal repair and 24 partial meniscectomy). Articular cartilage damage was worse in the ipsilateral reconstructed knee (P < .001). A meniscal injury requiring surgical treatment with ACLR predicted a worse CDS for medial meniscal treatment (medial compartment CDS: P = .005; whole joint CDS: P < .001) and lateral meniscal treatment (lateral compartment CDS: P = .038; whole joint CDS: P = .863). Other predictors of a worse relative CDS included age for the medial compartment (P < .001), surgically observed articular cartilage damage for the patellofemoral compartment (P = .048), and body mass index (P = .007) and age (P = .020) for the whole joint. CONCLUSION: A meniscal injury requiring surgical treatment with partial meniscectomy or meniscal repair at the time of ACLR predicted worse articular cartilage damage on MRI at 2 to 3 years after surgery. Further research is required to differentiate between the effects of partial meniscectomy and meniscal repair.

Statistical practice and transparent reporting in the neurosciences: Preclinical motor behavioral experiments.

Longitudinal and behavioral preclinical animal studies generate complex data, which may not be well matched to statistical approaches common in this literature. Analyses that do not adequately account for complexity may result in overly optimistic study conclusions, with consequences for reproducibility and translational decision-making. Recent work interrogating methodological shortcomings in animal research has not yet comprehensively investigated statistical shortcomings in the analysis of complex longitudinal and behavioral data. To this end, the current cross-sectional meta-research study rigorously reviewed published mouse or rat controlled experiments for motor rehabilitation in three neurologic conditions to evaluate statistical choices and reporting. Medline via PubMed was queried in February 2020 for English-language articles published January 1, 2017- December 31, 2019. Included were articles that used rat or mouse models of stroke, Parkinson's disease, or traumatic brain injury, employed a therapeutic controlled experimental design to determine efficacy, and assessed at least one functional behavioral assessment or global evaluation of function. 241 articles from 99 journals were evaluated independently by a team of nine raters. Articles were assessed for statistical handling of non-independence, animal attrition, outliers, ordinal data, and multiplicity. Exploratory analyses evaluated whether transparency or statistical choices differed as a function of journal factors. A majority of articles failed to account for sources of non-independence in the data (74-93%) and/or did not analytically account for mid-treatment animal attrition (78%). Ordinal variables were often treated as continuous (37%), outliers were predominantly not mentioned (83%), and plots often concealed the distribution of the data (51%) Statistical choices and transparency did not differ with regards to journal rank or reporting requirements. Statistical misapplication can result in invalid experimental findings and inadequate reporting obscures errors. Clinician-scientists evaluating preclinical work for translational promise should be mindful of commonplace errors. Interventions are needed to improve statistical decision-making in preclinical behavioral neurosciences research.

Sex-based differences in left ventricular remodeling in patients with chronic aortic regurgitation: a multi-modality study.

BACKGROUND: Significant aortic regurgitation (AR) leads to left ventricular (LV) remodeling; however, little data exist regarding sex-based differences in LV remodeling in this setting. We sought to compare LV remodeling and AR severity, assessed by echocardiography and cardiovascular magnetic resonance (CMR), to discern sex-based differences. METHODS: Patients with ≥ moderate chronic AR by echocardiography who underwent CMR within 90 days between December 2005 and October 2015 were included. Nonlinear regression models were built to assess the effect of AR regurgitant fraction (RF) on LV remodeling. A generalized linear model and Bland Altman analyses were constructed to evaluate differences between CMR and echocardiography. Referral for surgical intervention based on symptoms and LV remodeling was evaluated. RESULTS: Of the 243 patients (48.3 ± 16.6 years, 58 (24%) female), 119 (49%) underwent surgical intervention with a primary indication of severe AR, 97 (82%) men, 22 (18%) women. Significant sex differences in LV remodeling emerged on CMR. Women demonstrated significantly smaller LV end-diastolic volume index (LVEDVI) (96.8 ml/m2 vs 125.6 ml/m2, p < 0.001), LV end-systolic volume index (LVESVI) (41.1 vs 54.5 ml/m2, p < 0.001), blunted LV dilation in the setting of increasing AR severity (LVEDVI p value < 0.001, LVESVI p value 0.011), and LV length indexed (8.32 vs 9.69 cm, p < 0.001). On Bland Altman analysis, a significant interaction with sex and LV diameters was evident, demonstrating a significant increase in the difference between CMR and echocardiography measurements as the LV enlarged in women: LVEDVI (p = 0.006), LVESVI (p < 0.001), such that echocardiographic measurements increasingly underestimated LV diameters in women as the LV enlarged. LV length was higher for males with a linear effect from RF (p < 0.001), with LV length increasing at a higher rate with increasing RF for males compared to females (two-way interaction with sex p = 0.005). Sphericity volume index was higher for men after adjusting for a relative wall thickness (p = 0.033). CONCLUSIONS: CMR assessment of chronic AR revealed significant sex differences in LV remodeling and significant echocardiographic underestimation of LV dilation, particularly in women. Defining optimal sex-based CMR thresholds for surgical referral should be further developed. TRIAL REGISTRATION: NA.

Multireader Diagnostic Accuracy Imaging Studies: Fundamentals of Design and Analysis.

The design and analysis of multireader multicase (MRMC) studies are quite challenging. These studies differ from most medical studies because they need a reference standard and sampling from two populations (ie, reader and patient populations). They are quite expensive to conduct, requiring a good deal of readers' time for image interpretation. One common problem is the use of imperfect reference standards, often correlated with the test or tests being evaluated. Another common issue is oversimplification of the multidimensional MRMC data. In this study, the fundamentals of MRMC study design and analysis are reviewed. The goal is to provide investigators with a guide to the fundamentals of MRMC design and analysis, with references to more detailed discussions. In addition, readers are updated on newer areas of research, including correction for studies with multiple diagnostic accuracy end points and adjustment for location bias.

Estimating the Precision of Quantitative Imaging Biomarkers without Test-Retest Studies.

RATIONALE AND OBJECTIVES: A critical performance metric for any quantitative imaging biomarker is its ability to reliably generate similar values on repeat testing. This is known as the repeatability of the biomarker, and it is used to determine the minimum detectable change needed in order to show that a change over time is real change and not just due to measurement error. Test-retest studies are the classic approach for estimating repeatability; however, these studies can be infeasible when the imaging is expensive, time-consuming, invasive, or requires contrast agents. The objective of this study was to develop and test a method for estimating repeatability without a test-retest study. MATERIALS AND METHODS: We present a statistical method for estimating repeatability and testing whether an imaging method meets a specified criterion for repeatability in the absence of a test-retest study. The new method is applicable for the particular situation where a reference standard is available. A Monte Carlo simulation study was conducted to evaluate the performance of the new method. RESULTS: The proposed estimator is unbiased, and hypothesis tests with the new estimator have nominal type I error rate and power similar to a test-retest study. We considered the situation where the reference standard provides the true value, as well as when the reference standard itself has various magnitudes of measurement error. An example from CT imaging biomarkers of atherosclerosis illustrates the new method. CONCLUSION: Precision of a QIB can be measured without a test-retest study in the situation where a reference standard is available.

Improving the Robustness of Diagnostic Accuracy Results By Asking Study Readers to Further Distinguish Subjects Who Appear to be Without the Condition Of Interest.

RATIONALE AND OBJECTIVES: In diagnostic accuracy studies, cases in which a reader does not see the condition of interest are often given the same score for ROC analysis (e.g. confidence score of 0%). However, many of these cases can be further distinguished and doing so may result in more robust ROC results. MATERIALS AND METHODS: We examined two recent, real-world studies which used different procedures to encourage readers to further distinguish subjects who appear to be without the condition of interest. For each study, we calculated the results under two conditions. In the "zeros distinguished" (ZD) condition, we incorporated the confidence scores collected to further distinguish the normal-looking subjects. In the "zeros not distinguished" (ZND) condition, we disregarded these scores and simply gave the unit of analysis a score of zero whenever the reader did not identify the condition of interest in that unit. We compared the two conditions on (1) coverage of the ROC space and (2) discrepancy between parametric and nonparametric results. RESULTS: Compared to the ZND condition, coverage of the ROC space was improved in the ZD condition for all ROC curves in both studies. In the first study, there was a significant reduction in the discrepancy between parametric and nonparametric results (median discrepancy in ZND vs ZD condition: 0.033 vs 0.011, p = 0.012). A similar reduction was not seen in the second study, though the discrepancies were very low in both conditions (0.003 vs 0.006, p = 0.313). CONCLUSION: Prompting readers to further distinguish cases in which they do not see the condition of interest may result in more robust ROC results, though further exploration of this topic is warranted.

Development, validation, qualification, and dissemination of quantitative MR methods: Overview and recommendations by the ISMRM quantitative MR study group.

On behalf of the International Society for Magnetic Resonance in Medicine (ISMRM) Quantitative MR Study Group, this article provides an overview of considerations for the development, validation, qualification, and dissemination of quantitative MR (qMR) methods. This process is framed in terms of two central technical performance properties, i.e., bias and precision. Although qMR is confounded by undesired effects, methods with low bias and high precision can be iteratively developed and validated. For illustration, two distinct qMR methods are discussed throughout the manuscript: quantification of liver proton-density fat fraction, and cardiac T1 . These examples demonstrate the expansion of qMR methods from research centers toward widespread clinical dissemination. The overall goal of this article is to provide trainees, researchers, and clinicians with essential guidelines for the development and validation of qMR methods, as well as an understanding of necessary steps and potential pitfalls for the dissemination of quantitative MR in research and in the clinic.

The QIBA Profile for MRI-based Compositional Imaging of Knee Cartilage.

MRI-based cartilage compositional analysis shows biochemical and microstructural changes at early stages of osteoarthritis before changes become visible with structural MRI sequences and arthroscopy. This could help with early diagnosis, risk assessment, and treatment monitoring of osteoarthritis. Spin-lattice relaxation time constant in rotating frame (T1ρ) and T2 mapping are the MRI techniques best established for assessing cartilage composition. Only T2 mapping is currently commercially available, which is sensitive to water, collagen content, and orientation of collagen fibers, whereas T1ρ is more sensitive to proteoglycan content. Clinical application of cartilage compositional imaging is limited by high variability and suboptimal reproducibility of the biomarkers, which was the motivation for creating the Quantitative Imaging Biomarkers Alliance (QIBA) Profile for cartilage compositional imaging by the Musculoskeletal Biomarkers Committee of the QIBA. The profile aims at providing recommendations to improve reproducibility and to standardize cartilage compositional imaging. The QIBA Profile provides two complementary claims (summary statements of the technical performance of the quantitative imaging biomarkers that are being profiled) regarding the reproducibility of biomarkers. First, cartilage T1ρ and T2 values are measurable at 3.0-T MRI with a within-subject coefficient of variation of 4%-5%. Second, a measured increase or decrease in T1ρ and T2 of 14% or more indicates a minimum detectable change with 95% confidence. If only an increase in T1ρ and T2 values is expected (progressive cartilage degeneration), then an increase of 12% represents a minimum detectable change over time. The QIBA Profile provides recommendations for clinical researchers, clinicians, and industry scientists pertaining to image data acquisition, analysis, and interpretation and assessment procedures for T1ρ and T2 cartilage imaging and test-retest conformance. This special report aims to provide the rationale for the proposed claims, explain the content of the QIBA Profile, and highlight the future needs and developments for MRI-based cartilage compositional imaging for risk prediction, early diagnosis, and treatment monitoring of osteoarthritis.

Objective Task-Based Evaluation of Artificial Intelligence-Based Medical Imaging Methods:: Framework, Strategies, and Role of the Physician.

Artificial intelligence-based methods are showing promise in medical imaging applications. There is substantial interest in clinical translation of these methods, requiring that they be evaluated rigorously. We lay out a framework for objective task-based evaluation of artificial intelligence methods. We provide a list of available tools to conduct this evaluation. We outline the important role of physicians in conducting these evaluation studies. The examples in this article are proposed in the context of PET scans with a focus on evaluating neural network-based methods. However, the framework is also applicable to evaluate other medical imaging modalities and other types of artificial intelligence methods.