A Framework for Evaluating the Technical Performance of Multiparameter Quantitative Imaging Biomarkers (mp-QIBs).

Multiparameter quantitative imaging incorporates anatomical, functional, and/or behavioral biomarkers to characterize tissue, detect disease, identify phenotypes, define longitudinal change, or predict outcome. Multiple imaging parameters are sometimes considered separately but ideally are evaluated collectively. Often, they are transformed as Likert interpretations, ignoring the correlations of quantitative properties that may result in better reproducibility or outcome prediction. In this paper we present three use cases of multiparameter quantitative imaging: i) multidimensional descriptor, ii) phenotype classification, and iii) risk prediction. A fourth application based on data-driven markers from radiomics is also presented. We describe the technical performance characteristics and their metrics common to all use cases, and provide a structure for the development, estimation, and testing of multiparameter quantitative imaging. This paper serves as an overview for a series of individual articles on the four applications, providing the statistical framework for multiparameter imaging applications in medicine.

Multiparametric Quantitative Imaging Biomarkers for Phenotype Classification: A Framework for Development and Validation.

This manuscript is the third in a five-part series related to statistical assessment methodology for technical performance of multi-parametric quantitative imaging biomarkers (mp-QIBs). We outline approaches and statistical methodologies for developing and evaluating a phenotype classification model from a set of multiparametric QIBs. We then describe validation studies of the classifier for precision, diagnostic accuracy, and interchangeability with a comparator classifier. We follow with an end-to-end real-world example of development and validation of a classifier for atherosclerotic plaque phenotypes. We consider diagnostic accuracy and interchangeability to be clinically meaningful claims for a phenotype classification model informed by mp-QIB inputs, aiming to provide tools to demonstrate agreement between imaging-derived characteristics and clinically established phenotypes. Understanding that we are working in an evolving field, we close our manuscript with an acknowledgement of existing challenges and a discussion of where additional work is needed. In particular, we discuss the challenges involved with technical performance and analytical validation of mp-QIBs. We intend for this manuscript to further advance the robust and promising science of multiparametric biomarker development.

The RSNA QIBA Profile for Amyloid PET as an Imaging Biomarker for Cerebral Amyloid Quantification.

A standardized approach to acquiring amyloid PET images increases their value as disease and drug response biomarkers. Most 18F PET amyloid brain scans often are assessed only visually (per regulatory labels), with a binary decision indicating the presence or absence of Alzheimer disease amyloid pathology. Minimizing technical variance allows precise, quantitative SUV ratios (SUVRs) for early detection of ß-amyloid plaques and allows the effectiveness of antiamyloid treatments to be assessed with serial studies. Methods: The Quantitative Imaging Biomarkers Alliance amyloid PET biomarker committee developed and validated a profile to characterize and reduce the variability of SUVRs, increasing statistical power for these assessments. Results: On achieving conformance, sites can justify a claim that brain amyloid burden reflected by the SUVR is measurable to a within-subject coefficient of variation of no more than 1.94% when the same radiopharmaceutical, scanner, acquisition, and analysis protocols are used. Conclusion: This overview explains the claim, requirements, barriers, and potential future developments of the profile to achieve precision in clinical and research amyloid PET imaging.

Multiparametric Quantitative Imaging Biomarker as a Multivariate Descriptor of Health: A Roadmap.

Multiparametric quantitative imaging biomarkers (QIBs) offer distinct advantages over single, univariate descriptors because they provide a more complete measure of complex, multidimensional biological systems. In disease, where structural and functional disturbances occur across a multitude of subsystems, multivariate QIBs are needed to measure the extent of system malfunction. This paper, the first Use Case in a series of articles on multiparameter imaging biomarkers, considers multiple QIBs as a multidimensional vector to represent all relevant disease constructs more completely. The approach proposed offers several advantages over QIBs as multiple endpoints and avoids combining them into a single composite that obscures the medical meaning of the individual measurements. We focus on establishing statistically rigorous methods to create a single, simultaneous measure from multiple QIBs that preserves the sensitivity of each univariate QIB while incorporating the correlation among QIBs. Details are provided for metrological methods to quantify the technical performance. Methods to reduce the set of QIBs, test the superiority of the mp-QIB model to any univariate QIB model, and design study strategies for generating precision and validity claims are also provided. QIBs of Alzheimer's Disease from the ADNI merge data set are used as a case study to illustrate the methods described.

Impact of Alternate b-Value Combinations and Metrics on the Predictive Performance and Repeatability of Diffusion-Weighted MRI in Breast Cancer Treatment: Results from the ECOG-ACRIN A6698 Trial.

In diffusion-weighted MRI (DW-MRI), choice of b-value influences apparent diffusion coefficient (ADC) values by probing different aspects of the tissue microenvironment. As a secondary analysis of the multicenter ECOG-ACRIN A6698 trial, the purpose of this study was to investigate the impact of alternate b-value combinations on the performance and repeatability of tumor ADC as a predictive marker of breast cancer treatment response. The final analysis included 210 women who underwent standardized 4-b-value DW-MRI (b = 0/100/600/800 s/mm2) at multiple timepoints during neoadjuvant chemotherapy treatment and a subset (n = 71) who underwent test−retest scans. Centralized tumor ADC and perfusion fraction (fp) measures were performed using variable b-value combinations. Prediction of pathologic complete response (pCR) based on the mid-treatment/12-week percent change in each metric was estimated by area under the receiver operating characteristic curve (AUC). Repeatability was estimated by within-subject coefficient of variation (wCV). Results show that two-b-value ADC calculations provided non-inferior predictive value to four-b-value ADC calculations overall (AUCs = 0.60−0.61 versus AUC = 0.60) and for HR+/HER2− cancers where ADC was most predictive (AUCs = 0.75−0.78 versus AUC = 0.76), p < 0.05. Using two b-values (0/600 or 0/800 s/mm2) did not reduce ADC repeatability over the four-b-value calculation (wCVs = 4.9−5.2% versus 5.4%). The alternate metrics ADCfast (b ≤ 100 s/mm2), ADCslow (b ≥ 100 s/mm2), and fp did not improve predictive performance (AUCs = 0.54−0.60, p = 0.08−0.81), and ADCfast and fp demonstrated the lowest repeatability (wCVs = 6.71% and 12.4%, respectively). In conclusion, breast tumor ADC calculated using a simple two-b-value approach can provide comparable predictive value and repeatability to full four-b-value measurements as a marker of treatment response.

Magnetic resonance biomarkers in radiation oncology: The report of AAPM Task Group 294.

PURPOSE: A magnetic resonance (MR) biologic marker (biomarker) is a measurable quantitative characteristic that is an indicator of normal biological and pathogenetic processes or a response to therapeutic intervention derived from the MR imaging process. There is significant potential for MR biomarkers to facilitate personalized approaches to cancer care through more precise disease targeting by quantifying normal versus pathologic tissue function as well as toxicity to both radiation and chemotherapy. Both of which have the potential to increase the therapeutic ratio and provide earlier, more accurate monitoring of treatment response. The ongoing integration of MR into routine clinical radiation therapy (RT) planning and the development of MR guided radiation therapy systems is providing new opportunities for MR biomarkers to personalize and improve clinical outcomes. Their appropriate use, however, must be based on knowledge of the physical origin of the biomarker signal, the relationship to the underlying biological processes, and their strengths and limitations. The purpose of this report is to provide an educational resource describing MR biomarkers, the techniques used to quantify them, their strengths and weakness within the context of their application to radiation oncology so as to ensure their appropriate use and application within this field.

Repeatability of tumor perfusion kinetics from dynamic contrast-enhanced MRI in glioblastoma.

BACKGROUND: Dynamic contrast-enhanced MRI (DCE-MRI) parameters have been shown to be biomarkers for treatment response in glioblastoma (GBM). However, variations in analysis and measurement methodology complicate determination of biological changes measured via DCE. The aim of this study is to quantify DCE-MRI variations attributable to analysis methodology and image quality in GBM patients. METHODS: The Extended Tofts model (eTM) and Leaky Tracer Kinetic Model (LTKM), with manually and automatically segmented vascular input functions (VIFs), were used to calculate perfusion kinetic parameters from 29 GBM patients with double-baseline DCE-MRI data. DCE-MRI images were acquired 2-5 days apart with no change in treatment. Repeatability of kinetic parameters was quantified with Bland-Altman and percent repeatability coefficient (%RC) analysis. RESULTS: The perfusion parameter with the least RC was the plasma volume fraction (v p ), with a %RC of 53%. The extra-cellular extra-vascular volume fraction (v e ) %RC was 82% and 81%, for extended Tofts-Kety Model (eTM) and LTKM respectively. The %RC of the volume transfer rate constant (K trans ) was 72% for the eTM, and 82% for the LTKM, respectively. Using an automatic VIF resulted in smaller %RCs for all model parameters, as compared to manual VIF. CONCLUSIONS: As much as 72% change in K trans (eTM, autoVIF) can be attributable to non-biological changes in the 2-5 days between double-baseline imaging. Poor K trans repeatability may result from inferior temporal resolution and short image acquisition time. This variation suggests DCE-MRI repeatability studies should be performed institutionally, using an automatic VIF method and following quantitative imaging biomarkers alliance guidelines.

Standardization of [F-18]FDG PET/CT for response evaluation by the Radiologic Society of North America-Quantitative Imaging Biomarker Alliance (RSNA-QIBA) profile: preliminary results from the Japan-QIBA (J-QIBA) activities for Asian international multicenter phase II trial.

PURPOSE: In an Asian international multicenter phase II trial conducted in patients with peripheral T-cell lymphoma (PTCL), [F-18]FDG-PET/CT was used for evaluation of the therapeutic response. Standardization of the PET/CT scanners was necessary before patient enrollment. We therefore standardized the scanners by phantom tests based on the profile approved by the Quantitative Imaging Biomarkers Alliance (QIBA) of Radiological Society of North America (RSNA). MATERIALS AND METHODS: The tests were conducted on 12 scanners in 12 facilities in compliance with the QIBA Profile and used National Electrical Manufacturers Association (NEMA) International Electrotechnical Commission (IEC) body phantoms. We measured three parameters (standardized uptake value [SUV], resolution and noise) and adjusted the imaging parameter values. The indexes recommended in the Japanese Society of Nuclear Medicine (JSNM) guideline were also evaluated. RESULTS: In a total of 12 facilities, 6 facilities required no change in imaging conditions and 6 facilities required changes in imaging parameters. After revision, the three measurements (SUV, resolution and noise) met QIBA criteria at all sites, but 10 of the 12 scanners did not meet JSNM criteria. CONCLUSION: We standardized imaging conditions using phantoms as required in the RSNA-QIBA profile for response evaluation by [F-18]FDG PET/CT images in a multicenter study.

Variation and repeatability of measured standardized uptake values depending on actual values: a phantom study.

Standardized uptake values (SUVs) are the most widely used quantitative imaging biomarkers in positron emission tomography (PET); however, little is known about the changes in variation and repeatability of SUVs depending on the magnitude of the values. We hypothesized that low SUVs have larger variations than high SUVs, and attempted various kinds of experimental PET scans using a phantom. By adjusting the ratio of F-18 solution to tap water, a NEMA IEC body phantom was set for SUVs of 2.0, 4.0, and 8.0 inside six hot spheres. PET data were obtained for 4 hours, and the data reconstructed every 2 min. The SUVmax and SUVpeak of the spheres in all images were recorded. The relative SUVs were calculated by dividing the measured SUV by actual SUV, and used for the Bland-Altman plots. Some variation was observed for the measured SUVs. The measured SUVs for the actual SUV of 2.0 showed the largest variation among those of 2.0, 4.0, and 8.0, and those of 8.0 showed the smallest. Similarly, the relative SUVs showed significantly larger variations for lower values. In addition, the relative SUVmax showed larger variation and value than the relative SUVpeak. The Bland-Altman plots showed considerable variation and little agreement, but the degree of variation decreased as the measured value increased. We demonstrated some variation of the measured SUVs, which decreased for larger measured values. Clinicians should consider the inaccuracy of low SUVs not only in daily practice, but also for multi-institutional studies.

Repeatability and reproducibility of 2D and 3D hepatic MR elastography with rigid and flexible drivers at end-expiration and end-inspiration in healthy volunteers.

PURPOSE: To evaluate the repeatability and reproducibility of 2D and 3D hepatic MRE with rigid and flexible drivers at end-expiration and end-inspiration in healthy volunteers. MATERIALS AND METHODS: Nine healthy volunteers underwent two same-day MRE exams separated by a 5- to 10-min break. In each exam, 2D and 3D MRE scans were performed, each under four conditions (2 driver types [rigid, flexible] × 2 breath-hold phases [end-expiration, end-inspiration]). Repeatability (measurements under identical conditions) and reproducibility (measurements under different conditions) were analyzed by calculating bias, limit of agreement, repeatability coefficient (RC), reproducibility coefficient (RDC), intraclass correlation coefficient (ICC), and concordance correlation coefficient (CCC), as appropriate. RESULTS: For 2D MRE, RCs and ICCs range between 0.29-0.49 and 0.71-0.91, respectively. For 3D MRE, RCs and ICCs range between 0.16-0.26 and 0.84-0.96, respectively. Stiffness values were biased by breath-hold phase, being higher at end-inspiration than end-expiration, and the differences were significant for 3D MRE (p < 0.01). No bias was found between driver types. Inspiration vs. expiration RDCs and CCCs ranged between 0.30-0.54 and 0.61-0.72, respectively. Rigid vs. flexible driver RDCs and CCCs ranged between 0.10-0.44 and 0.79-0.94, respectively. CONCLUSION: This preliminary study suggests that 2D MRE and 3D MRE under most conditions potentially have good repeatability. Our result also points to the possibility that stiffness measured with the rigid and flexible drivers is reproducible. Reproducibility between breath-hold phases was modest, suggesting breath-hold phase might be a confounding factor in MRE-based stiffness measurement. However, larger studies are required to validate these preliminary results.

SUV measurement of normal vertebrae using SPECT/CT with Tc-99m methylene diphosphonate.

The purpose of this study is to perform quantitative measurement based on the standardized uptake value (SUV) of the uptake of Tc-99m methylene diphosphonate (MDP) in the normal vertebrae using a single photon emission tomography (SPECT)/computed tomography (CT) scanner. A retrospective study of patients with cancer or joint disorders was performed. We acquired data for a group of 29 patients (8 women and 21 men; mean age, 68.2 ± 6.7 years; age range, 44-87 years) undergoing bone SPECT/CT scans with Tc-99m MDP between September and October 2015. Various SUVs were calculated based on body-weight, lean-body-weight (lbw), Japanese lean-body-weight (jlbw) and Japanese bone-mineral-content (jbmc). SUVs of normal vertebrae showed a wide range of values. Among these, the maximum body-weight based SUV showed the lowest coefficient of variation. The SUVs also showed relatively small intra-subject variability. In addition, all SUVs showed moderate and significant correlation with height. Moreover, lbw-, jlbw-, and jbmc-based SUVs of men were significantly higher than those of women. In conclusions, SUVs of normal vertebrae showed a relatively large inter-individual variability and small intra-individual variability. As a quantitative imaging biomarker, SUVs might require standardization with adequate reference data for the same subject to minimize variability.

Algorithm Variability in the Estimation of Lung Nodule Volume From Phantom CT Scans: Results of the QIBA 3A Public Challenge.

RATIONALE AND OBJECTIVES: Quantifying changes in lung tumor volume is important for diagnosis, therapy planning, and evaluation of response to therapy. The aim of this study was to assess the performance of multiple algorithms on a reference data set. The study was organized by the Quantitative Imaging Biomarker Alliance (QIBA). MATERIALS AND METHODS: The study was organized as a public challenge. Computed tomography scans of synthetic lung tumors in an anthropomorphic phantom were acquired by the Food and Drug Administration. Tumors varied in size, shape, and radiodensity. Participants applied their own semi-automated volume estimation algorithms that either did not allow or allowed post-segmentation correction (type 1 or 2, respectively). Statistical analysis of accuracy (percent bias) and precision (repeatability and reproducibility) was conducted across algorithms, as well as across nodule characteristics, slice thickness, and algorithm type. RESULTS: Eighty-four percent of volume measurements of QIBA-compliant tumors were within 15% of the true volume, ranging from 66% to 93% across algorithms, compared to 61% of volume measurements for all tumors (ranging from 37% to 84%). Algorithm type did not affect bias substantially; however, it was an important factor in measurement precision. Algorithm precision was notably better as tumor size increased, worse for irregularly shaped tumors, and on the average better for type 1 algorithms. Over all nodules meeting the QIBA Profile, precision, as measured by the repeatability coefficient, was 9.0% compared to 18.4% overall. CONCLUSION: The results achieved in this study, using a heterogeneous set of measurement algorithms, support QIBA quantitative performance claims in terms of volume measurement repeatability for nodules meeting the QIBA Profile criteria.

Statistical Issues in Testing Conformance with the Quantitative Imaging Biomarker Alliance (QIBA) Profile Claims.

A major initiative of the Quantitative Imaging Biomarker Alliance is to develop standards-based documents called "Profiles," which describe one or more technical performance claims for a given imaging modality. The term "actor" denotes any entity (device, software, or person) whose performance must meet certain specifications for the claim to be met. The objective of this paper is to present the statistical issues in testing actors' conformance with the specifications. In particular, we present the general rationale and interpretation of the claims, the minimum requirements for testing whether an actor achieves the performance requirements, the study designs used for testing conformity, and the statistical analysis plan. We use three examples to illustrate the process: apparent diffusion coefficient in solid tumors measured by MRI, change in Perc 15 as a biomarker for the progression of emphysema, and percent change in solid tumor volume by computed tomography as a biomarker for lung cancer progression.