Can we predict discordant RECIST 1.1 evaluations in double read clinical trials?

Background: In lung clinical trials with imaging, blinded independent central review with double reads is recommended to reduce evaluation bias and the Response Evaluation Criteria In Solid Tumor (RECIST) is still widely used. We retrospectively analyzed the inter-reader discrepancies rate over time, the risk factors for discrepancies related to baseline evaluations, and the potential of machine learning to predict inter-reader discrepancies. Materials and methods: We retrospectively analyzed five BICR clinical trials for patients on immunotherapy or targeted therapy for lung cancer. Double reads of 1724 patients involving 17 radiologists were performed using RECIST 1.1. We evaluated the rate of discrepancies over time according to four endpoints: progressive disease declared (PDD), date of progressive disease (DOPD), best overall response (BOR), and date of the first response (DOFR). Risk factors associated with discrepancies were analyzed, two predictive models were evaluated. Results: At the end of trials, the discrepancy rates between trials were not different. On average, the discrepancy rates were 21.0%, 41.0%, 28.8%, and 48.8% for PDD, DOPD, BOR, and DOFR, respectively. Over time, the discrepancy rate was higher for DOFR than DOPD, and the rates increased as the trial progressed, even after accrual was completed. It was rare for readers to not find any disease, for less than 7% of patients, at least one reader selected non-measurable disease only (NTL). Often the readers selected some of their target lesions (TLs) and NTLs in different organs, with ranges of 36.0-57.9% and 60.5-73.5% of patients, respectively. Rarely (4-8.1%) two readers selected all their TLs in different locations. Significant risk factors were different depending on the endpoint and the trial being considered. Prediction had a poor performance but the positive predictive value was higher than 80%. The best classification was obtained with BOR. Conclusion: Predicting discordance rates necessitates having knowledge of patient accrual, patient survival, and the probability of discordances over time. In lung cancer trials, although risk factors for inter-reader discrepancies are known, they are weakly significant, the ability to predict discrepancies from baseline data is limited. To boost prediction accuracy, it would be necessary to enhance baseline-derived features or create new ones, considering other risk factors and looking into optimal reader associations.

Breaking down the RECIST 1.1 double read variability in lung trials: What do baseline assessments tell us?

Background: In clinical trials with imaging, Blinded Independent Central Review (BICR) with double reads ensures data blinding and reduces bias in drug evaluations. As double reads can cause discrepancies, evaluations require close monitoring which substantially increases clinical trial costs. We sought to document the variability of double reads at baseline, and variabilities across individual readers and lung trials. Material and methods: We retrospectively analyzed data from five BICR clinical trials evaluating 1720 lung cancer patients treated with immunotherapy or targeted therapy. Fifteen radiologists were involved. The variability was analyzed using a set of 71 features derived from tumor selection, measurements, and disease location. We selected a subset of readers that evaluated ≥50 patients in ≥two trials, to compare individual reader's selections. Finally, we evaluated inter-trial homogeneity using a subset of patients for whom both readers assessed the exact same disease locations. Significance level was 0.05. Multiple pair-wise comparisons of continuous variables and proportions were performed using one-way ANOVA and Marascuilo procedure, respectively. Results: Across trials, on average per patient, target lesion (TL) number ranged 1.9 to 3.0, sum of tumor diameter (SOD) 57.1 to 91.9 mm. MeanSOD=83.7 mm. In four trials, MeanSOD of double reads was significantly different. Less than 10% of patients had TLs selected in completely different organs and 43.5% had at least one selected in different organs. Discrepancies in disease locations happened mainly in lymph nodes (20.1%) and bones (12.2%). Discrepancies in measurable disease happened mainly in lung (19.6%). Between individual readers, the MeanSOD and disease selection were significantly different (p<0.001). In inter-trials comparisons, on average per patient, the number of selected TLs ranged 2.1 to 2.8, MeanSOD 61.0 to 92.4 mm. Trials were significantly different in MeanSOD (p<0.0001) and average number of selected TLs (p=0.007). The proportion of patients having one of the top diseases was significantly different only between two trials for lung. Significant differences were observed for all other disease locations (p<0.05). Conclusions: We found significant double read variabilities at baseline, evidence of reading patterns and a means to compare trials. Clinical trial reliability is influenced by the interplay of readers, patients and trial design.

Differences in sensitivity to new therapies between primary and metastatic breast cancer: A need to stratify the tumor response?

OBJECTIVE: We compared therapeutic response of Varlitinib + Capecitabine (VC) versus Lapatinib + Capecitabine (LC) in patients with human epidermal growth factor receptor 2-positive metastatic breast cancer after trastuzumab therapy by assessing changes in target lesion (TL) diameter and volume per location. METHODS: We retrospectively analyzed the CT data of the ASLAN001-003 study (NCT02338245). We analyzed TL size and number at each location focusing on therapeutic response from baseline to Week 12. We used TL diameter and volume to conduct an inter-arm comparison of the response according to: RECIST 1.1; stratified per TL location and considering TLs independently. Multiple pairwise intra-arm comparisons of therapeutic responses were performed. Considering TL independently, weighted models were designed by adding weighted mean TL responses grouped by location. RESULTS: We evaluated 42 patients (88 TL) and 35 patients (74 TL), respectively, at baseline and Week 12. We found reductions in breast TL burden in the VC arm compared to the LC arm (p = 0.002 (diameter), p < 0.001 (volume)). Responses and TL sizes at baseline were not correlated. Explained variabilities of volume change per TL location, patient and patient:TL interaction were 36%, 10% and 4% (VC), and 13%, 1% and 23%, (LC). A test of inter-arm difference of responses yielded p = 0.07 (diameter), and p < 0.001 (volume). CONCLUSIONS: The therapeutic responses differed across tumors' locations; the magnitude of the differences of responses across the tumors' locations were drug-dependent. Stratified analysis of the response by tumor location improved drug comparisons and is a powerful tool to understand TL heterogeneity.

Intra-scan inter-tissue variability can help harmonize radiomics features in CT.

OBJECTIVE: We studied the repeatability and the relative intra-scan variability across acquisition protocols in CT using phantom and unenhanced abdominal series. METHODS: We used 17 CT scans from the Credence Cartridge Radiomics Phantom database and 20 unenhanced multi-site non-pathologic abdominal patient series for which we measured spleen and liver tissues. We performed multiple measurements in extracting 9 radiomics features. We defined a "tandem" as the measurement of a given tissue (or material) by a given radiomics. For each tandem, we assessed the proportion of the variability attributable to repetitions, acquisition protocols, material, or patient. We analyzed the distribution of the intra-scan correlation between pairs of tandems and checked the impact of correlation coefficient greater than 0.90 in comparing paired and unpaired differences. RESULTS: The repeatability of radiomics features depends on the measured material; 56% of tandems were highly repeatable. Histogram-derived radiomics were generally less repeatable. Nearly 60% of relative radiomics measurements had a correlation coefficient higher than 0.90 allowing paired measurements to improve reliability in detecting the difference between two materials. The analysis of liver and spleen tissues showed that measurement variability was negligible with respect to other variabilities. As for phantom data, we found that gray level zone length matrix (GLZLM)-derived radiomics and gray level co-occurrence matrix (GLCM)-derived radiomics were the most correlating features. For these features, relative intra-scan measurements improved the detection of different materials or tissues. CONCLUSIONS: We identified radiomics features for which the intra-scan measurements between tissues are linearly correlated. This property represents an opportunity to improve tissue characterization and inter-site harmonization. KEY POINTS: • The repeatability of radiomics features on CT depends on the measured material or tissue. • Some tandems of radiomics features/tissues are linearly affected by the variability of acquisition protocols on CT. • Relative intra-scan measurements are an opportunity for improving quantitative imaging on CT.

Blinded Independent Central Review (BICR) in New Therapeutic Lung Cancer Trials.

BACKGROUND: Double reads in blinded independent central reviews (BICRs) are recommended to control the quality of trials but they are prone to discordances. We analyzed inter-reader discordances in a pool of lung cancer trials using RECIST 1.1. METHODS: We analyzed six lung cancer BICR trials that included 1833 patients (10,684 time points) involving 17 radiologists. We analyzed the rate of discrepancy of each trial at the time-point and patient levels as well as testing inter-trial differences. The analysis of adjudication made it possible to compute the readers' endorsement rates, the root causes of adjudications, and the proportions of "errors" versus "medically justifiable differences". RESULTS: The trials had significantly different discrepancy rates both at the time-point (average = 34.3%) and patient (average = 59.2%) levels. When considering only discrepancies for progressive disease, homogeneous discrepancy rates were found with an average of 32.9%, while readers' endorsement rates ranged between 27.7% and 77.8%. Major causes of adjudication were different per trial, with medically justifiable differences being the most common, triggering 74.2% of total adjudications. CONCLUSIONS: We provide baseline performances for monitoring reader performance in trials with double reads. Intelligent reading system implementation along with appropriate reader training and monitoring are solutions that could mitigate a large portion of the commonly encountered reading errors.

Letter to the editor: "Not all biases are bad: equitable and inequitable biases in machine learning and radiology".

Artificial intelligence algorithms are booming in medicine, and the question of biases induced or perpetuated by these tools is a very important topic. There is a greater risk of these biases in radiology, which is now the primary diagnostic tool in modern treatment. Some authors have recently proposed an analysis framework for social inequalities and the biases at risk of being introduced into future algorithms. In our paper, we comment on the different strategies for resolving these biases. We warn that there is an even greater risk in mixing the notion of equity, the definition of which is socio-political, into the design stages of these algorithms. We believe that rather than being beneficial, this could in fact harm the main purpose of these artificial intelligence tools, which is the care of the patient.

RECIST 1.1 and lesion selection: How to deal with ambiguity at baseline?

Response Evaluation Criteria In Solid Tumors (RECIST) is still the predominant criteria base for assessing tumor burden in oncology clinical trials. Despite several improvements that followed its first publication, RECIST continues to allow readers a lot of freedom in their evaluations. Notably in the selection of tumors at baseline. This subjectivity is the source of many suboptimal evaluations. When starting a baseline analysis, radiologists cannot always identify tumor malignancy with any certainty. Also, with RECIST, some findings can be deemed equivocal by radiologists with no confirmatory ground truth to rely on. In the specific case of Blinded Independent Central Review clinical trials with double reads using RECIST, the selection of equivocal tumors can have two major consequences: inter-reader variability and modified sensitivity of the therapeutic response. Apart from the main causes leading to the selection of an equivocal lesion, due to the uncertainty of the radiological characteristics or due to the censoring of on-site evaluations, several other situations can be described more precisely. These latter involve cases where an equivocal is selected as target or non-target lesions, the management of equivocal lymph nodes and the case of few target lesions. In all cases, awareness of the impact of selecting a non-malignant lesion will lead radiologists to make selections in the most rational way. Also, in clinical trials where the primary endpoint differs between phase 2 (response-related) and phase 3 (progression-related) trials, our impact analysis will help them to devise strategies for the management of equivocal lesions.

Harmonization of radiomic feature distributions: impact on classification of hepatic tissue in CT imaging.

OBJECTIVES: Following the craze for radiomic features (RF), their lack of reliability raised the question of the generalizability of classification models. Inter-site harmonization of images therefore becomes a central issue. We compared RF harmonization processing designed to detect liver diseases in CT images. METHODS: We retrospectively analyzed 76 multi-center portal CT series of non-diseased (NDL) and diseased liver (DL) patients. In each series, we positioned volumes of interest in spleen and liver, then extracted 9 RF (histogram and texture). We evaluated two RF harmonization approaches. First, in each series, we computed the Z-score of liver measurements based on those computed in the spleen. Second, we evaluated the ComBat method according to each imaging center; parameters were computed in the spleen and applied to the liver. We compared RF distributions and classification performances before/after harmonization. We classified NDL versus spleen and versus DL tissues. RESULTS: The RF distributions were all different between liver and spleen (p < 0.05). The Z-score harmonization outperformed for the detection of liver versus spleen: AUC = 93.1% (p < 0.001). For the detection of DL versus NDL, in a case/control setting, we found no differences between the harmonizations: mean AUC = 73.6% (p = 0.49). Using the whole datasets, the performances were improved using ComBat (p = 0.05) AUC = 82.4% and degraded with Z-score AUC = 67.4% (p = 0.008). CONCLUSIONS: Data harmonization requires to first focus on data structuring to not degrade the performances of subsequent classifications. Liver tissue classification after harmonization of spleen-based RF is a promising strategy for improving the detection of DL tissue. KEY POINTS: • Variability of acquisition parameter makes radiomics of CT features non-reproducible. • Data harmonization can help circumvent the inter-site variability of acquisition protocols. • Inter-site harmonization must be carefully implemented and requires designing consistent data sets.

Enhanced vertebra to disk ratio as a new semi-quantitative imaging biomarker for Gaucher disease patients.

PURPOSE: Gaucher disease (GD) is an inherited lysosomal storage disorder. The Vertebral Disk Ratio (VDR) is a semi-quantitative imaging biomarker designed to diagnose and monitor GD. Computed from standard T1 MRI images, the VDR is derived from 2D segmentations. This study aimed to evaluate the 3D version of VDR, namely eVDR, and analyze the performances of two eVDR-derived response criteria for GD patients. METHODS: Three datasets were used: 8 longitudinal GD patients, 13 non-GD patients, and 2 longitudinal GD patients with known Bone Marrow Burden (BMB) scores. Two eVDR-derived response criteria were tested: 1) a parametric version (PeVDR) averaging all eVDR measures recorded for the 5 lumbar vertebrae; and 2) a non-parametric version (NPeVDR), considering all eVDR measures as independent and evaluating therapeutic response in a paired fashion. Analyses included assessment of reader variability in eVDR (3D) versus VDR (2D) and comparison with BMB response criteria. RESULTS: The repeatability of eVDR (3D) versus VDR (2D) demonstrated no difference in mean values but a lower variance (p < 0.004). The PeVDR intra-reader variability had a standard deviation < 0.1 with a coefficient of variation < 5%; the inter-reader variability featured a Limit of Agreement < 5% and a Bias < 3%. Observational comparison of eVDR and BMB scoring and sensitivity indicated a correlation between PeVDR and BMB, with an improved sensitivity with the NPeVDR version. CONCLUSIONS: Based on a standard MRI sequence, the eVDR imaging biomarker and its derived response criteria improved GD assessments and could help assessing other bone marrow diseases.

Radiology workflow for RECIST assessment in clinical trials: Can we reconcile time-efficiency and quality?

PURPOSE: In oncology clinical trials, nonconformity issues are frequently reported. Radiological workload is increasing, thus reducing radiologists' availability and affecting diagnostic quality. We compared performances of a standard radiological workflow (SW) and a novel "hybrid workflow" (HW). METHOD: We prospectively studied imaging data of 40 patients included in RECIST 1.1 clinical trials. Ninety-six time-points were reviewed by 7 radiologists and one trained technologist. Nonconformities using the SW were retrieved from hospital archives. For the HW, radiologists performed all baseline evaluations; the technologist made subsequent measurements. Finally, the radiologists checked the technologist's findings before confirming the evaluations. The HW enabled implementation of an electronic reporting system. An independent body compared SW and HW reading times and nonconformity occurrences. RESULTS: Using SW, 19 types of nonconformity were found: blank report (13%); unsigned report (11%); undocumented change of tumor burden (10%); undocumented new lesions (9%); missing/wrong patients' appointment dates (7%); undocumented tumor location (5%); error in tumor burden change (5%). SW and HW nonconformities affected 55% (179/323) and 5% (2/40) of reports, respectively (p < 0.001). HW nonconformities were: one inaccurate login name was used on the platform, and one erroneous time-point number. On average, SW required 11'30″ [10'06″; 13'20″] per time-point. HW required 1'35″ [40″; 5'08″] for radiologists, and 12'18″ [11'12″; 14'18″] for the technologist. CONCLUSIONS: HW significantly reduced the number of trial nonconformities and saved 87% of radiologists' time while enabling them to apply their expertise to final decisions. HW could offer an effective opportunity for cost reduction associated with improved imaging trial quality.

Focused ultrasound for the treatment of bone metastases: effectiveness and feasibility.

BACKGROUND: To evaluate the effectiveness and feasibility of high-intensity focused ultrasound (HIFU) for the treatment of bone metastases. METHODS: A single-center prospective study was made involving 17 consecutive patients with symptomatic bone metastases. Patients were treated by Focused Ultrasound (FUs) performed with magnetic resonance (MR) guidance. Surgical treatment or radiotherapy treatment was not indicated for patients who underwent FUs. Lesions were located in the appendicular and axial skeleton and consisted of secondary symptomatic lesions. The clinical course of pain was evaluated using the Visual Analog Scale (VAS) before treatment, at 1 week, and at 1 month after treatment and the Oral Morphine Equivalent Daily Dose (OMEDD) was also recorded. We used Wilcoxon signed rank test to assess change in patient pain (R CRAN software V 3.1.1). RESULTS: We observed a significant decrease in the pain felt by patients between pre- procedure and 1 week post-procedure (p = 2.9.10-4), and pre-procedure and 1 month post-procedure (p = 3.10-4). The proportion of responders according to the International Bone Metastases Consensus Working Party was: Partial Response 50% (8/16) and Complete Response 37.5% (6/16). CONCLUSIONS: HIFU under MR-guidance seems to be an effective and safe procedure in the treatment of symptomatic bone lesions for patients suffering from metastatic disease. A significant decrease of patient pain was observed. TRIAL REGISTRATION: NCT01091883. Registered 24 March 2010. Level of evidence: Level 3.

Discrepancies of assessments in a RECIST 1.1 phase II clinical trial - association between adjudication rate and variability in images and tumors selection.

BACKGROUND: In imaging-based clinical trials, it is common practice to perform double reads for each image, discrepant interpretations can result from these two different evaluations. In this study we analyzed discrepancies that occurred between local investigators (LI) and blinded independent central review (BICR) by comparing reader-selected imaging scans and lesions. Our goal was to identify the causes of discrepant declarations of progressive disease (PD) between LI and BICR in a clinical trial. METHODS: We retrospectively analyzed imaging data from a RECIST 1.1-based, multi-sites, phase II clinical trial of 179 patients with adult small cell lung cancer, treated with Cabazitaxel compared to Topotecan. Any discrepancies in the determination of PD between LI and BICR readers were reviewed by a third-party adjudicator. For each imaging time point and reader, we recorded the selected target lesions, non-target lesions, and new lesions. Odds ratios were calculated to measure the association between discrepant declarations of PD and the differences in reviewed imaging scans (e.g. same imaging modality but with different reconstruction parameters) and selected lesions. Reasons for discrepancies were analyzed. RESULTS: The average number of target lesions found by LI and BICR was respectively 2.9 and 3.4 per patient (p < 0.05), 18.4% of these target lesions were actually non-measurable. LI and BICR performed their evaluations based on different baseline imaging scans for 59% of the patients, they selected at least one different target lesion in 85% of patients. A total of 36.7% of patients required adjudication. Reasons of adjudication included differences in 1) reporting new lesions (53.7%), 2) the measured change of the tumor burden (18.5%), and 3) the progression of non-target lesions (11.2%). The rate of discrepancy was not associated with the selection of non-measurable target lesions or with the readers' assessment of different images. Paradoxically, more discrepancies occurred when LI and BICR selected exactly the same target lesions at baseline compared to when readers selected not exactly the same lesions. CONCLUSIONS: For a large proportion of evaluations, LI and BICR did not select the same imaging scans and target lesions but with a limited impact on the rate of discrepancy. The majority of discrepancies were explained by the difference in detecting new lesions. TRIAL REGISTRATION: ARD12166 ( https://clinicaltrials.gov/ct2/show/NCT01500720 ).

Effects of relaxing therapies on patient's pain during percutaneous interventional radiology procedures.

BACKGROUND: Interventional radiology procedures in cancer patients cause stress and anxiety. Our objective was to relate our experience in the use of sophrology techniques during interventional radiology procedures and evaluate the effects on patient's pain and anxiety. METHODS: We present a prospective observational study on 60 consecutive patients who underwent interventional radiology procedures in a context of oncologic management from September 2017 to March 2018. Forty-two patients were asked if they wished to benefit from the sophrology and hypnosis techniques during their procedure. A control group was also made including 18 patients. Anxiety level and pain were evaluated using the visual analog scale (VAS) before and during procedures. RESULTS: We observed a significant decrease in anxiety experienced by patients during interventional radiology procedures compared to before procedures in the sophrology group (P=3.318E-08), and a level of anxiety and pain during gestures inferior to that of the control group (P=2.035E-06 and 7.03E-05 respectively). CONCLUSIONS: Relaxing therapies, such as sophrology and hypnosis, seems to be an interesting additional tool for the management of patients in interventional oncology, inducing a decrease of stress, pain, and anxiety in patients.

How reliable are ADC measurements? A phantom and clinical study of cervical lymph nodes.

OBJECTIVE: To assess the reliability of ADC measurements in vitro and in cervical lymph nodes of healthy volunteers. METHODS: We used a GE 1.5 T MRI scanner and a first ice-water phantom according to recommendations released by the Quantitative Imaging Biomarker Alliance (QIBA) for assessing ADC against reference values. We analysed the target size effect by using a second phantom made of six inserted spheres with diameters ranging from 10 to 37 mm. Thirteen healthy volunteers were also scanned to assess the inter- and intra-observer reproducibility of volumetric ADC measurements of cervical lymph nodes. RESULTS: On the ice-water phantom, the error in ADC measurements was less than 4.3 %. The spatial bias due to the non-linearity of gradient fields was found to be 24 % at 8 cm from the isocentre. ADC measure reliability decreased when addressing small targets due to partial volume effects (up to 12.8 %). The mean ADC value of cervical lymph nodes was 0.87.10-3 ± 0.12.10-3 mm2/s with a good intra-observer reliability. Inter-observer reproducibility featured a bias of -5.5 % due to segmentation issues. CONCLUSION: ADC is a potentially important imaging biomarker in oncology; however, variability issues preclude its broader adoption. Reliable use of ADC requires technical advances and systematic quality control. KEY POINTS: • ADC is a promising quantitative imaging biomarker. • ADC has a fair inter-reader variability and good intra-reader variability. • Partial volume effect, post-processing software and non-linearity of scanners are limiting factors. • No threshold values for detecting cervical lymph node malignancy can be drawn.

Computer tomography-based body surface area evaluation for drug dosage: Quantitative radiology versus anthropomorphic evaluation.

OBJECTIVE: The measure of body surface area (BSA) is a standard for planning optimal dosing in oncology. This index is derived from a model having questionable performances. In this study, we proposed measurement of BSA from whole body CT images (iBSA). We tested the reliability of iBSA assessments and simulated the impact of our approach on patient chemotherapy dosage planning. METHODS: We first evaluated accuracy and precision of iBSA in measuring 14 phantom and 11 CT test-retest images.Secondly, we retrospectively analyzed 26 whole body PET-CT scans to evaluate inter-method variability between iBSA and the most used anthropomorphic models, notably the "Du Bois and Du Bois" model. Finally, we simulated the impact on chemotherapy dose planning of capecitabine based on iBSA. RESULTS: Precision and accuracy of iBSA measurement featured a standard deviation of 1.11% and a mean error of 1.53%. Inter-method variability between iBSA and "Du Bois and Du Bois" assessment featured a standard deviation of 4.11% leading to a reclassification rate of capecitabine of 32.5%. CONCLUSIONS: iBSA could help the oncologist in standardizing assessments for chemotherapy planning. iBSA could also be relevant for applications such as comprehensive body composition and provide a sensitive measurement for changes related to nutritional intake or other metabolism.

Algorithm Variability in the Estimation of Lung Nodule Volume From Phantom CT Scans: Results of the QIBA 3A Public Challenge.

RATIONALE AND OBJECTIVES: Quantifying changes in lung tumor volume is important for diagnosis, therapy planning, and evaluation of response to therapy. The aim of this study was to assess the performance of multiple algorithms on a reference data set. The study was organized by the Quantitative Imaging Biomarker Alliance (QIBA). MATERIALS AND METHODS: The study was organized as a public challenge. Computed tomography scans of synthetic lung tumors in an anthropomorphic phantom were acquired by the Food and Drug Administration. Tumors varied in size, shape, and radiodensity. Participants applied their own semi-automated volume estimation algorithms that either did not allow or allowed post-segmentation correction (type 1 or 2, respectively). Statistical analysis of accuracy (percent bias) and precision (repeatability and reproducibility) was conducted across algorithms, as well as across nodule characteristics, slice thickness, and algorithm type. RESULTS: Eighty-four percent of volume measurements of QIBA-compliant tumors were within 15% of the true volume, ranging from 66% to 93% across algorithms, compared to 61% of volume measurements for all tumors (ranging from 37% to 84%). Algorithm type did not affect bias substantially; however, it was an important factor in measurement precision. Algorithm precision was notably better as tumor size increased, worse for irregularly shaped tumors, and on the average better for type 1 algorithms. Over all nodules meeting the QIBA Profile, precision, as measured by the repeatability coefficient, was 9.0% compared to 18.4% overall. CONCLUSION: The results achieved in this study, using a heterogeneous set of measurement algorithms, support QIBA quantitative performance claims in terms of volume measurement repeatability for nodules meeting the QIBA Profile criteria.

Changes of lung tumour volume on CT - prediction of the reliability of assessments.

BACKGROUND: For oncological evaluations, quantitative radiology gives clinicians significant insight into patients' response to therapy. In regard to the Response Evaluation Criteria in Solid Tumours (RECIST), the classification of disease evolution partly consists in applying thresholds to the measurement of the relative change of tumour. In the case of tumour volumetry, response thresholds have not yet been established. This study proposes and validates a model for calculating thresholds for the detection of minimal tumour change when using the volume of pulmonary lesions on CT as imaging biomarker. METHODS: Our work is based on the reliability analysis of tumour volume measurements documented by the Quantitative Imaging Biomarker Alliance. Statistics of measurements were entered into a multi-parametric mathematical model of the relative changes derived from the Geary-Hinkley transformation. The consistency of the model was tested by comparing modelled thresholds against Monte Carlo simulations of tumour volume measurements with additive random error. The model has been validated by repeating measurements on real patient follow ups. RESULTS: For unchanged tumour volume, relying on a normal distribution of error, the agreement between model and simulations featured a type I error of 5.25%. Thus, we established that a threshold of 35% of volume reduction corresponds to a partial response (PR) and a 55% volume increase corresponds to progressive disease (PD). Changes between -35 and +55% are categorized as stable disease (SD). Tested on real clinical data, 97.1% [95.7; 98.0] of assessments fall into the range of variability predicted by our model of confidence interval. CONCLUSIONS: Our study indicates that the Geary Hinkley model, using published statistics, is appropriate to predict response thresholds for the volume of pulmonary lesions on CT.

Inter-Method Performance Study of Tumor Volumetry Assessment on Computed Tomography Test-Retest Data.

RATIONALE AND OBJECTIVES: Tumor volume change has potential as a biomarker for diagnosis, therapy planning, and treatment response. Precision was evaluated and compared among semiautomated lung tumor volume measurement algorithms from clinical thoracic computed tomography data sets. The results inform approaches and testing requirements for establishing conformance with the Quantitative Imaging Biomarker Alliance (QIBA) Computed Tomography Volumetry Profile. MATERIALS AND METHODS: Industry and academic groups participated in a challenge study. Intra-algorithm repeatability and inter-algorithm reproducibility were estimated. Relative magnitudes of various sources of variability were estimated using a linear mixed effects model. Segmentation boundaries were compared to provide a basis on which to optimize algorithm performance for developers. RESULTS: Intra-algorithm repeatability ranged from 13% (best performing) to 100% (least performing), with most algorithms demonstrating improved repeatability as the tumor size increased. Inter-algorithm reproducibility was determined in three partitions and was found to be 58% for the four best performing groups, 70% for the set of groups meeting repeatability requirements, and 84% when all groups but the least performer were included. The best performing partition performed markedly better on tumors with equivalent diameters greater than 40 mm. Larger tumors benefitted by human editing but smaller tumors did not. One-fifth to one-half of the total variability came from sources independent of the algorithms. Segmentation boundaries differed substantially, not ony in overall volume but also in detail. CONCLUSIONS: Nine of the 12 participating algorithms pass precision requirements similar to what is indicated in the QIBA Profile, with the caveat that the present study was not designed to explicitly evaluate algorithm profile conformance. Change in tumor volume can be measured with confidence to within ±14% using any of these nine algorithms on tumor sizes greater than 10 mm. No partition of the algorithms was able to meet the QIBA requirements for interchangeability down to 10 mm, although the partition comprising best performing algorithms did meet this requirement for a tumor size of greater than approximately 40 mm.