Multivariable prognostic modelling to improve prediction of colorectal cancer recurrence: the PROSPeCT trial.

OBJECTIVE: Improving prognostication to direct personalised therapy remains an unmet need. This study prospectively investigated promising CT, genetic, and immunohistochemical markers to improve the prediction of colorectal cancer recurrence. MATERIAL AND METHODS: This multicentre trial (ISRCTN 95037515) recruited patients with primary colorectal cancer undergoing CT staging from 13 hospitals. Follow-up identified cancer recurrence and death. A baseline model for cancer recurrence at 3 years was developed from pre-specified clinicopathological variables (age, sex, tumour-node stage, tumour size, location, extramural venous invasion, and treatment). Then, CT perfusion (blood flow, blood volume, transit time and permeability), genetic (RAS, RAF, and DNA mismatch repair), and immunohistochemical markers of angiogenesis and hypoxia (CD105, vascular endothelial growth factor, glucose transporter protein, and hypoxia-inducible factor) were added to assess whether prediction improved over tumour-node staging alone as the main outcome measure. RESULTS: Three hundred twenty-six of 448 participants formed the final cohort (226 male; mean 66 ± 10 years. 227 (70%) had ≥ T3 stage cancers; 151 (46%) were node-positive; 81 (25%) developed subsequent recurrence. The sensitivity and specificity of staging alone for recurrence were 0.56 [95% CI: 0.44, 0.67] and 0.58 [0.51, 0.64], respectively. The baseline clinicopathologic model improved specificity (0.74 [0.68, 0.79], with equivalent sensitivity of 0.57 [0.45, 0.68] for high vs medium/low-risk participants. The addition of prespecified CT perfusion, genetic, and immunohistochemical markers did not improve prediction over and above the clinicopathologic model (sensitivity, 0.58-0.68; specificity, 0.75-0.76). CONCLUSION: A multivariable clinicopathological model outperformed staging in identifying patients at high risk of recurrence. Promising CT, genetic, and immunohistochemical markers investigated did not further improve prognostication in rigorous prospective evaluation. CLINICAL RELEVANCE STATEMENT: A prognostic model based on clinicopathological variables including age, sex, tumour-node stage, size, location, and extramural venous invasion better identifies colorectal cancer patients at high risk of recurrence for neoadjuvant/adjuvant therapy than stage alone. KEY POINTS: Identification of colorectal cancer patients at high risk of recurrence is an unmet need for treatment personalisation. This model for recurrence, incorporating many patient variables, had higher specificity than staging alone. Continued optimisation of risk stratification schema will help individualise treatment plans and follow-up schedules.

Colorectal Cancer: Performance and Evaluation for CT Colonography Screening- A Multicenter Cluster-randomized Controlled Trial.

Background Most radiologists reporting CT colonography (CTC) do not undergo compulsory performance accreditation, potentially lowering diagnostic sensitivity. Purpose To determine whether 1-day individualized training in CTC reporting improves diagnostic sensitivity of experienced radiologists for 6-mm or larger lesions, the durability of any improvement, and any associated factors. Materials and Methods This prospective, multicenter cluster-randomized controlled trial was performed in National Health Service hospitals in England and Wales between April 2017 and January 2020. CTC services were cluster randomized into intervention (1-day training plus feedback) or control (no training or feedback) arms. Radiologists in the intervention arm attended a 1-day workshop focusing on CTC reporting pitfalls with individualized feedback. Radiologists in the control group received no training. Sensitivity for 6-mm or larger lesions was tested at baseline and 1, 6, and 12 months thereafter via interpretation of 10 CTC scans at each time point. The primary outcome was the mean difference in per-lesion sensitivity between arms at 1 month, analyzed using multilevel regression after adjustment for baseline sensitivity. Secondary outcomes included per-lesion sensitivity at 6- and 12-month follow-up, sensitivity for flat neoplasia, and effect of prior CTC experience. Results A total of 69 hospitals were randomly assigned to the intervention (31 clusters, 80 radiologists) or control (38 clusters, 59 radiologists) arm. Radiologists were experienced (median, 500-999 CTC scans interpreted) and reported CTC scans routinely (median, 151-200 scans per year). One-month sensitivity improved after intervention (66.4% [659 of 992]) compared with sensitivity in the control group (42.4% [278 of 655]; difference = 20.8%; 95% CI: 14.6, 27.0; P < .001). Improvements were maintained at 6 (66.4% [572 of 861] vs 50.5% [283 of 560]; difference = 13.0%; 95% CI: 7.4, 18.5; P < .001) and 12 (63.7% [310 of 487] vs 44.4% [187 of 421]; difference = 16.7%; 95% CI: 10.3, 23.1; P < .001) months. This beneficial effect applied to flat lesions (difference = 22.7%; 95% CI: 15.5, 29.9; P < .001) and was independent of career experience (≥1500 CTC scans: odds ratio = 1.09; 95% CI: 0.88, 1.36; P = .22). Conclusion For radiologists evaluating CT colonography studies, a 1-day training intervention yielded sustained improvement in detection of clinically relevant colorectal neoplasia, independent of previous career experience. Clinical trial registration no. NCT02892721 © RSNA, 2022 Online supplemental material is available for this article. See also the editorial by Pickhardt in this issue. An earlier incorrect version appeared online and in print. This article was corrected on February 28, 2022.

Serum HE4 and diagnosis of ovarian cancer in postmenopausal women with adnexal masses.

BACKGROUND: Transvaginal ultrasound and serum CA125 are routinely used for differential diagnosis of pelvic adnexal mass. Use of human epididymis 4 was approved in the United States in 2011. However, there is scarcity of studies evaluating the additional value of human epididymis 4. OBJECTIVE: The objective of the study was to evaluate the performance characteristics of transvaginal ultrasound, CA125, and human epididymis 4 for differential diagnosis of ovarian cancer in postmenopausal women with adnexal masses. STUDY DESIGN: This was a cohort study nested within the screen arms of the multicenter randomized controlled trial, United Kingdom Collaborative Trial of Ovarian Cancer Screening, based in England, Wales, and Northern Ireland. In United Kingdom Collaborative Trial of Ovarian Cancer Screening, 48,230 women randomized to transvaginal ultrasound screening and 50,078 to multimodal screening (serum CA125 interpreted by Risk of Ovarian Cancer Algorithm with second line transvaginal ultrasound) underwent the first (prevalence) screen. Women with adnexal lesions and/or persistently elevated risk were clinically assessed and underwent surgery or follow-up for a median of 10.9 years. Banked samples taken within 6 months of transvaginal ultrasound from all clinically assessed women were assayed for human epididymis 4 and CA125. Area under the curve and sensitivity for diagnosing ovarian cancer of multiple penalized logistic regression models incorporating logCA125, log human epididymis 4, age, and simple ultrasound features of the adnexal mass were compared. RESULTS: Of 1590 (158 multimodal, 1432 ultrasound) women with adnexal masses, 78 were diagnosed with ovarian cancer (48 invasive epithelial ovarian, 14 type I, 34 type II; 24 borderline epithelial; 6 nonepithelial) within 1 year of scan. The area under the curve (0.893 vs 0.896; P = .453) and sensitivity (74.4% vs 75.6% ;P = .564) at fixed specificity of 90% of the model incorporating age, ultrasound, and CA125 were similar to that also including human epididymis 4. Both models had high sensitivity for invasive epithelial ovarian (89.6%) and type II (>91%) cancers. CONCLUSION: Our population cohort study suggests that human epididymis 4 adds little value to concurrent use of CA125 and transvaginal ultrasound in the differential diagnosis of adnexal masses in postmenopausal women.

How to avoid describing your radiological research study incorrectly.

This review identifies and examines terms used to describe a radiological research "study" or "trial". A taxonomy of clinical research descriptions is explained with reference to medical imaging examples. Because many descriptive terms have precise methodological implications, it is important that these terms are understood by readers and used correctly by researchers, so that the reader is not misled. KEY POINTS: • Multiple different terms are being used to describe radiological research "studies" and "trials", and many of these terms have precise methodological implications. • Radiological researchers sometimes use titles that describe their research incorrectly. This can mislead the reader as to what was actually done. • It is important that readers and researchers understand the correct taxonomy of clinical research and that researchers adopt the correct description for their work.

The cardiac output optimisation following liver transplant (COLT) trial: a feasibility randomised controlled trial.

BACKGROUND: Perioperative goal directed fluid therapy (GDFT) has been shown to reduce postoperative complications following major surgery; this intervention has not been formally evaluated in the setting of liver transplantation. METHODS: We conducted a prospective trial of GDFT following liver transplantation randomising patients with liver cirrhosis to either 12 h of GDFT using non-invasive cardiac output monitoring or standard care (SC). The primary outcome was feasibility. Secondary outcomes included survival, postoperative complications (Clavien-Dindo), quality of life (by EQ-5D-5L) and resource use. Trial specific follow up occurred at 90 and 180 days after surgery. RESULTS: The study was feasible. Of 224 eligible patients, 122 were approached, 114 consented to participate and 60 were enrolled into the trial. The mean (SD) volume of IV crystalloid administered to the GDFT group during the 12-h study period was 3968 (2073) ml for the GDFT group and 2510 (1026) ml for the SC group. As regards secondary outcomes there was no difference in survival or overall complication rates. There was no significant difference in quality of life scores and resource use between the groups. CONCLUSION: A randomised study of GDFT following liver transplantation is feasible. A post-trial stakeholder meeting supported proceeding with a full multi-centre trial.

Urinary Neutrophil Gelatinase Associated Lipocalins (NGALs) predict acute kidney injury post liver transplant.

BACKGROUND: Acute Kidney Injury, a common complication of liver transplant, is associated with a significant increase in the risk of morbidity, mortality and graft loss. Current diagnostic criteria leaves a delay in diagnosis allowing further potential irreversible damage. Early biomarkers of renal injury are of clinical importance and Neutrophil Gelatinase Associated Lipocalins (NGALs) and Syndecan-1 were investigated. METHODS: AKI was defined according to the Acute Kidney Injury Network criteria. Urine and blood samples were collected pre-operatively, immediately post-op and 24 h post reperfusion to allow measurement of NGAL and Syndecan-1 levels. RESULTS: 13 of 27 patients developed an AKI. Patients who developed AKI had significantly higher peak transaminases. Urinary NGAL, plasma NGAL and Syndecan-1 levels were significantly elevated in all patients post reperfusion. Urinary NGAL levels immediately post-op were significantly higher in patients who developed an AKI than those that didn't [1319 ng/ml vs 46.56 ng/ml, p ≤ 0.001]. ROC curves were performed and urinary NGAL levels immediately post-op were an excellent biomarker for AKI with an area under the curve of 0.948 (0.847-1.00). CONCLUSIONS: Urinary NGAL levels measured immediately post-op accurately predict the development of AKI and their incorporation into clinical practise could allow early protocols to be developed to treat post transplant AKI.

Increasing Navigation Speed at Endoluminal CT Colonography Reduces Colonic Visualization and Polyp Identification.

Purpose To investigate the effect of increasing navigation speed on the visual search and decision making during polyp identification for computed tomography (CT) colonography Materials and Methods Institutional review board permission was obtained to use deidentified CT colonography data for this prospective reader study. After obtaining informed consent from the readers, 12 CT colonography fly-through examinations that depicted eight polyps were presented at four different fixed navigation speeds to 23 radiologists. Speeds ranged from 1 cm/sec to 4.5 cm/sec. Gaze position was tracked by using an infrared eye tracker, and readers indicated that they saw a polyp by clicking a mouse. Patterns of searching and decision making by speed were investigated graphically and by multilevel modeling. Results Readers identified polyps correctly in 56 of 77 (72.7%) of viewings at the slowest speed but in only 137 of 225 (60.9%) of viewings at the fastest speed (P = .004). They also identified fewer false-positive features at faster speeds (42 of 115; 36.5%) of videos at slowest speed, 89 of 345 (25.8%) at fastest, P = .02). Gaze location was highly concentrated toward the central quarter of the screen area at faster speeds (mean gaze points at slowest speed vs fastest speed, 86% vs 97%, respectively). Conclusion Faster navigation speed at endoluminal CT colonography led to progressive restriction of visual search patterns. Greater speed also reduced both true-positive and false-positive colorectal polyp identification. © RSNA, 2017 Online supplemental material is available for this article.

Point-of-Care Testing in Liver Disease and Liver Surgery.

The alterations in coagulation and hemostasis that accompany liver disease are complex, and while patients with this disease have traditionally been perceived as having a bleeding diathesis, it is now understood that in stable patients hemostasis is "re-balanced." Hepatic surgery, and particularly liver transplantation, can be associated with large fluid shifts, massive bleeding, and coagulopathy, as well as postoperative thrombosis. Point-of-care tests (POCTs) of coagulation facilitate goal-directed treatments and hemostatic monitoring in dynamic environments where the coagulation status can alter rapidly and often unpredictably. POCTs reflect more accurately the re-balanced hemostatic system than do conventional coagulation tests (CCTs). Viscoelastic POCT-guided transfusion algorithms permit a reduction in blood product administration and are a key component of patient blood management programs. Moreover, viscoelastic POCTs are better able to identify patients with hypercoagulability than CCTs. With thrombosis increasingly recognized to be a problem in patients with liver disease, POCTs hold promise for both individualized bleeding and thrombosis management.

Evaluation of treatment response and resistance in metastatic renal cell cancer (mRCC) using integrated 18F-Fluorodeoxyglucose (18F-FDG) positron emission tomography/magnetic resonance imaging (PET/MRI); The REMAP study.

BACKGROUND: Tyrosine kinase inhibitors are the first line standard of care for treatment of metastatic renal cell carcinoma (RCC). Accurate response assessment in the setting of antiangiogenic therapies remains suboptimal as standard size-related response criteria do not necessarily accurately reflect clinical benefit, as they may be less pronounced or occur later in therapy than devascularisation. The challenge for imaging is providing timely assessment of disease status allowing therapies to be tailored to ensure ongoing clinical benefit. We propose that combined assessment of morphological, physiological and metabolic imaging parameters using 18F-fluorodeoxyglucose positron emission tomography/magnetic resonance imaging (18F-FDG PET/MRI) will better reflect disease behaviour, improving assessment of response/non-response/relapse. METHODS/DESIGN: The REMAP study is a single-centre prospective observational study. Eligible patients with metastatic renal cell carcinoma, planned for systemic therapy, with at least 2 lesions will undergo an integrated 18F-FDG PET and MRI whole body imaging with diffusion weighted and contrast-enhanced multiphasic as well as standard anatomical MRI sequences at baseline, 12 weeks and 24 weeks of systemic therapy allowing 18F-FDG standardised uptake value (SUV), apparent diffusion co-efficient (ADC) and normalised signal intensity (SI) parameters to be obtained. Standard of care contrast-enhanced computed tomography CT scans will be performed at equivalent time-points. CT response categorisation will be performed using RECIST 1.1 and alternative (modified)Choi and MASS criteria. The reference standard for disease status will be by consensus panel taking into account clinical, biochemical and conventional imaging parameters. Intra- and inter-tumoural heterogeneity in vascular, diffusion and metabolic response/non-response will be assessed by image texture analysis. Imaging will also inform the development of computational methods for automated disease status categorisation. DISCUSSION: The REMAP study will demonstrate the ability of integrated 18F-FDG PET-MRI to provide a more personalised approach to therapy. We suggest that 18F-FDG PET/MRI will provide superior sensitivity and specificity in early response/non-response categorisation when compared to standard CT (using RECIST 1.1 and alternative (modified)Choi or MASS criteria) thus facilitating more timely and better informed treatment decisions. TRIAL REGISTRATION: The trial is approved by the Southeast London Research Ethics Committee reference 16/LO/1499 and registered on the NIHR clinical research network portfolio ISRCTN12114913 .

Management of severe perioperative bleeding: guidelines from the European Society of Anaesthesiology: First update 2016.

: The management of perioperative bleeding involves multiple assessments and strategies to ensure appropriate patient care. Initially, it is important to identify those patients with an increased risk of perioperative bleeding. Next, strategies should be employed to correct preoperative anaemia and to stabilise macrocirculation and microcirculation to optimise the patient's tolerance to bleeding. Finally, targeted interventions should be used to reduce intraoperative and postoperative bleeding, and so prevent subsequent morbidity and mortality. The objective of these updated guidelines is to provide healthcare professionals with an overview of the most recent evidence to help ensure improved clinical management of patients. For this update, electronic databases were searched without language restrictions from 2011 or 2012 (depending on the search) until 2015. These searches produced 18 334 articles. All articles were assessed and the existing 2013 guidelines were revised to take account of new evidence. This update includes revisions to existing recommendations with respect to the wording, or changes in the grade of recommendation, and also the addition of new recommendations. The final draft guideline was posted on the European Society of Anaesthesiology website for four weeks for review. All comments were collated and the guidelines were amended as appropriate. This publication reflects the output of this work.

Intraoperative frozen section analysis for the diagnosis of early stage ovarian cancer in suspicious pelvic masses.

BACKGROUND: Women with suspected early-stage ovarian cancer need surgical staging which involves taking samples from areas within the abdominal cavity and retroperitoneal lymph nodes in order to inform further treatment. One potential strategy is to surgically stage all women with suspicious ovarian masses, without any histological information during surgery. This avoids incomplete staging, but puts more women at risk of potential surgical over-treatment.A second strategy is to perform a two-stage procedure to remove the pelvic mass and subject it to paraffin sectioning, which involves formal tissue fixing with formalin and paraffin embedding, prior to ultrathin sectioning and multiple site sampling of the tumour. Surgeons may then base further surgical staging on this histology, reducing the rate of over-treatment, but conferring additional surgical and anaesthetic morbidity.A third strategy is to perform a rapid histological analysis on the ovarian mass during surgery, known as 'frozen section'. Tissues are snap frozen to allow fine tissue sections to be cut and basic histochemical staining to be performed. Surgeons can perform or avoid the full surgical staging procedure depending on the results. However, this is a relatively crude test compared to paraffin sections, which take many hours to perform. With frozen section there is therefore a risk of misdiagnosing malignancy and understaging women subsequently found to have a presumed early-stage malignancy (false negative), or overstaging women without a malignancy (false positive). Therefore it is important to evaluate the accuracy and usefulness of adding frozen section to the clinical decision-making process. OBJECTIVES: To assess the diagnostic test accuracy of frozen section (index test) to diagnose histopathological ovarian cancer in women with suspicious pelvic masses as verified by paraffin section (reference standard). SEARCH METHODS: We searched MEDLINE (January 1946 to January 2015), EMBASE (January 1980 to January 2015) and relevant Cochrane registers. SELECTION CRITERIA: Studies that used frozen section for intraoperative diagnosis of ovarian masses suspicious of malignancy, provided there was sufficient data to construct 2 x 2 tables. We excluded articles without an available English translation. DATA COLLECTION AND ANALYSIS: Authors independently assessed the methodological quality of included studies using the Quality Assessment of Diagnostic Accuracy Studies tool (QUADAS-2) domains: patient selection, index test, reference standard, flow and timing. Data extraction converted 3 x 3 tables of per patient results presented in articles into 2 x 2 tables, for two index test thresholds. MAIN RESULTS: All studies were retrospective, and the majority reported consecutive sampling of cases. Sensitivity and specificity results were available from 38 studies involving 11,181 participants (3200 with invasive cancer, 1055 with borderline tumours and 6926 with benign tumours, determined by paraffin section as the reference standard). The median prevalence of malignancy was 29% (interquartile range (IQR) 23% to 36%, range 11% to 63%). We assessed test performance using two thresholds for the frozen section test. Firstly, we used a test threshold for frozen sections, defining positive test results as invasive cancer and negative test results as borderline and benign tumours. The average sensitivity was 90.0% (95% confidence interval (CI) 87.6% to 92.0%; with most studies typically reporting range of 71% to 100%), and average specificity was 99.5% (95% CI 99.2% to 99.7%; range 96% to 100%).Similarly, we analysed sensitivity and specificity using a second threshold for frozen section, where both invasive cancer and borderline tumours were considered test positive and benign cases were classified as negative. Average sensitivity was 96.5% (95% CI 95.5% to 97.3%; typical range 83% to 100%), and average specificity was 89.5% (95% CI 86.6% to 91.9%; typical range 58% to 99%).Results were available from the same 38 studies, including the subset of 3953 participants with a frozen section result of either borderline or invasive cancer, based on final diagnosis of malignancy. Studies with small numbers of disease-negative cases (borderline cases) had more variation in estimates of specificity. Average sensitivity was 94.0% (95% CI 92.0% to 95.5%; range 73% to 100%), and average specificity was 95.8% (95% CI 92.4% to 97.8%; typical range 81% to 100%).Our additional analyses showed that, if the frozen section showed a benign or invasive cancer, the final diagnosis would remain the same in, on average, 94% and 99% of cases, respectively.In cases where the frozen section diagnosis was a borderline tumour, on average 21% of the final diagnoses would turn out to be invasive cancer.In three studies, the same pathologist interpreted the index and reference standard tests, potentially causing bias. No studies reported blinding pathologists to index test results when reporting paraffin sections.In heterogeneity analyses, there were no statistically significant differences between studies with pathologists of different levels of expertise. AUTHORS' CONCLUSIONS: In a hypothetical population of 1000 patients (290 with cancer and 80 with a borderline tumour), if a frozen section positive test result for invasive cancer alone was used to diagnose cancer, on average 261 women would have a correct diagnosis of a cancer, and 706 women would be correctly diagnosed without a cancer. However, 4 women would be incorrectly diagnosed with a cancer (false positive), and 29 with a cancer would be missed (false negative).If a frozen section result of either an invasive cancer or a borderline tumour was used as a positive test to diagnose cancer, on average 280 women would be correctly diagnosed with a cancer and 635 would be correctly diagnosed without. However, 75 women would be incorrectly diagnosed with a cancer and 10 women with a cancer would be missed.The largest discordance is within the reporting of frozen section borderline tumours. Investigation into factors leading to discordance within centres and standardisation of criteria for reporting borderline tumours may help improve accuracy. Some centres may choose to perform surgical staging in women with frozen section diagnosis of a borderline ovarian tumour to reduce the number of false positives. In their interpretation of this review, readers should evaluate results from studies most typical of their population of patients.

Clinical Utility of Viscoelastic Tests of Coagulation (TEG/ROTEM) in Patients with Liver Disease and during Liver Transplantation.

The concept that patients with stable liver disease are at an increased risk of bleeding, based solely on abnormalities of conventional coagulation tests such as prothrombin time (PT) and international normalized ratio (INR), is now recognized to be an overly simplistic interpretation of an extremely complex situation. These tests are in fact very poor predictors of bleeding in patients with liver disease who undergo invasive or surgical procedures. Commercially available whole blood viscoelastic tests (thromboelastography [TEG] and thromboelastometry [ROTEM]) evaluate the kinetics of coagulation from initial clot formation to final clot strength. These dynamic tests provide a composite picture reflecting the interaction of plasma, blood cells, and platelets, and more closely reflect the situation in vivo than do PT/INR, which are performed on plasma samples and measure isolated end points. Despite prolonged PT/INR and low platelet counts, viscoelastic tests are within normal range in many patients with both acute and chronic liver disease, commensurate with the concept of rebalanced hemostasis, and in keeping with the fact that an increasing number of these patients undergo liver transplantation without the need for blood or blood products. In addition, these tests reveal important additional information, such as the presence of hypercoagulability and a prothrombotic state, and also information about the presence of endogenous heparinoids associated with vascular endothelial damage, due to sepsis or acute inflammation. This review provides an overview of the current literature on the potential clinical utility of viscoelastic tests of coagulation in patients with liver disease.

Disadvantages of using the area under the receiver operating characteristic curve to assess imaging tests: a discussion and proposal for an alternative approach.

OBJECTIVES: The objectives are to describe the disadvantages of the area under the receiver operating characteristic curve (ROC AUC) to measure diagnostic test performance and to propose an alternative based on net benefit. METHODS: We use a narrative review supplemented by data from a study of computer-assisted detection for CT colonography. RESULTS: We identified problems with ROC AUC. Confidence scoring by readers was highly non-normal, and score distribution was bimodal. Consequently, ROC curves were highly extrapolated with AUC mostly dependent on areas without patient data. AUC depended on the method used for curve fitting. ROC AUC does not account for prevalence or different misclassification costs arising from false-negative and false-positive diagnoses. Change in ROC AUC has little direct clinical meaning for clinicians. An alternative analysis based on net benefit is proposed, based on the change in sensitivity and specificity at clinically relevant thresholds. Net benefit incorporates estimates of prevalence and misclassification costs, and it is clinically interpretable since it reflects changes in correct and incorrect diagnoses when a new diagnostic test is introduced. CONCLUSIONS: ROC AUC is most useful in the early stages of test assessment whereas methods based on net benefit are more useful to assess radiological tests where the clinical context is known. Net benefit is more useful for assessing clinical impact. KEY POINTS: • The area under the receiver operating characteristic curve (ROC AUC) measures diagnostic accuracy. • Confidence scores used to build ROC curves may be difficult to assign. • False-positive and false-negative diagnoses have different misclassification costs. • Excessive ROC curve extrapolation is undesirable. • Net benefit methods may provide more meaningful and clinically interpretable results than ROC AUC.

The effect of computer-aided detection markers on visual search and reader performance during concurrent reading of CT colonography.

OBJECTIVE: We aimed to identify the effect of computer-aided detection (CAD) on visual search and performance in CT Colonography (CTC) of inexperienced and experienced readers. METHODS: Fifteen endoluminal CTC examinations were recorded, each with one polyp, and two videos were generated, one with and one without a CAD mark. Forty-two readers (17 experienced, 25 inexperienced) interpreted the videos during infrared visual search recording. CAD markers and polyps were treated as regions of interest in data processing. This multi-reader, multi-case study was analysed using multilevel modelling. RESULTS: CAD drew readers' attention to polyps faster, accelerating identification times: median 'time to first pursuit' was 0.48 s (IQR 0.27 to 0.87 s) with CAD, versus 0.58 s (IQR 0.35 to 1.06 s) without. For inexperienced readers, CAD also held visual attention for longer. All visual search metrics used to assess visual gaze behaviour demonstrated statistically significant differences when "with" and "without" CAD were compared. A significant increase in the number of correct polyp identifications across all readers was seen with CAD (74 % without CAD, 87 % with CAD; p < 0.001). CONCLUSIONS: CAD significantly alters visual search and polyp identification in readers viewing three-dimensional endoluminal CTC. For polyp and CAD marker pursuit times, CAD generally exerted a larger effect on inexperienced readers. KEY POINTS: • Visual gaze is attracted by computer-assisted detection (CAD) marks on polyps • Inexperienced readers' gaze is affected more by CAD than experienced readers. • CAD marks could mean that the unannotated endoluminal surface is relatively neglected. • Correct polyp identification is increased significantly by CAD.

Small Polyps at Endoluminal CT Colonography Are Often Seen But Ignored by Radiologists.

OBJECTIVE: The objective of our study was to describe the characteristics of polyps viewed but then dismissed incorrectly by radiologists at endoluminal CT colonography (CTC), eye movements during these errors, and features provoking false-positive diagnoses. MATERIALS AND METHODS: Forty-two radiologists viewed 30 endoluminal CTC videos, each depicting a polyp, while their eye movements were tracked. Half of the videos had computer-assisted detection (CAD), and half did not. Classification errors were defined when proven polyps were seen but dismissed. Eye movements during these errors and during correct polyp identifications were compared with multilevel modeling. Polyps were divided subsequently into "difficult to classify" and "easy to classify" using a classification error threshold of more than 15%. Polyp diameter, height, and subjective conspicuity and the proportion of time viewed were compared between groups. RESULTS: Eye tracking revealed that 97% of false-negative polyp diagnoses were nonetheless preceded by the reader observing the polyp. The difficult polyps were significantly smaller than the easy polyps (mean diameter, 5.4 vs 8.2 mm, respectively p = 0.014) and were subjectively less conspicuous (median score, 4 vs 2; p = 0.0032). Readers spent proportionally less time viewing difficult polyps than viewing easy polyps (29.0% of the time they were on-screen vs 42.6%, respectively; p = 0.01) regardless of the presence of CAD. CONCLUSION: Even small and subjectively inconspicuous polyps attract reader gaze, but they are nonetheless ignored. These errors are made rapidly even with CAD. Efforts to improve reader performance at CTC should focus on decision making rather than detection alone.

Blood CEA levels for detecting recurrent colorectal cancer.

BACKGROUND: Testing for carcino-embryonic antigen (CEA) in the blood is a recommended part of follow-up to detect recurrence of colorectal cancer following primary curative treatment. There is substantial clinical variation in the cut-off level applied to trigger further investigation. OBJECTIVES: To determine the diagnostic performance of different blood CEA levels in identifying people with colorectal cancer recurrence in order to inform clinical practice. SEARCH METHODS: We conducted all searches to January 29 2014. We applied no language limits to the searches, and translated non-English manuscripts. We searched for relevant reviews in the MEDLINE, EMBASE, MEDION and DARE databases. We searched for primary studies (including conference abstracts) in the Cochrane Central Register of Controlled Trials (CENTRAL), in MEDLINE, EMBASE, and the Science Citation Index & Conference Proceedings Citation Index - Science. We identified ongoing studies by searching WHO ICTRP and the ASCO meeting library. SELECTION CRITERIA: We included cross-sectional diagnostic test accuracy studies, cohort studies, and randomised controlled trials (RCTs) of post-resection colorectal cancer follow-up that compared CEA to a reference standard. We included studies only if we could extract 2 x 2 accuracy data. We excluded case-control studies, as the ratio of cases to controls is determined by the study design, making the data unsuitable for assessing test accuracy. DATA COLLECTION AND ANALYSIS: Two review authors (BDN, IP) assessed the quality of all articles independently, discussing any disagreements. Where we could not reach consensus, a third author (BS) acted as moderator. We assessed methodological quality against QUADAS-2 criteria. We extracted binary diagnostic accuracy data from all included studies as 2 x 2 tables. We conducted a bivariate meta-analysis. We used the xtmelogit command in Stata to produce the pooled estimates of sensitivity and specificity and we also produced hierarchical summary ROC plots. MAIN RESULTS: In the 52 included studies, sensitivity ranged from 41% to 97% and specificity from 52% to 100%. In the seven studies reporting the impact of applying a threshold of 2.5 µg/L, pooled sensitivity was 82% (95% confidence interval (CI) 78% to 86%) and pooled specificity 80% (95% CI 59% to 92%). In the 23 studies reporting the impact of applying a threshold of 5 µg/L, pooled sensitivity was 71% (95% CI 64% to 76%) and pooled specificity 88% (95% CI 84% to 92%). In the seven studies reporting the impact of applying a threshold of 10 µg/L, pooled sensitivity was 68% (95% CI 53% to 79%) and pooled specificity 97% (95% CI 90% to 99%). AUTHORS' CONCLUSIONS: CEA is insufficiently sensitive to be used alone, even with a low threshold. It is therefore essential to augment CEA monitoring with another diagnostic modality in order to avoid missed cases. Trying to improve sensitivity by adopting a low threshold is a poor strategy because of the high numbers of false alarms generated. We therefore recommend monitoring for colorectal cancer recurrence with more than one diagnostic modality but applying the highest CEA cut-off assessed (10 µg/L).

Critical appraisal and data extraction for systematic reviews of prediction modelling studies: the CHARMS checklist.

Carl Moons and colleagues provide a checklist and background explanation for critically appraising and extracting data from systematic reviews of prognostic and diagnostic prediction modelling studies. Please see later in the article for the Editors' Summary.

Tracking eye gaze during interpretation of endoluminal three-dimensional CT colonography: visual perception of experienced and inexperienced readers.

PURPOSE: To identify and compare key stages of the visual process in experienced and inexperienced readers and to examine how these processes are used to search a moving three-dimensional ( 3D three-dimensional ) image and their relationship to false-negative errors. MATERIALS AND METHODS: Institutional review board research ethics approval was granted to use anonymized computed tomographic (CT) colonographic data from previous studies and to obtain eye-tracking data from volunteers. Sixty-five radiologists (27 experienced, 38 inexperienced) interpreted 23 endoluminal 3D three-dimensional CT colonographic videos. Eye movements were recorded by using eye tracking with a desk-mounted tracker. Readers indicated when they saw a polyp by clicking a computer mouse. Polyp location and boundary on each video frame were quantified and gaze data were related to the polyp boundary for each individual reader and case. Predefined metrics were quantified and used to describe and compare visual search patterns between experienced and inexperienced readers by using multilevel modeling. RESULTS: Time to first pursuit was significantly shorter in experienced readers (hazard ratio, 1.22 [95% confidence interval: 1.04, 1.44]; P = .017) but other metrics were not significantly different. Regardless of expertise, metrics such as assessment, identification period, and pursuit times were extended in videos where polyps were visible on screen for longer periods of time. In 97% (760 of 787) of observations, readers correctly pursued polyps. CONCLUSION: Experienced readers had shorter time to first eye pursuit, but many other characteristics of eye tracking were similar between experienced and inexperienced readers. Readers pursued polyps in 97% of observations, which indicated that errors during interpretation of 3D three-dimensional CT colonography in this study occurred in either the discovery or the recognition phase, but rarely in the scanning phase of radiologic image inspection.

Detection of extracolonic pathologic findings with CT colonography: a discrete choice experiment of perceived benefits versus harms.

PURPOSE: To determine the maximum rate of false-positive diagnoses that patients and health care professionals were willing to accept in exchange for detection of extracolonic malignancy by using computed tomographic (CT) colonography for colorectal cancer screening. MATERIALS AND METHODS: After obtaining ethical approval and informed consent, 52 patients and 50 health care professionals undertook two discrete choice experiments where they chose between unrestricted CT colonography that examined intra- and extracolonic organs or CT colonography restricted to the colon, across different scenarios. The first experiment detected one extracolonic malignancy per 600 cases with a false-positive rate varying across scenarios from 0% to 99.8%. One experiment examined radiologic follow-up generated by false-positive diagnoses while the other examined invasive follow-up. Intracolonic performance was identical for both tests. The median tipping point (maximum acceptable false-positive rate for extracolonic findings) was calculated overall and for both groups by bootstrap analysis. RESULTS: The median tipping point for radiologic follow-up occurred at a false-positive rate greater than 99.8% (interquartile ratio [IQR], 10 to >99.8%). Participants would tolerate at least a 99.8% rate of unnecessary radiologic tests to detect an additional extracolonic malignancy. The median tipping-point for invasive follow-up occurred at a false-positive rate of 10% (IQR, 2 to >99.8%). Tipping points were significantly higher for patients than for health care professionals for both experiments (>99.8 vs 40% for radiologic follow-up and >99.8 vs 5% for invasive follow-up, both P < .001). CONCLUSION: Patients and health care professionals are willing to tolerate high rates of false-positive diagnoses with CT colonography in exchange for diagnosis of extracolonic malignancy. The actual specificity of screening CT colonography for extracolonic findings in clinical practice is likely to be highly acceptable to both patients and health care professionals. Online supplemental material is available for this article.