Reproducibility of radiomics quality score: an intra- and inter-rater reliability study.

OBJECTIVES: To investigate the intra- and inter-rater reliability of the total radiomics quality score (RQS) and the reproducibility of individual RQS items' score in a large multireader study. METHODS: Nine raters with different backgrounds were randomly assigned to three groups based on their proficiency with RQS utilization: Groups 1 and 2 represented the inter-rater reliability groups with or without prior training in RQS, respectively; group 3 represented the intra-rater reliability group. Thirty-three original research papers on radiomics were evaluated by raters of groups 1 and 2. Of the 33 papers, 17 were evaluated twice with an interval of 1 month by raters of group 3. Intraclass coefficient (ICC) for continuous variables, and Fleiss' and Cohen's kappa (k) statistics for categorical variables were used. RESULTS: The inter-rater reliability was poor to moderate for total RQS (ICC 0.30-055, p < 0.001) and very low to good for item's reproducibility (k - 0.12 to 0.75) within groups 1 and 2 for both inexperienced and experienced raters. The intra-rater reliability for total RQS was moderate for the less experienced rater (ICC 0.522, p = 0.009), whereas experienced raters showed excellent intra-rater reliability (ICC 0.91-0.99, p < 0.001) between the first and second read. Intra-rater reliability on RQS items' score reproducibility was higher and most of the items had moderate to good intra-rater reliability (k - 0.40 to 1). CONCLUSIONS: Reproducibility of the total RQS and the score of individual RQS items is low. There is a need for a robust and reproducible assessment method to assess the quality of radiomics research. CLINICAL RELEVANCE STATEMENT: There is a need for reproducible scoring systems to improve quality of radiomics research and consecutively close the translational gap between research and clinical implementation. KEY POINTS: • Radiomics quality score has been widely used for the evaluation of radiomics studies. • Although the intra-rater reliability was moderate to excellent, intra- and inter-rater reliability of total score and point-by-point scores were low with radiomics quality score. • A robust, easy-to-use scoring system is needed for the evaluation of radiomics research.

Defining tumor growth in vestibular schwannomas: a volumetric inter-observer variability study in contrast-enhanced T1-weighted MRI.

PURPOSE: For patients with vestibular schwannomas (VS), a conservative observational approach is increasingly used. Therefore, the need for accurate and reliable volumetric tumor monitoring is important. Currently, a volumetric cutoff of 20% increase in tumor volume is widely used to define tumor growth in VS. The study investigates the tumor volume dependency on the limits of agreement (LoA) for volumetric measurements of VS by means of an inter-observer study. METHODS: This retrospective study included 100 VS patients who underwent contrast-enhanced T1-weighted MRI. Five observers volumetrically annotated the images. Observer agreement and reliability was measured using the LoA, estimated using the limits of agreement with the mean (LOAM) method, and the intraclass correlation coefficient (ICC). RESULTS: The 100 patients had a median average tumor volume of 903 mm3 (IQR: 193-3101). Patients were divided into four volumetric size categories based on tumor volume quartile. The smallest tumor volume quartile showed a LOAM relative to the mean of 26.8% (95% CI: 23.7-33.6), whereas for the largest tumor volume quartile this figure was found to be 7.3% (95% CI: 6.5-9.7) and when excluding peritumoral cysts: 4.8% (95% CI: 4.2-6.2). CONCLUSION: Agreement limits within volumetric annotation of VS are affected by tumor volume, since the LoA improves with increasing tumor volume. As a result, for tumors larger than 200 mm3, growth can reliably be detected at an earlier stage, compared to the currently widely used cutoff of 20%. However, for very small tumors, growth should be assessed with higher agreement limits than previously thought.

Comparison of different radiographic methods to measure the slip angle in children with slipped capital femoral epiphysis (SCFE).

BACKGROUND: The management of patients with slipped capital femoral epiphysis (SCFE) requires imaging diagnostics of good quality and accurate measurement of the degree of slippage. In Sweden, three different radiological methods are commonly used: the calcar femorale method; the Billing method; and the Head-shaft angle described by Southwick. PURPOSE: To evaluate whether any of the three most common methods used in Sweden to measure the slip angle was more useful and reproducible than the others. MATERIAL AND METHODS: Two experienced orthopaedists measured the slip angle in preoperative hip radiographs. Intra- and inter-observer variability between the two experienced observers and the reported value by clinicians who treated the child with SCFE was evaluated. RESULTS: The intraclass correlation coefficient (ICC) confidence interval (CI) between the two experienced observers and the reporting clinicians overlapped for the three methods. In 37% of the cases, the difference was more than 5° between the experienced observers' measurement and the reported value by clinicians. The two experienced orthopaedists' intra- and inter-observer variability was low. CONCLUSION: The observer's experience is more important than the method of choice when measuring the slip angle in SCFE. The research group recommends the calcar femorale method due to its feasibility on the versatile and commonly used frog leg lateral view.

Organ-contour-driven auto-matching algorithm in image-guided radiotherapy.

PURPOSE: This study aimed to demonstrate the potential clinical applicability of an organ-contour-driven auto-matching algorithm in image-guided radiotherapy. METHODS: This study included eleven consecutive patients with cervical cancer who underwent radiotherapy in 23 or 25 fractions. Daily and reference magnetic resonance images were converted into mesh models. A weight-based algorithm was implemented to optimize the distance between the mesh model vertices and surface of the reference model during the positioning process. Within the cost function, weight parameters were employed to prioritize specific organs for positioning. In this study, three scenarios with different weight parameters were prepared. The optimal translation and rotation values for the cervix and uterus were determined based on the calculated translations alone or in combination with rotations, with a rotation limit of ±3°. Subsequently, the coverage probabilities of the following two planning target volumes (PTV), an isotropic 5 mm and anisotropic margins derived from a previous study, were evaluated. RESULTS: The percentage of translations exceeding 10 mm varied from 9% to 18% depending on the scenario. For small PTV sizes, more than 80% of all fractions had a coverage of 80% or higher. In contrast, for large PTV sizes, more than 90% of all fractions had a coverage of 95% or higher. The difference between the median coverage with translational positioning alone and that with both translational and rotational positioning was 1% or less. CONCLUSION: This algorithm facilitates quantitative positioning by utilizing a cost function that prioritizes organs for positioning. Consequently, consistent displacement values were algorithmically generated. This study also revealed that the impact of rotational corrections, limited to ±3°, on PTV coverage was minimal.

Evaluating the impact of delayed-phase imaging in Contrast-Enhanced Mammography on breast cancer staging: A comparative study of abbreviated versus complete protocol.

PURPOSE: Contrast-enhanced mammography (CEM) is an innovative imaging tool for breast cancer detection, involving intravenous injection of a contrast medium and the assessment of lesion enhancement in two phases: early and delayed. The aim of the study was to analyze the topographic concordance of lesions detected in the early- versus delayed phase acquisitions. MATERIALS AND METHODS: Approved by the Ethics Committee (No. 118/20), this prospective study included 100 women with histopathological confirmed breast neoplasia (B6) at the Radiodiagnostics Department of the Maggiore della Carità Hospital of Novara, Italy from May 1, 2021, to October 17, 2022. Participants underwent CEM examinations using a complete protocol, encompassing both early- and delayed image acquisitions. Three experienced radiologists blindly analyzed the CEM images for contrast enhancement to determine the topographic concordance of the identified lesions. Two readers assessed the complete study (protocol A), while one reader assessed the protocol without the delayed phase (protocol B). The average glandular dose (AGD) of the entire procedure was also evaluated. RESULTS: The analysis demonstrated high concordance among the three readers in the topographical identification of lesions within individual quadrants of both breasts, with a Cohen's κ > 0.75, except for the lower inner quadrant of the right breast and the retro-areolar region of the left breast. The mean whole AGD was 29.2 mGy. The mean AGD due to CEM amounted to 73% of the whole AGD (21.2 mGy). The AGD attributable to the delayed phase of CEM contributed to 36% of the whole AGD (10.5 mGy). CONCLUSIONS: As we found no significant discrepancy between the readings of the two protocols, we conclude that delayed-phase image acquisition in CEM does not provide essential diagnostic benefits for effective disease management. Instead, it contributes to unnecessary radiation exposure.

Validation of the updated Bosniak classification (2019) in pathologically confirmed CT-categorised cysts.

INTRODUCTION: The updated Bosniak classification in 2019 (v2019) addresses vague imaging terms and revises the criteria with the intent to categorise a higher proportion of cysts in lower-risk groups and reduce benign cyst resections. The aim of the present study was to compare the diagnostic accuracy and inter-observer agreement rate of the original (v2005) and updated classifications (v2019). METHOD: Resected/biopsied cysts were categorised according to Bosniak classifications (v2005 and v2019) and the diagnostic accuracy was assessed with reference to histopathological analysis. The inter-observer agreement of v2005 and v2019 was determined. RESULTS: The malignancy rate of the cohort was 83.6% (51/61). Using v2019, a higher proportion of malignant cysts were categorised as Bosniak ≥ III (88.2% vs 84.3%) and a significantly higher percentage were categorised as Bosniak IV (68.9% vs 47.1%; p = 0.049) in comparison to v2005. v2019 would have resulted in less benign cyst resections (13.5% vs 15.7%). Calcified versus non-calcified cysts had lower rates of malignancy (57.1% vs 91.5%; RR,0.62; p = 0.002). The inter-observer agreement of v2005 was higher than that of v2019 (kappa, 0.70 vs kappa, 0.43). DISCUSSION: The updated classification improves the categorisation of malignant cysts and reduces benign cyst resection. The low inter-observer agreement remains a challenge to the updated classification system.

Delineation and agreement of FET PET biological volumes in glioblastoma: results of the nuclear medicine credentialing program from the prospective, multi-centre trial evaluating FET PET In Glioblastoma (FIG) study-TROG 18.06.

PURPOSE: The O-(2-[18F]-fluoroethyl)-L-tyrosine (FET) PET in Glioblastoma (FIG) trial is an Australian prospective, multi-centre study evaluating FET PET for glioblastoma patient management. FET PET imaging timepoints are pre-chemoradiotherapy (FET1), 1-month post-chemoradiotherapy (FET2), and at suspected progression (FET3). Before participant recruitment, site nuclear medicine physicians (NMPs) underwent credentialing of FET PET delineation and image interpretation. METHODS: Sites were required to complete contouring and dynamic analysis by ≥ 2 NMPs on benchmarking cases (n = 6) assessing biological tumour volume (BTV) delineation (3 × FET1) and image interpretation (3 × FET3). Data was reviewed by experts and violations noted. BTV definition includes tumour-to-background ratio (TBR) threshold of 1.6 with crescent-shaped background contour in the contralateral normal brain. Recurrence/pseudoprogression interpretation (FET3) required assessment of maximum TBR (TBRmax), dynamic analysis (time activity curve [TAC] type, time to peak), and qualitative assessment. Intraclass correlation coefficient (ICC) assessed volume agreement, coefficient of variation (CoV) compared maximum/mean TBR (TBRmax/TBRmean) across cases, and pairwise analysis assessed spatial (Dice similarity coefficient [DSC]) and boundary agreement (Hausdorff distance [HD], mean absolute surface distance [MASD]). RESULTS: Data was accrued from 21 NMPs (10 centres, n ≥ 2 each) and 20 underwent review. The initial pass rate was 93/119 (78.2%) and 27/30 requested resubmissions were completed. Violations were found in 25/72 (34.7%; 13/12 minor/major) of FET1 and 22/74 (29.7%; 14/8 minor/major) of FET3 reports. The primary reasons for resubmission were as follows: BTV over-contour (15/30, 50.0%), background placement (8/30, 26.7%), TAC classification (9/30, 30.0%), and image interpretation (7/30, 23.3%). CoV median and range for BTV, TBRmax, and TBRmean were 21.53% (12.00-30.10%), 5.89% (5.01-6.68%), and 5.01% (3.37-6.34%), respectively. BTV agreement was moderate to excellent (ICC = 0.82; 95% CI, 0.63-0.97) with good spatial (DSC = 0.84 ± 0.09) and boundary (HD = 15.78 ± 8.30 mm; MASD = 1.47 ± 1.36 mm) agreement. CONCLUSION: The FIG study credentialing program has increased expertise across study sites. TBRmax and TBRmean were robust, with considerable variability in BTV delineation and image interpretation observed.

Automated Breast Density Assessment in MRI Using Deep Learning and Radiomics: Strategies for Reducing Inter-Observer Variability.

BACKGROUND: Accurate breast density evaluation allows for more precise risk estimation but suffers from high inter-observer variability. PURPOSE: To evaluate the feasibility of reducing inter-observer variability of breast density assessment through artificial intelligence (AI) assisted interpretation. STUDY TYPE: Retrospective. POPULATION: Six hundred and twenty-one patients without breast prosthesis or reconstructions were randomly divided into training (N = 377), validation (N = 98), and independent test (N = 146) datasets. FIELD STRENGTH/SEQUENCE: 1.5 T and 3.0 T; T1-weighted spectral attenuated inversion recovery. ASSESSMENT: Five radiologists independently assessed each scan in the independent test set to establish the inter-observer variability baseline and to reach a reference standard. Deep learning and three radiomics models were developed for three classification tasks: (i) four Breast Imaging-Reporting and Data System (BI-RADS) breast composition categories (A-D), (ii) dense (categories C, D) vs. non-dense (categories A, B), and (iii) extremely dense (category D) vs. moderately dense (categories A-C). The models were tested against the reference standard on the independent test set. AI-assisted interpretation was performed by majority voting between the models and each radiologist's assessment. STATISTICAL TESTS: Inter-observer variability was assessed using linear-weighted kappa (κ) statistics. Kappa statistics, accuracy, and area under the receiver operating characteristic curve (AUC) were used to assess models against reference standard. RESULTS: In the independent test set, five readers showed an overall substantial agreement on tasks (i) and (ii), but moderate agreement for task (iii). The best-performing model showed substantial agreement with reference standard for tasks (i) and (ii), but moderate agreement for task (iii). With the assistance of the AI models, almost perfect inter-observer variability was obtained for tasks (i) (mean κ = 0.86), (ii) (mean κ = 0.94), and (iii) (mean κ = 0.94). DATA CONCLUSION: Deep learning and radiomics models have the potential to help reduce inter-observer variability of breast density assessment. LEVEL OF EVIDENCE: 3 TECHNICAL EFFICACY: Stage 1.

Inter-observer variability of expert-derived morphologic risk predictors in aortic dissection.

OBJECTIVES: Establishing the reproducibility of expert-derived measurements on CTA exams of aortic dissection is clinically important and paramount for ground-truth determination for machine learning. METHODS: Four independent observers retrospectively evaluated CTA exams of 72 patients with uncomplicated Stanford type B aortic dissection and assessed the reproducibility of a recently proposed combination of four morphologic risk predictors (maximum aortic diameter, false lumen circumferential angle, false lumen outflow, and intercostal arteries). For the first inter-observer variability assessment, 47 CTA scans from one aortic center were evaluated by expert-observer 1 in an unconstrained clinical assessment without a standardized workflow and compared to a composite of three expert-observers (observers 2-4) using a standardized workflow. A second inter-observer variability assessment on 30 out of the 47 CTA scans compared observers 3 and 4 with a constrained, standardized workflow. A third inter-observer variability assessment was done after specialized training and tested between observers 3 and 4 in an external population of 25 CTA scans. Inter-observer agreement was assessed with intraclass correlation coefficients (ICCs) and Bland-Altman plots. RESULTS: Pre-training ICCs of the four morphologic features ranged from 0.04 (-0.05 to 0.13) to 0.68 (0.49-0.81) between observer 1 and observers 2-4 and from 0.50 (0.32-0.69) to 0.89 (0.78-0.95) between observers 3 and 4. ICCs improved after training ranging from 0.69 (0.52-0.87) to 0.97 (0.94-0.99), and Bland-Altman analysis showed decreased bias and limits of agreement. CONCLUSIONS: Manual morphologic feature measurements on CTA images can be optimized resulting in improved inter-observer reliability. This is essential for robust ground-truth determination for machine learning models. KEY POINTS: • Clinical fashion manual measurements of aortic CTA imaging features showed poor inter-observer reproducibility. • A standardized workflow with standardized training resulted in substantial improvements with excellent inter-observer reproducibility. • Robust ground truth labels obtained manually with excellent inter-observer reproducibility are key to develop reliable machine learning models.

Anastomotic perfusion assessment with indocyanine green in robot-assisted low-anterior resection, a multicenter study of interobserver variation.

BACKGROUND: Securing sufficient blood perfusion to the anastomotic area after low-anterior resection is a crucial factor in preventing anastomotic leakage (AL). Intra-operative indocyanine green fluorescent imaging (ICG-FI) has been suggested as a tool to assess perfusion. However, knowledge of inter-observer variation among surgeons in the interpretation of ICG-FI is sparse. Our primary objective was to evaluate inter-observer variation among surgeons in the interpretation of bowel blood-perfusion assessed visually by ICG-FI. Our secondary objective was to compare the results both from the visual assessment of ICG and from computer-based quantitative analyses of ICG-FI between patients with and without the development of AL. METHOD: A multicenter study, including patients undergoing robot-assisted low anterior resection with stapled anastomosis. ICG-FI was evaluated visually by the surgeon intra-operatively. Postoperatively, recorded videos were anonymized and exchanged between centers for inter-observer evaluation. Time to visibility (TTV), time to maximum visibility (TMV), and time to wash-out (TWO) were visually assessed. In addition, the ICG-FI video-recordings were analyzed using validated pixel analysis software to quantify blood perfusion. RESULTS: Fifty-five patients were included, and five developed clinical AL. Bland-Altman plots (BA plots) demonstrated wide inter-observer variation for visually assessed fluorescence on all parameters (TTV, TMV, and TWO). Comparing leak-group with no-leak group, we found no significant differences for TTV: Hazard Ratio; HR = 0.82 (CI 0.32; 2.08), TMV: HR = 0.62 (CI 0.24; 1.59), or TWO: HR = 1.11 (CI 0.40; 3.11). In the quantitative pixel analysis, a lower slope of the fluorescence time-curve was found in patients with a subsequent leak: median 0.08 (0.07;0.10) compared with non-leak patients: median 0.13 (0.10;0.17) (p = 0.04). CONCLUSION: The surgeon's visual assessment of the ICG-FI demonstrated wide inter-observer variation, there were no differences between patients with and without AL. However, quantitative pixel analysis showed a significant difference between groups. TRIAL REGISTRATION: ClinicalTrials.gov Identifier: NCT04766060.

Intra- and interobserver agreement of rectal cancer staging with MRI.

BACKGROUND: Reliable preoperative staging of rectal cancers is crucial for treatment decision making. PURPOSE: To assess the intra- and inter-observer agreement of rectal cancer staging, including the sub-categories, with magnetic resonance imaging (MRI). MATERIAL AND METHODS: The study includes 85 patients (35.3% women; mean age = 62.2 ± 11.2 years) who underwent MRI for rectal cancer staging between August 2020 and April 2021. All the stored images were evaluated independently by two radiologists with 10-15 years of experience. For intra-observer agreement, the evaluations were done two months apart. Analyses were made using kappa, prevalence and bias-adjusted kappa (PABAK), and intraclass correlation coefficient (ICC), where appropriate. RESULTS: There was a substantial inter-observer agreement for tumor localization (kappa = 0.665, PABAK = 0.682), mesorectal fascia invasion (kappa = 0.663, PABAK = 0.822), internal and external sphincter involvement (kappa 0.804 and 0.751, PABAK 0.859 and 0.929, respectively), and moderate to substantial agreement for M-staging (kappa = 0.451, PABAK = 0.742) and extramural vascular invasion (kappa = 0.569, PABAK = 0.741). There was also a good inter-observer agreement for T staging and N staging (ICC = 0.862, 95% confidence interval [CI] = 0.788-0.911; and ICC = 0.841, 95% CI = 0.595-0.922, respectively). As expected, intra-observer agreement was better than inter-observer agreement. CONCLUSION: Intra- and inter-observer agreement for MRI staging of rectal cancers using the structured reporting template is good.

Ultrasound attenuation imaging: a reproducible alternative for the noninvasive quantitative assessment of hepatic steatosis in children.

BACKGROUND: Pediatric hepatic steatosis is a global public health concern, as an increasing number of children are affected by this condition. Liver biopsy is the gold standard diagnostic method; however, this procedure is invasive. Magnetic resonance imaging (MRI)-derived proton density fat fraction has been accepted as an alternative to biopsy. However, this method is limited by cost and availability. Ultrasound (US) attenuation imaging is an upcoming tool for noninvasive quantitative assessment of hepatic steatosis in children. A limited number of publications have focused on US attenuation imaging and the stages of hepatic steatosis in children. OBJECTIVE: To analyze the usefulness of ultrasound attenuation imaging for the diagnosis and quantification of hepatic steatosis in children. MATERIAL AND METHODS: Between July and November 2021, 174 patients were included and divided into two groups: group 1, patients with risk factors for steatosis (n = 147), and group 2, patients without risk factors for steatosis (n = 27). In all cases, age, sex, weight, body mass index (BMI), and BMI percentile were determined. B-mode US (two observers) and US attenuation imaging with attenuation coefficient acquisition (two independent sessions, two different observers) were performed in both groups. Steatosis was classified into four grades (0: absent, 1: mild, 2: moderate and 3: severe) using B-mode US. Attenuation coefficient acquisition was correlated with steatosis score according to Spearman's correlation. Attenuation coefficient acquisition measurements' interobserver agreement was assessed using intraclass correlation coefficients (ICC). RESULTS: All attenuation coefficient acquisition measurements were satisfactory without technical failures. The median values for group 1 for the first session were 0.64 (0.57-0.69) dB/cm/MHz and 0.64 (0.60-0.70) dB/cm/MHz for the second session. The median values for group 2 for the first session were 0.54 (0.51-0.56) dB/cm/MHz and 0.54 (0.51-0.56) dB/cm/MHz for the second. The average attenuation coefficient acquisition was 0.65 (0.59-0.69) dB/cm/MHz for group 1 and 0.54 (0.52-0.56) dB/cm/MHz for group 2. There was excellent interobserver agreement at 0.94 (95% CI 0.92-0.96). There was substantial agreement between both observers (κ = 0.77, with a P < 0.001). There was a positive correlation between ultrasound attenuation imaging and B-mode scores for both observers (r = 0.87, P < 0.001 for observer 1; r = 0.86, P < 0.001 for observer 2). Attenuation coefficient acquisition median values were significantly different for each steatosis grade (P < 0.001). In the assessment of steatosis by B-mode US, the agreement between the two observers was moderate (κ = 0.49 and κ = 0.55, respectively, with a P < 0.001 in both cases). CONCLUSION: US attenuation imaging is a promising tool for the diagnosis and follow-up of pediatric steatosis, which provides a more repeatable form of classification, especially at low levels of steatosis detectable in B-mode US.

Inter-observer variability in library plan selection on iterative CBCT and synthetic CT images of cervical cancer patients.

INTRODUCTION: In the Library-of-Plans (LoP) approach, correct plan selection is essential for delivering radiotherapy treatment accurately. However, poor image quality of the cone-beam computed tomography (CBCT) may introduce inter-observer variability and thereby hamper accurate plan selection. In this study, we investigated whether new techniques to improve the CBCT image quality and improve consistency in plan selection, affects the accuracy of LoP selection in cervical cancer patients. MATERIALS AND METHODS: CBCT images of 12 patients were used to investigate the inter-observer variability of plan selection based on different CBCT image types. Six observers were asked to individually select a plan based on clinical X-ray Volumetric Imaging (XVI) CBCT, iterative reconstructed CBCT (iCBCT) and synthetic CTs (sCT). Selections were performed before and after a consensus meeting with the entire group, in which guidelines were created. A scoring by all observers on the image quality and plan selection procedure was also included. For plan selection, Fleiss' kappa (κ) statistical test was used to determine the inter-observer variability within one image type. RESULTS: The agreement between observers was significantly higher on sCT compared to CBCT. The consensus meeting improved the duration and inter-observer variability. In this manuscript, the guidelines attributed the overall results in the plan selection. Before the meeting, the gold standard was selected in 76% of the cases on XVI CBCT, 74% on iCBCT, and 76% on sCT. After the meeting, the gold standard was selected in 83% of the cases on XVI CBCT, 81% on iCBCT, and 90% on sCT. CONCLUSION: The use of sCTs can increase the agreement of plan selection among observers and the gold standard was indicated to be selected more often. It is important that clear guidelines for plan selection are implemented in order to benefit from the increased image quality, accurate selection, and decrease inter-observer variability.

Classifying Bone Loss in Failed Stemmed Total Knee Arthroplasty: Determining Reliability.

BACKGROUND: Failed stemmed total knee arthroplasty (TKA) components present with varying degrees of bone loss and technical challenges. A classification system has been proposed based upon metaphyseal bone loss and diaphyseal cortical integrity. A validation study was performed to determine interobserver and intraobserver reliability at multiple institutions and with different levels of training. METHODS: An online survey with digital anteroposterior and lateral radiographs was sent to 5 arthroplasty surgeons and 5 adult reconstruction fellows. The survey included 62 cases with stemmed femoral and tibial components, considered failures and pending revision, and scored by each reviewer independently using the classification system. Each case was scored in 2 separate sessions. Interobserver and intraobserver reliability was assessed using the intraclass correlation coefficient (ICC). RESULTS: Interobserver grading for both the femur (0.69) and tibia (0.72) showed strong reliability among the attendings and fellows, with slightly stronger reliability in tibia cases. The intraclass correlation coefficient (ICC) for attendings and fellows was similar overall, demonstrating consistency of the grading regardless of training level. Intraobserver comparisons showed a strong ICC for attendings and fellows in femoral cases, while fellows had near-perfect ICC in tibia cases. Across all reviewers there was on average 93% agreement within 1 grade per case with the majority of the discrepancy occurring at the metaphyseal-diaphyseal junction. CONCLUSION: This classification demonstrated overall strong interobserver and intraobserver reliability, with 93% agreement within 1 grade of bone loss. With further education, this classification system can ultimately be used to standardize the degree of bone loss in failed stemmed components and help with preoperative planning.

Pancreatic cancer detection with a non-contrast MR protocol: is it reliable?

PURPOSE: The pancreatic cancer (PC) is the 4th leading cancer-related death, becoming the second one by 2030, with a 5 year survival rate of 8%. Considering its increased incidence in high-risk categories compared to the general population, we aimed to validate a non-contrast MR protocol, to detect PC in its earliest phase, which could be suitable as a screening tool in high-risk patients. MATERIALS AND METHODS: In this retrospective study, we selected 200 patients (> 40 years) from our radiological database, which performed upper abdominal MRI between 2012 and 2017. 100 were negative for pancreatic lesions and 100 positive for pancreatic lesion (< 30 mm). The latter group included: 40 PDAC (pancreatic adenocarcinoma), 42 BD-IPMN (Branch Duct- Intraductal Papillary Mucinous Neoplasm), 10 PNET(pancreatic neuroendocrine tumor), 4 SCN(serous cystic neoplasm), 3 IPS(intrapancreatic spleen), 1 MCN(mucinous cystic neoplasm). Three readers (R1, R2 and R3) with a high, medium and low experience, respectively, analysed, first, the non-contrast MR sequences (single-shot T2w breath-hold, GE T1w FS, DWI and 2D/3D MRCP), and then the standard MR protocol, independently, randomly and anonymously. Readers identified or excluded the presence of pancreatic lesion, in both reading sessions. These results were compared with the histopathological diagnosis, and then divided into 3 different classes of lesions: all lesions, pancreatic adenocarcinoma and solid lesion. Mcnemar's test was used to compare the results. The inter-observer agreement was determined according to the kappa statistic in both protocols, and then the inter-protocol agreement was calculated. RESULTS: The non-contrast MR protocol has reached statistical parameters values ranging between 83% in SE (sensitivity) by R3 and 99% in NPV (negative predictive value) by R1. The standard MR protocol has reported slight increasing statistical parameters compared to those of the proposed one. However, there are not significant statistical differences between the both protocols. The proposed non-contrast MR protocol has reported the highest NPVs in the PDAC group detection (R1: 99%, R2: 99%, R3: 98%). In all groups of lesions, the agreement between the two protocols was excellent for each Reader ranging from 96 to 98%. CONCLUSION: The proposed non-contrast MR protocol showed high PC detection values and a time execution ≤ 20 min. Therefore, it can be proposed as a screening tool in high-risk patients.

What is the failure rate of constrained liners in complex revision total hip arthroplasty?

BACKGROUND: Recurrent hip dislocation after multiple revision total hip arthroplasty is a severe complication. Therefore, constrained acetabular liners (CL) have been used during salvage procedures. We report our experience of constrained liners in a re-revision setting with focus on re-dislocation. We also evaluated acetabular and femoral bone loss as potential risk factor. METHODS: Between January 2013 and December 2016, 65 patients were treated in a single institution for revision and re-revision hip arthroplasty using CL. The indication for using a CL was a high risk of re-redislocation after multiple recurrent hip dislocation including failed Dual Mobility Cups (DMC). Compromising soft tissue defects as well as severe bone defect were therefore regarded as high risks. Thirty-eight patients (77.6%) underwent a minimum of three surgical procedures before the index revision procedure. Sixteen patients (24.6%) were excluded as they were lost to follow-up, expired before minimum follow-up or refused study participation, leaving 49 patients in the analysis (75.4%). The mean follow-up was 62 months (44-74; SD = 7.7). We assessed the following potential risk factors for revision or dislocation: type of surgical setting (septic/aseptic), BMI, cup inclination angle, size of liner used and acetabular and femoral bone loss according to Paprosky classification. The primary endpoints were dislocation or repeat revision for any reason. RESULTS: Of the 49 patients, we found an overall re-revision rate of 40.8% (20/49) and a dislocation rate of 30.6% (15/49). There were no significant differences among the surgical re-revision rate or dislocation rate as a factor of patient characteristics. In terms of bone loss, there was a trend towards higher revision rates for increasing acetabular and femoral bone loss, but without statistical significance. CONCLUSIONS: We found the use of a constrained liner in a re-revision setting still bears a high risk of re-revision and re-dislocation. Therefore we restrained from using constrained liners in favour of Dual mobility cups. In this study there was no significant higher dislocation rate in the subgroup of periprosthetic infection. Furthermore the rigid design of a constrained liner bears the known risk of structural failure of acetabular reconstruction implants. Severe acetabular or femoral bone defects seem to have an impact on the revision rate, but not on the dislocation rate with regards to the restored offset and center of the hip. Results have to be taken into context such that the study population inherently has a predisposition for poorer outcomes. Indications should be strongly filtered for patients at high risk for recurrent hip joint dislocation including failed DMCs with only limited bone loss and moderate soft tissue defects. Our modification to the existing classification with a high inter and intraobserver reliability will make future studies more comparable regarding revisions and bone stock loss. Still further research using objective and reproducible parameters is needed to better analyze data especially in the background of complex revision hip arthroplasty.

Inter observer reliability for peritoneal carcinomatosis at computed tomography.

OBJECTIVE: To determine whether there is inter-observer reliability between radiologists for reporting peritoneal carcinomatosis and computed tomography peritoneal carcinomatosis index estimation. METHODS: The retrospective, cross-sectional study was conducted at Dow Institute of Radiology, Dow University of Health Sciences (DUHS), Ojha campus and comprised computed tomography scans done between December 1, 2019, to May 31, 2020, that were extracted from the institutional database searched using key words 'peritoneal carcinomatosis' and/or 'serosal deposits'. The first readers had 1-4 years of post-fellowship experience, while the 2nd readers were senior radiologists. Inter-observer reliability was assessed quantitatively and qualitatively for 15 peritoneal sites using, among other tolls, the Sugarbaker computed tomography peritoneal carcinomatosis index. Data was analysed using SPSS 21. RESULTS: Out of 236 subjects with mean age 53.6±13.6 years, there were 173(73.3%) females and 63(26.7%) males. The most common primary cancer was ovarian 145(61.4%), followed by colon 26(11%). The size of peritoneal deposit was not reported in 75(31.8%) cases. Of the 15 sites analysed, good agreement was not found in 7(46.7%). There was an excellent intra-class correlation for measuring computed tomography peritoneal carcinomatosis index scores among the radiologists irrespective of the grade of the faculty (>0.90). CONCLUSIONS: The inter-observer reliability was low, but good agreement for computed tomography peritoneal carcinomatosis index would encourage radiologists to use it in peritoneal cancer reporting.

Direct Comparison of US and MR Elastography for Staging Liver Fibrosis in Patients With Nonalcoholic Fatty Liver Disease.

BACKGROUND & AIMS: As alternatives to the expensive liver biopsy for assessing liver fibrosis stage in patients with nonalcoholic fatty liver disease (NAFLD), we directly compared the diagnostic abilities of magnetic resonance elastography (MRE), vibration-controlled transient elastography (VCTE), and two-dimensional shear wave elastography (2D-SWE). METHODS: Overall, 231 patients with biopsy-proven NAFLD were included. Intra- and inter-observer reproducibility was analyzed using intraclass correlation coefficient in a sub-group of 70 participants, in whom liver stiffness measurement (LSM) was performed by an elastography expert and an ultrasound expert who was an elastography trainee on the same day. RESULTS: Valid LSMs were obtained for 227, 220, 204, and 201 patients using MRE, VCTE, 2D-SWE, and all three modalities combined, respectively. Although the area under the curve did not differ between the modalities for detecting stage ≥1, ≥2, and ≥3 liver fibrosis, it was higher for MRE than VCTE and 2D-SWE for stage 4. Sex was a significant predictor of discordance between VCTE and liver fibrosis stage. Skin-capsule distance and the ratio of the interquartile range of liver stiffness to the median were significantly associated with discordance between 2D-SWE and liver fibrosis stage. However, no factors were associated with discordance between MRE and liver fibrosis stage. Intra- and inter-observer reproducibility in detecting liver fibrosis was higher for MRE than VCTE and 2D-SWE. CONCLUSIONS: MRE, VCTE, and 2D-SWE demonstrated excellent diagnostic accuracy in detecting liver fibrosis in patients with NAFLD. MRE demonstrated the highest diagnostic accuracy for stage 4 detection and intra- and inter-observer reproducibility. UMIN Clinical Trials Registry No. UMIN000031491.

Evaluation of inter-observer variability regarding aortic and mitral valve findings on transesophageal echocardiograms ordered for suspected endocarditis.

BACKGROUND: Transesophageal echocardiography (TEE) is the gold standard for the detection of valvular vegetations (VV). Differentiating small VV from degenerative changes is challenging and prone to inter-observer variability. We evaluated inter-observer agreement regarding aortic (AV) and mitral valve (MV) findings on TEEs ordered for suspected infective endocarditis (IE). METHODS: A total of 349 consecutive TEEs were evaluated. Studies were classified as "definite, possible, or no" IE with valve masses classified further by morphology. Nine faculty echocardiographers scored randomly selected TEEs of the AV (N = 38) and MV (N = 35). Inter-reader variability was calculated using the Fleiss/Scott Kappa (Kf). RESULTS: Positive blood cultures were present in 81% and 45% had definite IE by the modified Duke criteria. There was moderate reader agreement regarding the presence of a valvular mass for both the AV (Kf = .41, 95% CI [.30-.53]) and MV (Kf = .49, 95% CI [.34-.65]). For diagnosis of IE, there was fair agreement for the AV (Kf = .29, 95% CI [.18-.42]) and moderate agreement for the MV (Kf = .53, 95% CI [.36-.70]). Masses described as large, multi-lobulated, or pedunculated were more frequently categorized as clinical IE, (p < .006, both valves), however those with filamentous lesions were not (p < .001, both valves). CONCLUSIONS: In a large academic center, the inter-observer agreement for the presence of a left sided valvular mass was moderate and agreement regarding the final diagnosis of IE was fair to moderate, with better agreement among readers evaluating the MV. Lesion morphology is associated with the clinical diagnosis of IE.

Different chest CT scoring systems in patients with COVID-19: could baseline CT be a helpful tool in predicting survival in patients with matched ages and co-morbid conditions?

BACKGROUND: Computed tomography (CT) gives an idea about the prognosis in patients with COVID-19 lung infiltration. PURPOSE: To evaluate the success rates of various scoring methods utilized in order to predict survival periods, on the basis of the imaging findings of COVID-19. Another purpose, on the other hand, was to evaluate the agreements among the evaluating radiologists. MATERIAL AND METHODS: A total of 100 cases of known COVID-19 pneumonia, of which 50 were deceased and 50 were living, were included in the study. Pre-existing scoring systems, which were the Total Severity Score (TSS), Chest Computed Tomography Severity Score (CT-SS), and Total CT Score, were utilized, together with the Early Decision Severity Score (ED-SS), which was developed by our team, to evaluate the initial lung CT scans of the patients obtained at their initial admission to the hospital. The scans were evaluated retrospectively by two radiologists. Area under the curve (AUC) values were acquired for each scoring system, according to their performances in predicting survival times. RESULTS: The mean age of the patients was 61 ± 14.85 years (age range = 18-87 years). There was no difference in co-morbidities between the living and deceased patients. The survival predicted AUC values of ED-SS, CT-SS, TSS, and Total CT Score systems were 0.876, 0.823, 0.753, and 0.744, respectively. CONCLUSION: Algorithms based on lung infiltration patterns of COVID-19 may be utilized for both survival prediction and therapy planning.