How reliable is the distinction between thoracolumbar AO type A3 and A4 fractures? A systematic literature review.

PURPOSE: The AOSpine classification divides thoracolumbar burst fractures into A3 and A4 fractures; nevertheless, past research has found inconsistent interobserver reliability in detecting those two fracture patterns. This systematic analysis aims to synthesize data on the reliability of discriminating between A3 and A4 fractures. METHODS: We searched PubMed, Scopus, and the Web of Science for studies reporting the inter- and intra-observer reliability of detecting thoracolumbar AO A3 and A4 fractures using computed tomography (CT). The search spanned 2013 to 2023 and included both primarily reliability and observational comparative studies. We followed the PRISMA guidelines and used the modified COSMIN checklist to assess the studies' quality. Kappa coefficient (k) values were categorized according to Landis and Koch, from slight to excellent. RESULTS: Of the 396 identified studies, nine met the eligibility criteria; all were primarily reliability studies except one observational study. Interobserver k values for A3/A4 fractures varied widely among studies (0.19-86). The interobserver reliability was poor in two studies, fair in one study, moderate in four studies, and excellent in two studies. Only two studies reported intra-observer reliability, showing fair and excellent agreement. The included studies revealed significant heterogeneity in study design, sample size, and interpretation methods. CONCLUSION: Considerable variability exists in interobserver reliability for distinguishing A3 and A4 fractures from slight to excellent agreement. This variability might be attributed to methodological heterogeneity among studies, limitations of reliability analysis, or diagnostic pitfalls in differentiating between A3 and A4. Most observational studies comparing the outcome of A3 and A4 fractures do not report interobserver agreement, and this should be considered when interpreting their results.

Comparison and interobserver reliability between a visual analog scale and the Wisconsin Calf Health Scoring Chart for detection of respiratory disease in dairy calves.

Respiratory disease is an ongoing challenge for calves in the dairy sector with a relatively high prevalence and impact on welfare and economics. Applying scoring protocols for detecting respiratory disease requires that they are easily implemented, consistent between observers and fast to use in daily management. This study was conducted in one Danish dairy farm from September 2020 through January 2021. The study included 126 heifer calves enrolled in the age of 17 to 24 d. All calves were observed every second day for a period of 46 d. At each visit all calves were scored with a new visual analog scale (VAS) and the Wisconsin Calf Health Scoring Chart (WCHSC). We calculated agreement between the 2 scoring systems based on conditional probability to score higher or lower than a cutoff in the VAS compared with a specified cutoff in WCHSC used as reference test. A generalized mixed effects regression model was developed to estimate the prevalence of respiratory disease and the overall agreement between the 2 scoring systems. The overall agreement between the VAS and WCHSC was 89.6%. The second part of the study assessed interobserver reliability between 2 experienced observers and between an experienced observer and veterinary students. The interobserver reliability was calculated by intraclass correlation coefficient and was 0.58 between experienced observers and was 0.34 between an experienced observer and veterinary students indicating a moderate to poor reliability between the observers. It was possible to use VAS as an alternative clinical scoring method, which primarily focuses on the general condition of the individual calf rather than specific categories of clinical signs. Our study set up lacked a comparison to other diagnostic tools i.e., thoracic ultrasound to confirm the findings which should be considered in future studies when exploring VAS as a screening tool for detection of respiratory disease in dairy calves.

Reliability of Wrist Arthroscopy in the Diagnosis and Treatment of Triangular Fibrocartilage Complex Tears.

PURPOSE: Wrist arthroscopy is considered the gold standard for diagnosis of intra-articular wrist conditions including triangular fibrocartilage complex (TFCC) tears; however, the ability to reliably diagnose TFCC pathology during wrist arthroscopy is unclear. The purpose of this study was to assess the reliability of the diagnosis of TFCC tears on wrist arthroscopy videos. METHODS: Five hand surgeons reviewed 43 cases on a secure web-based platform at two time points separated by 4-6 weeks. Each case included a deidentified clinical vignette and wrist radiographs with ulnar variance measurements and an arthroscopy video of ulnar wrist pathology, averaging 42 seconds. Surgeons were queried on the presence of TFCC tear and Palmer and Atzei classifications. Interobserver/intraobserver reliabilities were determined using kappa coefficients. RESULTS: The interobserver reliability regarding the presence/absence of TFCC tear was fair-good, with kappa coefficients of 0.596 in the first round of case review and 0.708 in the second round. The overall intraobserver reliability for the presence/absence of TFCC tear was also fair-good, with a kappa coefficient of 0.567. For cases with central TFCC tears, the interobserver reliability regarding the presence of TFCC tear was perfect, with a kappa coefficient of 1.0. When central tears were excluded, the kappa coefficients decreased to 0.322 and 0.368 in each round. The interobserver reliability for the Palmer and Atzei classifications was low, with kappa coefficients of 0.220 and 0.121 in the first round and 0.222 and 0.123 in the second round. CONCLUSIONS: Experienced wrist arthroscopy surgeons demonstrated fair-good interobserver reliability for the diagnosis of TFCC tear on wrist arthroscopy. However, when central TFCC tears were excluded, interobserver reliability was poor. These findings demonstrate a need for a focus on education for wrist arthroscopy anatomy. CLINICAL RELEVANCE: This investigation demonstrated poor agreement in surgeon identification and classification of TFCC tears, aside from central TFCC tears. The low reliability has a direct bearing on treatment considerations for TFCC tears.

Reliability of the Clinical Measurement of Upper and Lower Extremity Joint Motion in the Pediatric Population: A Systematic Review.

AIM: To examine intraobserver and interobserver reliability of the clinical measurement of pediatric joint motion of upper and lower extremities, based on participant condition and measurement technique. METHODS: PubMed, CINAHL, and Web of Science were searched using combinations of children or adolescents, range of motion, and reliability. Reference lists and citations of reviewed studies were searched for additional publications. RESULTS: Thirty-one studies of pediatric samples of developing typically, orthopedic injuries, athletes, cerebral palsy, and other diagnoses were reviewed. For techniques, measurements were made most frequently with a goniometer followed by visual estimation, inclinometer, smartphone apps, and specialized devices. The reliability of hip abduction measurements of participants with cerebral palsy was evaluated most often and varied widely for both intraobserver and interobserver. In general, goniometric results indicated greater reliability for upper than lower extremities and for intraobserver than interobserver. As the other techniques were each utilized in only a few studies, involving different participant conditions, joint motions and statistics, the analysis of their reliability was limited. CONCLUSIONS: Intraobserver and interobserver reliability have not been established for pediatric joint motion measurements. Further research should include various joint motion measurements for different pediatric conditions using appropriate statistics. Results would provide important information for making clinical decisions.

Classification of intraarticular calcaneal fractures: comparison of two classification systems.

PURPOSE: Accurately classifying displaced intraarticular calcaneal fractures (DIACFs) is essential for orthopedic surgeons to choose optimal treatment methods and provide results evaluation and communication. Many authors studying used Sanders classification reported moderate intra- and interobserver reliability. Taking the software opportunity of 3D virtual exarticulation, Goldzak updated French tri-dimensional Utheza classification, providing an alternative framework for classifying DIACFs. The aim of this study was to compare the intra- and interobserver reliability of Sanders versus Goldzak classification systems. METHODS: The CT scans of 30 patients with displaced intraarticular calcaneal fractures, treated in the same trauma center between 2014-2018, were analyzed by 16 medical doctors (specialists and residents in orthopedic surgery, specialists and residents in radiology), and classified according to Sanders and Goldzak classifications. The same images were sent on two separate sessions, in a randomized order. Interobserver reliability and intraobserver reproducibility were assessed using Kappa statistics and Gwet's AC1 coefficient. RESULTS: Interobserver reliability using Gwet reported a value of 0.36 for Goldzak classification and 0.30 for Sanders classification (corresponding to "fair assessment" in both cases). In absence of subclasses, "substantial assessment" was reported for Goldzak classification (Gwet of 0.61) and "moderate assessment" for Sanders classification (Gwet of 0.46). Goldzak system had a greater interobserver reliability in the group of radiology residents. Intraobserver reliability coefficient was 0.60 for Goldzak classification and 0.69 for Sanders classification, indicating a substantial agreement for both classifications. CONCLUSION: Despite the better view of the fracture lines provided by 3D reconstructions, this study failed to prove the superiority of Goldzak classification compared to Sanders classification for DIACFs.

Exploration of Caregiver Interrater Agreement and Test-Retest Reliability on the Infant Cleft Observer Outcomes (iCOO).

Caregiver and observer-reported measures are frequently used as outcomes for research on infants and young children who are unable to report on their own health. Our team developed the Infant with Clefts Observation Outcomes Instrument (iCOO) for infants with cleft lip with or without cleft palate. This exploratory study compared test-retest and interrater reliabilities to inform whether differences in caregiver perspective might affect the iCOO.This study is a secondary analysis comparing caregiver interrater agreement to test-retest reliability. Twenty-five pairs of caregivers completed the iCOO before surgery, 1 week later for test-retest reliability, 2 days after surgery, and 2 months after surgery. Reliability was assessed using intraclass correlations (ICCs) and t-tests were used to compare ratings between caregivers.Infants had cleft lip (28%) or cleft lip and palate (72%). Primary caregivers were predominantly mothers (92%) and secondary caregivers were predominantly fathers (80%). Test-retest reliability met psychometric standards for most items on the iCOO (81%-86% of items). Caregiver agreement on the iCOO items was lower than test-retest reliability (33%-46% of items met psychometric standards). Caregivers did not systematically differ in whether they rated infants as healthier or less healthy than the other caregiver (5%-16% of items had statistically significant differences).Caregivers used the measure consistently, but had different experiences and perceptions of their infant's health and functioning. Future studies are needed to explore mechanisms for the differences in test-retest and interrater reliability. Whenever possible, the same caregiver should provide ratings of the infant, including on the iCOO.

Fracture displacement of lateral compression type 1 (LC1) Pelvic Ring Injuries: Which measurement methods are reliable and does displacement correlate with adverse events?

PURPOSE: To determine the interobserver reliability of perceived displacement (PD) ≥ 1 cm of lateral compression type 1 (LC1) pelvic ring fractures and to determine if PD correlated with published methods of measuring fracture displacement and adverse events. METHODS: The injury and follow-up radiographs of 10 patients with nonoperative minimally displaced LC1 injuries were reviewed by 27 orthopaedic trauma surgeons to determine if they perceived an interval fracture displacement ≥ 1 cm. Perceived displacement (PD) was compared to 11 measurements of displacement: the difference in heights of iliac crests (IC), sourcils, and ischial tuberosities on AP/outlet radiographs; the distance from femoral head (FH) to the sacral midline and the distance between the radiographic teardrops (TD) on the AP; the distance from the TD to sacral midline and the difference in distances between the SI joints and the contralateral TD on the inlet. The interobserver reliability and correlation of PD and measured displacement was calculated. The association between PD and adverse events was also evaluated. RESULTS: PD had weak interobserver reliability (kappa = 0.46). Many of the measurements of displacement were poorly sensitive for PD. The magnification-corrected AP TD measurement had the highest sensitivity (100%) for PD and excellent interobserver reliability (Intraclass correlation 0.97), but had a low specificity (57.1%). All three patients with PD had adverse events while patients without PD had uneventful recoveries (p = 0.008). CONCLUSIONS: The AP TD measurement detected all cases of PD and had excellent reliability. PD was associated with adverse events, suggesting that accurate and reliable measurements of displacement are warranted. LEVEL OF EVIDENCE III: Diagnostic.

Interobserver reliability of Masei index validation by a multicenter collaborative group of rheumatologists.

MASEI is the main validated ultrasound score for the evaluation of enthesis. The lack of studies facing the agreement to achieve for the interpretation of the MAdrid Sonographic Enthesis Index (MASEI) among researchers from different centers in multicenter studies is of concern. The aim of this multicenter was to evaluate the interobserver reliability of MASEI. An experienced ultrasonographer-rheumatologist performed ultrasound scans of the areas included in MASEI index in three patients with Ankylosing Spondylitis and Psoriatic Arthritis. Videos were captured. The videos were then evaluated by 24 rheumatologists of the ultrasound working group of the Catalan Society of Rheumatology (EcoCAT). A face-to-face training meeting was held. Ten days after the workshop, the study participants evaluated the videos. A reliability assessment was performed. The ICC for the MASEI scores after the workshop was of 0.97 (95% CI 89-99). Reliability did not vary statistically with examiner experience. Globally, no problems of reliability by structures were seen, and all the ICCs were above 0.90 and improved slightly after the educational program. However, the correlation observed between examiners at plantar aponeursis and triceps tendon was weak. The small variability observed in the results of the index validation in our study, suggests that the MASEI index is reproducible by different observers when those are well trained and show awesome results of the enthesis when examined by ultrasound.

Validity and interobserver agreement of a new radiographic grading system for intervertebral disc degeneration: Part III. Thoracic spine.

PURPOSE: The aim of this study was to assess the validity and objectivity of a new quantitative radiographic grading system for thoracic intervertebral disc degeneration. METHODS: The new grading system involves the measurement variables "Height loss" and "Osteophyte formation", which are determined from lateral radiographs, resulting in the "Overall degree of degeneration" on a four-point scale from 0 (no degeneration) to 3 (severe degeneration). Validation was performed by comparing the radiographic degrees of degeneration of 54 human intervertebral discs to the respective macroscopic degrees, which were defined as the "real" degrees of degeneration. Interobserver agreement was examined using radiographs of 135 human thoracic intervertebral discs. Agreement was quantified by means of quadratically weighted Kappa coefficients with 95% confidence limits (CL). RESULTS: Validation revealed almost perfect agreement between the radiographic and the macroscopic overall degrees of degeneration (Kappa 0.968, CL 0.944-0.991), while the macroscopic grades tended to be underestimated in low degeneration grades. Radiographic grading of two independent observers also exhibited almost perfect agreement (Kappa 0.883, CL 0.824-0.941) as well as tendencies towards rater-dependent differences in low degeneration grades. CONCLUSION: The new quantitative radiographic grading scheme represents a valid, reliable, and almost objective method for assessing the degree of degeneration of individual thoracic intervertebral discs. Potential effects of interindividual variations and the radiographic superimposition of anatomical structures represent a limitation of this method should be taken into account when using the grading system for clinical and experimental purposes, especially with regard to specific morphological as well as patient- and donor-specific characteristics.

Inter- and intraobserver reliabilities and critical analysis of the osteoporotic fracture classification of osteoporotic vertebral body fractures.

PURPOSE: The Osteoporotic Fracture Working Group (Spine Division of the German Orthopaedic and Trauma Society) has developed a classification system for osteoporotic thoracolumbar fractures, namely the osteoporotic fracture (OF) classification system. The purpose of this study was to determine the inter- and intraobserver reliabilities of the OF classification system for osteoporotic vertebral body fractures (VFs) at a level-one trauma centre. METHODS: Conventional radiography, magnetic resonance imaging (MRI), and computed tomography (CT) scans of 54 consecutive women who sustained an osteoporotic VF were analysed by six orthopaedic traumatologists with varying levels of experience. The inter- and intraobserver reliabilities of the OF classification system were determined using intraclass correlation coefficients (ICCs) and Cohen's kappa. RESULTS: The overall interobserver reliability of the OF classification system was good (ICC, 0.62 [0.51, 0.72]). The intraobserver reliability was found to be substantial (overall weighted Cohen's kappa estimate [95% confidence interval {CI}] = 0.74 [0.67, 0.80]) and better when the radiography, MRI, and CT scans were assessed together than when only the radiography and MRI scans were evaluated, although the difference was not significant. CONCLUSION: The OF classification system is easy to use. It shows good interobserver reliability and substantial intraobserver reliability if diagnostic prerequisites (conventional radiography, MRI, and CT scans) are met.

Inter-operator and inter-device reproducibility of shear wave elastography in healthy muscle tissues.

PURPOSE: The study aimed to assess whether the more limiting factor in reproducibility of shear wave elastography (SWE) would be the operator dependency or the incompatibility of different ultrasound (US) devices. The interrater agreement with less experienced operators was studied. METHODS: A total of 24 healthy volunteers participated in the study (18 females, 6 males; range of age 27-55 years). SWE of biceps brachii (BB) and tibialis anterior (TA) muscles was performed on both sides from all participants in both longitudinal and transverse orientation of the transducer in respect to muscle fibers. Two operators repeated the SWE with two different US devices from different manufacturers (scanners 1 and 2). RESULTS: Intraclass correlation coefficient between the two operators was 0.91 (CI 0.88-0.93) for scanner 1 and 0.81 (CI 0.74-0.86) for scanner 2, respectively. Instead, there were significant differences in the SWE measurements between the two scanners, emphasizing in transverse orientation of the transducer. In the transverse transducer orientation, the mean shear wave velocity (SWV) in TA was 1.45 m/s (standard deviation [SD] ± 0.35 m/s) with scanner 1 and 2.35 m/s (SD ± 0.83 m/s) with scanner 2 (p < 0.001). In BB, the mean transverse SWV was 1.49 m/s (SD ± 0.35 m/s) with scanner 1 and 2.29 m/s (SD ± 0.63 m/s) with scanner 2 (p < 0.001). In longitudinal transducer orientation, the mean SWV in TA was 3.00 m/s (SD ± 0.73 m/s) with scanner 1 and 3.26 m/s (SD ± 0.42 m/s) with scanner 2 (p = 0.050). In BB, the mean longitudinal SWV was 3.60 m/s (SD ± 0.77 m/s) with scanner 1 and 3.96 m/s (SD ± 0.62 m/s) with scanner 2 (p = 0.019). The presented mean values were obtained by operator 1, there were no significant differences in the SWE measurements performed by the two operators. CONCLUSION: The results implicate that the reproducibility of the SWE measurements depends rather on the used US device than on the operator. It is recommendable that clinics collect reference values with their own US device and consider threshold values presented in previous studies only directional.

Standard radiographic assessments of distal radius fractures miss involvement of the distal radioulnar joint: a diagnostic study.

INTRODUCTION: Distal radius fractures account for one-fifth of all fractures in the emergency department. Their classification based on standard radiographs is common practice although low inter-observer reliabilities and superiority of computer tomography (CT) scanning in evaluation of joint congruency have been reported. MATERIALS AND METHODS: We retrospectively analyzed 96 displaced distal radius fractures scheduled for open reduction and internal fixation using standard radiographic assessment. The radiographs were classified with the Arbeitsgemeinschaft für Osteosynthesefragen/Orthopaedic Trauma Association (AO/OTA), Fernandez and Frykman classifications by three observers and inter-rater reliabilities were calculated. Additional CT scanning was performed in all cases and the following parameters were assessed: radiocarpal joint involvement, fracture extent into the radial sigmoid notch, i.e. the distal radio-ulnar joint, comminution of the metaphysis, and concomitant ulnar styloid fracture. The CT scans were used as a reference standard to determine sensitivity and accuracy of standard radiographic assessment in evaluation of distal radius fractures. RESULTS: The inter-rater agreement for the AO classification was 35.4%, 68.8% for the Fernandez and 38.5% for the Frykman classification. Fracture extension into the radiocarpal joint was present in 81 cases (84.4%). Sigmoid notch involvement was found in 81 fractures (84.4%). Involvement of both joints was present in 72 cases (75%). The sensitivity of standard radiographs regarding radiocarpal joint involvement was 93.8%. Considering involvement of the distal radio-ulnar joint the false-negative rate using standard radiographs was 61.7% and the test's accuracy for sigmoid notch involvement was 45.8%. CONCLUSION: This study demonstrates that involvement of the sigmoid notch is frequently missed in standard radiographs. The presented data support the frequent use of CT imaging to allow the holistic illustration of a fracture's complexion and to ensure optimal pre-operative planning.

CT-Based Classification Systems for Intra-Articular Calcaneal Fractures: The Inter- and Intraobserver Variations as well as Integrality.

The aim of this study was to measure the inter- and intraobserver variations as well as integrality of the Zwipp, Crosby-Fitzgibbons, Sanders, and Eastwood-Atkins classification systems based on more accurate CT scans. Five hundred and forty-nine patients with intra-articular calcaneal fractures from January 2018 to December 2019 taken from a database in our level-I trauma center (3 affiliated hospitals) were included. For each case, normative CT (1 mm slices) scans were available. Four different observers reviewed all CT scans 2 times according to these 4 most prevalent fracture classification systems (FCSs) within a 2-month interval. For these 4 FCSs, the kappa [κ] coefficient was used to evaluate interobserver reliability and intraobserver reproducibility, and the percentage that can be classified was used to indicate integrality. The κ values were measured for Zwipp (κ = 0.38 interobserver, κ = 0.61 intraobserver), Crosby-Fitzgibbons (κ = 0.48 interobserver, κ = 0.79 intraobserver), Sanders (κ = 0.40 interobserver, κ = 0.57 intraobserver), and Eastwood-Atkins (κ = 0.44 interobserver, κ = 0.72 intraobserver). Furthermore, the integralities were calculated for Zwipp (100%), Crosby-Fitzgibbons (100%), Sanders (92%) as well as Eastwood-Atkins (89.6%). Compared with previous literatures, CT scanning with higher accuracy can significantly improve intraobserver reproducibility of Zwipp and Eastwood-Atkins FCSs, but it has no positive effect on variability of Sanders FCS and interobserver reliability of Crosby-Fitzgibbons FCS. In terms of integrality, Zwipp and Crosby-Fitzgibbons FCSs appear to be superior to the other 2 FCSs.

Three-dimensional classification of the Lenke 1 adolescent idiopathic scoliosis using coronal and lateral spinal radiographs.

BACKGROUND: Classification of the spinal deformity in adolescent idiopathic scoliosis (AIS) remains two-dimensional (2D) as the spinal radiographs remain the mainstay in clinical evaluation of the disease. 3D classification systems are proposed, however are time consuming. We here aim to evaluate the clinical application of a 3D classification system by the use of only posterior-anterior and lateral radiographs in Lenke 1 adolescent idiopathic scoliosis (AIS). METHODS: Forty Lenke 1 AIS were classified by five observers following a three-step flowchart, developed based on our previous 3D classification system. This 3D classification characterizes the curve in the frontal and sagittal views and infers the third dimension with rules based on prior data to determine the 3D subtypes of the curve. Repeated rating was performed for 20 randomly selected patients in the same cohort. In addition to the classification by the raters, the 3D model of the spines were generated to determine the actual curve subtype based on the algorithm that was originally used to develop the 3D classification system. The interobserver and intraobserver reliability and the classification accuracy were determined for both 3D and axial classifications of the cohort. RESULTS: The interobserver reliability was moderate to strong with a kappa value between 0.61-0.89 for 3D and axial classifications. Comparing the mathematical classification and the raters' classification, the classification accuracy among all raters ranged between 56 and 89%. CONCLUSION: We evaluated the reliability of a previously developed 3D classification system for Lenke 1 AIS patients when only two-view spinal radiographs are available. Radiologists and orthopedic surgeons were able to identify the 3D subtypes of Lenke 1 AIS from the patients' radiographs with moderate to strong reliability. The new 3D classification has the potential to identify the subtypes of the Lenke 1 AIS without a need for quantitative 3D image post-processing.

Comparison of best-fit circle versus contralateral comparison methods to quantify glenoid bone defect.

BACKGROUND: Several measurement techniques have been reported to quantify glenoid bone defect in patients with anterior shoulder instability. Among them, the method that uses a best-fit circle and another that uses the contralateral glenoid as a control are most commonly used. However, to our knowledge, no study has been reported that compared the reliability of these methods. The purpose of this study, therefore, was to determine which of these methods has higher reproducibility. METHOD: In this study, 3-dimensional computed tomography data from 94 patients (mean age 29 years) with unilateral anterior shoulder instability were used. Three examiners measured the glenoid bone defect of each patient 3 times using 2 techniques: the best-fit circle method and the contralateral comparison method. Intra- and interobserver reliabilities were measured using intraclass correlation coefficient (ICC). RESULTS: The intraobserver reliability was found to be 0.91 for the best-fit circle method and 0.98 for the contralateral comparison method. The interobserver reliability was 0.77 for the best-fit circle method and 0.88 for the contralateral method. The percentage of glenoid defect was 11.5% when using the best-fit circle and 10.7% with the contralateral method. CONCLUSION: The contralateral comparison method was more reliable than the best-fit circle method for quantifying the amount of glenoid bone loss.

Intraclass Correlations of Measured Magnetic Resonance Imaging Volumes of Laser Interstitial Thermal Therapy-Treated High-Grade Gliomas.

BACKGROUND AND OBJECTIVES: Laser interstitial thermal therapy (LITT) is a minimally invasive therapeutic option for the treatment of brain tumors. Previous studies have quantitatively followed the ablated volumes of high-grade gliomas. Reported treatment volumes range from 28% to 100%, with no reported interobserver analysis. Because these volumes are subjectively measured, it is necessary to establish concordance between clinicians. STUDY DESIGN/MATERIALS AND METHODS: Utilizing Brainlab tumor analysis software (Brainlab, Munich, Germany), five physician users traced out tumor volumes slice-by-slice on 10 treated tumors in eight patients. The participants were briefed with specific instructions and a demonstration on how to trace the enhancing borders of the tumor slice-by-slice. Volumes automatically calculated by the Brainlab software included preoperative, intraoperative ablation and postoperative enhancing volumes. Data regarding size, cystic appearance, pathology, previous surgery, and demographics were included. RESULTS: The intraclass correlation coefficient (ICC) for preoperative, intraoperative, and postoperative volumes was 0.92 (95% confidence interval, [CI] 0.81-0.97), 0.90 (0.77-0.96), and 0.89 (0.74-0.96), respectively. The overall ICC was 0.72 (0.50-0.87). ICC comparisons were also made for each pair of readers (neuroradiologist, neuro-oncologist, senior neurosurgery resident, neurosurgery junior resident) which resulted in pretreatment ICC scores of 0.97, 0.91, 0.66, 0.94; intratreatment scores of 0.97, 0.78, 0.90, 0.96; and posttreatment scores of 0.96, 0.81, 0.89, and 0.87. A Bland-Altman plot was also used to assess the differences in volumes. CONCLUSIONS: The ICC gives a composite of the consistency of measurements made by multiple observers measuring the same quantity. The overall ICC of 0.72 means there is good correlation between observers in our study between measured volumes. Lasers Surg. Med. © 2019 Wiley Periodicals, Inc.

Differences in Two- and Three-Dimensional Assessment of the Mitral Valve by Novices and Experts, Illustrated Using Anterior Mitral Valve Leaflet Length.

OBJECTIVES: In this measurement validation study, the authors evaluated agreement between 2-dimensional (2D) and three-dimensional (3D) transesophageal echocardiography (TEE), measuring anterior mitral valve leaflet length by both novice and experienced echocardiographers. DESIGN: This was a retrospective, observational study. SETTING: Single university hospital. PARTICIPANTS: Analyses on datasets from 44 patients. INTERVENTIONS: None. MEASUREMENTS AND MAIN RESULTS: Fifty datasets from 44 patients with mitral regurgitation were analyzed by 4 observers (2 novices, 2 experts). All observers measured the anterior mitral valve leaflet length from end-systolic 2D TEE images from the midesophageal longitudinal axis view and 3D software-augmented TEE images. The overall mean anterior mitral valve leaflet length was significantly shorter with 3D versus 2D TEE measurements (24.6 ± 4.5 mm v 26.2 ± 5.3 mm; p < 0.001), with novices measuring shorter leaflets than experts for both techniques (p < 0.001 and p = 0.005, respectively). Bland-Altman plots of 3D and 2D TEE measurements showed mean biases (95% limits of agreement) of -1.6 mm (-9.0 to 5.9 mm), -1.8 mm (-9.6 to 6.0 mm), and -1.3 mm (-8.4 to 5.7 mm) for all observers, novices, and experts, respectively. For 2D measurements, interobserver reliability was very strong among experts and strong among novices (Pearson's r = 0.83 v 0.66; p = 0.055). For 3D measurements, interobserver reliability was strong in experts and moderate in novices (Pearson's r = 0.69 v 0.51; p = 0.168). CONCLUSION: For both novices and experts, 3D TEE measurements of the anterior mitral valve leaflet were significantly shorter than 2D measurements. Interobserver reliability was lowest for novices making 3D TEE measurements, indicating that reliable, quantitative evaluation of 3D TEE may require a greater amount of practice.

Tumor budding in colorectal carcinoma: An institutional interobserver reliability and prognostic study of colorectal adenocarcinoma cases.

BACKGROUND: Colorectal carcinomas are one of the most commonly diagnosed malignancies. There are many prognostic factors relating to clinical course and disease progression, including tumor stage, metastasis, and tumor budding. In 2016, the International Tumor Budding Consensus Conference (ITBCC) created a system to uniformly assess tumor budding. This system includes a 3-tier system for the grading of tumor budding. In the past, there lacked uniform consensus, however the general grading practice was based on a 2-tiered system. Given that tumor budding is considered to have prognostic value, the accuracy and reproducibility of its assessment is vital. Our study aims to look at interobserver agreement in the scoring of tumor budding. DESIGN: A total of 233 cases of colorectal carcinoma diagnosed in our health system were retrospectively analyzed and routine H&E stained slides of these cases were collected. A representative slide for tumor budding was selected per case. Four investigators with different levels of experience and expertise evaluated the selected slide of each case for tumor budding. Scoring was based on the ITBCC protocol. Clinico-pathological data was collected for each case and analyzed with tumor budding scores. Tumor budding scores per individual investigator and consensus tumor budding score were compared to patient and tumor characteristics including patient survival, tumor grade, tumor stage, and lymph node status. RESULTS: Inter-observer agreement was calculated using Gwet's Agreement Coefficient (AC1) and associated 95% confidence intervals was used to compare the ratings made by 4 pathologists. Overall, there was variation among pathologists in tumor budding score (Gwet's agreement coefficient = 0.25 and 0.326 for 3-tier and 2-tier grading system, respectively). Results show higher reliability with the 2-tier system compared to the 3-tier system. Tumor stage was significantly associated with budding score for all individual investigators and the consensus value (p value < 0.001). CONCLUSION: There is low inter-observer agreement in the assessment of tumor budding in colorectal carcinoma. This suggests that it is difficult to uniformly grade tumor budding and that our classification system needs improvement. We found that the older 2-tier system (Hase et al.) results in slightly higher inter-observer agreement than the recently proposed 3-tier grading system (ITBCC, 2016), though both systems lead to suboptimal agreement. Worth noting is that observers with subspecialty GI training and more work experience had higher inter-observer agreement. Our results showed that subspecialty training tends to increase agreement more than overall work experience. In addition, our exploratory results showed that there is an association of tumor budding score to tumor stage. While increasing refinement in classification, the 3-tiered system resulted in decreased agreement in tumor budding assessment. Clearly, there is more work to be done in the identification and quantification of tumor buds.

The intra- and interobserver reliability of the Tile AO, the Young and Burgess, and FFP classifications in pelvic trauma.

INTRODUCTION: Several different systems of classification have been developed to understand the complexity of pelvic ring fractures, to facilitate communication between physicians and to support the selection of appropriate therapeutic measures. The purpose of this study was to measure the inter- and intraobserver reliability of Tile AO, Young and Burgess, and FFP classification in pelvic ring fractures. The Rommens classification system (FFP) is analyzed for the first time. MATERIALS AND METHODS: Four reviewers (2 × senior pelvic trauma surgeon, 1 × resident, 1 × medical student) separately analyzed and classified 154 CT scans of patients with pelvic fracture. The Tile AO, the Young and Burgess, and the FFP classifications (subgroup with patients ≥ 60 years) were compared. Another blinded re-evaluation was carried out after 2 months to determine intraobserver reliability. RESULTS: The overall interobserver agreement was fair for all classification systems (ICC: OTA 0.55, Young and Burgess 0.42, FFP 0.54). For specific categories, (e.g. type B or C fractures), there was a substantial agreement between the experienced surgeons (kappa: OTA 0.64, Young and Burgess 0.62, FFP 0.68). For inexperienced observers, there was a fair agreement in all systems (kappa: OTA 0.23, Young and Burgess 0.23, FFP 0.36). CONCLUSIONS: All three classifications reach their maximum reliability with advanced expertise in the surgery of pelvic fractures. The novel FFP classification has proved to be at least equivalent when directly compared to the established systems. The FFP classification system showed substantial reliability in patients older than 60 years.

Computed tomography in ventral hernia diagnosis: have we improved? A quality improvement initiative.

BACKGROUND: Previous studies suggest that agreement between readers of computed tomography (CT) scans for the diagnosis of a ventral hernia (VH) is poor (32% agreement, κ = 0.21). Recommendations were developed by surgeons and radiologists after determining common reasons for disagreement among CT reviewers; however, the long-term effect of adoption of these recommendations has not been assessed. The aim of this quality improvement (QI) project was to determine whether the incorporation of recommendations developed by surgeons and radiologists improves agreement among reviewers of CT scans in diagnosing a VH. METHODS: A prospective cohort of patients, with a CT scan of the abdomen and pelvis in the past 1 y, attending a surgery clinic at a single institution was enrolled. Enrolled subjects underwent a standardized physical examination by a trained hernia surgeon to determine the likelihood of a clinical VH (no, indeterminate, or yes). The QI intervention was the distribution and implementation of previously described recommendations. After a year of intervention, independent radiologists assessed patients' CT scans for the presence or absence of a VH. Percent agreement and kappa were calculated to determine interobserver reliability. In-person discussion on scans with disagreement was held, and the results were used as a "gold standard" to calculate sensitivity, specificity, positive, and negative predictive values for CT scan diagnosis of a VH. RESULTS: A total of 79 patients were included in the study. After QI intervention, seven radiologists agreed on 43% of the scans, and κ was 0.50 (P < 0.001). Agreement was highest among patients with a high clinical likelihood of a VH and lowest among patients with an indeterminate clinical likelihood. Sensitivity and specificity were 0.369 and 0.833, respectively. CONCLUSIONS: After the implementation of recommendations, there is improved agreement among radiologists reading CT scans for the diagnosis of a VH. However, there is substantial room for improvement, and CT scans for the diagnosis of VH is not ready for widespread use.