Interobserver variability in the histopathological evaluation of melanoma: analysis of 60 case reports.

INTRODUCTION: Since the field of dermatopathology is not an exact science, it is prone to personal subjectivity, which sometimes causes disagreements on the diagnosis and assessment of some histological features. In the case of melanoma, some variables such as regression are associated with low interobserver agreement. On the contrary, other variables such as the measurement of Breslow thickness show high reproducibility. OBJECTIVE: The main objective of our study was to investigate multiple features of 60 consecutive cases of melanoma to establish interobserver reproducibility. METHODS AND MAIN RESULTS: We conducted an observational and descriptive study at Hospital de Manises, Valencia, Spain, IVO Foundation, Valencia, Spain, and Hospital 12 de Octubre, Madrid, Spain. The mean level of agreement of all study variables was moderate (Cohen's kappa coefficient statistic = 0.5). The highest agreement corresponded to polypoid morphology, pigmentation, ulceration, and solar elastosis. On the other hand, the lowest level agreement was reached for the presence of cellular pleomorphism and tumor necrosis. CONCLUSIONS: Our mean level of agreement was moderate, which reflects that some of the measured characteristics such as cellular pleomorphism or the presence of necrosis cannot be used for future studies or must be redefined and their reproducibility, reestablished. When conducting a research study, it is necessary to analyze the study variables to demonstrate their validity to measure or classify a certain feature. It is also advisable to warrant that that the variables are reproducible to be able to use them for other studies or in the routine clinical practice.

Procedural Fidelity Reporting in The Analysis of Verbal Behavior from 2007-2021.

Procedural fidelity refers to the degree to which procedures for an assessment or intervention (i.e., independent variables) are implemented consistent with the prescribed protocols. Procedural fidelity is an important factor in demonstrating the internal validity of an experiment and clinical treatments. Previous reviews evaluating the inclusion of procedural fidelity in published empirical articles demonstrated underreporting of procedural fidelity procedures and measures within specific journals. We conducted a systematic review of The Analysis of Verbal Behavior (TAVB) to evaluate the trends in procedural fidelity reporting from 2007 to 2021. Of the 253 articles published in TAVB during the reporting period, 144 of the articles (168 studies) met inclusionary criteria for further analysis. Our results showed that 54% of studies reported procedural fidelity data, which is slightly higher than previous reviews. In comparison, interobserver-agreement data were reported for a high percentage of studies reviewed (i.e., 93%). Further discussion of results and applied research implications are included.

Interobserver agreement and diagnostic challenges of Congo red staining for amyloid detection on fat pad aspiration biopsies.

INTRODUCTION: Congo red staining of fat pad fine needle aspiration specimens is a method utilized for evaluation of amyloid deposition. However, these specimens can pose diagnostic challenges for cytopathologists. As part of ongoing internal quality improvement measures, the objective of this study was to evaluate the intradepartmental interobserver agreement of these specimens and to identify factors that affect the variability of the interpretations. MATERIALS AND METHODS: There were 7 participants, which included 3 trainees, 3 cytopathologists, and 1 cytotechnologist. Each participant reviewed 50 Congo red stained fat pad fine needle aspiration slides. The interpretations were categorized into 3 groups: negative, indeterminate/suspicious, and positive. The participants also noted any interpretation challenges they encountered for each case. RESULTS: There was only slight interobserver agreement among all participants (κ = 0.133). Stratified by participant group, the interobserver agreement among the trainees was slight bordering on poor (κ = 0.028) and among cytopathologists was fair (κ = 0.249). The highest agreement between 2 observers was between 2 cytopathologists and the level of agreement was moderate bordering on fair (κ = 0.426). There were only 3 cases (6.0%) with full agreement among observers, while in 25 cases (50.0%), there were 2 category differences in interpretations. The primary diagnostic challenge reported by participants was when weak or focal birefringence was encountered as well as cases complicated by poor stain quality and overstaining. CONCLUSIONS: We found only slight interobserver agreement among all study participants. A major area of challenge was cases with weak birefringence resulting in high variance of interpretation among participants.

Verifying a novel bile cytology scoring system.

BACKGROUND: The scoring system for bile cytology (SSBC) aims to improve bile cytology diagnostic accuracy. Here, the practicality of SSBC was verified by multiple cytotechnologists. METHODS: Bile cytological specimens were evaluated by 24 cytotechnologists using SSBC. The samples were assessed before using the SSBC (first-time assessment) according to three categories: benign, indeterminate, and malignant. A first scoring evaluation (FSE) was then performed using SSBC; each item in the scoring system was classified as present or absent. After distributing an instruction sheet with diagnostic criteria, a second scoring evaluation (SSE) was performed using SSBC. Each method was evaluated using diagnostic accuracy and interobserver and intraobserver agreement. RESULTS: Several samples were assessed as indeterminate in the first-time assessment. Although the specificity of the SSE improved, the sensitivity and accuracy decreased compared with those of the FSE. The overall interobserver agreement was fair for all parameters, including abnormal chromatin, irregular internuclear distances, irregularly overlapped nuclei, irregular cluster margins, and final evaluation in the FSE and SSE. The final evaluation by histological type exhibited slight agreement for well-differentiated tubular adenocarcinoma and almost perfect agreement for poorly differentiated tubular adenocarcinoma in the FSE and SSE. For moderately differentiated tubular adenocarcinoma, agreement was moderate in the FSE and fair in the SSE. For cholangitis, a slight agreement was observed in the FSE, which improved to fair in the SSE. CONCLUSIONS: Although the SSBC is expected to improve specificity, there exists ambiguity regarding SSBC criteria and interindividual assessment differences. Therefore, the objective assessment method should be revised.

World Health Organization Reporting System for Soft Tissue Cytopathology: Risk of malignancy and reproducibility of categories among observers.

INTRODUCTION: In 2024, the World Health Organization (WHO) is scheduled to publish the WHO Reporting System for Soft Tissue Cytopathology (WHORSSTC). This system establishes categories with well-defined definitions, criteria, and estimated risks of malignancy (ROMs) for soft tissue tumors. The estimates of ROM are based on a relatively small number of published studies. Interobserver reproducibility is not addressed in the reporting system even though reproducibility of a reporting system is highly important. METHODS: A manual search of one authors personal consultation files and teaching set (L.J.L.) was conducted for all cytologic specimens of soft tissue tumors accessioned between January 1, 1985 and December 31, 2022. Only cases with documented surgical pathology follow-up were included in the study. Slides from each case were evaluated independently by three cytopathologists with each case assigned to one of the WHORSSTC categories. A ROM for each of the WHORSSTC categories was calculated. Interobserver agreement was evaluated by the kappa and weighted kappa statistics. RESULTS: Risk for malignancy by category were: Category 1: 0%, Category 2: 28%, Category 3: 57%, Category 4: 47%, Category 5: 63%, and Category 6: 88%. Kappa statistics for agreement between raters varied from 0.2183 to 0.3465 and weighted kappa varied from 0.3778 to 0.5217. CONCLUSIONS: The WHORSSTC showed a progression of malignancy risk from the category "benign" (28%) to the category "malignant" (88%). Interobserver agreement was only fair.

Diagnostic interobserver agreement for thyroid fine-needle aspirates: Effects of reviewer experience and molecular diagnostics.

OBJECTIVES: Few cytologically indeterminate thyroid fine-needle aspirations (FNAs) harbor BRAF V600E. Here, we assess interobserver agreement for The Bethesda System for Reporting Thyroid Cytopathology (TBSRTC) category III (atypia of undetermined significance [AUS]) FNAs harboring BRAF V600E and contrast their features with those harboring non-BRAF V600E alterations, with attention to cytopathology experience. METHODS: Seven reviewers evaluated 5 AUS thyroid FNAs harboring BRAF V600E. To blind reviewers, cases were intermixed with 19 FNAs falling within other TBSRTC categories and in which genetic alterations other than BRAF V600E had been identified (24 FNAs total). Interobserver agreement against both "index" and most popular ("mode") diagnoses was calculated. Four additional BRAF V600E cases were independently reviewed. RESULTS: Reviewers included 3 trainees and 3 American Board of Pathology (board)-certified cytopathologists. Board-certified cytopathologists, whose experience ranged from 2 to more than 15 subspecialty practice years, had known AUS rates. BRAF V600E was identified in 5 of 260 (2%) AUS FNAs. Interobserver agreement was higher among cytopathologists with more experience. Mode diagnosis differed from index diagnosis in 6 of 11 cases harboring RAS-like alterations; mode diagnosis was AUS in 4 of 5 BRAF V600E FNAs. CONCLUSIONS: Atypia of undetermined significance of thyroid FNAs harboring BRAF V600E is uncommon yet relatively reproducible, particularly among pathologists with experience. It is advisable to sequence BRAF across V600 in such cases.

Interobserver agreement in the interpretation of anal cytology.

BACKGROUND: Anal cytology represents a tool for anal cancer screening in high-risk populations. In addition to accuracy, the reproducibility of the interpretation is of key importance. The authors evaluated the agreement of anal cytologic interpretation between two cytopathologists. METHODS: Liquid-based cytologic slides from human immunodeficiency virus (HIV)-negative men who have sex with men (MSM) were evaluated by two readers with at least 10 years of expertise in cervical cytology. Cases with a discordant interpretation were reviewed, and a consensus was reached. Human papillomavirus (HPV) genotyping was performed using a proprietary HPV genotyping test. Unweighted and weighted Cohen kappa and 95% confidence interval (CI) values were calculated. RESULTS: Overall, 713 slides that were adequate for interpretation were evaluated (MSM: median age, 33 years). An HPV test was performed on 620 samples (87.0%). Considering a dichotomous interpretation (negative for intraepithelial lesion or malignancy vs. atypical squamous cells of undetermined significance or worse), the crude agreement between the two readers was 93.3% (kappa = 0.82; 95% CI, 0.77-0.87). Once a consensus for discordant cases was reached, the best agreement was found for the negative for intraepithelial lesion or malignancy category (511 of 528 samples; 96.8%), whereas the atypical squamous cells of undetermined significance category showed the lowest agreement (90 of 117 samples, 76.9%). Considering the individual cytologic categories, overall agreement was 92.1% (kappa = 0.85; 95% CI, 0.81-0.89). The discordant interpretations were not associated with high-risk HPV infection, HPV16 infection, or MSM age. CONCLUSIONS: The results indicating excellent interobserver agreement in this study substantiate the use of anal cytology in the setting of human immunodeficiency virus-negative MSM.

Interobserver agreement of current and new proposed endoscopic scores for postoperative recurrence in Crohn's disease.

BACKGROUND AND AIMS: The modified Rutgeerts' score (mRS) is widely used for the assessment of endoscopic postoperative recurrence (ePOR) in Crohn's disease (CD) after ileocolic resection to guide therapeutic decisions. To improve the validity and prognostic value of this endoscopic assessment, two new scores have been proposed. This study assessed the interobserver agreement of the current (mRS) and new endoscopic scores for ePOR in CD. METHODS: Sixteen Dutch academic and non-academic IBD specialists assessed endoscopic videos (n=71) of postoperative CD patients (n=66) retrieved from nine Dutch centers. Each video was assessed for the degree of inflammation by four gastroenterologists using the mRS and the new proposed endoscopic score: REMIND score (separate score of anastomosis and neoterminal ileum) and updated Rutgeerts score (assessment of lesions at the anastomotic line, ileal inlet, ileal body and neoterminal ileum). In addition, lesions at the ileal body, ileal inlet, neoterminal ileum, colonic and/or ileal blind loop were separately assessed. Interobserver agreement was assessed using Fleiss' weighted kappa. RESULTS: Fleiss' weighted kappa for the mRS was 0.67 (95% confidence interval [CI] 0.59-0.74). The weighted kappa for the REMIND score was 0.73 (95% CI 0.65-0.80) for lesions in the neoterminal ileum and 0.46 (95% CI 0.35-0.58) for anastomotic lesions. The weighted kappa for the updated Rutgeerts' score was 0.69 (95% CI 0.62-0.77). The weighted kappa for lesions in the ileal body, ileal inlet, neoterminal ileum, colonic and ileal blind loop was 0.61 (95% CI 0.49-0.73), 0.63 (95% CI 0.54-0.72), 0.61 (95% CI 0.49-0.74), 0.83 (95% CI 0.62-1.00) and 0.68 (95% CI 0.46-0.89). CONCLUSION: The interobserver agreement of the mRS is substantial. Similarly, the interobserver agreement is substantial for the updated Rutgeerts' score. According to the REMIND score, the interobserver agreement was substantial for lesions in the neoterminal ileum, whereas only moderate for anastomotic lesions. Since therapeutic decisions in clinical practice are based on these assessments and these scores are used as outcome measure in clinical studies, further improvement of the interobserver agreement is essential.

Inter- and intraobserver agreement of antenatal cardiotocography assessments by maternity care professionals: A prospective study.

OBJECTIVE: In the Netherlands, antenatal cardiotocography (aCTG) to assess fetal well-being is performed in obstetrician-led care. An innovative initiative was started to evaluate whether aCTG for specific indications-reduced fetal movements, external cephalic version, or postdate pregnancy-is feasible in non-obstetrician-led care settings by independent primary care midwives. Quality assessment is essential when reorganizing and shifting tasks and responsibilities. Therefore, we aimed to assess the inter- and intraobserver agreement for aCTG assessments between and within four professional groups involved in Dutch maternity care regarding the overall classification and assessment of the various components of aCTG. METHOD: This was a prospective study among 47 Dutch primary care midwives, hospital-based midwives, residents, and obstetricians. Ten aCTG traces were assessed twice at a 1 month interval. To ensure a representative sample, we used two different sets of 10 aCTG traces each. We calculated the degree of agreement using the proportions of agreement. RESULTS: The proportions of agreement for interobserver agreement on the classification of aCTG between and within the four professional groups varied from 0.82 to 0.94. The proportions of agreement for each professional group were slightly higher for intraobserver (0.86-0.94) than for interobserver agreement. For the various aCTG components, the proportions of agreement for interobserver agreement varied from 0.64 (presence of contractions) to 0.98 (baseline heart frequency). CONCLUSION: The proportion of agreement levels between and within the maternity care professionals in the classification of aCTG traces among healthy women were comparable. This means that these professional groups are equally well able to classify aCTGs in healthy pregnant women.

Interobserver variability in thyroid ultrasound.

PURPOSE: Ultrasound evaluation of thyroid nodules is the preferred technique, but it is dependent on operator interpretation, leading to inter-observer variability. The current study aimed to determine the inter-physician consensus on nodular characteristics, risk categorization in the classification systems, and the need for fine needle aspiration puncture. METHODS: Four endocrinologists from the same center blindly evaluated 100 ultrasound images of thyroid nodules from 100 different patients. The following ultrasound features were evaluated: composition, echogenicity, margins, calcifications, and microcalcifications. Nodules were also classified according to ATA, EU-TIRADS, K-TIRADS, and ACR-TIRADS classifications. Krippendorff's alpha test was used to assess interobserver agreement. RESULTS: The interobserver agreement for ultrasound features was: Krippendorff's coefficient 0.80 (0.71-0.89) for composition, 0.59 (0.47-0.72) for echogenicity, 0.73 (0.57-0.88) for margins, 0.55 (0.40-0.69) for calcifications, and 0.50 (0.34-0.67) for microcalcifications. The concordance for the classification systems was 0.7 (0.61-0.80) for ATA, 0.63 (0.54-0.73) for EU-TIRADS, 0.64 (0.55-0.73) for K-TIRADS, and 0.68 (0.60-0.77) for K-TIRADS. The concordance in the indication of fine needle aspiration puncture (FNA) was 0.86 (0.71-1), 0.80 (0.71-0.88), 0.77 0.67-0.87), and 0.73 (0.64-0.83) for systems previously described respectively. CONCLUSIONS: Interobserver agreement was acceptable for the identification of nodules requiring cytologic study using various classification systems. However, limited concordance was observed in risk stratification and many ultrasonographic characteristics of the nodules.

Large-scale analysis of interobserver agreement and reliability in cardiotocography interpretation during labor using an online tool.

BACKGROUND: While the effectiveness of cardiotocography in reducing neonatal morbidity is still debated, it remains the primary method for assessing fetal well-being during labor. Evaluating how accurately professionals interpret cardiotocography signals is essential for its effective use. The objective was to evaluate the accuracy of fetal hypoxia prediction by practitioners through the interpretation of cardiotocography signals and clinical variables during labor. MATERIAL AND METHODS: We conducted a cross-sectional online survey, involving 120 obstetric healthcare providers from several countries. One hundred cases, including fifty cases of fetal hypoxia, were randomly assigned to participants who were invited to predict the fetal outcome (binary criterion of pH with a threshold of 7.15) based on the cardiotocography signals and clinical variables. After describing the participants, we calculated (with a 95% confidence interval) the success rate, sensitivity and specificity to predict the fetal outcome for the whole population and according to pH ranges, professional groups and number of years of experience. Interobserver agreement and reliability were evaluated using the proportion of agreement and Cohen's kappa respectively. RESULTS: The overall ability to predict a pH level below 7.15 yielded a success rate of 0.58 (95% CI 0.56-0.60), a sensitivity of 0.58 (95% CI 0.56-0.60) and a specificity of 0.63 (95% CI 0.61-0.65). No significant difference in the success rates was observed with respect to profession and number of years of experience. The success rate was higher for the cases with a pH level below 7.05 (0.69) and above 7.20 (0.66) compared to those falling between 7.05 and 7.20 (0.48). The proportion of agreement between participants was good (0.82), with an overall kappa coefficient indicating substantial reliability (0.63). CONCLUSIONS: The use of an online tool enabled us to collect a large amount of data to analyze how practitioners interpret cardiotocography data during labor. Despite a good level of agreement and reliability among practitioners, the overall accuracy is poor, particularly for cases with a neonatal pH between 7.05 and 7.20. Factors such as profession and experience level do not present notable impact on the accuracy of the annotations. The implementation and use of a computerized cardiotocography analysis software has the potential to enhance the accuracy to detect fetal hypoxia, especially for ambiguous cardiotocography tracings.

Repeatability of diffusion-based stiffness prediction - A healthy volunteer study.

INTRODUCTION: The study investigated the repeatability of brain diffusion-based stiffness prediction (DWIstiff) in healthy volunteers. METHODS: Thirty-one healthy volunteers were examined with DWIstiff using two different sets of b-values: b200-1500 s/mm2 (DWIstiff, 1500) and b200-1000 s/mm2 (DWIstiff, 1000). Each b-value set was scanned twice per imaging session without repositioning the participants. DWIstiff images were reconstructed from each set. Two observers delineated regions of interest (ROIs) on each DWIstiff image. The repeatability coefficient (RC), coefficient of variation (CV), inter- and intraobserver agreement were calculated. RESULTS: After excluding three participants due to image artifacts, the study included twenty-eight volunteers (mean age (range)) 37 years (24-62), 10 males, 18 females). For DWIstiff, 1500, the lowest and the highest RCs were in the parietal lobe (0.52) and respectively the brain stem (1.17). The lowest RC for DWIstiff, 1000 was in the frontal lobe (0.42) and the highest in the brain stem (1.58). The CV for whole brain measurements was 3.83 % for DWIstiff, 1500 and 4.93 % for DWIstiff, 1000. The Bland‒Altman (BA) limits of agreement (LoA) for the intraobserver agreement of DWIstiff, 1500 were -0.90 to 1.06 and respectively -0.78 to 0.88 for DWIstiff, 1000. Regarding interobserver agreement, the LoA were -0.85 to 0.94 for DWIstiff, 1500 and -0.61 to 0.66 for DWIstiff, 1000. CONCLUSION: DWIstiff is a precise technique with some observer dependence. Repeatability is higher for DWIstiff, 1000 s/mm2 than for DWIstiff 1500 s/mm2. IMPLICATIONS FOR PRACTICE: Our findings suggest that DWIstiff can reliably detect stiffness changes larger than 4.93 % in healthy volunteers. Further studies should investigate whether the repeatability of DWIstiff may be affected by the presence of pathology such as a brain tumor.

The effect of addition of 2DCT scans and 3DCT scans for the classification of tibial plateau fractures: a systematic review.

PURPOSE: In this systematic review, we evaluate the effect of radiographs and 2D and 3D imaging techniques on the interobserver agreement of six commonly used classification systems for tibial plateau fractures. METHODS: In accordance with PRISMA guidelines, PubMed, Cochrane, Embase and Web of Science were searched for studies regarding the effect of 2D and 3D imaging techniques on the interobserver agreement of tibial plateau classification systems. Studies validating new classification systems, not providing own data or only providing information on the interobserver agreement for radiographs were excluded. Studies were scored based on the ROBINS-I risk of bias tool. RESULTS: Our review analysed 14 studies on different classification systems used for tibial plateau fractures in clinical practice, with the Schatzker classification being the most commonly used classification system. The results showed that the addition of 2D CT led to a significant improvement of interobserver agreement for one study. However, other included studies showed varying levels of interobserver agreement, ranging from fair to substantial according to the interpretation by Landis and Koch. The addition of 3D CT resulted in a significant deterioration in one study for the Schatzker classification. Similar to the addition of 2D CT, the interobserver agreement for the Schatzker classification with the addition of 3D CT were heterogeneous ranging from fair to almost perfect according to the interpretation by Landis and Koch. CONCLUSIONS: The use of 2D CT can be recommended for classifying tibial plateau fractures with the Schatzker classification, AO/OTA classification and Hohl classification. The value of 3D CT on the interobserver agreement of commonly used classification systems remains uncertain and unproven. Therefore, we do not recommend the use of 3D CT for the classification of tibial plateau fractures. Overall, the advancement of imaging techniques is not in line with the advancement in interobserver agreement on fracture classification.

Is the fetus fit for labor? Introducing fast-and-frugal trees (FFTrees) to simplify triage of women for STAN monitoring: An interobserver agreement comparison with traditional classification.

INTRODUCTION: It is a shortcoming of traditional cardiotocography (CTG) classification table formats that CTG traces are frequently classified differently by different users, resulting in poor interobserver agreements. A fast-and-frugal tree (FFTree) flow chart may help provide better concordance because it is straightforward and has clearly structured binary questions with understandable "yes" or "no" responses. The initial triage to determine whether a fetus is suitable for labor when utilizing fetal ECG ST analysis (STAN) is very important, since a fetus with restricted capacity to respond to hypoxic stress may not generate STAN events and therefore may become falsely negative. This study aimed to compare physiology-focused FFTree CTG interpretation with FIGO classification for assessing the suitability for STAN monitoring. MATERIAL AND METHODS: A retrospective study of 36 CTG traces with a high proportion of adverse outcomes (17/36) selected from a European multicenter study database. Eight experienced European obstetricians evaluated the initial 40 minutes of the CTG recordings and judged whether STAN was a suitable fetal surveillance method and whether intervention was indicated. The experts rated the CTGs using the FFTree and FIGO classifications at least 6 weeks apart. Interobserver agreements were calculated using proportions of agreement and Fleiss' kappa (κ). RESULTS: The proportions of agreement for "not suitable for STAN" were for FIGO 47% (95% confidence interval [CI] 42%-52%) and for FFTree 60% (95% CI 56-64), ie a significant difference; the corresponding figures for "yes, suitable" were 74% (95% CI 71-77) and 70% (95% CI 67-74). For "intervention needed" the figures were 52% (95% CI 47-56) vs 58% (95% CI 54-62) and for "expectant management" 74% (95% CI 71-77) vs 72% (95% CI 69-75). Fleiss' κ agreement on "suitability for STAN" was 0.50 (95% CI 0.44-0.56) for the FIGO classification and 0.57 (95% CI 0.51-0.63) for the FFTree classification; the corresponding figures for "intervention or expectancy" were 0.53 (95% CI 0.47-0.59) and 0.57 (95% CI 0.51-0.63). CONCLUSIONS: The proportion of agreement among expert obstetricians using the FFTree physiological approach was significantly higher compared with the traditional FIGO classification system in rejecting cases not suitable for STAN monitoring. That might be of importance to avoid false negative STAN recordings. Other agreement figures were similar. It remains to be shown whether the FFTree simplicity will benefit less experienced users and how it will work in real-world clinical scenarios.

Interobserver agreement and prognostic value of image-based scoring systems in patients with primary sclerosing cholangitis.

BACKGROUND: Primary sclerosing cholangitis (PSC) is a cholestatic liver disease that progresses to cirrhosis and liver failure. The Anali and Amsterdam scores are based upon imaging features on MRI and ERCP, respectively. AIMS: We aimed to compare the interobserver variability and performances of these scores. METHODS: Patients with PSC with at least 1 MRCP were included. Images were independently scored by 2 experts. Agreement and prognostic performance with a primary end point of hepatic decompensation was assessed. RESULTS: Fifty-nine patients were included (67.8% male, 86.4% IBD). Interobserver agreement for the Anali and Amsterdam scores were moderate (k = 0.49; 95% CI 0.35-0.64 and k = 0.43; 95% CI 0.30-0.56, respectively). Among the Anali components, dysmorphy (caudate/right lobe ratio > 0.9) had fair agreement (k = 0.37; 95% CI 0.14-0.60) and portal hypertension (k = 0.64, 95% CI 0.32-0.89) and intrahepatic dilation (k = 0.70; 95% CI 0.53-0.87) had substantial agreement. The Amsterdam extrahepatic and intrahepatic scores had fair agreement (k = 0.38; 95% CI 0.23-0.52) and moderate agreement (k = 0.50; 95% CI 0.34-0.67), respectively. Anali score (HR 5.90, 95% CI 1.64-21.21), total bilirubin (HR = 3.23; 95% Cl 1.06-9.91), and age (HR = 1.05; 95% CI 1.00-1.11) were independent predictors of hepatic decompensation. Mayo risk score and Anali score had good discriminative ability with c-statistics of 0.78 (CI 0.59-0.96) and 0.76 (CI 0.56-0.91). Anali score remained an independent predictor after adjusting for Mayo risk score. CONCLUSION: Anali score adds additional predictive value for hepatic decompensation in patients with PSC.

Reliability assessment of methylthioadenosine phosphorylase immunohistochemistry as a surrogate biomarker for CDKN2A homozygous deletion in adult-type IDH-mutant diffuse gliomas.

According to the 2021 World Health Organization classification of brain tumors, astrocytomas containing a CDKN2A/B homozygous deletion (HD) are designated as grade 4 even when no microvascular proliferation and/or necrosis is present. In this study, we aimed to investigate the relationship between CDKN2A HD and loss of methylthioadenosine phosphorylase (MTAP) expression in adult-type IDH-mutant gliomas and to assess the sensitivity and specificity of MTAP immunohistochemistry (IHC) along with interobserver agreement as a surrogate biomarker for CDKN2A HD. Eighty-eight astrocytomas and 71 oligodendrogliomas cases that were diagnosed between 2014 and 2021 at Hacettepe University were selected and tissue microarrays were conducted to perform CDKN2A fluorescence in situ hybridization and MTAP IHC. Twenty-five (15.7%) cases harbored CDKN2A HD. MTAP loss was detected in 28 (15.7%) cases by the first observer and 27 (17%) cases by the second observer. The sensitivity and specificity of MTAP were calculated as 88% and 95.52%-96.27% for 2 observers. A very good/perfect agreement was noted between the observers (Cohen kappa coefficient = 0.938). Intratumoral heterogeneity was observed in 4 cases. MTAP IHC was found to be a reliable surrogate biomarker as a possible alternative to CDKN2A HD identification with a high sensitivity and specificity along with high interobserver agreement.

Assessment of Interobserver Agreement in the Interpretation of 99mTc-PYP SPECT Imaging of Cardiac Amyloidosis.

Background: Technetium-99m pyrophosphate single photon emission computed tomography (99mTc-PYP SPECT) imaging is widely used to diagnose cardiac amyloidosis, a disease characterized by amyloid protein deposits in the myocardium. The effects of viewing perspectives on interobserver agreement in the interpretation of 99mTc-PYP SPECT images for the diagnosis of cardiac amyloidosis remain unclear. Methods: A retrospective analysis of 32 consecutive patients who underwent 99mTc-PYP imaging for the diagnosis of cardiac amyloidosis at Nagasaki University Hospital between October 2017 and February 2020 was performed. Four evaluators independently reviewed coronal, sagittal, and transaxial images and then all images together and made a categorical diagnosis based on predefined criteria. Interobserver agreement was analyzed using Cohen's Kappa analysis. Results: Kappa coefficient values in the four-grade grading system (grades 0-3) ranged between 0.31 and 0.95, while those in the binary grading system (positive/negative) ranged between 0.88 and 1. The sagittal view showed the highest value in the four-grade grading system (0.95) and the lowest in the binary grading system (0.88). The transaxial view was more likely to show a consistently high kappa value in both the four-grade and binary grading systems. The use of the multiplanar view reduced the number of subjects classified as grade 1. Conclusion: Our study demonstrates that the transaxial view provides the most consistent interpretation of 99mTc-PYP SPECT images for the diagnosis of cardiac amyloidosis. The use of the multiplanar view may also reduce equivocal interpretations, which are graded as grade 1. Further studies with larger sample sizes and a quantitative analysis are needed to confirm the present results.

Interobserver Agreement in Automatic Segmentation Annotation of Prostate Magnetic Resonance Imaging.

We aimed to compare the performance and interobserver agreement of radiologists manually segmenting images or those assisted by automatic segmentation. We further aimed to reduce interobserver variability and improve the consistency of radiomics features. This retrospective study included 327 patients diagnosed with prostate cancer from September 2016 to June 2018; images from 228 patients were used for automatic segmentation construction, and images from the remaining 99 were used for testing. First, four radiologists with varying experience levels retrospectively segmented 99 axial prostate images manually using T2-weighted fat-suppressed magnetic resonance imaging. Automatic segmentation was performed after 2 weeks. The Pyradiomics software package v3.1.0 was used to extract the texture features. The Dice coefficient and intraclass correlation coefficient (ICC) were used to evaluate segmentation performance and the interobserver consistency of prostate radiomics. The Wilcoxon rank sum test was used to compare the paired samples, with the significance level set at p < 0.05. The Dice coefficient was used to accurately measure the spatial overlap of manually delineated images. In all the 99 prostate segmentation result columns, the manual and automatic segmentation results of the senior group were significantly better than those of the junior group (p < 0.05). Automatic segmentation was more consistent than manual segmentation (p < 0.05), and the average ICC reached >0.85. The automatic segmentation annotation performance of junior radiologists was similar to that of senior radiologists performing manual segmentation. The ICC of radiomics features increased to excellent consistency (0.925 [0.888~0.950]). Automatic segmentation annotation provided better results than manual segmentation by radiologists. Our findings indicate that automatic segmentation annotation helps reduce variability in the perception and interpretation between radiologists with different experience levels and ensures the stability of radiomics features.

Increased Thyroidal Activity on Routine FDG-PET/CT after Combination Immune Checkpoint Inhibition: Temporal Associations with Clinical and Biochemical Thyroiditis.

BACKGROUND: FDG-PET/CT used for immune checkpoint inhibitor (ICI) response assessment can incidentally identify immune-related adverse events (irAEs), including thyroiditis. This study aimed to correlate the time course of FDG-PET/CT evidence of thyroiditis with clinical and biochemical evolution of thyroid dysfunction. METHODS: A retrospective review was performed by two independent blinded nuclear medicine physicians (NMPs) of thyroidal FDG uptake in 127 patients who underwent PET/CT between January 2016 and January 2019 at baseline and during treatment monitoring of combination ICI therapy for advanced melanoma. Interobserver agreement was assessed and FDG-PET/CT performance defined by a receiver-operating characteristic (ROC) curve using thyroid function tests (TFTs) as the standard of truth. Thyroid maximum standardized uptake value (SUVmax) and its temporal changes with respect to the longitudinal biochemistry were serially recorded. RESULTS: At a median of 3 weeks after commencing ICI, 43/127 (34%) had a diagnosis of thyroiditis established by abnormal TFTs. FDG-PET/CT was performed at baseline and at a median of 11 weeks (range 3-32) following the start of therapy. ROC analysis showed an area under the curve of 0.87 (95% CI 0.80, 0.94) for FDG-PET/CT for detection of thyroiditis with a positive predictive value of 93%. Among patients with biochemical evidence of thyroiditis, those with a positive FDG-PET/CT were more likely to develop overt hypothyroidism (77% versus 35%, p < 0.01). In the evaluation of the index test, there was an almost perfect interobserver agreement between NMPs of 93.7% (95% CI 89.4-98.0), kappa 0.83. CONCLUSION: Increased metabolic activity of the thyroid on routine FDG-PET/CT performed for tumoral response of patients undergoing ICI therapy is generally detected well after routine biochemical diagnosis. Elevation of FDG uptake in the thyroid is predictive of overt clinical hypothyroidism and suggests that an ongoing robust inflammatory response beyond the initial thyrotoxic phase may be indicative of thyroid destruction.

Agreement between pharmacists and physicians on the assessment of appropriateness of antimicrobial prescribing.

INTRODUCTION: Audits for monitoring the quality of antimicrobial prescribing are a main tool in antimicrobial stewardship programs; however, interobserver reliability has not been conclusively assessed. Our objective was to measure the level of agreement between pharmacists and physicians on the appropriateness of antimicrobials prescribing in hospitals. METHODS: A national multicenter, cross-sectional study was conducted of patients who were receiving antimicrobials one day of April 2021. Hospital participation was voluntary, and the study population was randomly selected. Pharmacists and physicians performed a simultaneous, independent assessment of the quality of antimicrobial prescriptions. The observers used an assessment method by which all indicators of the quality of antimicrobial use were considered. Finally, an algorithm was used to rate overall antimicrobial prescribing as appropriate, suboptimal, inappropriate, or not assessable. Gwet's AC1 coefficient was used to assess interobserver agreement. RESULTS: In total, 101 hospitals participated, and 411 hospital antimicrobial prescriptions were reviewed. The strength of agreement was moderate regarding the overall quality of prescribing (AC1=0.51; 95%CI=[0.44-0.58]). A very good level of agreement (AC1>0.80) was observed between pharmacists and physicians in all indicators of the quality, except for duration of treatment, rated as good (AC1=0.79; 95%CI=[0.75-0.83]), and registration on the medical record, rated as fair (AC1=0.34; 95%CI=[0.26-0.43]). The agreement was greater in critical care, onco-hematology, and pediatric units than in medical and surgery units. CONCLUSIONS: In this point prevalence study, a moderate level of agreement was observed between pharmacists and physicians in the evaluation of the appropriateness of antimicrobials prescribing in hospitals.