One-tissue compartment model for myocardial perfusion quantification with N-13 ammonia PET provides matching results: A cross-comparison between Carimas, FlowQuant, and PMOD.

PURPOSE: To cross-compare three software packages (SPs)-Carimas, FlowQuant, and PMOD-to quantify myocardial perfusion at global, regional, and segmental levels. MATERIALS AND METHODS: Stress N-13 ammonia PET scans of 48 patients with HCM were analyzed in three centers using Carimas, FlowQuant, and PMOD. Values agreed if they had an ICC > 0.75 and a difference < 20% of the median across all observers. RESULTS: When using 1TCM on the global level, the agreement was good, and the maximum difference between 1TCM MBF values was 17.2% (ICC = 0.83). On the regional level, the agreement was acceptable except in the LCx region (25.5% difference, ICC = 0.74) between FlowQuant and PMOD. Carimas-1TCM agreed well with PMOD-1TCM and FlowQuant-1TCM. Values obtained with FlowQuant-1TCM had a somewhat lesser agreement with PMOD-1TCM, especially at the segmental level. CONCLUSIONS: The global and regional MBF values (with one exception) agree well between the different software packages. There is significant variability in segmental values, mainly located in the LCx region and segments. Out of the studied tools, Carimas can be used interchangeably with both PMOD and FlowQuant for 1TCM implementation on all levels-global, regional, and segmental.

A comparison of Cohen's Kappa and Gwet's AC1 when calculating inter-rater reliability coefficients: a study conducted with personality disorder samples.

BACKGROUND: Rater agreement is important in clinical research, and Cohen's Kappa is a widely used method for assessing inter-rater reliability; however, there are well documented statistical problems associated with the measure. In order to assess its utility, we evaluated it against Gwet's AC1 and compared the results. METHODS: This study was carried out across 67 patients (56% males) aged 18 to 67, with a mean SD of 44.13 ± 12.68 years. Nine raters (7 psychiatrists, a psychiatry resident and a social worker) participated as interviewers, either for the first or the second interviews, which were held 4 to 6 weeks apart. The interviews were held in order to establish a personality disorder (PD) diagnosis using DSM-IV criteria. Cohen's Kappa and Gwet's AC1 were used and the level of agreement between raters was assessed in terms of a simple categorical diagnosis (i.e., the presence or absence of a disorder). Data were also compared with a previous analysis in order to evaluate the effects of trait prevalence. RESULTS: Gwet's AC1 was shown to have higher inter-rater reliability coefficients for all the PD criteria, ranging from .752 to 1.000, whereas Cohen's Kappa ranged from 0 to 1.00. Cohen's Kappa values were high and close to the percentage of agreement when the prevalence was high, whereas Gwet's AC1 values appeared not to change much with a change in prevalence, but remained close to the percentage of agreement. For example a Schizoid sample revealed a mean Cohen's Kappa of .726 and a Gwet's AC1of .853 , which fell within the different level of agreement according to criteria developed by Landis and Koch, and Altman and Fleiss. CONCLUSIONS: Based on the different formulae used to calculate the level of chance-corrected agreement, Gwet's AC1 was shown to provide a more stable inter-rater reliability coefficient than Cohen's Kappa. It was also found to be less affected by prevalence and marginal probability than that of Cohen's Kappa, and therefore should be considered for use with inter-rater reliability analysis.

Myocardial perfusion quantification with Rb-82 PET: good interobserver agreement of Carimas software on global, regional, and segmental levels.

PURPOSE: To estimate the interobserver agreement of the Carimas software package (SP) on global, regional, and segmental levels for the most widely used myocardial perfusion PET tracer-Rb-82. MATERIALS AND METHODS: Rest and stress Rb-82 PET scans of 48 patients with suspected or known coronary artery disease (CAD) were analyzed in four centers using the Carimas SP. We considered values to agree if they simultaneously had an intraclass correlation coefficient (ICC) > 0.75 and a difference < 20% of the median across all observers. RESULTS: The median values on the segmental level were 1.08 mL/min/g for rest myocardial blood flow (MBF), 2.24 mL/min/g for stress MBF, and 2.17 for myocardial flow reserve (MFR). For the rest MBF and MFR, all the values at all the levels fulfilled were in excellent agreement. For stress MBF, at the global and regional levels, all the 24 comparisons showed excellent agreement. Only 1 out of 102 segmental comparisons (seg. 14) was over the adequate agreement limit-23.5% of the median value (ICC = 0.95). CONCLUSION: Interobserver agreement for Rb-82 PET myocardial perfusion quantification analyzed with Carimas is good at any LV segmentation level-global, regional, and segmental. It is good for all the estimates-rest MBF, stress MBF, and MFR.

Large-Sample Variance of Fleiss Generalized Kappa.

Cohen's kappa coefficient was originally proposed for two raters only, and it later extended to an arbitrarily large number of raters to become what is known as Fleiss' generalized kappa. Fleiss' generalized kappa and its large-sample variance are still widely used by researchers and were implemented in several software packages, including, among others, SPSS and the R package "rel." The purpose of this article is to show that the large-sample variance of Fleiss' generalized kappa is systematically being misused, is invalid as a precision measure for kappa, and cannot be used for constructing confidence intervals. A general-purpose variance expression is proposed, which can be used in any statistical inference procedure. A Monte-Carlo experiment is presented, showing the validity of the new variance estimation procedure.

Testing the Difference of Correlated Agreement Coefficients for Statistical Significance.

This article addresses the problem of testing the difference between two correlated agreement coefficients for statistical significance. A number of authors have proposed methods for testing the difference between two correlated kappa coefficients, which require either the use of resampling methods or the use of advanced statistical modeling techniques. In this article, we propose a technique similar to the classical pairwise t test for means, which is based on a large-sample linear approximation of the agreement coefficient. We illustrate the use of this technique with several known agreement coefficients including Cohen's kappa, Gwet's AC1, Fleiss's generalized kappa, Conger's generalized kappa, Krippendorff's alpha, and the Brenann-Prediger coefficient. The proposed method is very flexible, can accommodate several types of correlation structures between coefficients, and requires neither advanced statistical modeling skills nor considerable computer programming experience. The validity of this method is tested with a Monte Carlo simulation.

Quantification of myocardial blood flow in absolute terms using (82)Rb PET imaging: the RUBY-10 Study.

OBJECTIVES: The purpose of this study was to compare myocardial blood flow (MBF) and myocardial flow reserve (MFR) estimates from rubidium-82 positron emission tomography ((82)Rb PET) data using 10 software packages (SPs) based on 8 tracer kinetic models. BACKGROUND: It is unknown how MBF and MFR values from existing SPs agree for (82)Rb PET. METHODS: Rest and stress (82)Rb PET scans of 48 patients with suspected or known coronary artery disease were analyzed in 10 centers. Each center used 1 of 10 SPs to analyze global and regional MBF using the different kinetic models implemented. Values were considered to agree if they simultaneously had an intraclass correlation coefficient >0.75 and a difference <20% of the median across all programs. RESULTS: The most common model evaluated was the Ottawa Heart Institute 1-tissue compartment model (OHI-1-TCM). MBF values from 7 of 8 SPs implementing this model agreed best. Values from 2 other models (alternative 1-TCM and Axially distributed) also agreed well, with occasional differences. The MBF results from other models (e.g., 2-TCM and retention) were less in agreement with values from OHI-1-TCM. CONCLUSIONS: SPs using the most common kinetic model-OHI-1-TCM-provided consistent results in measuring global and regional MBF values, suggesting that they may be used interchangeably to process data acquired with a common imaging protocol.