Simulating and estimating agreement in the presence of multiple raters and covariates.

Cohen's and Fleiss's kappa are popular estimators for assessing agreement among two and multiple raters, respectively, for a binary response. While additional methods have been developed to account for multiple raters and covariates, they are not always applicable, rarely used, and none simplify to Cohen's kappa. Furthermore, there are no methods to simulate Bernoulli observations under the kappa agreement structure such that the developed methods could be adequately assessed. This manuscript overcomes these shortfalls. First, we developed a model-based estimator for kappa that accommodates multiple raters and covariates through a generalized linear mixed model and encompasses Cohen's kappa as a special case. Second, we created a framework to simulate dependent Bernoulli observations that upholds all 2-tuple pair of rater's kappa agreement structure and includes covariates. We used this framework to assess our method when kappa was nonzero. Simulations showed that Cohen's and Fleiss's kappa estimates were inflated unlike our model-based kappa. We analyzed an Alzheimer's disease neuroimaging study and the classic cervical cancer pathology study. The proposed model-based kappa and advancement in simulation methodology demonstrates that the popular approaches of Cohen's and Fleiss's kappa are poised to yield invalid conclusions while our work overcomes shortfalls, leading to improved inferences.

Broad Therapeutic Time Window for Driving Motor Recovery After TBI Using Activity-Dependent Stimulation.

BACKGROUND: After an acquired injury to the motor cortex, the ability to generate skilled movements is impaired, leading to long-term motor impairment and disability. While rehabilitative therapy can improve outcomes in some individuals, there are no treatments currently available that are able to fully restore lost function. OBJECTIVE: We previously used activity-dependent stimulation (ADS), initiated immediately after an injury, to drive motor recovery. The objective of this study was to determine if delayed application of ADS would still lead to recovery and if the recovery would persist after treatment was stopped. METHODS: Rats received a controlled cortical impact over primary motor cortex, microelectrode arrays were implanted in ipsilesional premotor and somatosensory areas, and a custom brain-machine interface was attached to perform the ADS. Stimulation was initiated either 1, 2, or 3 weeks after injury and delivered constantly over a 4-week period. An additional group was monitored for 8 weeks after terminating ADS to assess persistence of effect. Results were compared to rats receiving no stimulation. RESULTS: ADS was delayed up to 3 weeks from injury onset and still resulted in significant motor recovery, with maximal recovery occurring in the 1-week delay group. The improvements in motor performance persisted for at least 8 weeks following the end of treatment. CONCLUSIONS: ADS is an effective method to treat motor impairments following acquired brain injury in rats. This study demonstrates the clinical relevance of this technique as it could be initiated in the post-acute period and could be explanted/ceased once recovery has occurred.

A semi-automated pipeline for fulfillment of resource requests from a longitudinal Alzheimer's disease registry.

OBJECTIVE: Managing registries with continual data collection poses challenges, such as following reproducible research protocols and guaranteeing data accessibility. The University of Kansas (KU) Alzheimer's Disease Center (ADC) maintains one such registry: Curated Clinical Cohort Phenotypes and Observations (C3PO). We created an automated and reproducible process by which investigators have access to C3PO data. MATERIALS AND METHODS: Data was input into Research Electronic Data Capture. Monthly, data part of the Uniform Data Set (UDS), that is data also collected at other ADCs, was uploaded to the National Alzheimer's Coordinating Center (NACC). Quarterly, NACC cleaned, curated, and returned the UDS to the KU Data Management and Statistics (DMS) Core, where it was stored in C3PO with other quarterly curated site-specific data. Investigators seeking to utilize C3PO submitted a research proposal and requested variables via the publicly accessible and searchable data dictionary. The DMS Core used this variable list and an automated SAS program to create a subset of C3PO. RESULTS: C3PO contained 1913 variables stored in 15 datasets. From 2017 to 2018, 38 data requests were completed for several KU departments and other research institutions. Completing data requests became more efficient; C3PO subsets were produced in under 10 seconds. DISCUSSION: The data management strategy outlined above facilitated reproducible research practices, which is fundamental to the future of research as it allows replication and verification to occur. CONCLUSION: We created a transparent, automated, and efficient process of extracting subsets of data from a registry where data was changing daily.

Relationship between caffeine intake and autosomal dominant polycystic kidney disease progression: a retrospective analysis using the CRISP cohort.

BACKGROUND: Caffeine has been proposed, based on in vitro cultured cell studies, to accelerate progression of autosomal dominant polycystic kidney disease (ADPKD) by increasing kidney size. Since ADPKD patients are advised to minimize caffeine intake, we investigated the effect of caffeine on disease progression in the Consortium for Radiologic Imaging Studies of Polycystic Kidney Disease (CRISP), a prospective, observational cohort study. METHODS: Our study included 239 patients (mean age = 32.3 ± 8.9 ys; 188 caffeine consumers) with a median follow-up time of 12.5 years. Caffeine intake reported at baseline was dichotomized (any vs. none). Linear mixed models, unadjusted and adjusted for age, race, sex, BMI, smoking, hypertension, genetics and time, were used to model height-adjusted total kidney volume (htTKV) and iothalamate clearance (mGFR). Cox proportional hazards models and Kaplan-Meier plots examined the effect of caffeine on time to ESRD or death. RESULTS: Caffeine-by-time was statistically significant when modeling ln(htTKV) in unadjusted and adjusted models (p <  0.01) indicating that caffeine consumers had slightly faster kidney growth (by 0.6% per year), but htTKV remained smaller from baseline throughout the study. Caffeine consumption was not associated with a difference in mGFR, or in the time to ESRD or death (p > 0.05). Moreover the results were similar when outcomes were modeled as a function of caffeine dose. CONCLUSION: We conclude that caffeine does not have a significant detrimental effect on disease progression in ADPKD.

A genome-wide association study of red-blood cell fatty acids and ratios incorporating dietary covariates: Framingham Heart Study Offspring Cohort.

Recent analyses have suggested a strong heritable component to circulating fatty acid (FA) levels; however, only a limited number of genes have been identified which associate with FA levels. In order to expand upon a previous genome wide association study done on participants in the Framingham Heart Study Offspring Cohort and FA levels, we used data from 2,400 of these individuals for whom red blood cell FA profiles, dietary information and genotypes are available, and then conducted a genome-wide evaluation of potential genetic variants associated with 22 FAs and 15 FA ratios, after adjusting for relevant dietary covariates. Our analysis found nine previously identified loci associated with FA levels (FADS, ELOVL2, PCOLCE2, LPCAT3, AGPAT4, NTAN1/PDXDC1, PKD2L1, HBS1L/MYB and RAB3GAP1/MCM6), while identifying four novel loci. The latter include an association between variants in CALN1 (Chromosome 7) and eicosapentaenoic acid (EPA), DHRS4L2 (Chromosome 14) and a FA ratio measuring delta-9-desaturase activity, as well as two loci associated with less well understood proteins. Thus, the inclusion of dietary covariates had a modest impact, helping to uncover four additional loci. While genome-wide association studies continue to uncover additional genes associated with circulating FA levels, much of the heritable risk is yet to be explained, suggesting the potential role of rare genetic variation, epistasis and gene-environment interactions on FA levels as well. Further studies are needed to continue to understand the complex genetic picture of FA metabolism and synthesis.

Histopathologic features of melanoma in difficult-to-diagnose lesions: A case-control study.

BACKGROUND: Dermatopathology is considered the gold standard for melanoma diagnosis, but a subset of cases is difficult to diagnose by histopathology. OBJECTIVE: The goals of this study were to measure the accuracy of histopathologic features in difficult-to-diagnose melanocytic tumors and the interobserver agreement of those features. METHODS: This is a case-control study of histopathologic features of melanoma in 100 difficult-to-diagnose melanocytic neoplasms (40 melanomas and 60 nevi). Slides were blindly evaluated by 5 dermatopathologists. Frequencies, predictive values, and interobserver agreement were calculated. Univariate and multivariate logistic regression analyses were performed to identify the most influential features in arriving at a diagnosis of melanoma. RESULTS: Asymmetry, single-cell melanocytosis, solar elastosis, pagetoid melanocytosis, and broad surface diameter were most influential in arriving at a diagnosis of melanoma. Asymmetry and single-cell melanocytosis were most predictive of melanoma. Fleiss kappa was <0.6 for interobserver agreement in 9/10 histopathologic features of melanoma. LIMITATIONS: This study is limited by the small sample size, selection bias, and binary classification of melanocytic lesions. CONCLUSION: Our results indicate histopathologic features of melanoma in difficult-to-diagnose lesions vary in accuracy and reproducibility.