Reliability of study endpoint adjudication in a pragmatic trial on brain arteriovenous malformations.

BACKGROUND: The results of a clinical trial are given in terms of primary and secondary outcomes that are obtained for each patient. Just as an instrument should provide the same result when the same object is measured repeatedly, the agreement of the adjudication of a clinical outcome between various raters is fundamental to interpret study results. The reliability of the adjudication of study endpoints determined by examination of the electronic case report forms of a pragmatic trial has not previously been tested. METHODS: The electronic case report forms of 62/434 (14%) patients selected to be observed in a study on brain AVMs were independently examined twice (4 weeks apart) by 8 raters who judged whether each patient had reached the following study endpoints: (1) new intracranial hemorrhage related to AVM or to treatment; (2) new non-hemorrhagic neurological event; (3) increase in mRS ≥1; (4) serious adverse events (SAE). Inter and intra-rater reliability were assessed using Gwet's AC1 (κG) statistics, and correlations with mRS score using Cramer's V test. RESULTS: There was almost perfect agreement for intracranial hemorrhage (92% agreement; κG = 0.84 (95%CI: 0.76-0.93), and substantial agreement for SAEs (88% agreement; κG = 0.77 (95%CI: 0.67-0.86) and new non-hemorrhagic neurological event (80% agreement; κG = 0.61 (95%CI: 0.50-0.72). Most endpoints correlated (V = 0.21-0.57) with an increase in mRS of ≥1, an endpoint which was itself moderately reliable (76% agreement; κG = 0.54 (95%CI: 0.43-0.64). CONCLUSION: Study endpoints of a pragmatic trial were shown to be reliable. More studies on the reliability of pragmatic trial endpoints are needed.

Clinical uncertainty in large vessel occlusion ischemic stroke: does automated perfusion imaging make a difference? An intra-rater and inter-rater agreement study.

BACKGROUND: Limited research exists regarding the impact of neuroimaging on endovascular thrombectomy (EVT) decisions for late-window cases of large vessel occlusion (LVO) stroke. OBJECTIVE: T0 assess whether perfusion CT imaging: (1) alters the proportion of recommendations for EVT, and (2) enhances the reliability of EVT decision-making compared with non-contrast CT and CT angiography. METHODS: We conducted a survey using 30 patients drawn from an institutional database of 3144 acute stroke cases. These were presented to 29 Canadian physicians with and without perfusion imaging. We used non-overlapping 95% confidence intervals and difference in agreement classification as criteria to suggest a difference between the Gwet AC1 statistics (κG). RESULTS: The percentage of EVT recommendations differed by 1.1% with or without perfusion imaging. Individual decisions changed in 21.4% of cases (11.3% against EVT and 10.1% in favor). Inter-rater agreement (κG) among the 29 raters was similar between non-perfusion and perfusion CT neuroimaging (κG=0.487; 95% CI 0.327 to 0.647 and κG=0.552; 95% CI 0.430 to 0.675). The 95% CIs overlapped with moderate agreement in both. Intra-rater agreement exhibited overlapping 95% CIs for all 28 raters. κG was either substantial or excellent (0.81-1) for 71.4% (20/28) of raters in both groups. CONCLUSIONS: Despite the minimal difference in overall EVT recommendations with either neuroimaging protocol one in five decisions changed with perfusion imaging. Regarding agreement we found that the use of automated CT perfusion images does not significantly impact the reliability of EVT decisions for patients with late-window LVO.

Nonaneurysmal perimesencephalic subarachnoid hemorrhage on noncontrast head CT: An accuracy, inter-rater, and intra-rater reliability study.

BACKGROUND AND PURPOSE: To evaluate the reliability and accuracy of nonaneurysmal perimesencephalic subarachnoid hemorrhage (NAPSAH) on Noncontrast Head CT (NCCT) between numerous raters. MATERIALS AND METHODS: 45 NCCT of adult patients with SAH who also had a catheter angiography (CA) were independently evaluated by 48 diverse raters; 45 raters performed a second assessment one month later. For each case, raters were asked: 1) whether they judged the bleeding pattern to be perimesencephalic; 2) whether there was blood anterior to brainstem; 3) complete filling of the anterior interhemispheric fissure (AIF); 4) extension to the lateral part of the sylvian fissure (LSF); 5) frank intraventricular hemorrhage; 6) whether in the hypothetical presence of a negative CT angiogram they would still recommend CA. An automatic NAPSAH diagnosis was also generated by combining responses to questions 2-5. Reliability was estimated using Gwet's AC1 (κG), and the relationship between the NCCT diagnosis of NAPSAH and the recommendation to perform CA using Cramer's V test. Multi-rater accuracy of NCCT in predicting negative CA was explored. RESULTS: Inter-rater reliability for the presence of NAPSAH was moderate (κG = 0.58; 95%CI: 0.47, 0.69), but improved to substantial when automatically generated (κG = 0.70; 95%CI: 0.59, 0.81). The most reliable criteria were the absence of AIF filling (κG = 0.79) and extension to LSF (κG = 0.79). Mean intra-rater reliability was substantial (κG = 0.65). NAPSAH weakly correlated with CA decision (V = 0.50). Mean sensitivity and specificity were 58% (95%CI: 44%, 71%) and 83 % (95%CI: 72 %, 94%), respectively. CONCLUSION: NAPSAH remains a diagnosis of exclusion. The NCCT diagnosis was moderately reliable and its impact on clinical decisions modest.

Clinical Uncertainty in Large Vessel Occlusion ischemic stroke (CULVO): Does automated perfusion scanning make a difference? Protocol of an intrarater and interrater agreement study.

BACKGROUND: Guidelines recommend the use of perfusion computed tomography (CT) to identify emergent large vessel ischemic stroke (ELVIS) patients who are likely to benefit from endovascular thrombectomy (EVT) if they present within 6-24 hour (late window) of stroke onset. We aim to determine if the interrater and intrarater reliability among physicians when recommending EVT is significantly different when perfusion CT or non-perfusion CT is reviewed. METHODS: A total of 30 non-consecutive patients will be selected from our institutional database comprising 3144 cranial CT scans performed for acute stroke symptoms January 2018 to August 2022. The clinical and radiologic data of the 30 patients will be presented in random order to a group of 29 physicians in two separate sessions at least three weeks apart. In each session, the physicians will evaluate each patient once with automated perfusion images and once without. We will use non-overlapping 95% confidence intervals and difference in agreement classification as criteria to suggest a difference between the Gwet AC1 statistics (κG). DISCUSSION: The results obtained from this study, combined with the clinical outcomes data of patients categorized through the two imaging techniques and a cost-effectiveness analysis, will offer a comprehensive evaluation of the clinical utility of perfusion CT neuroimaging. Should there be no significant disparity in the reliability of decisions made by clinicians using the two neuroimaging protocols, it may be necessary to revise existing recommendations regarding neuroimaging in the later time window to align with these findings.

Patient Selection in a Pragmatic Study on the Management of Patients with Brain Arteriovenous Malformations.

BACKGROUND: The Treatment of Brain Arteriovenous Malformations Study (TOBAS) is an all-inclusive pragmatic study comprising 2 randomized clinical trials (RCTs). Patients excluded from the RCTs are followed in parallel treatment and observation registries, allowing a comparison between RCT and registry patients. METHODS: The first randomized clinical trial (RCT-1) offers 1:1 randomized allocation of intervention versus conservative management for patients with arteriovenous malformation (AVM). The second randomized clinical trial (RCT-2) allocates 1:1 pre-embolization or no pre-embolization to surgery or radiosurgery patients judged treatable with or without embolization. Characteristics of RCT patients are reported and compared to registry patients. RESULTS: From June 2014 to May 2021, 1010 patients with AVM were recruited; 498 patients were observed and 373 were included in the treatment registries. Randomized allocation in RCT-1 was applied to 139 (26%) of the 512 patients (including 127 of 222 [57%] with unruptured AVMs) considered for curative treatment. RCT-1 AVM patients differed (in rupture status, Spetzler-Martin grade and baseline modified Rankin Score) from those in the observation or treatment registries (P < 0.001). Most patients had small (<3 cm; 71%) low-grade (Spetzler-Martin I-II; 64%) unruptured (91%) AVMs. The allocated management was conservative (n = 71) or curative (n = 68), using surgery (n = 39), embolization (n = 16), or stereotactic radiosurgery (n = 13). Pre-embolization was considered for 179/309 (58%) patients allocated/assigned to surgery or stereotactic radiosurgery; 87/179 (49%) were included in RCT-2. RCT-2 patient AVMs differed in size, eloquence and grade from patients of the pre-embolization registry (P < 0.01). Most had small (<3 cm in 82%) low-grade (83%) AVMs in non-eloquent brain (64%). CONCLUSIONS: Patients included in the RCTs differ significantly from registry patients. Meaningful results can be obtained if multiple centers actively participate in the TOBAS RCTs.

Angiographic results of surgical or endovascular treatment of intracranial aneurysms: a systematic review and inter-observer reliability study.

PURPOSE: Results of surgical or endovascular treatment of intracranial aneurysms are often assessed using angiography. A reliable method to report results irrespective of treatment modality is needed to enable comparisons. Our goals were to systematically review existing classification systems, and to propose a 3-point classification applicable to both treatments and assess its reliability. METHODS: We conducted two systematic reviews on classification systems of angiographic results after clipping or coiling to select a simple 3-category scale that could apply to both treatments. We then circulated an electronic portfolio of angiograms of clipped (n=30) or coiled (n=30) aneurysms, and asked raters to evaluate the degree of occlusion using this scale. Raters were also asked to choose an appropriate follow-up management for each patient based on the degree of occlusion. Agreement was assessed using Krippendorff's α statistics (αK), and relationship between occlusion grade and clinical management was analyzed using Fisher's exact and Cramer's V tests. RESULTS: The systematic reviews found 70 different grading scales with heterogeneous reliability (kappa values from 0.12 to 1.00). The 60-patient portfolio was independently evaluated by 19 raters of diverse backgrounds (neurosurgery, radiology, and neurology) and experience. There was substantial agreement (αK=0.76, 95%CI, 0.67-0.83) between raters, regardless of background, experience, or treatment used. Intra-rater agreement ranged from moderate to almost perfect. A strong relationship was found between angiographic grades and management decisions (Cramer's V: 0.80±0.12). CONCLUSION: A simple 3-point scale demonstrated sufficient reliability to be used in reporting aneurysm treatments or in evaluating treatment results in comparative randomized trials.

Atlantoaxial wiring hardware failure resulting in intracranial hemorrhage and hydrocephalus: illustrative case.

BACKGROUND: Atlantoaxial sublaminar wiring has many known complications related to hardware failure, but intracranial hemorrhage is a rare complication. OBSERVATIONS: A 61-year-old female patient with prior atlantoaxial sublaminar wiring for odontoid fracture nonunion experienced decreased level of consciousness due to a subarachnoid and subdural hemorrhage of the posterior fossa with intraventricular extension and hydrocephalus. Rupture of the sublaminar wire with intramedullary protrusion was the cause of the hemorrhage. The patient was treated with ventriculostomy for hydrocephalus and occipital cervical fusion for spinal instability, along with removal of the broken wire and drainage of a hematoma. LESSONS: This uncommon cause of intracranial hemorrhage highlights an additional risk of atlantoaxial sublaminar wiring compared with other atlantoaxial fusion techniques. In addition, this case suggests cervical instrumentation failure as a differential diagnosis of subarachnoid and subdural hemorrhage of the posterior fossa when a history of prior instrumentation is known.

Scleroderma's Possible Dual Role in the Pathophysiology of Intracranial Aneurysms: Case Report and Literature Review.

BACKGROUND: Scleroderma, or systemic sclerosis, is an autoimmune disorder affecting connective tissues, including blood vessels. Although the exact mechanism is not understood, it results in the production of an abnormal amount of collagen. Cases have been reported in which patients with scleroderma also had intracranial aneurysms. We wish to gain insight into any potential association between the 2 diseases. CASE DESCRIPTION: We reviewed the literature of scleroderma cases with cerebral aneurysms and added our own case, focusing on patient and aneurysm characteristics. Including the present case, this paper pertains to 11 cases with 26 aneurysms. Fifty-five percent had multiple aneurysms, of which two thirds had >2. When data were available, 35% of aneurysms were ≥1 cm, 12% of which were giant. Four aneurysms were fusiform (15%). Patients presented with subarachnoid hemorrhage in 45% of cases. CONCLUSIONS: Scleroderma may lead to the formation of an aneurysm, and the abnormal growth of collagen may protect it from early rupture, allowing it to reach a large size.

Excessive laboratory monitoring to prevent adolescent's refeeding syndrome: opportunities for enhancement.

PURPOSE: Anorexia nervosa (AN) is a complex medical condition affecting mainly adolescents and young adults. To monitor and prevent refeeding syndrome, current guidelines recommend daily laboratory testing in the first week of hospitalization and 2-3 times/week for the following 3 weeks. The aims of this study were to determine the proportion of abnormal results of the blood tests done during the first week of nutritional rehabilitation in adolescents with AN, the proportion of test having led to supplementation and the cost of all these tests. METHOD: A retrospective chart review of admissions for eating disorders between May 2014 and May 2015 in a tertiary Pediatric University Hospital center was performed. Patients were included if they were younger than 18 years, admitted for protocol-based refeeding and met criteria for AN (DSM 5). RESULTS: Among the 99 hospitalizations included in the study, the mean age was 14.6 years (± 1.7), with a female predominance (97%). The mean admission BMI was 15.3 ± 2 kg/m2 (Z-score - 2.6 ± 1.4). The mean length of hospitalization was 40.3 days ± 21.8. Of the 1289 laboratory tests performed, only 1.5% revealed abnormal values and 0.85% led to supplementation. No critical value was identified. The total cost for the tests performed was 148,926.80 CAD$, 1504$/admitted patient, instead of 3890$/admitted patient had we followed the recommendations. CONCLUSION: More precise criteria should be developed regarding the frequency of laboratory tests needed to monitor and prevent refeeding syndrome. At present, the recommendations could lead to unnecessary testing and expenses. LEVEL OF EVIDENCE: Level IV: Dramatic results in uncontrolled trials.