Building reliable and generalizable clerkship competency assessments: Impact of 'hawk-dove' correction.

PURPOSE: Systematic differences among raters' approaches to student assessment may result in leniency or stringency of assessment scores. This study examines the generalizability of medical student workplace-based competency assessments including the impact of rater-adjusted scores for leniency and stringency. METHODS: Data were collected from summative clerkship assessments completed for 204 students during 2017-2018 the clerkship at a single institution. Generalizability theory was used to explore variance attributed to different facets (rater, learner, item, and competency domain) through three unbalanced random-effects models by clerkship including applying assessor stringency-leniency adjustments. RESULTS: In the original assessments, only 4-8% of the variance was attributed to the student with the remainder being rater variance and error. Aggregating items to create a composite score increased variability attributable to the student (5-13% of variance). Applying a stringency-leniency ('hawk-dove') correction substantially increased the variance attributed to the student (14.8-17.8%) and reliability. Controlling for assessor leniency/stringency reduced measurement error, decreasing the number of assessments required for generalizability from 16-50 to 11-14. CONCLUSIONS: Similar to prior research, most of the variance in competency assessment scores was attributable to raters, with only a small proportion attributed to the student. Making stringency-leniency corrections using rater-adjusted scores improved the psychometric characteristics of assessment scores.

Bringing clerkship grading back to the bedside.

BACKGROUND: The hallmark of medical education is learning from patients through interactions at the bedside. However, many medical schools incorporate clinical clerkship grading systems that incentivize time away from patient care activities to focus on examinations of medical knowledge. The purpose of this innovation was to develop a grading system that encouraged development of competencies beyond medical knowledge, including patient care, communication, and professionalism skills. METHODS: In 2016, the authors convened a diverse workgroup to reform the clerkship grading approach at their institution. The group reviewed relevant literature and discussed approaches used by other institutions. They developed a competency-based criterion system. For each of four competency domains (patient care, medical knowledge (examination score), professionalism, and communication/teamwork), students received a designation of inadequate, competent, or exemplary. The highest grade ("honors") was awarded to students who met the criteria for exemplary performance in at least two domains. It was a primarily compensatory model, although to achieve competency (pass) scoring was non-compensatory (acceptable performance in all categories). RESULTS: A total of 231 medical students received 1499 clerkship grades during the 2018-2019 year. Compared to previous years, more students (40% vs. 15%) received honors. A substantial proportion (43%) received honors without achieving an exemplary designation in the medical knowledge domain (i.e., standardized examination). FINDINGS AND DISCUSSION: The revised grading system provided several avenues for students to excel and receive honors in their clerkships allowing for recognition of students who excelled in patient care-related areas other than standardized examinations.

Evaluating the Reliability and Validity Evidence of the RIME (Reporter-Interpreter-Manager-Educator) Framework for Summative Assessments Across Clerkships.

PURPOSE: The ability of medical schools to accurately and reliably assess medical student clinical performance is paramount. The RIME (reporter-interpreter-manager-educator) schema was originally developed as a synthetic and intuitive assessment framework for internal medicine clerkships. Validity evidence of this framework has not been rigorously evaluated outside of internal medicine. This study examined factors contributing to variability in RIME assessment scores using generalizability theory and decision studies across multiple clerkships, thereby contributing to its internal structure validity evidence. METHOD: Data were collected from RIME-based summative clerkship assessments during 2018-2019 at Virginia Commonwealth University. Generalizability theory was used to explore variance attributed to different facets through a series of unbalanced random-effects models by clerkship. For all analyses, decision (D-) studies were conducted to estimate the effects of increasing the number of assessments. RESULTS: From 231 students, 6,915 observations were analyzed. Interpreter was the most common RIME designation (44.5%-46.8%) across all clerkships. Variability attributable to students ranged from 16.7% in neurology to 25.4% in surgery. D-studies showed the number of assessments needed to achieve an acceptable reliability (0.7) ranged from 7 in pediatrics and surgery to 11 in internal medicine and 12 in neurology. However, depending on the clerkship each student received between 3 and 8 assessments. CONCLUSIONS: This study conducted generalizability- and D-studies to examine the internal structure validity evidence of RIME clinical performance assessments across clinical clerkships. Substantial proportion of variance in RIME assessment scores was attributable to the rater, with less attributed to the student. However, the proportion of variance attributed to the student was greater than what has been demonstrated in other generalizability studies of summative clinical assessments. Overall, these findings support the use of RIME as a framework for assessment across clerkships and demonstrate the number of assessments required to obtain sufficient reliability.

Cortical superficial siderosis multifocality in cerebral amyloid angiopathy: A prospective study.

OBJECTIVE: In order to explore the mechanisms of cortical superficial siderosis (cSS) multifocality and its clinical implications for recurrent intracerebral hemorrhage (ICH) risk in patients with cerebral amyloid angiopathy (CAA), we used a new rating method that we developed specifically to evaluate cSS extent at spatially separated foci. METHODS: Consecutive patients with CAA-related ICH according to Boston criteria from a single-center prospective cohort were analyzed. The new score that assesses cSS multifocality (total range 0-4) showed excellent interrater reliability (k = 0.87). The association of cSS with markers of CAA and acute ICH was investigated. Patients were followed prospectively for recurrent symptomatic ICH. RESULTS: The cohort included 313 patients with CAA. Multifocal cSS prevalence was 21.1%. APOE ε2 allele prevalence was higher in patients with multifocal cSS. In probable/definite CAA, cSS multifocality was independently associated with neuroimaging markers of CAA severity, including lobar microbleeds, but not with acute ICH features, which conversely, were determinants of cSS in possible CAA. During a median follow-up of 2.6 years (interquartile range 0.9-5.1 years), the annual ICH recurrence rates per cSS scores (0-4) were 5%, 6.5%, 13.5%, 16.2%, and 26.9%, respectively. cSS multifocality (presence and spread) was the only independent predictor of increased symptomatic ICH risk (hazard ratio 3.19; 95% confidence interval 1.77-5.75; p < 0.0001). CONCLUSIONS: The multifocality of cSS correlates with disease severity in probable CAA; therefore cSS is likely to be caused by discrete hemorrhagic foci. The new cSS scoring system might be valuable for clinicians in determining annual risk of ICH recurrence.

Developing multidisciplinary clinics for neuromuscular care and research.

Multidisciplinary care is considered the standard of care for both adult and pediatric neuromuscular disorders and has been associated with improved quality of life, resource utilization, and health outcomes. Multidisciplinary care is delivered in multidisciplinary clinics that coordinate care across multiple specialties by reducing travel burden and streamlining care. In addition, the multidisciplinary care setting facilitates the integration of clinical research, patient advocacy, and care innovation (e.g., telehealth). Yet, multidisciplinary care requires substantial commitment of staff time and resources. We calculated personnel costs in our ALS clinic in 2015 and found an average cost per patient visit of $580, of which only 45% was covered by insurance reimbursement. In this review, we will describe classic and emerging concepts in multidisciplinary care models for adult and pediatric neuromuscular disease. We will then explore the financial impact of multidisciplinary care with emphasis on sustainability and metrics to demonstrate quality and value. Muscle Nerve 56: 848-858, 2017.

Distribution of lacunes in cerebral amyloid angiopathy and hypertensive small vessel disease.

OBJECTIVE: To evaluate whether the burden of deep and lobar lacunes differs between patients with intracerebral hemorrhage (ICH) with definite/probable cerebral amyloid angiopathy (CAA) per the Boston criteria and hypertensive small vessel disease (HTN-SVD; ICH in basal ganglia, thalami, brainstem). METHODS: We defined lobar and deep lacunes similar to the topographic distribution used for ICH and cerebral microbleeds (CMBs). We then compared their distribution between patients with CAA-ICH and those with strictly deep CMB and ICH (HTN-ICH). The independent associations of lacune location with the diagnosis of CAA-ICH and HTN-ICH were evaluated with multivariable models. The relationship between lobar lacunes and white matter hyperintensity (WMH) volume was evaluated by means of partial correlation analyses adjusted for age and a validated visual scale. RESULTS: In our final cohort of 316 patients with ICH, lacunes were frequent (24.7%), with similar rates in 191 patients with CAA and 125 with HTN-ICH (23% vs 27.2%, p = 0.4). Lobar lacunes were more commonly present in CAA (20.4% vs 5.7%, p < 0.001), while deep lacunes were more frequent in HTN-ICH (15.2% vs 2.1%, p < 0.001). After correction for demographics and clinical and neuroimaging markers of SVD, lobar lacunes were associated with CAA (p = 0.003) and deep lacunes with HTN-ICH (p < 0.001). Lobar lacunes in 80% of the cases were at least in contact with WMH, and after adjustment for age, they were highly correlated to WMH volume (r = 0.42, p < 0.001). CONCLUSIONS: Lobar lacunes are associated with CAA, whereas deep lacunes are more frequent in HTN-SVD. Lobar lacunes seem to have a close relationship with WMH, suggesting a possible common origin.

MRI-visible perivascular spaces in cerebral amyloid angiopathy and hypertensive arteriopathy.

OBJECTIVE: To assess MRI-visible enlarged perivascular spaces (EPVS) burden and different topographical patterns (in the centrum semiovale [CSO] and basal ganglia [BG]) in 2 common microangiopathies: cerebral amyloid angiopathy (CAA) and hypertensive arteriopathy (HA). METHODS: Consecutive patients with spontaneous intracerebral hemorrhage (ICH) from a prospective MRI cohort were included. Small vessel disease MRI markers, including cerebral microbleeds (CMBs), cortical superficial siderosis (cSS), and white matter hyperintensities (WMH), were rated. CSO-EPVS/BG-EPVS were assessed on a validated 4-point visual rating scale (0 = no EPVS, 1 = <10, 2 = 11-20, 3 = 21-40, and 4 = >40 EPVS). We tested associations of predefined high-degree (score >2) CSO-EPVS and BG-EPVS with other MRI markers in multivariable logistic regression. We subsequently evaluated associations with CSO-EPVS predominance (i.e., CSO-EPVS > BG-EPVS) and BG-EPVS predominance pattern (i.e., BG-EPVS > CSO-EPVS) in adjusted multinomial logistic regression (reference group, BG-EPVS = CSO-EPVS). RESULTS: We included 315 patients with CAA-ICH and 137 with HA-ICH. High-degree CSO-EPVS prevalence was greater in CAA-related ICH vs HA-related ICH (43.8% vs 17.5%, p < 0.001). In multivariable logistic regression, high-degree CSO-EPVS was associated with lobar CMB (odds ratio [OR] 1.33, 95% confidence interval [CI] 1.10-1.61, p = 0.003) and cSS (OR 2.08, 95% CI 1.30-3.32, p = 0.002). Deep CMBs (OR 2.85, 95% CI 1.75-4.64, p < 0.0001) and higher WMH volume (OR 1.02, 95% CI 1.01-1.04, p = 0.010) were predictors of high-degree BG-EPVS. A CSO-EPVS-predominant pattern was more common in CAA-ICH than in HA-ICH (75.9% vs 39.4%, respectively, p < 0.0001). CSO-PVS predominance was associated with lobar CMB burden and cSS, while BG-EPVS predominance was associated with HA-ICH and WMH volumes. CONCLUSIONS: Different patterns of MRI-visible EPVS provide insights into the dominant underlying microangiopathy type in patients with spontaneous ICH.

Association of Key Magnetic Resonance Imaging Markers of Cerebral Small Vessel Disease With Hematoma Volume and Expansion in Patients With Lobar and Deep Intracerebral Hemorrhage.

IMPORTANCE: Hematoma expansion is an important determinant of outcome in spontaneous intracerebral hemorrhage (ICH) due to small vessel disease (SVD), but the association between the severity of the underlying SVD and the extent of bleeding at the acute phase is unknown to date. OBJECTIVE: To investigate the association between key magnetic resonance imaging (MRI) markers of SVD (as per the Standards for Reporting Vascular Changes on Neuroimaging [STRIVE] guidelines) and hematoma volume and expansion in patients with lobar or deep ICH. DESIGN, SETTING, AND PARTICIPANTS: Analysis of data collected from 418 consecutive patients admitted with primary lobar or deep ICH to a single tertiary care medical center between January 1, 2000, and October 1, 2012. Data were analyzed on March 4, 2016. Participants were consecutive patients with computed tomographic images allowing ICH volume calculation and MRI allowing imaging markers of SVD assessment. MAIN OUTCOMES AND MEASURES: The ICH volumes at baseline and within 48 hours after symptom onset were measured in 418 patients with spontaneous ICH without anticoagulant therapy, and hematoma expansion was calculated. Cerebral microbleeds, cortical superficial siderosis, and white matter hyperintensity volume were assessed on MRI. The associations between these SVD markers and ICH volume, as well as hematoma expansion, were investigated using multivariable models. RESULTS: This study analyzed 254 patients with lobar ICH (mean [SD] age, 75 [11] years and 140 [55.1%] female) and 164 patients with deep ICH (mean [SD] age 67 [14] years and 71 [43.3%] female). The presence of cortical superficial siderosis was an independent variable associated with larger ICH volume in the lobar ICH group (odds ratio per quintile increase in final ICH volume, 1.49; 95% CI, 1.14-1.94; P = .004). In multivariable models, the absence of cerebral microbleeds was associated with larger ICH volume for both the lobar and deep ICH groups (odds ratios per quintile increase in final ICH volume, 1.41; 95% CI, 1.11-1.81; P = .006 and 1.43; 95% CI, 1.04-1.99; P = .03; respectively) and with hematoma expansion in the lobar ICH group (odds ratio, 1.70; 95% CI, 1.07-2.92; P = .04). The white matter hyperintensity volumes were not associated with either hematoma volume or expansion. CONCLUSIONS AND RELEVANCE: In patients admitted with primary lobar or deep ICH to a single tertiary care medical center, the presence of cortical superficial siderosis was an independent variable associated with larger lobar ICH volume, and the absence of cerebral microbleeds was associated with larger lobar and deep ICHs. The absence of cerebral microbleeds was independently associated with more frequent hematoma expansion in patients with lobar ICH. We provide an analytical framework for future studies aimed at limiting hematoma expansion.

Intracranial atherosclerosis and cerebral small vessel disease in intracerebral hemorrhage patients.

BACKGROUND: The association between cerebral small vessel diseases (cSVD) and intracranial atherosclerosis is debated and conflicting results have been reported. We sought to investigate this association in patients with intracerebral hemorrhage (ICH), due to severe cSVD. METHODS: Consecutive ICH patients were divided into those meeting criteria for cerebral amyloid angiopathy (CAA) and those with deep hypertensive ICH consistent with hypertensive cSVD (HTN-SVD). White matter hyperintensity volumes (WMH) and microbleed counts (MB) were measured on MRI. CTA was rated for severity of intracranial carotid calcifications and for presence of >50% intracranial stenosis (ICS). Associations of intracranial atherosclerosis severity with type of SVD (CAA vs HTN-cSVD) and with imaging and clinical markers of cSVD burden were analyzed. RESULTS: The cohort included 253 CAA and 90 HTN-SVD patients. In multivariable models, the type of cSVD (CAA vs. HTN-cSVD) was not associated with calcification severity (OR=1.04, 95% CI [0.62-3.5], p=0.37) or presence of ICS (OR=0.84, 95% CI [0.21-2.74], p=0.78). We found no association between intracranial atherosclerosis (calcifications and stenoses) and parenchymal markers of cSVD severity (WMH and MB, adjusted p≥0.2 for all comparisons) and no association with presence of dementia before ICH (adjusted p≥0.2 for both comparisons). CONCLUSIONS: We found no association between intracranial atherosclerosis and parenchymal or clinical consequences of cSVD, suggesting that cSVDs while sharing some risk factors are not influenced by upstream larger vessel pathologies.

White matter hyperintensity patterns in cerebral amyloid angiopathy and hypertensive arteriopathy.

OBJECTIVE: To identify different white matter hyperintensity (WMH) patterns between 2 hemorrhage-prone cerebral small vessel diseases (SVD): cerebral amyloid angiopathy (CAA) and hypertensive arteriopathy (HA). METHODS: Consecutive patients with SVD-related intracerebral hemorrhage (ICH) from a single-center prospective cohort were analyzed. Four predefined subcortical WMH patterns were compared between the CAA and HA groups. These WMH patterns were (1) multiple subcortical spots; (2) peri-basal ganglia (BG); (3) large posterior subcortical patches; and (4) anterior subcortical patches. Their associations with other imaging (cerebral microbleeds [CMBs], enlarged perivascular spaces [EPVS]) and clinical markers of SVD were investigated using multivariable logistic regression. RESULTS: The cohort included 319 patients with CAA and 137 patients with HA. Multiple subcortical spots prevalence was higher in the CAA compared to the HA group (29.8% vs 16.8%; p = 0.004). Peri-BG WMH pattern was more common in the HA- vs the CAA-ICH group (19% vs 7.8%; p = 0.001). In multivariable logistic regression, presence of multiple subcortical spots was associated with lobar CMBs (odds ratio [OR] 1.23; 95% confidence interval [CI] 1.01-1.50, p = 0.039) and high degree of centrum semiovale EPVS (OR 2.43; 95% CI 1.56-3.80, p < 0.0001). By contrast, age (OR 1.05; 95% CI 1.02-1.09, p = 0.002), deep CMBs (OR 2.46; 95% CI 1.44-4.20, p = 0.001), total WMH volume (OR 1.02; 95% CI 1.01-1.04, p = 0.002), and high BG EPVS degree (OR 8.81; 95% CI 3.37-23.02, p < 0.0001) were predictors of peri-BG WMH pattern. CONCLUSION: Different patterns of subcortical leukoaraiosis visually identified on MRI might provide insights into the dominant underlying microangiopathy type as well as mechanisms of tissue injury in patients with ICH.

Incidence of symptomatic hemorrhage in patients with lobar microbleeds.

BACKGROUND AND PURPOSE: Lobar microbleeds suggestive of cerebral amyloid angiopathy (CAA) are often identified on MRI in the absence of lobar intracerebral hemorrhage (ICH). We compared the baseline characteristics and risk of subsequent ICH among such patients to those presenting with CAA-related lobar ICH. METHODS: Clinical data (demographics, risk factors), apolipoprotein E genotype, neuroimaging markers of CAA severity (microbleed counts, leukoaraiosis volume), and clinical outcomes (incidence rates of ICH and death during a mean follow-up of 5.3±3.8 years) were compared between 63 patients enrolled because of incidentally found microbleeds and 316 with CAA-related ICH, in our prospectively enrolled cohort. Predictors of incident ICH were explored in the microbleed-only patients using multivariable Cox regression models. RESULTS: Microbleed-only patients shared similar demographic, apolipoprotein E, and vascular risk profiles with lobar ICH patients, but had more lobar microbleeds (median, 10 versus 2; P<0.001) and higher leukoaraiosis volumes (median, 31 versus 23 mL; P=0.02). Microbleed-only patients had a nontrivial incidence rate of ICH, not different from patients presenting with ICH (5 versus 8.9 per 100 person-years; adjusted hazard ratio, 0.58; 95% confidence interval, 0.31-1.06; P=0.08). Microbleed-only patients had a higher mortality rate (hazard ratio, 1.67; 95% confidence interval, 1.1-2.6) compared with ICH survivors. Warfarin use and increasing age were independent predictors of future ICH among microbleed-only patients after correction for other covariates. CONCLUSIONS: Patients presenting with isolated lobar microbleeds on MRI have a genetic, neuroimaging, and hemorrhagic risk profile suggestive of severe CAA pathology. They have a substantial risk of incident ICH, potentially affecting decisions regarding anticoagulation in clinical situations.

Cerebral microbleeds and macrobleeds: should they influence our recommendations for antithrombotic therapies?

Intracerebral hemorrhage (ICH, or macrobleeds) and cerebral microbleeds-smaller foci of hemosiderin deposits commonly detected by magnetic resonance imaging of older adults with or without ICH-are both associated with an increased risk of future ICH. These hemorrhagic pathologies also share risk factors with ischemic thromboembolic conditions that may require antithrombotic therapy, requiring specialists in cardiology, internal medicine, and neurology to weigh the benefits vs hemorrhagic risks of antithrombotics in individual patients. This paper will review recent advances in our understanding of hemorrhage prone cerebrovascular pathologies with a particular emphasis on use of these markers in decision making for antithrombotic use.