Exploring the use of Rasch modelling in "common content" items for multi-site and multi-year assessment.

Rasch modelling is a powerful tool for evaluating item performance, measuring drift in difficulty over time, and comparing students who sat assessments at different times or at different sites. Here, we use data from thirty UK medical schools to describe the benefits of Rasch modelling in quality assurance and the barriers to using it. Sixty "common content" multiple choice items were offered to all UK medical schools in 2016-17, and a further sixty in 2017-18, with five available in both years. Thirty medical schools participated, for sixty total datasets across two sessions, and 14,342 individual sittings. Schools selected items to embed in written assessment near the end of their programmes. We applied Rasch modelling to evaluate unidimensionality, model fit statistics and item quality, horizontal equating to compare performance across schools, and vertical equating to compare item performance across time. Of the sixty sittings, three provided non-unidimensional data, and eight violated goodness of fit measures. Item-level statistics identified potential improvements in item construction and provided quality assurance. Horizontal equating demonstrated large differences in scores across schools, while vertical equating showed item characteristics were stable across sessions. Rasch modelling provides significant advantages in model- and item- level reporting compared to classical approaches. However, the complexity of the analysis and the smaller number of educators familiar with Rasch must be addressed locally for a programme to benefit. Furthermore, due to the comparative novelty of Rasch modelling, there is greater ambiguity on how to proceed when a Rasch model identifies misfitting or problematic data.

Endothelin blockade prevents the long-term cardiovascular and renal sequelae of acute kidney injury in mice.

Acute kidney injury (AKI) is common and associated with increased risks of cardiovascular and chronic kidney disease. Causative molecular/physiological pathways are poorly defined. There are no therapies to improve long-term outcomes. An activated endothelin system promotes cardiovascular and kidney disease progression. We hypothesized a causal role for this in the transition of AKI to chronic disease. Plasma endothelin-1 was threefold higher; urine endothelin-1 was twofold higher; and kidney preproendothelin-1, endothelin-A, and endothelin-B receptor message up-regulated in patients with AKI. To show causality, AKI was induced in mice by prolonged ischemia with a 4-week follow-up. Ischemic injury resulted in hypertension, endothelium-dependent and endothelium-independent macrovascular and microvascular dysfunction, and an increase in circulating inflammatory Ly6Chigh monocytes. In the kidney, we observed fibrosis, microvascular rarefaction, and inflammation. Administration of endothelin-A antagonist, but not dual endothelin-A/B antagonist, normalized blood pressure, improved macrovascular and microvascular function, and prevented the transition of AKI to CKD. Endothelin-A blockade reduced circulating and renal proinflammatory Ly6Chigh monocytes and B cells, and promoted recruitment of anti-inflammatory Ly6Clow monocytes to the kidney. Blood pressure reduction alone provided no benefits; blood pressure reduction alongside blockade of the endothelin system was as effective as endothelin-A antagonism in mitigating the long-term sequelae of AKI in mice. Our studies suggest up-regulation of the endothelin system in patients with AKI and show in mice that existing drugs that block the endothelin system, particularly those coupling vascular support and anti-inflammatory action, can prevent the transition of AKI to chronic kidney and cardiovascular disease.

Utility of interval kidney biopsy in ANCA-associated vasculitis.

OBJECTIVES: ANCA-associated vasculitis (AAV) is a rare autoimmune disorder that commonly involves the kidney. Early identification of kidney involvement, assessing treatment-response and predicting outcome are important clinical challenges. Here, we assessed the potential utility of interval kidney biopsy in AAV. METHODS: In a tertiary referral centre with a dedicated vasculitis service, we identified patients with AAV who had undergone interval kidney biopsy, defined as a repeat kidney biopsy (following an initial biopsy showing active AAV) undertaken to determine the histological response in the kidney following induction immunosuppression. We analysed biochemical, histological and outcome data, including times to kidney failure and death for all patients. RESULTS: We identified 57 patients with AAV who underwent at least one interval kidney biopsy (59 interval biopsies in total; median time to interval biopsy ∼130 days). Of the 59 interval biopsies performed, 24 (41%) patients had clinically suspected active disease at time of biopsy which was confirmed histologically in only 42% of cases; 35 (59%) patients were in clinical disease-remission, and this was correct in 97% of cases. The clinician's impression was incorrect in one in four patients. Hematuria at interval biopsy did not correlate with histological activity. Interval biopsy showed fewer acute lesions and more chronic damage compared with initial biopsy and led to immunosuppressive treatment-change in 75% (44/59) of patients. Clinical risk prediction tools tended to operate better using interval biopsy data. CONCLUSION: Interval kidney biopsy is useful for determining treatment-response and subsequent disease management in AAV. It may provide better prognostic information than initial kidney biopsy and should be considered for inclusion into future clinical trials and treatment protocols for patients with AAV.

Variation in performance on common content items at UK medical schools.

BACKGROUND: Due to differing assessment systems across UK medical schools, making meaningful cross-school comparisons on undergraduate students' performance in knowledge tests is difficult. Ahead of the introduction of a national licensing assessment in the UK, we evaluate schools' performances on a shared pool of "common content" knowledge test items to compare candidates at different schools and evaluate whether they would pass under different standard setting regimes. Such information can then help develop a cross-school consensus on standard setting shared content. METHODS: We undertook a cross-sectional study in the academic sessions 2016-17 and 2017-18. Sixty "best of five" multiple choice 'common content' items were delivered each year, with five used in both years. In 2016-17 30 (of 31 eligible) medical schools undertook a mean of 52.6 items with 7,177 participants. In 2017-18 the same 30 medical schools undertook a mean of 52.8 items with 7,165 participants, creating a full sample of 14,342 medical students sitting common content prior to graduation. Using mean scores, we compared performance across items and carried out a "like-for-like" comparison of schools who used the same set of items then modelled the impact of different passing standards on these schools. RESULTS: Schools varied substantially on candidate total score. Schools differed in their performance with large (Cohen's d around 1) effects. A passing standard that would see 5 % of candidates at high scoring schools fail left low-scoring schools with fail rates of up to 40 %, whereas a passing standard that would see 5 % of candidates at low scoring schools fail would see virtually no candidates from high scoring schools fail. CONCLUSIONS: Candidates at different schools exhibited significant differences in scores in two separate sittings. Performance varied by enough that standards that produce realistic fail rates in one medical school may produce substantially different pass rates in other medical schools - despite identical content and the candidates being governed by the same regulator. Regardless of which hypothetical standards are "correct" as judged by experts, large institutional differences in pass rates must be explored and understood by medical educators before shared standards are applied. The study results can assist cross-school groups in developing a consensus on standard setting future licensing assessment.

Effect of remote online exam delivery on student experience and performance in applied knowledge tests.

BACKGROUND: The use of remote online delivery of summative assessments has been underexplored in medical education. Due to the COVID-19 pandemic, all end of year applied knowledge multiple choice question (MCQ) tests at one UK medical school were switched from on campus to remote assessments. METHODS: We conducted an online survey of student experience with remote exam delivery and compared test performance in remote versus invigilated campus-based forms of similar assessments for Year 4 and 5 students across two academic years. RESULTS: Very few students experienced technical or practical problems in completing their exam remotely. Test anxiety was reduced for some students but increased for others. The majority of students preferred the traditional setting of invigilated exams in a computer lab, feeling this ensured an even playing field for all candidates. Mean score was higher for Year 4 students in the remotely-delivered versus campus-based form of the same exam (76.53% [SD 6.57] vs. 72.81% [6.64]; t438.38 = 5.94, p = 0.001; d = 0.56), whereas candidate performance was equivalent across both forms for Year 5 students. CONCLUSIONS: Remote online MCQ exam delivery is an effective and generally acceptable approach to summative assessment, and could be used again in future without detriment to students if onsite delivery is not possible.

Deletion of the myeloid endothelin-B receptor confers long-term protection from angiotensin II-mediated kidney, eye and vessel injury.

The endothelin system may be an important player in hypertensive end-organ injury as endothelin-1 increases blood pressure and is pro-inflammatory. The immune system is emerging as an important regulator of blood pressure and we have shown that the early hypertensive response to angiotensin-II infusion was amplified in mice deficient of myeloid endothelin-B (ETB) receptors (LysM-CreEdnrblox/lox). Hypothesizing that these mice would display enhanced organ injury, we gave angiotensin-II to LysM-CreEdnrblox/lox and littermate controls (Ednrblox/lox) for six weeks. Unexpectedly, LysM-CreEdnrblox/lox mice were significantly protected from organ injury, with less proteinuria, glomerulosclerosis and inflammation of the kidney compared to controls. In the eye, LysM-CreEdnrblox/lox mice had fewer retinal hemorrhages, less microglial activation and less vessel rarefaction. Cardiac remodeling and dysfunction were similar in both groups at week six but LysM-CreEdnrblox/lox mice had better endothelial function. Although blood pressure was initially higher in LysM-CreEdnrblox/lox mice, this was not sustained. A natriuretic switch at about two weeks, due to enhanced ETB signaling in the kidney, induced a hypertensive reversal. By week six, blood pressure was lower in LysM-CreEdnrblox/lox mice than in controls. At six weeks, macrophages from LysM-CreEdnrblox/lox mice were more anti-inflammatory and had greater phagocytic ability compared to the macrophages of Ednrblox/lox mice. Thus, myeloid cell ETB receptor signaling drives this injury both through amplifying hypertension and by inflammatory polarization of macrophages.

Long-term outcomes in elderly patients with ANCA-associated vasculitis.

OBJECTIVE: ANCA-associated vasculitis (AAV) is a small vessel vasculitis that commonly presents in the elderly. However, there are few long-term outcome data for these patients. Here, we assessed long-term outcomes in a single-centre cohort of elderly patients with AAV. Additionally, we tested whether a pre-morbid frailty score could aid prognosis. METHODS: Using a prospectively-compiled dataset, we investigated patients over the age of 65 who presented with AAV between 2005 and 2017 to a regional vasculitis centre. We used a Cox model to determine the factors associated with mortality. We also compared outcomes in pre-specified subgroups stratified by baseline frailty score, ANCA serotype and induction immunosuppression (with cyclophosphamide, rituximab or mycophenolate mofetil used as the main glucocorticoid-sparing agent). RESULTS: 83 patients were included in the study and were followed for a median of 1203 days. Median age was 74 years (range 65-92). Two- and five-year survival in the overall cohort were 83% (95% CI 75, 92%) and 75% (95% CI 65, 86%), respectively. The median cumulative dose of oral prednisolone was 2030 mg during the first three months. Only one patient received intravenous glucocorticoids. Age, frailty score and CRP at presentation were independently associated with mortality; all deaths occurred in patients aged over 75 at presentation. Patients treated with a cyclophosphamide-based induction regimen tended to be younger than those treated with rituximab or mycophenolate mofetil. Survival was better in the cyclophosphamide-treated group. CONCLUSION: In the contemporary era, the overall prognosis of AAV in elderly patients is good. Baseline frailty associates with disease outcomes including mortality. A low-dose glucocorticoid regimen (avoiding intravenous methylprednisolone) can be used to treat AAV effectively in elderly patients.

Using prescribing very short answer questions to identify sources of medication errors: a prospective study in two UK medical schools.

OBJECTIVE: To assess the utility and ability of the novel prescribing very short answer (VSA) question format to identify the sources of undergraduate prescribing errors when compared with the conventional single best answer (SBA) question format and assess the acceptability of machine marking prescribing VSAs. DESIGN: A prospective study involving analysis of data generated from a pilot two-part prescribing assessment. SETTING: Two UK medical schools. PARTICIPANTS: 364 final year medical students took part. Participation was voluntary. There were no other inclusion or exclusion criteria. OUTCOMES: (1) Time taken to mark and verify VSA questions (acceptability), (2) differences between VSA and SBA scores, (3) performance in VSA and (4) SBA format across different subject areas and types of prescribing error made in the VSA format. RESULTS: 18 200 prescribing VSA questions were marked and verified in 91 min. The median percentage score for the VSA test was significantly lower than the SBA test (28% vs 64%, p<0.0001). Significantly more prescribing errors were detected in the VSA format than the SBA format across all domains, notably in prescribing insulin (96.4% vs 50.3%, p<0.0001), fluids (95.6% vs 55%, p<0.0001) and analgesia (85.7% vs 51%, p<0.0001). Of the incorrect VSA responses, 33.1% were due to the medication prescribed, 6.0% due to the dose, 1.4% due to the route and 4.8% due to the frequency. CONCLUSIONS: Prescribing VSA questions represent an efficient tool for providing detailed insight into the sources of significant prescribing errors, which are not identified by SBA questions. This makes the prescribing VSA a valuable formative assessment tool to enhance students' skills in safe prescribing and to potentially reduce prescribing errors.

A novel role for myeloid endothelin-B receptors in hypertension.

AIMS: Hypertension is common. Recent data suggest that macrophages (Mφ) contribute to, and protect from, hypertension. Endothelin-1 (ET-1) is the most potent endogenous vasoconstrictor with additional pro-inflammatory properties. We investigated the role of the ET system in experimental and clinical hypertension by modifying Mφ number and phenotype. METHODS AND RESULTS: In vitro, Mφ ET receptor function was explored using pharmacological, gene silencing, and knockout approaches. Using the CD11b-DTR mouse and novel mice with myeloid cell-specific endothelin-B (ETB) receptor deficiency (LysMETB-/-), we explored the effects of modifying Mφ number and phenotype on the hypertensive effects of ET-1, angiotensin II (ANG II), a model that is ET-1 dependent, and salt. In patients with small vessel vasculitis, the impacts of Mφ depleting and non-depleting therapies on blood pressure (BP) and endothelial function were examined. Mouse and human Mφ expressed both endothelin-A and ETB receptors and displayed chemokinesis to ET-1. However, stimulation of Mφ with exogenous ET-1 did not polarize Mφ phenotype. Interestingly, both mouse and human Mφ cleared ET-1 through ETB receptor mediated, and dynamin-dependent, endocytosis. Mφ depletion resulted in an augmented chronic hypertensive response to both ET-1 and salt. LysMETB-/- mice displayed an exaggerated hypertensive response to both ET-1 and ANG II. Finally, in patients who received Mφ depleting immunotherapy BP was higher and endothelial function worse than in those receiving non-depleting therapies. CONCLUSION: Mφ and ET-1 may play an important role in BP control and potentially have a critical role as a therapeutic target in hypertension.

Granulocyte macrophage-colony stimulating factor: A key modulator of renal mononuclear phagocyte plasticity.

Macrophage-colony stimulating factor (M-CSF) and granulocyte macrophage-colony stimulating factor (GM-CSF) play key roles in the differentiation of macrophages and dendritic cells (DCs). We examined the effect of treatment with M-CSF-containing macrophage medium or GM-CSF-containing DC medium upon the phenotype of murine bone marrow-derived macrophages and DCs. Culture of macrophages for 5 days in DC medium reduced F4/80 expression and increased CD11c expression with cells effectively stimulating T cell proliferation in a mixed lymphocyte reaction. DC medium treatment of macrophages significantly reduced phagocytosis of both apoptotic cells and latex beads and strongly induced the expression of the chemokine receptor CCR7 known to be involved in DC trafficking to lymph nodes. Lysates of obstructed murine kidneys expressed both M-CSF and GM-CSF though M-CSF expression was dominant (M-CSF:GM-CSF ratio ∼30:1). However, combination treatment with both M-CSF and GM-CSF (ratio 30:1) indicated that small amounts of GM-CSF skewed macrophages towards a DC-like phenotype. To determine whether macrophage phenotype might be modulated in vivo we tracked CD45.1+ bone marrow-derived macrophages intravenously administered to CD45.2+ mice with unilateral ureteric obstruction. Flow cytometry of enzyme dissociated kidneys harvested 3 days later indicated CD11c and MHC Class II upregulation by adoptively transferred CD45.1+ cells with CD45.1+ cells evident in draining renal lymph nodes. Our data suggests that GM-CSF modulates mononuclear phagocyte plasticity, which likely promotes resolution of injury and healing in the injured kidney.

Variation in passing standards for graduation-level knowledge items at UK medical schools.

OBJECTIVES: Given the absence of a common passing standard for students at UK medical schools, this paper compares independently set standards for common 'one from five' single-best-answer (multiple-choice) items used in graduation-level applied knowledge examinations and explores potential reasons for any differences. METHODS: A repeated cross-sectional study was conducted. Participating schools were sent a common set of graduation-level items (55 in 2013-2014; 60 in 2014-2015). Items were selected against a blueprint and subjected to a quality review process. Each school employed its own standard-setting process for the common items. The primary outcome was the passing standard for the common items by each medical school set using the Angoff or Ebel methods. RESULTS: Of 31 invited medical schools, 22 participated in 2013-2014 (71%) and 30 (97%) in 2014-2015. Schools used a mean of 49 and 53 common items in 2013-2014 and 2014-2015, respectively, representing around one-third of the items in the examinations in which they were embedded. Data from 19 (61%) and 26 (84%) schools, respectively, met the inclusion criteria for comparison of standards. There were statistically significant differences in the passing standards set by schools in both years (effect sizes (f2 ): 0.041 in 2013-2014 and 0.218 in 2014-2015; both p < 0.001). The interquartile range of standards was 5.7 percentage points in 2013-2014 and 6.5 percentage points in 2014-2015. There was a positive correlation between the relative standards set by schools in the 2 years (Pearson's r = 0.57, n = 18, p = 0.014). Time allowed per item, method of standard setting and timing of examination in the curriculum did not have a statistically significant impact on standards. CONCLUSIONS: Independently set standards for common single-best-answer items used in graduation-level examinations vary across UK medical schools. Further work to examine standard-setting processes in more detail is needed to help explain this variability and develop methods to reduce it.

Choice of Illumination System & Fluorophore for Multiplex Immunofluorescence on FFPE Tissue Sections.

The recent availability of novel dyes and alternative light sources to facilitate complex tissue immunofluorescence studies such as multiplex labelling has not been matched by reports critically evaluating the considerations and relative benefits of these new tools, particularly in combination. Product information is often limited to wavelengths used for older fluorophores (FITC, TRITC & corresponding Alexa dyes family). Consequently, novel agents such as Quantum dots are not widely appreciated or used, despite highly favourable properties including extremely bright emission, stability and potentially reduced tissue autofluorescence at the excitation wavelength. Using spectral analysis, we report here a detailed critical appraisal and comparative evaluation of different light sources and fluorophores in multiplex immunofluorescence of clinical biopsy sections. The comparison includes mercury light, metal halide and 3 different LED-based systems, using 7 Qdots (525, 565, 585, 605, 625, 705), Cy3 and Cy5. We discuss the considerations relevant to achieving the best combination of light source and fluorophore for accurate multiplex fluorescence quantitation. We highlight practical limitations and confounders to quantitation with filter-based approaches.

Working with Commercially Available Quantum Dots for Immunofluorescence on Tissue Sections.

Quantum dots are semiconductor fluorescent nanocrystals that exhibit excellent characteristics compared with more commonly used organic fluorescent dyes. For many years quantum dot conjugated products have been available in multiple forms for fluorescence imaging of tissue sections under the trademark name Qdot®. They have much increased brightness, narrow emission spectrum, large Stokes shift and photostability compared with conventional organic fluorescent dyes, which together make them the fluorophores of choice for demanding requirements. Vivid Qdots are recent replacements for original Qdots, modified to improve brightness, however this has affected the fluorescence stability in commonly used conditions for immunohistochemistry. We present here our investigation of the stability of original and Vivid Qdots in solution and in immunohistochemistry, highlight the potential pitfalls and propose a protocol for stable and reliable multiplex staining with current commercially available original and Vivid Qdots.

Hemin Preconditioning Upregulates Heme Oxygenase-1 in Deceased Donor Renal Transplant Recipients: A Randomized, Controlled, Phase IIB Trial.

BACKGROUND: The enzyme heme oxygenase-1 (HO-1) degrades heme and protects against ischemia-reperfusion injury. Monocytes/macrophages are the major source of HO-1 and higher levels improve renal transplant outcomes. Heme arginate (HA) safely induces HO-1 in humans. METHODS: The Heme Oxygenase-1 in renal Transplantation study was a randomized, placebo-controlled, IIb trial to evaluate HA effect on HO-1 upregulation after deceased donor kidney transplantation. 40 recipients were randomized to either 3 mg kg HA or placebo (0.9% NaCl), given preoperatively (day 0) and again on day 2. Recipient blood and urine were collected daily. Graft biopsies were taken preoperatively and on day 5. Primary outcome was HO-1 upregulation in peripheral blood mononuclear cells (PBMCs). Secondary outcomes were graft HO-1 upregulation and injury, urinary biomarkers, and renal function. RESULTS: The HA upregulated PBMC HO-1 protein more than placebo at 24 hours: HA 11.1 ng/mL versus placebo 0.14 ng/mL (P = < 0.0001). The PBMC HO-1 messenger RNA also increased: HA 2.73-fold versus placebo 1.41-fold (P = 0.02). Heme arginate increased day 5 tissue HO-1 protein immunopositivity compared with placebo: HA 0.21 versus placebo -0.03 (P = 0.02) and % HO-1-positive renal macrophage also increased: HA 50.8 cells per high power field versus placebo 22.3 (P = 0.012). Urinary biomarkers were reduced after HA but not significantly. Histological injury and renal function were similar but the study was not powered for this. Adverse events were equivalent between groups. CONCLUSIONS: The primary outcome was achieved and demonstrated for the first time that HA safely induces HO-1 in transplant recipients. Planned larger studies will determine the impact of HO-1 upregulation on clinical outcomes and evaluate the benefit to patients at risk of ischemia-reperfusion injury.

Benefits of an expanded use of plasma exchange for anti-neutrophil cytoplasmic antibody-associated vasculitis within a dedicated clinical service.

BACKGROUND: Current recommendations for ANCA-associated vasculitis (AAV) support its management within a dedicated clinical service. Therapies for AAV are imperfect with many patients failing to achieve disease control and others experiencing disease relapse. Plasma exchange (PEX) may be beneficial especially when the kidney is involved. METHODS: Within a new, dedicated service we retrospectively assessed, over a 6-year period, the benefits of PEX in two patient cohorts, discriminated by PEX treatment alone. Patients received PEX alongside standard of care if they fulfilled any of the following criteria: 1. serum creatinine >500 µmol/l or dialysis-requiring renal failure, 2. alveolar haemorrhage, 3. renal biopsy showing ≥30 % focal and necrotising lesions ± cellular crescents. Outcome measures included disease remission and relapse, cumulative immunosuppression, and morbidity and mortality. RESULTS: Of 104 new patients, 58 patients received PEX at presentation, 46 did not. Cyclophosphamide and/or rituximab dosing was similar for both groups. Although patients receiving PEX had poorer renal function, a higher C-reactive protein and disease activity score at presentation disease remission rate was similar in both groups (no PEX vs. PEX: 96 % vs. 98 %). The PEX group entered remission quicker (no PEX vs. PEX: 3.9 ± 4.0 vs. 2.8 ± 1.3 months, p < 0.05), with a lower 3-month cumulative glucocorticoid dose (no PEX vs. PEX: 2.5 ± 0.4 vs. 2.3 ± 0.2 g, p < 0.001). Relapse was similar between groups but adverse events lower in the PEX group. CONCLUSIONS: PEX may be of benefit in AAV. Larger, longer randomised controlled trials are now needed.