A Library of Logic Models to Explain How Interventions to Reduce Diagnostic Errors Work.

OBJECTIVES: We aimed to create a library of logic models for interventions to reduce diagnostic error. This library can be used by those developing, implementing, or evaluating an intervention to improve patient care, to understand what needs to happen, and in what order, if the intervention is to be effective. METHODS: To create the library, we modified an existing method for generating logic models. The following five ordered activities to include in each model were defined: preintervention; implementation of the intervention; postimplementation, but before the immediate outcome can occur; the immediate outcome (usually behavior change); and postimmediate outcome, but before a reduction in diagnostic errors can occur. We also included reasons for lack of progress through the model. Relevant information was extracted about existing evaluations of interventions to reduce diagnostic error, identified by updating a previous systematic review. RESULTS: Data were synthesized to create logic models for four types of intervention, addressing five causes of diagnostic error in seven stages in the diagnostic pathway. In total, 46 interventions from 43 studies were included and 24 different logic models were generated. CONCLUSIONS: We used a novel approach to create a freely available library of logic models. The models highlight the importance of attending to what needs to occur before and after intervention delivery if the intervention is to be effective. Our work provides a useful starting point for intervention developers, helps evaluators identify intermediate outcomes, and provides a method to enable others to generate libraries for interventions targeting other errors.

Offenders With Personality Disorder Who Fail to Progress: A Case-Control Study Using Partial Least Squares Structural Equation Modeling Path Analysis.

BACKGROUND: Offenders with personality disorder can be challenging to engage and retain in treatment. The UK Offender Personality Disorder (OPD) pathway aims to proactively and responsively identify and engage offenders with personality disorder. However, a subpopulation of offenders on the pathway have been found to not be accepted into any OPD service and therefore fail to progress. OBJECTIVE: This study aims to identify and describe offenders on the OPD pathway who fail to progress and to understand the causal drivers by which individuals fail to progress in the pathway. METHODS: A sample of 50 offenders on the OPD pathway who had been refused from at least two OPD services (nonprogression group) were compared to 100 offenders accepted into OPD services (control group). Partial least squares structural equation modeling was used to model the causal factors involved in not being accepted into OPD services. RESULTS: The path coefficients in the structural model showed that the most influential factor in nonprogression was attitude toward treatment (ß=.41; P<.001; f2=0.25) alongside those with psychopathology (ß=.41; P<.001; f2=0.25), specifically, psychopathy, psychosis, and co-occurring personality disorder. CONCLUSIONS: The findings of the study provide a basis of how to work with this population in the future to increase the likelihood of acceptance into OPD services.

Revising ethical guidance for the evaluation of programmes and interventions not initiated by researchers.

Public health and service delivery programmes, interventions and policies (collectively, 'programmes') are typically developed and implemented for the primary purpose of effecting change rather than generating knowledge. Nonetheless, evaluations of these programmes may produce valuable learning that helps determine effectiveness and costs as well as informing design and implementation of future programmes. Such studies might be termed 'opportunistic evaluations', since they are responsive to emergent opportunities rather than being studies of interventions that are initiated or designed by researchers. However, current ethical guidance and registration procedures make little allowance for scenarios where researchers have played no role in the development or implementation of a programme, but nevertheless plan to conduct a prospective evaluation. We explore the limitations of the guidance and procedures with respect to opportunistic evaluations, providing a number of examples. We propose that one key missing distinction in current guidance is moral responsibility: researchers can only be held accountable for those aspects of a study over which they have control. We argue that requiring researchers to justify an intervention, programme or policy that would occur regardless of their involvement prevents or hinders research in the public interest without providing any further protections to research participants. We recommend that trial consent and ethics procedures allow for a clear separation of responsibilities for the intervention and the evaluation.

"Who needs secure services for personality disorder?" Results of an expert Delphi study with professional staff.

BACKGROUND: Personality Disorder (PD) is an enduring, multi-faceted mental disorder, associated with adverse health effects, difficulties with interpersonal relationships and in some cases increased risk to others. A limited number of dedicated forensic mental health services are available for serious offenders with severe personality disorder. The recent Offender Personality Disorder (OPD) strategy aims to ensure that most such offenders are treated in prison rather than secure psychiatric services, except in highly complex cases where this is not possible. While the strategy sets out very broad criteria relating to this, greater clarity is needed to support decisions about appropriate transfer and hence enhance public protection. This study explored which characteristics professional experts associate with appropriate transfer from prison to forensic mental health services for high-risk offenders with PD. METHOD: A modified Delphi survey distributed through an online survey system was conducted in two-rounds with a group of professional experts recruited from forensic mental healthcare; criminal justice and specialist commissioning. RESULTS: Fifty-one (56%) respondents completed stage one of the Delphi and 34 (61%) of these completed stage two. Consensus was reached for a total of 22 items indicating complexity, including co-morbid mental illness, high level of risk, lack of progress in prison and high motivation for treatment. A preliminary checklist for these factors was developed. Panel members consistently emphasised the importance of the individual's presenting need, the overall clinical picture and formulation in their free text responses. CONCLUSIONS: Professionals face a complex picture when making decisions regarding suitability for hospital admission for high-risk male offenders with PD, with varied opinions amongst professional experts as to priorities for intervention and a focus on individual needs through formulation. It was, nevertheless, possible to condense these views into a set of consistent variables that can be used to highlight the need for transfer into hospital-based treatment services.

Prioritising allocation of donor human breast milk amongst very low birthweight infants in middle-income countries.

The use of donor human breast milk instead of formula reduces the risk of necrotising enterocolitis in preterm infants when their mother's own milk is insufficient. Use of donor milk is limited by the cost of establishing a milk bank and a lack of donors, but the optimal rationing of limited donor milk is unclear. This paper uses an economic model to explore how a limited donor milk supply should be allocated across very low birthweight infants in South Africa considering 2 outcomes: maximising lives saved and minimising costs. We developed a probabilistic cohort Markov decision model with 10,000 infants across 4 birthweight groups. We evaluated allocation scenarios in which infants in each group could be exclusively formula-fed or fed donor milk for 14 or 28 days and thereafter formula until death or discharge. Prioritising infants in the lowest birthweight groups would save the most lives, whereas prioritising infants in the highest birthweight groups would result in the highest cost savings. All allocation scenarios would be considered very cost-effective in South Africa compared to the use of formula; the "worst case" was $619 per Disability Adjusted Life Year averted. There is a compelling argument to increase the supply of donor milk in middle-income countries. Our analysis could be extended by taking a longer term perspective, using data from more than one country and exploring the use of donor milk as an adjunct to mother's own milk, rather than a pure substitute for it.

The predictive validity of the Living Goods selection tools for community health workers in Kenya: cohort study.

BACKGROUND: Ensuring that selection processes for Community Health Workers (CHWs) are effective is important due to the scale and scope of modern CHW programmes. However they are relatively understudied. While community involvement in selection should never be eliminated entirely, there are other complementary methods that could be used to help identify those most likely to be high-performing CHWs. This study evaluated the predictive validity of three written tests and two individual sections of a one-to-one interview used for selection into CHW posts in eight areas of Kenya. METHODS: A cohort study of CHWs working for Living Goods in eight local areas of Kenya was undertaken. Data on the selection scores, post-training assessment scores and subsequent on-the-job performance (number of household and pregnancy registrations, number of child assessments, proportion of on-time follow-ups and value of goods sold) were obtained for 547 CHWs. Kendall's tau-b correlations between each selection score and performance outcome were calculated. RESULTS: None of the correlations between selection scores and outcomes reached the 0.3 threshold of an "adequate" predictor of performance. Correlations were higher for the written components of the selection process compared to the interview components, with some small negative correlations found for the latter. CONCLUSIONS: If the measures of performance included in this study are considered critical, then further work to develop the CHW selection tools is required. This could include modifying the content of both tools or increasing the length of the written tests to make them more reliable, for if a test is not reliable then it cannot be valid. Other important outcomes not included in this study are retention in post and quality of care. Other CHW programme providers should consider evaluating their own selection tools in partnership with research teams.

The use of low-cost Android tablets to train community health workers in Mukono, Uganda, in the recognition, treatment and prevention of pneumonia in children under five: a pilot randomised controlled trial.

BACKGROUND: Since 2012, The World Health Organization and UNICEF have advocated for community health workers (CHWs) to be trained in Integrated Community Case Management (iCCM) of common childhood illnesses, such as pneumonia. Despite the effectiveness of iCCM, CHWs face many barriers to accessing training. This pilot study compares traditional training with using locally made videos loaded onto low-cost Android tablets to train CHWs on the pneumonia component of iCCM. METHODS: We conducted a pilot randomised controlled trial with CHWs in the Mukono District of Uganda. The unit of randomisation was the sub-county level, and the unit of analysis was at the level of the individual CHW. Eligible CHWs had completed basic iCCM training but had not received any refresher training on the pneumonia component of iCCM in the preceding 2 years. CHWs in the control group received training in the recognition, treatment, and prevention of pneumonia as it is currently delivered, through a 1-day, in-person workshop. CHWs allocated to the intervention group received training via locally made educational videos hosted on low-cost Android tablets. The primary outcome was change in knowledge acquisition, assessed through a multiple choice questionnaire before and after training, and a post-training clinical assessment. The secondary outcome was a qualitative evaluation of CHW experiences of using the tablet platform. RESULTS: In the study, 129 CHWs were enrolled, 66 and 63 in the control and intervention groups respectively. CHWs in both groups demonstrated an improvement in multiple choice question test scores before and after training; however, there was no statistically significant difference in the improvement between groups (t = 1.15, p = 0.254). There was a statistically significant positive correlation (Pearson's r = 0.26, p = 0.03) linking years of education to improvement in test scores in the control group, which was not present in the intervention group. The majority of CHWs expressed satisfaction with the use of tablets as a training tool; however, some reported technical issues (n = 9). CONCLUSION: Tablet-based training is comparable to traditional training in terms of knowledge acquisition. It also proved to be feasible and a satisfactory means of delivering training to CHWs. Further research is required to understand the impacts of scaling such an intervention. TRIAL REGISTRATION: Registered on 23/11/2016 at clinicaltrials.gov ( NCT02971449 ).

Formative evaluation of a training intervention for community health workers in South Africa: A before and after study.

BACKGROUND: Community Health Workers (CHWs) have a crucial role in improving health in their communities and their role is being expanded in many parts of the world. However, the effectiveness of CHWs is limited by poor training and the education of CHWs has received little scientific attention. METHODS: Our study was carried out in two districts of KwaZulu-Natal, South Africa. We developed and piloted an inexpensive (two day) training intervention covering national government priorities: HIV/AIDS, sexually transmitted disease and Tuberculosis; and Women's Sexual and Reproductive Health and Rights. Sixty-four CHWs consented to participate in the main study which measured knowledge gains using a modified Solomon design of four different testing schedules to distinguish between the effects of the intervention, testing and any interaction between intervention and testing. We also measured confidence, satisfaction and costs. RESULTS: Following the training intervention, improvements in knowledge scores were seen across topics and across districts. These changes in knowledge were statistically significant (p<0.001) and of large magnitude (over 45 percentage points or four standard deviations). However, the CHWs assigned to the test-test-train schedule in one district showed high gains in knowledge prior to receiving the training. All CHWs reported high levels of satisfaction with the training and marked improvements in their confidence in advising clients. The training cost around US$48 per CHW per day and has the potential to be cost-effective if the large gains in knowledge are translated into improved field-based performance and thus health outcomes. CONCLUSION: Training CHWs can result in large improvements in knowledge with a short intervention. However, improvements seen in other studies could be due to test 'reactivity'. Further work is needed to measure the generalisability of our results, retention of knowledge and the extent to which improved knowledge is translated into improved practice.

Evaluating the impact of a community health worker programme on non-communicable disease, malnutrition, tuberculosis, family planning and antenatal care in Neno, Malawi: protocol for a stepped-wedge, cluster randomised controlled trial.

INTRODUCTION: This protocol concerns the implementation and evaluation of an intervention designed to realign the existing cadre of community health workers (CHWs) in Neno district, Malawi to better support the care needs of the clients they serve. The proposed intervention is a 'Household Model' where CHWs will be reassigned to households, rather than to specific patients with HIV and/or tuberculosis (TB). METHODS AND ANALYSIS: Using a stepped-wedge, cluster-randomised design, this study investigates whether high HIV retention rates can be replicated for non-communicable diseases (NCDs), and the model's impact on TB and paediatric malnutrition case finding, as well as the uptake of family planning and antenatal care. Eleven sites (health centres and hospitals) were arranged into six clusters (average cluster population 21 800). Primary outcomes include retention in care for HIV and chronic NCDs, TB case finding, paediatric malnutrition case finding, and utilisation of early and complete antenatal care. Clinical outcomes are based on routinely collected data from the Ministry of Health's District Health Information System 2 and an OpenMRS electronic medical record supported by Partners In Health. Additionally, semistructured qualitative interviews with various stakeholders will assess community perceptions and context of the Household Model. ETHICS AND DISSEMINATION: Ethics approval has been obtained from the Malawian National Health Science Research Committee (#16/11/1694) in Lilongwe, Malawi; Partners Healthcare Human Research Committee (#2017P000548/PHS) in Somerville, Massachusetts; and the Biomedical and Scientific Research Ethics Sub-Committee (REGO-2017-2060) at the University of Warwick in Coventry, UK. Dissemination will include manuscripts for peer-reviewed publication as well as a full report detailing the findings of the intervention for the Malawian Ministry of Health. TRIAL REGISTRATION NUMBER: NCT03106727. PRIMARY SPONSOR: Partners In Health | Abwenzi Pa Za Umoyo P.O. Box 56, Neno, Malawi. Protocol Version 4, March 2018.

Cost-effectiveness of health care service delivery interventions in low and middle income countries: a systematic review.

BACKGROUND: Low and middle income countries (LMICs) face severe resource limitations but the highest burden of disease. There is a growing evidence base on effective and cost-effective interventions for these diseases. However, questions remain about the most cost-effective method of delivery for these interventions. We aimed to review the scope, quality, and findings of economic evaluations of service delivery interventions in LMICs. METHODS: We searched PUBMED, MEDLINE, EconLit, and NHS EED for studies published between 1st January 2000 and 30th October 2016 with no language restrictions. We included all economic evaluations that reported incremental costs and benefits or summary measures of the two such as an incremental cost effectiveness ratio. Studies were grouped by both disease area and outcome measure and permutation plots were completed for similar interventions. Quality was judged by the Drummond checklist. RESULTS: Overall, 3818 potentially relevant abstracts were identified of which 101 studies were selected for full text review. Thirty-seven studies were included in the final review. Twenty-three studies reported on interventions we classed as "changing by whom and where care was provided", specifically interventions that entailed task-shifting from doctors to nurses or community health workers or from facilities into the community. Evidence suggests this type of intervention is likely to be cost-effective or cost-saving. Nine studies reported on quality improvement initiatives, which were generally found to be cost-effective. Quality and methods differed widely limiting comparability of the studies and findings. CONCLUSIONS: There is significant heterogeneity in the literature, both methodologically and in quality. This renders further comparisons difficult and limits the utility of the available evidence to decision makers.

Would changing the selection process for GP trainees stem the workforce crisis? A cohort study using multiple-imputation and simulation.

BACKGROUND: There is currently a shortage of qualified GPs in the UK and not all of the training posts available each year are filled. Changing the way in which GP trainees are selected could help increase the training post fill rate and the number of new entrants to the GP Register. The aim of this study was to model the impact of changing the selection process for GP training on the number of trainees obtaining GP Registration, either with or without extensions. METHOD: This was a cohort study using UK applications for GP training in 2011-14. Application data were linked using GMC numbers to training outcome data where available, and imputed using multiple imputation where missing. The number of trainees appointed and GP Registrations within three and five years' full-time-equivalent were estimated for four different selection processes. RESULTS: The cut scores used in the actual 2015 selection process makes it impossible to fill all training posts. Random selection is the worst option, but the difference between this and other processes modelled falls as more trainees are selected. There are large marginal effects on outcomes: those with the highest selection scores are more likely to obtain GP Registration than those with the lowest scores. CONCLUSIONS: Changing the selection process alone would have a small impact on the number of GP Registrations; reducing/removing cut scores would have a much larger impact. This would also increase the number of trainees requiring extensions and being released from training which would have adverse consequences for the profession.

The Dose-Response Relationship Between Training Load and Aerobic Fitness in Academy Rugby Union Players.

PURPOSE: To identify the dose-response relationship between measures of training load (TL) and changes in aerobic fitness in academy rugby union players. METHOD: Training data from 10 academy rugby union players were collected during a 6-wk in-season period. Participants completed a lactate-threshold test that was used to assess VO2max, velocity at VO2max, velocity at 2 mmol/L (lactate threshold), and velocity at 4 mmol/L (onset of lactate accumulation; vOBLA) as measures of aerobic fitness. Internal-TL measures calculated were Banister training impulse (bTRIMP), Edwards TRIMP, Lucia TRIMP, individualized TRIMP (iTRIMP), and session RPE (sRPE). External-TL measures calculated were total distance, PlayerLoad™, high-speed distance >15 km/h, very-high-speed distance >18 km/h, and individualized high-speed distance based on each player's vOBLA. RESULTS: A second-order-regression (quadratic) analysis found that bTRIMP (R2 = .78, P = .005) explained 78% of the variance and iTRIMP (R2 = .55, P = .063) explained 55% of the variance in changes in VO2max. All other HR-based internal-TL measures and sRPE explained less than 40% of variance with fitness changes. External TL explained less than 42% of variance with fitness changes. CONCLUSIONS: In rugby players, bTRIMP and iTRIMP display a curvilinear dose-response relationship with changes in maximal aerobic fitness.

Determining whether Community Health Workers are 'Deployment Ready' Using Standard Setting.

BACKGROUND: Community Health Workers (CHWs) provide basic health screening and advice to members of their own communities. Although CHWs are trained, no CHW programmes have used a formal method to identify the level of achievement on post-training assessments that distinguishes "safe" from "unsafe". Objectives: The aim of this study was to use Ebel method of standard setting for a post-training written knowledge assessment for CHWs in Neno, Malawi. METHODS: 12 participants agreed the definitions of a "just-deployment ready" and an "ideal" CHW. Participants rated the importance and difficulty of each question on a three-point scale and also indicated the proportion of "just-deployment ready" CHWs expected to answer each of the nine question types correctly. Mean scores were used to determine the passing standard, which was reduced by one standard error of measurement (SEM) as this was the first time any passing standard had been employed.The level of agreement across participants' ratings of importance and difficulty was calculated using Krippendorf's alpha. The assessment results from the first cohort of CHW trainees were analysed using classical test theory. FINDINGS: There was poor agreement between participants on item ratings of both importance and difficulty (Krippendorf's alphas of 0.064 and 0.074 respectively). The pass mark applied to the assessment, following adjustment using the SEM, was 53.3%. Based on this pass mark, 68% of 129 CHW trainees were 'clear passes', 11% 'borderline passes', 9% 'borderline fails' and 12% 'clear fails'. CONCLUSIONS: Determining whether a CHW is deployment-ready is an important, but difficult exercise, as evidenced by a lack of agreement regarding question importance and difficulty. Future exercises should allow more time for training, discussion and modification of ratings. Based on the assessment, most CHWs trained could be considered deployment-ready, but following-up their performance in the field will be vital to validate the pass mark set.

Affordability of comprehensive community health worker programmes in rural sub-Saharan Africa.

INTRODUCTION: Community health worker (CHW) programmes have low costs per person served and are central to achieving universal healthcare. However, their total cost is high and the target of one million CHWs for sub-Saharan Africa by 2015 was not met. We consider the affordability of rural CHW programmes by estimating total programme costs relative to national healthcare expenditure at different CHW salaries and resources available for healthcare. METHODS: We combine an existing source of rural CHW programme costs with World Bank data to estimate relative CHW programme costs in 37 countries. We consider three 'salaries' (CHWs as volunteers, paid the local equivalent of US$80 per month and paid the national minimum wage) and four potential healthcare budgets (both actual and Abuja declaration allocations alone and increased by external funding received and potential foreign aid, respectively). Costs are shown in 2012 nominal US$. RESULTS: With CHWs paid the local equivalent of US$80 per month and financed from existing central government healthcare budgets, the median relative cost of a CHW programme would be 27% of the healthcare budget. While less than 2.5% in five countries (Botswana, Equatorial Guinea, Gabon, Namibia and South Africa), this relative cost would exceed 100% in three (Chad, Eritrea and Niger). There is a strong negative linear relationship (R2=0.83, p<0.001) between the natural logs of gross domestic product (GDP) per capita and affordability. In 23 countries with GDP per capita under US$1200, the cost of a CHW programme would exceed 12% of actual healthcare spending if CHWs were paid US$80 per month. CONCLUSION: CHWs may be a stepping stone to universal access to professional healthcare, but there is high variability in the affordability of CHW programmes across sub-Saharan Africa. In many countries, such programmes are not yet affordable unless significant foreign aid is received.

Cost and Cost-Effectiveness of Donor Human Milk to Prevent Necrotizing Enterocolitis: Systematic Review.

BACKGROUND: Necrotizing enterocolitis (NEC) is a costly gastrointestinal disorder that mainly affects preterm and low-birth-weight infants and can lead to considerable morbidity and mortality. Mother's own milk is protective against NEC but is not always available. In such cases, donor human milk has also been shown to be protective (although to a lesser extent) compared with formula milk, but it is more expensive. This systematic review aimed at evaluating the cost of donor milk, the cost of treating NEC, and the cost-effectiveness of exclusive donor milk versus formula milk feeding to reduce the short-term health and treatment costs of NEC. MATERIALS AND METHODS: We systematically searched five relevant databases to find studies with verifiable costs or charges of donor milk and/or treatment of NEC and any economic evaluations comparing exclusive donor milk with exclusive formula milk feeding. All search results were double screened. RESULTS: Seven studies with verifiable donor milk costs and 17 with verifiable NEC treatment costs were included. The types of cost or charge included varied considerably across studies, so quantitative synthesis was not attempted. Estimates of the incremental length of stay associated with NEC were ∼18 days for medical NEC and 50 days for surgical NEC. Two studies claimed to report economic evaluations but did not do so in practice. CONCLUSIONS: It is likely that donor milk provides short-term cost savings by reducing the incidence of NEC. Future studies should provide more details on cost components included and a full economic evaluation, including long-term outcomes, should be undertaken.

Prescribing Safety Assessment 2016: Delivery of a national prescribing assessment to 7343 UK final-year medical students.

AIMS: Newly graduated doctors write a large proportion of prescriptions in UK hospitals but recent studies have shown that they frequently make prescribing errors. The prescribing safety assessment (PSA) has been developed as an assessment of competence in relation to prescribing and supervising the use of medicines. This report describes the delivery of the PSA to all UK final-year medical students in 2016 (PSA2016). METHODS: The PSA is a 2-hour online assessment comprising eight sections which cover various aspects of prescribing defined within the outcomes of undergraduate education identified by the UK General Medical Council. Students sat one of four PSA 'papers', which had been standard-set using a modified Angoff process. RESULTS: A total of 7343 final-year medical students in all 31 UK medical schools sat the PSA. The overall pass rate was 95% with the pass rates for the individual papers ranging from 93 to 97%. The PSA was re-sat by 261 students who had failed and 80% of those candidates passed. The internal consistency (Cronbach's alpha) of the four papers ranged from 0.74 to 0.77 (standard error of measurement 4.13-4.24%). There was a statistically significant variation in performance between medical school cohorts (F = 32.6, P < 0.001) and a strongly positive correlation in performance for individual schools between PSA2015 and PSA2016 (r = 0.79, 95% CI 0.61-0.90; P < 0.01). CONCLUSIONS: PSA2016 demonstrated the feasibility of delivering a standardized national prescribing assessment online. The vast majority of UK final-year medical students were able to meet a prespecified standard of prescribing competence.

Variation in passing standards for graduation-level knowledge items at UK medical schools.

OBJECTIVES: Given the absence of a common passing standard for students at UK medical schools, this paper compares independently set standards for common 'one from five' single-best-answer (multiple-choice) items used in graduation-level applied knowledge examinations and explores potential reasons for any differences. METHODS: A repeated cross-sectional study was conducted. Participating schools were sent a common set of graduation-level items (55 in 2013-2014; 60 in 2014-2015). Items were selected against a blueprint and subjected to a quality review process. Each school employed its own standard-setting process for the common items. The primary outcome was the passing standard for the common items by each medical school set using the Angoff or Ebel methods. RESULTS: Of 31 invited medical schools, 22 participated in 2013-2014 (71%) and 30 (97%) in 2014-2015. Schools used a mean of 49 and 53 common items in 2013-2014 and 2014-2015, respectively, representing around one-third of the items in the examinations in which they were embedded. Data from 19 (61%) and 26 (84%) schools, respectively, met the inclusion criteria for comparison of standards. There were statistically significant differences in the passing standards set by schools in both years (effect sizes (f2 ): 0.041 in 2013-2014 and 0.218 in 2014-2015; both p < 0.001). The interquartile range of standards was 5.7 percentage points in 2013-2014 and 6.5 percentage points in 2014-2015. There was a positive correlation between the relative standards set by schools in the 2 years (Pearson's r = 0.57, n = 18, p = 0.014). Time allowed per item, method of standard setting and timing of examination in the curriculum did not have a statistically significant impact on standards. CONCLUSIONS: Independently set standards for common single-best-answer items used in graduation-level examinations vary across UK medical schools. Further work to examine standard-setting processes in more detail is needed to help explain this variability and develop methods to reduce it.

Integrating emotional and psychological support into the end-stage renal disease pathway: a protocol for mixed methods research to identify patients' lower-level support needs and how these can most effectively be addressed.

BACKGROUND: As a result of difficulties related to their illness, diagnosis and treatment, patients with end-stage renal disease experience significant emotional and psychological problems, which untreated can have considerable negative impact on their health and wellbeing. Despite evidence that patients desire improved support, management of their psychosocial problems, particularly at the lower-level, remains sub-optimal. There is limited understanding of the specific support that patients need and want, from whom, and when, and also a lack of data on what helps and hinders renal staff in identifying and responding to their patients' support needs, and how barriers to doing so might be overcome. Through this research we therefore seek to determine what, when, and how, support for patients with lower-level emotional and psychological problems should be integrated into the end-stage renal disease pathway. METHODS/DESIGN: The research will involve two linked, multicentre studies, designed to identify and consider the perspectives of patients at five different stages of the end-stage renal disease pathway (Study 1), and renal staff working with them (Study 2). A convergent, parallel mixed methods design will be employed for both studies, with quantitative and qualitative data collected separately. For each study, the data sets will be analysed separately and the results then compared or combined using interpretive analysis. A further stage of synthesis will employ data-driven thematic analysis to identify: triangulation and frequency of themes across pathway stages; patterns and plausible explanations of effects. DISCUSSION: There is an important need for this research given the high frequency of lower-level distress experienced by end-stage renal disease patients and lack of progress to date in integrating support for their lower-level psychosocial needs into the care pathway. Use of a mixed methods design across the two studies will generate a holistic patient and healthcare professional perspective that is more likely to identify viable solutions to enable implementation of timely and integrated care. Based on the research outputs, appropriate support interventions will be developed, implemented and evaluated in a linked follow-on study.