Revising ethical guidance for the evaluation of programmes and interventions not initiated by researchers.

Public health and service delivery programmes, interventions and policies (collectively, 'programmes') are typically developed and implemented for the primary purpose of effecting change rather than generating knowledge. Nonetheless, evaluations of these programmes may produce valuable learning that helps determine effectiveness and costs as well as informing design and implementation of future programmes. Such studies might be termed 'opportunistic evaluations', since they are responsive to emergent opportunities rather than being studies of interventions that are initiated or designed by researchers. However, current ethical guidance and registration procedures make little allowance for scenarios where researchers have played no role in the development or implementation of a programme, but nevertheless plan to conduct a prospective evaluation. We explore the limitations of the guidance and procedures with respect to opportunistic evaluations, providing a number of examples. We propose that one key missing distinction in current guidance is moral responsibility: researchers can only be held accountable for those aspects of a study over which they have control. We argue that requiring researchers to justify an intervention, programme or policy that would occur regardless of their involvement prevents or hinders research in the public interest without providing any further protections to research participants. We recommend that trial consent and ethics procedures allow for a clear separation of responsibilities for the intervention and the evaluation.

"Who needs secure services for personality disorder?" Results of an expert Delphi study with professional staff.

BACKGROUND: Personality Disorder (PD) is an enduring, multi-faceted mental disorder, associated with adverse health effects, difficulties with interpersonal relationships and in some cases increased risk to others. A limited number of dedicated forensic mental health services are available for serious offenders with severe personality disorder. The recent Offender Personality Disorder (OPD) strategy aims to ensure that most such offenders are treated in prison rather than secure psychiatric services, except in highly complex cases where this is not possible. While the strategy sets out very broad criteria relating to this, greater clarity is needed to support decisions about appropriate transfer and hence enhance public protection. This study explored which characteristics professional experts associate with appropriate transfer from prison to forensic mental health services for high-risk offenders with PD. METHOD: A modified Delphi survey distributed through an online survey system was conducted in two-rounds with a group of professional experts recruited from forensic mental healthcare; criminal justice and specialist commissioning. RESULTS: Fifty-one (56%) respondents completed stage one of the Delphi and 34 (61%) of these completed stage two. Consensus was reached for a total of 22 items indicating complexity, including co-morbid mental illness, high level of risk, lack of progress in prison and high motivation for treatment. A preliminary checklist for these factors was developed. Panel members consistently emphasised the importance of the individual's presenting need, the overall clinical picture and formulation in their free text responses. CONCLUSIONS: Professionals face a complex picture when making decisions regarding suitability for hospital admission for high-risk male offenders with PD, with varied opinions amongst professional experts as to priorities for intervention and a focus on individual needs through formulation. It was, nevertheless, possible to condense these views into a set of consistent variables that can be used to highlight the need for transfer into hospital-based treatment services.

The use of low-cost Android tablets to train community health workers in Mukono, Uganda, in the recognition, treatment and prevention of pneumonia in children under five: a pilot randomised controlled trial.

BACKGROUND: Since 2012, The World Health Organization and UNICEF have advocated for community health workers (CHWs) to be trained in Integrated Community Case Management (iCCM) of common childhood illnesses, such as pneumonia. Despite the effectiveness of iCCM, CHWs face many barriers to accessing training. This pilot study compares traditional training with using locally made videos loaded onto low-cost Android tablets to train CHWs on the pneumonia component of iCCM. METHODS: We conducted a pilot randomised controlled trial with CHWs in the Mukono District of Uganda. The unit of randomisation was the sub-county level, and the unit of analysis was at the level of the individual CHW. Eligible CHWs had completed basic iCCM training but had not received any refresher training on the pneumonia component of iCCM in the preceding 2 years. CHWs in the control group received training in the recognition, treatment, and prevention of pneumonia as it is currently delivered, through a 1-day, in-person workshop. CHWs allocated to the intervention group received training via locally made educational videos hosted on low-cost Android tablets. The primary outcome was change in knowledge acquisition, assessed through a multiple choice questionnaire before and after training, and a post-training clinical assessment. The secondary outcome was a qualitative evaluation of CHW experiences of using the tablet platform. RESULTS: In the study, 129 CHWs were enrolled, 66 and 63 in the control and intervention groups respectively. CHWs in both groups demonstrated an improvement in multiple choice question test scores before and after training; however, there was no statistically significant difference in the improvement between groups (t = 1.15, p = 0.254). There was a statistically significant positive correlation (Pearson's r = 0.26, p = 0.03) linking years of education to improvement in test scores in the control group, which was not present in the intervention group. The majority of CHWs expressed satisfaction with the use of tablets as a training tool; however, some reported technical issues (n = 9). CONCLUSION: Tablet-based training is comparable to traditional training in terms of knowledge acquisition. It also proved to be feasible and a satisfactory means of delivering training to CHWs. Further research is required to understand the impacts of scaling such an intervention. TRIAL REGISTRATION: Registered on 23/11/2016 at clinicaltrials.gov ( NCT02971449 ).

The predictive validity of the Living Goods selection tools for community health workers in Kenya: cohort study.

BACKGROUND: Ensuring that selection processes for Community Health Workers (CHWs) are effective is important due to the scale and scope of modern CHW programmes. However they are relatively understudied. While community involvement in selection should never be eliminated entirely, there are other complementary methods that could be used to help identify those most likely to be high-performing CHWs. This study evaluated the predictive validity of three written tests and two individual sections of a one-to-one interview used for selection into CHW posts in eight areas of Kenya. METHODS: A cohort study of CHWs working for Living Goods in eight local areas of Kenya was undertaken. Data on the selection scores, post-training assessment scores and subsequent on-the-job performance (number of household and pregnancy registrations, number of child assessments, proportion of on-time follow-ups and value of goods sold) were obtained for 547 CHWs. Kendall's tau-b correlations between each selection score and performance outcome were calculated. RESULTS: None of the correlations between selection scores and outcomes reached the 0.3 threshold of an "adequate" predictor of performance. Correlations were higher for the written components of the selection process compared to the interview components, with some small negative correlations found for the latter. CONCLUSIONS: If the measures of performance included in this study are considered critical, then further work to develop the CHW selection tools is required. This could include modifying the content of both tools or increasing the length of the written tests to make them more reliable, for if a test is not reliable then it cannot be valid. Other important outcomes not included in this study are retention in post and quality of care. Other CHW programme providers should consider evaluating their own selection tools in partnership with research teams.

Would changing the selection process for GP trainees stem the workforce crisis? A cohort study using multiple-imputation and simulation.

BACKGROUND: There is currently a shortage of qualified GPs in the UK and not all of the training posts available each year are filled. Changing the way in which GP trainees are selected could help increase the training post fill rate and the number of new entrants to the GP Register. The aim of this study was to model the impact of changing the selection process for GP training on the number of trainees obtaining GP Registration, either with or without extensions. METHOD: This was a cohort study using UK applications for GP training in 2011-14. Application data were linked using GMC numbers to training outcome data where available, and imputed using multiple imputation where missing. The number of trainees appointed and GP Registrations within three and five years' full-time-equivalent were estimated for four different selection processes. RESULTS: The cut scores used in the actual 2015 selection process makes it impossible to fill all training posts. Random selection is the worst option, but the difference between this and other processes modelled falls as more trainees are selected. There are large marginal effects on outcomes: those with the highest selection scores are more likely to obtain GP Registration than those with the lowest scores. CONCLUSIONS: Changing the selection process alone would have a small impact on the number of GP Registrations; reducing/removing cut scores would have a much larger impact. This would also increase the number of trainees requiring extensions and being released from training which would have adverse consequences for the profession.

Prioritising allocation of donor human breast milk amongst very low birthweight infants in middle-income countries.

The use of donor human breast milk instead of formula reduces the risk of necrotising enterocolitis in preterm infants when their mother's own milk is insufficient. Use of donor milk is limited by the cost of establishing a milk bank and a lack of donors, but the optimal rationing of limited donor milk is unclear. This paper uses an economic model to explore how a limited donor milk supply should be allocated across very low birthweight infants in South Africa considering 2 outcomes: maximising lives saved and minimising costs. We developed a probabilistic cohort Markov decision model with 10,000 infants across 4 birthweight groups. We evaluated allocation scenarios in which infants in each group could be exclusively formula-fed or fed donor milk for 14 or 28 days and thereafter formula until death or discharge. Prioritising infants in the lowest birthweight groups would save the most lives, whereas prioritising infants in the highest birthweight groups would result in the highest cost savings. All allocation scenarios would be considered very cost-effective in South Africa compared to the use of formula; the "worst case" was $619 per Disability Adjusted Life Year averted. There is a compelling argument to increase the supply of donor milk in middle-income countries. Our analysis could be extended by taking a longer term perspective, using data from more than one country and exploring the use of donor milk as an adjunct to mother's own milk, rather than a pure substitute for it.

Prescribing Safety Assessment 2016: Delivery of a national prescribing assessment to 7343 UK final-year medical students.

AIMS: Newly graduated doctors write a large proportion of prescriptions in UK hospitals but recent studies have shown that they frequently make prescribing errors. The prescribing safety assessment (PSA) has been developed as an assessment of competence in relation to prescribing and supervising the use of medicines. This report describes the delivery of the PSA to all UK final-year medical students in 2016 (PSA2016). METHODS: The PSA is a 2-hour online assessment comprising eight sections which cover various aspects of prescribing defined within the outcomes of undergraduate education identified by the UK General Medical Council. Students sat one of four PSA 'papers', which had been standard-set using a modified Angoff process. RESULTS: A total of 7343 final-year medical students in all 31 UK medical schools sat the PSA. The overall pass rate was 95% with the pass rates for the individual papers ranging from 93 to 97%. The PSA was re-sat by 261 students who had failed and 80% of those candidates passed. The internal consistency (Cronbach's alpha) of the four papers ranged from 0.74 to 0.77 (standard error of measurement 4.13-4.24%). There was a statistically significant variation in performance between medical school cohorts (F = 32.6, P < 0.001) and a strongly positive correlation in performance for individual schools between PSA2015 and PSA2016 (r = 0.79, 95% CI 0.61-0.90; P < 0.01). CONCLUSIONS: PSA2016 demonstrated the feasibility of delivering a standardized national prescribing assessment online. The vast majority of UK final-year medical students were able to meet a prespecified standard of prescribing competence.

Variation in passing standards for graduation-level knowledge items at UK medical schools.

OBJECTIVES: Given the absence of a common passing standard for students at UK medical schools, this paper compares independently set standards for common 'one from five' single-best-answer (multiple-choice) items used in graduation-level applied knowledge examinations and explores potential reasons for any differences. METHODS: A repeated cross-sectional study was conducted. Participating schools were sent a common set of graduation-level items (55 in 2013-2014; 60 in 2014-2015). Items were selected against a blueprint and subjected to a quality review process. Each school employed its own standard-setting process for the common items. The primary outcome was the passing standard for the common items by each medical school set using the Angoff or Ebel methods. RESULTS: Of 31 invited medical schools, 22 participated in 2013-2014 (71%) and 30 (97%) in 2014-2015. Schools used a mean of 49 and 53 common items in 2013-2014 and 2014-2015, respectively, representing around one-third of the items in the examinations in which they were embedded. Data from 19 (61%) and 26 (84%) schools, respectively, met the inclusion criteria for comparison of standards. There were statistically significant differences in the passing standards set by schools in both years (effect sizes (f2 ): 0.041 in 2013-2014 and 0.218 in 2014-2015; both p < 0.001). The interquartile range of standards was 5.7 percentage points in 2013-2014 and 6.5 percentage points in 2014-2015. There was a positive correlation between the relative standards set by schools in the 2 years (Pearson's r = 0.57, n = 18, p = 0.014). Time allowed per item, method of standard setting and timing of examination in the curriculum did not have a statistically significant impact on standards. CONCLUSIONS: Independently set standards for common single-best-answer items used in graduation-level examinations vary across UK medical schools. Further work to examine standard-setting processes in more detail is needed to help explain this variability and develop methods to reduce it.

Integrating emotional and psychological support into the end-stage renal disease pathway: a protocol for mixed methods research to identify patients' lower-level support needs and how these can most effectively be addressed.

BACKGROUND: As a result of difficulties related to their illness, diagnosis and treatment, patients with end-stage renal disease experience significant emotional and psychological problems, which untreated can have considerable negative impact on their health and wellbeing. Despite evidence that patients desire improved support, management of their psychosocial problems, particularly at the lower-level, remains sub-optimal. There is limited understanding of the specific support that patients need and want, from whom, and when, and also a lack of data on what helps and hinders renal staff in identifying and responding to their patients' support needs, and how barriers to doing so might be overcome. Through this research we therefore seek to determine what, when, and how, support for patients with lower-level emotional and psychological problems should be integrated into the end-stage renal disease pathway. METHODS/DESIGN: The research will involve two linked, multicentre studies, designed to identify and consider the perspectives of patients at five different stages of the end-stage renal disease pathway (Study 1), and renal staff working with them (Study 2). A convergent, parallel mixed methods design will be employed for both studies, with quantitative and qualitative data collected separately. For each study, the data sets will be analysed separately and the results then compared or combined using interpretive analysis. A further stage of synthesis will employ data-driven thematic analysis to identify: triangulation and frequency of themes across pathway stages; patterns and plausible explanations of effects. DISCUSSION: There is an important need for this research given the high frequency of lower-level distress experienced by end-stage renal disease patients and lack of progress to date in integrating support for their lower-level psychosocial needs into the care pathway. Use of a mixed methods design across the two studies will generate a holistic patient and healthcare professional perspective that is more likely to identify viable solutions to enable implementation of timely and integrated care. Based on the research outputs, appropriate support interventions will be developed, implemented and evaluated in a linked follow-on study.

Evaluation of the effect of socio-economic status on performance in a Multiple Mini Interview for admission to medical school.

INTRODUCTION: Widening participation in Medicine is a key policy priority as it helps promote a diverse and representative workforce and improves patient care. The selection process employed can influence the socio-economic composition of the student cohort and this study therefore evaluated whether Multiple Mini Interview (MMI) performance was influenced by school type or area-level Higher Education (HE) participation rates. METHODS: MMI performance for all UK applicants was recorded and consent to link performance with socio-economic data was requested using an applicant questionnaire. Station-level and total MMI scores, and offer rates were compared between applicants from non-selective, non-fee-paying schools and from selective and/or fee-paying schools; and between applicants from each quintile of area-level HE participation. RESULTS: 793 applicants were included in the analysis. MMI performance and offer rates were slightly higher for applicants from non-selective, non-fee-paying schools and/or from lower HE participation quintiles, but the effects were small and not statistically significant. CONCLUSIONS: The MMI did not favour applicants from selective and/or fee-paying schools, or from areas with high HE participation rates. Work to evaluate whether the relationship between MMI and medical school performance is the same for all sub-groups of students is now required.

Factor structure of the Essen Climate Evaluation Schema measure of social climate in a UK medium-security setting.

BACKGROUND: Social climate has an influence on a number of treatment-related factors, including service users' behaviour, staff morale and treatment outcomes. Reliable assessment of social climate is, therefore, beneficial within forensic mental health settings. The Essen Climate Evaluation Schema (EssenCES) has been validated in forensic mental health services in the UK and Germany. Preliminary normative data have been produced for UK high-security national health services and German medium-security and high-security services. AIMS: We aim to validate the use of the EssenCES scale (English version) and provide preliminary normative data in UK medium-security hospital settings. METHODS: The EssenCES scale was completed in a medium-security mental health service as part of a service-wide audit. A total of 89 patients and 112 staff completed the EssenCES. RESULTS: The three-factor structure of the EssenCES and its internal construct validity were maintained within the sample. Scores from this medium-security hospital sample were significantly higher than those from earlier high-security hospital data, with three exceptions--'patient cohesion' according to the patients and 'therapeutic hold' according to staff and patients. CONCLUSION: Our data support the use of the EssenCES scale as a valid measure for assessing social climate within medium-security hospital settings. Significant differences between the means of high-security and medium-security service samples imply that degree of security is a relevant factor affecting the ward climate and that in monitoring quality of secure services, it is likely to be important to apply different scores to reflect standards.

Assessments and admissions during the first 6 years of a UK medium secure DSPD service.

BACKGROUND: Offenders with personality disorder represent a considerable burden on mental health and criminal justice services. Millfields Unit was one of the pilot medium secure units (MSUs) established by the UK Department of Health and Ministry of Justice to provide dedicated services for such individuals. AIMS: Our study aimed to describe a complete cohort of referrals made to the service and to explore decisions on admissions to the Unit. METHODS: File review was used to collect a structured data set of information on the first 158 referrals. These were then analysed descriptively and inferentially. RESULTS: Previous psychological treatment in prison and evidence of motivation for further treatment were the strongest predictors of admission to the service. People admitted showed evidence of more extensive criminal histories, prior treatment failure and higher level of risk than those accepted into conventional medium secure units. Despite the stated purpose of the units being for people with personality disorder, a high number of patients with a primary diagnosis of psychotic illness were referred. CONCLUSION: The new service accepted a high-risk group of people with personality disorder whose needs would not otherwise have been met.

Development of a modified Cohen method of standard setting.

BACKGROUND: A new 'Cohen' approach to standard setting was recently described where the pass mark is calculated as 60% of the score of the student at the 95th percentile, after correcting for guessing. AIM: This article considers how two potential criticisms of the Cohen method can be addressed and proposes a modified version, with the assumptions tested using local data. METHODS: The modified version removes the correction for guessing and uses the score of the 90th, rather than the 95th percentile student as the reference point, based on the cumulative density functions for 32 modules from one medical school; and incorporates an indirect criterion-referenced passing standard by changing the 60% multiplier to the ratio of the cut score to the score of the student at the 90th percentile on exams that have been standard set using modified Angoff. RESULTS: The assumption that the performance of the 90th percentile student is consistent over time holds for multiple choice questions. Applying the modified Cohen method to the 32 modules generally reduced the variation in failure rate across modules, compared to a fixed pass mark of 50%. CONCLUSION: The results suggest that the modified Cohen method holds much promise as an economical approach to standard setting.

The effects of laryngeal mask airway passage simulation training on the acquisition of undergraduate clinical skills: a randomised controlled trial.

BACKGROUND: Effective use of the laryngeal mask airway (LMA) requires learning proper insertion technique in normal patients undergoing routine surgical procedures. However, there is a move towards simulation training for learning practical clinical skills, such as LMA placement. The evidence linking different amounts of mannequin simulation training to the undergraduate clinical skill of LMA placement in real patients is limited. The purpose of this study was to compare the effectiveness in vivo of two LMA placement simulation courses of different durations. METHODS: Medical students (n = 126) enrolled in a randomised controlled trial. Seventy-eight of these students completed the trial. The control group (n = 38) received brief mannequin training while the intervention group (n = 40) received additional more intensive mannequin training as part of which they repeated LMA insertion until they were proficient. The anaesthetists supervising LMA placements in real patients rated the participants' performance on assessment forms. Participants completed a self-assessment questionnaire. RESULTS: Additional mannequin training was not associated with improved performance (37% of intervention participants received an overall placement rating of > 3/5 on their first patient compared to 48% of the control group, X2 = 0.81, p = 0.37). The agreement between the participants and their instructors in terms of LMA placement success rates was poor to fair. Participants reported that mannequins were poor at mimicking reality. CONCLUSIONS: The results suggest that the value of extended mannequin simulation training in the case of LMA placement is limited. Educators considering simulation for the training of practical skills should reflect on the extent to which the in vitro simulation mimics the skill required and the degree of difficulty of the procedure.

Offenders With Personality Disorder Who Fail to Progress: A Case-Control Study Using Partial Least Squares Structural Equation Modeling Path Analysis.

BACKGROUND: Offenders with personality disorder can be challenging to engage and retain in treatment. The UK Offender Personality Disorder (OPD) pathway aims to proactively and responsively identify and engage offenders with personality disorder. However, a subpopulation of offenders on the pathway have been found to not be accepted into any OPD service and therefore fail to progress. OBJECTIVE: This study aims to identify and describe offenders on the OPD pathway who fail to progress and to understand the causal drivers by which individuals fail to progress in the pathway. METHODS: A sample of 50 offenders on the OPD pathway who had been refused from at least two OPD services (nonprogression group) were compared to 100 offenders accepted into OPD services (control group). Partial least squares structural equation modeling was used to model the causal factors involved in not being accepted into OPD services. RESULTS: The path coefficients in the structural model showed that the most influential factor in nonprogression was attitude toward treatment (ß=.41; P<.001; f2=0.25) alongside those with psychopathology (ß=.41; P<.001; f2=0.25), specifically, psychopathy, psychosis, and co-occurring personality disorder. CONCLUSIONS: The findings of the study provide a basis of how to work with this population in the future to increase the likelihood of acceptance into OPD services.

A Library of Logic Models to Explain How Interventions to Reduce Diagnostic Errors Work.

OBJECTIVES: We aimed to create a library of logic models for interventions to reduce diagnostic error. This library can be used by those developing, implementing, or evaluating an intervention to improve patient care, to understand what needs to happen, and in what order, if the intervention is to be effective. METHODS: To create the library, we modified an existing method for generating logic models. The following five ordered activities to include in each model were defined: preintervention; implementation of the intervention; postimplementation, but before the immediate outcome can occur; the immediate outcome (usually behavior change); and postimmediate outcome, but before a reduction in diagnostic errors can occur. We also included reasons for lack of progress through the model. Relevant information was extracted about existing evaluations of interventions to reduce diagnostic error, identified by updating a previous systematic review. RESULTS: Data were synthesized to create logic models for four types of intervention, addressing five causes of diagnostic error in seven stages in the diagnostic pathway. In total, 46 interventions from 43 studies were included and 24 different logic models were generated. CONCLUSIONS: We used a novel approach to create a freely available library of logic models. The models highlight the importance of attending to what needs to occur before and after intervention delivery if the intervention is to be effective. Our work provides a useful starting point for intervention developers, helps evaluators identify intermediate outcomes, and provides a method to enable others to generate libraries for interventions targeting other errors.

Determining whether Community Health Workers are 'Deployment Ready' Using Standard Setting.

BACKGROUND: Community Health Workers (CHWs) provide basic health screening and advice to members of their own communities. Although CHWs are trained, no CHW programmes have used a formal method to identify the level of achievement on post-training assessments that distinguishes "safe" from "unsafe". Objectives: The aim of this study was to use Ebel method of standard setting for a post-training written knowledge assessment for CHWs in Neno, Malawi. METHODS: 12 participants agreed the definitions of a "just-deployment ready" and an "ideal" CHW. Participants rated the importance and difficulty of each question on a three-point scale and also indicated the proportion of "just-deployment ready" CHWs expected to answer each of the nine question types correctly. Mean scores were used to determine the passing standard, which was reduced by one standard error of measurement (SEM) as this was the first time any passing standard had been employed.The level of agreement across participants' ratings of importance and difficulty was calculated using Krippendorf's alpha. The assessment results from the first cohort of CHW trainees were analysed using classical test theory. FINDINGS: There was poor agreement between participants on item ratings of both importance and difficulty (Krippendorf's alphas of 0.064 and 0.074 respectively). The pass mark applied to the assessment, following adjustment using the SEM, was 53.3%. Based on this pass mark, 68% of 129 CHW trainees were 'clear passes', 11% 'borderline passes', 9% 'borderline fails' and 12% 'clear fails'. CONCLUSIONS: Determining whether a CHW is deployment-ready is an important, but difficult exercise, as evidenced by a lack of agreement regarding question importance and difficulty. Future exercises should allow more time for training, discussion and modification of ratings. Based on the assessment, most CHWs trained could be considered deployment-ready, but following-up their performance in the field will be vital to validate the pass mark set.

Cost-effectiveness of health care service delivery interventions in low and middle income countries: a systematic review.

BACKGROUND: Low and middle income countries (LMICs) face severe resource limitations but the highest burden of disease. There is a growing evidence base on effective and cost-effective interventions for these diseases. However, questions remain about the most cost-effective method of delivery for these interventions. We aimed to review the scope, quality, and findings of economic evaluations of service delivery interventions in LMICs. METHODS: We searched PUBMED, MEDLINE, EconLit, and NHS EED for studies published between 1st January 2000 and 30th October 2016 with no language restrictions. We included all economic evaluations that reported incremental costs and benefits or summary measures of the two such as an incremental cost effectiveness ratio. Studies were grouped by both disease area and outcome measure and permutation plots were completed for similar interventions. Quality was judged by the Drummond checklist. RESULTS: Overall, 3818 potentially relevant abstracts were identified of which 101 studies were selected for full text review. Thirty-seven studies were included in the final review. Twenty-three studies reported on interventions we classed as "changing by whom and where care was provided", specifically interventions that entailed task-shifting from doctors to nurses or community health workers or from facilities into the community. Evidence suggests this type of intervention is likely to be cost-effective or cost-saving. Nine studies reported on quality improvement initiatives, which were generally found to be cost-effective. Quality and methods differed widely limiting comparability of the studies and findings. CONCLUSIONS: There is significant heterogeneity in the literature, both methodologically and in quality. This renders further comparisons difficult and limits the utility of the available evidence to decision makers.