Validity Evidence for a Knowledge Assessment Tool for a Mastery Learning Scrub Training Curriculum.

PURPOSE: To examine the validity evidence for a scrub training knowledge assessment tool to demonstrate the utility and robustness of a multimodal, entrustable professional activity (EPA)-aligned, mastery learning scrub training curriculum. METHOD: Validity evidence was collected for the knowledge assessment used in the scrub training curriculum at Stanford University School of Medicine from April 2017 to June 2018. The knowledge assessment had 25 selected response items that mapped to curricular objectives, EPAs, and operating room policies. A mastery passing standard was established using the Mastery Angoff and Patient-Safety approaches. Learners were assessed pre curriculum, post curriculum, and 6 months after the curriculum. RESULTS: From April 2017 to June 2018, 220 medical and physician assistant students participated in the scrub training curriculum. The mean pre- and postcurriculum knowledge scores were 74.4% (standard deviation [SD] = 15.6) and 90.1% (SD = 8.3), respectively, yielding a Cohen's d = 1.10, P < .001. The internal reliability of the assessment was 0.71. Students with previous scrub training performed significantly better on the precurriculum knowledge assessment than those without previous training (81.9% [SD = 12.6] vs 67.0% [SD = 14.9]; P < .001). The mean item difficulty was 0.74, and the mean item discrimination index was 0.35. The Mastery Angoff overall cut score was 92.0%. CONCLUSIONS: This study describes the administration of and provides validity evidence for a knowledge assessment tool for a multimodal, EPA-aligned, mastery-based curriculum for scrub training. The authors support the use of scores derived from this test for assessing scrub training knowledge among medical and physician assistant students.

Validation of the Fracture Mobility Score against the Parker Mobility Score in hip fracture patients.

INTRODUCTION: The Parker Mobility Score has proven to be a valid and reliable measurement of hip fracture patient mobility. For hip fracture registries the Fracture Mobility Score is advised and used, although this score has never been validated. This study aims to validate the Fracture Mobility Score against the Parker Mobility Score. PATIENTS AND METHODS: The Dutch Hip Fracture Audit uses the Fracture Mobility Score (categorical scale). For the purpose of this study, five hospitals registered both the Fracture Mobility Score and the Parker Mobility Score (0-9 scale) for every admitted hip fracture patient in 2018. The Spearman correlation between the two scores was calculated. To test whether the correlation coefficient remained stable among different patient subgroups, analyses were stratified according to baseline patient characteristics. RESULTS: In total 1,201 hip fracture patients were included. The Spearman correlation between the Fracture Mobility Score and Parker Mobility Score was strong: 0.73 (p = < 0.001). Stratified for gender, age, ASA score, dementia, Index of Activities of Daily Living (KATZ-6 ADL score), living situation and nutritional status, the correlation coefficient varied between 0.40-0.84. For patients aged 90 and over and having an ASA score of III-IV who suffered from dementia, had a KATZ-6 ADL score of 1-6, lived in an institution and/or were malnourished, the correlation was moderate. CONCLUSION: The Fracture Mobility Score is overall strongly correlated with the Parker Mobility Score and can be considered as a valid score to measure hip fracture patient mobility. This may encourage other hip fracture audits to also use the Fracture Mobility Score, which would increase the uniformity of mobility score results among national hip fracture audits and decrease the overall registration load.

The prescription of lidocaine patches in osteoarthritis-a complete audit cycle.

OBJECTIVES: Nationally, 5% of the population have been prescribed topical lidocaine patches (TLPs). These patches cost €77-230 per month. The only licensed indication for TLP is post-herpetic neuralgia (PHN). It has not proven to be effective for any other indication including musculoskeletal and post-surgical pain. It is estimated that only 5-10% of patients on TLP have PHN. The aim of this study was to audit the number of inpatients currently prescribed TLP-appropriately and inappropriately-and to examine the prescribing patterns. METHODS: The acute inpatient population was audited in August 2017 to determine TLP prevalence. Demographics, indication, prescriber grade, and whether TLP was commenced during the current admission were recorded. Education was given surrounding indications and licensed usage of TLP. The acute inpatient population was subsequently re-audited in February 2018. RESULTS: There were 304 patients included in the initial study and 300 in the repeat study. The most common indication was musculoskeletal pain in each study, 75 and 77.78%, respectively. The number of inpatients on TLP fell from 17.1% (n = 52) to 6% (n = 18) after the intervention. The potential savings between the study periods are therefore €31,418-93,840 in total and €23,100-69,000 in musculoskeletal patients. CONCLUSION: A large proportion of inpatients are inappropriately prescribed TLP predominantly for musculoskeletal pain, resulting in substantial avoidable cost to the hospital. Education of prescribers and implementation of policies is required to limit inappropriate prescribing.

External inspection of compliance with standards for improved healthcare outcomes.

BACKGROUND: Inspection systems are used in healthcare to promote quality improvements (i.e. to achieve changes in organisational structures or processes, healthcare provider behaviour and patient outcomes). These systems are based on the assumption that externally promoted adherence to evidence-based standards (through inspection/assessment) will result in higher quality of healthcare. However, the benefits of external inspection in terms of organisational-, provider- and patient-level outcomes are not clear. This is the first update of the original Cochrane review, published in 2011. OBJECTIVES: To evaluate the effectiveness of external inspection of compliance with standards in improving healthcare organisation behaviour, healthcare professional behaviour and patient outcomes. SEARCH METHODS: We searched the following electronic databases for studies up to 1 June 2015: the Cochrane Central Register of Controlled Trials (CENTRAL), MEDLINE, Embase, Database of Abstracts of Reviews of Effectiveness, HMIC, ClinicalTrials.gov and the World Health Organization International Clinical Trials Registry Platform. There was no language restriction and we included studies regardless of publication status. We also searched the reference lists of included studies and contacted authors of relevant papers, accreditation bodies and the International Organization for Standardization (ISO), regarding any further published or unpublished work. We also searched an online database of systematic reviews (PDQ-evidence.org). SELECTION CRITERIA: We included randomised controlled trials (RCTs), non-randomised trials (NRCTs), interrupted time series (ITSs) and controlled before-after studies (CBAs) evaluating the effect of external inspection against external standards on healthcare organisation change, healthcare professional behaviour or patient outcomes in hospitals, primary healthcare organisations and other community-based healthcare organisations. DATA COLLECTION AND ANALYSIS: Two review authors independently applied eligibility criteria, extracted data and assessed the risk of bias of each included study. Since meta-analysis was not possible, we produced a narrative results summary. We used the GRADE tool to assess the certainty of the evidence. MAIN RESULTS: We did not identify any new eligible studies in this update. One cluster RCT involving 20 South African public hospitals and one ITS involving all acute hospital trusts in England, met the inclusion criteria. A trust is a National Health Service hospital which has opted to withdraw from local authority control and be managed by a trust instead.The cluster RCT reported mixed effects of external inspection on compliance with COHSASA (Council for Health Services Accreditation for South Africa) accreditation standards and eight indicators of hospital quality. Improved total compliance score with COHSASA accreditation standards was reported for 21/28 service elements: mean intervention effect was 30% (95% confidence interval (CI) 23% to 37%) (P < 0.001). The score increased from 48% to 78% in intervention hospitals, while remaining the same in control hospitals (43%). The median intervention effect for the indicators of hospital quality of care was 2.4% (range -1.9% to +11.8%).The ITS study evaluated compliance with policies to address healthcare-acquired infections and reported a mean reduction in MRSA (methicillin-resistant Staphylococcus aureus) infection rates of 100 cases per quarter (95% CI -221.0 to 21.5, P = 0.096) at three months' follow-up and an increase of 70 cases per quarter (95% CI -250.5 to 391.0; P = 0.632) at 24 months' follow-up. Regression analysis showed similar MRSA rates before and after the external inspection (difference in slope 24.27, 95% CI -10.4 to 58.9; P = 0.147).Neither included study reported data on unanticipated/adverse consequences or economic outcomes. The cluster RCT reported mainly outcomes related to healthcare organisation change, and no patient reported outcomes other than patient satisfaction.The certainty of the included evidence from both studies was very low. It is uncertain whether external inspection accreditation programmes lead to improved compliance with accreditation standards. It is also uncertain if external inspection infection programmes lead to improved compliance with standards, and if this in turn influences healthcare-acquired MRSA infection rates. AUTHORS' CONCLUSIONS: The review highlights the paucity of high-quality controlled evaluations of the effectiveness and the cost-effectiveness of external inspection systems. If policy makers wish to understand the effectiveness of this type of intervention better, there needs to be further studies across a range of settings and contexts and studies reporting outcomes important to patients.

Developing entrustable professional activities as the basis for assessment of competence in an internal medicine residency: a feasibility study.

BACKGROUND: Graduate medical education programs assess trainees' performance to determine readiness for unsupervised practice. Entrustable professional activities (EPAs) are a novel approach for assessing performance of core professional tasks. AIM: To describe a pilot and feasibility evaluation of two EPAs for competency-based assessment in internal medicine (IM) residency. SETTING/PARTICIPANTS: Post-graduate year-1 interns (PGY-1s) and attendings at a large internal medicine (IM) residency program. PROGRAM DESCRIPTION: Two Entrustable professional activities (EPA) assessments (Discharge, Family Meeting) were piloted. PROGRAM FEASIBILITY EVALUATION: Twenty-eight out of 43 (65.1 %) PGY-1 s and 32/43 (74.4 %) attendings completed surveys about the Discharge EPA experience. Most who completed the EPA assessment (10/12, 83.8 %, PGY-1s; 9/11, 83.3 %, attendings) agreed it facilitated useful feedback discussions. For the Family Meeting EPA, 16/26 (61.5 %) PGY-1s completed surveys, and most who participated (9/12 PGY1s, 75 %) reported it improved attention to family meeting education, although only half recommended continuing the EPA assessment. DISCUSSION: From piloting two EPA assessments in a large IM residency, we recognized our reminder systems and time dedicated for completing EPA requirements as inadequate. Collaboration around patient safety and palliative care with relevant clinical services has enhanced implementation and buy-in. We will evaluate how well EPA-based assessment serves the intended purpose of capturing trainees' trustworthiness to conduct activities unsupervised.

Effectiveness of external inspection of compliance with standards in improving healthcare organisation behaviour, healthcare professional behaviour or patient outcomes.

BACKGROUND: Inspection systems are used in health care to promote quality improvements, i.e. to achieve changes in organisational structures or processes, healthcare provider behaviour and patient outcomes. These systems are based on the assumption that externally promoted adherence to evidence-based standards (through inspection/assessment) will result in higher quality of health care. However, the benefits of external inspection in terms of organisational, provider and patient level outcomes are not clear. OBJECTIVES: To evaluate the effectiveness of external inspection of compliance with standards in improving healthcare organisation behaviour, healthcare professional behaviour and patient outcomes. SEARCH METHODS: We searched the following electronic databases for studies: the Cochrane Central Register of Controlled Trials (CENTRAL), MEDLINE, EMBASE, CINAHL, Cochrane Database of Systematic Reviews, Database of Abstracts of Reviews of Effectiveness, Scopus, HMIC, Index to Theses and Intute from their inception dates up to May 2011. There was no language restriction and studies were included regardless of publication status. We searched the reference lists of included studies and contacted authors of relevant papers, accreditation bodies and the International Organization for Standardisation (ISO), regarding any further published or unpublished work. SELECTION CRITERIA: We included randomised controlled trials (RCTs), controlled clinical trials (CCTs), interrupted time-series (ITSs) and controlled before and after studies (CBAs) evaluating the effect of external inspection against external standards on healthcare organisation change, healthcare professional behaviour or patient outcomes in hospitals, primary healthcare organisations and other community-based healthcare organisations. DATA COLLECTION AND ANALYSIS: Two review authors independently applied eligibility criteria, extracted data and assessed the risk of bias of each included study. Since meta-analysis was not possible, we produced a narrative results summary. MAIN RESULTS: We identified one cluster-RCT involving 20 South African public hospitals (Salmon 2003) and one ITS involving all acute trusts in England (OPM 2009) for inclusion in this review.Salmon and colleagues (Salmon 2003) showed mixed effects of a hospital accreditation system on the compliance with COHSASA (the Council for Health Services Accreditation for South Africa) accreditation standards and eight indicators of hospital quality. Significantly improved total mean compliance score with COHSASA accreditation standards was found for 21/28 service elements: mean intervention effect (95% confidence interval (CI)) was 30% (23% to 57%) (P < 0.001). The score increased from 48% to 78% in intervention hospitals, while remaining the same in control hospitals (43%). A sub-analysis of 424 a priori identified critical criteria (19 service elements) showed significantly improved compliance with the critical standards (P < 0.001). The score increased from 41% (21% to 46%) to 75% (55% to 96%) in intervention hospitals, but was unchanged in control hospitals (37%). Only one of the nine intervention hospitals gained full accreditation status at the end of the study period, with two others reached pre-accreditation status.The median intervention effect (range) for the indicators of hospital quality of care was 2.4 (-1.9 to +11.8) and only one of the eight indicators: 'nurses perception of clinical quality, participation and teamwork' was significantly improved (mean intervention effect 5.7, P = 0.03).Re-analysis of the MRSA (methicillin-resistant Staphylococcus aureus) data showed statistically non-significant effects of the Healthcare Commissions Infection Inspection programme. AUTHORS' CONCLUSIONS: We only identified two studies for inclusion in this review, which highlights the paucity of high-quality controlled evaluations of the effectiveness of external inspection systems. No firm conclusions could therefore be drawn about the effectiveness of external inspection on compliance with standards.

Using theory to synthesise evidence from behaviour change interventions: the example of audit and feedback.

Evidence syntheses are used to inform health care policy and practice. Behaviour change theories offer frameworks for categorising and evaluating interventions and identifying likely mechanisms through which effects are achieved. Yet systematic reviews rarely explicitly classify intervention components using theory, which may result in evidence syntheses and health care practice recommendations that are less than optimal. This paper outlines a method for applying theory to evidence syntheses of behaviour change interventions. We illustrate this method with an analysis of 'audit and feedback' interventions, based on data from a Cochrane review. Our analysis is based on Control Theory, which suggests that behaviour change is most likely if feedback is accompanied by comparison with a behavioural target and by action plans, and we coded interventions for these three techniques. Multivariate meta-regression was performed on 85 comparisons from 61 studies. However, few interventions incorporated targets or action plans, and so meta-regression models were likely to be underfitted due to insufficient power. The utility of our approach could not be tested via our analysis because of the limited nature of the audit and feedback interventions. However, we show that conceptualising and categorising interventions using behaviour change theory can reveal the theoretical coherence of interventions and so point towards improvements in intervention design, evaluation and synthesis. The results demonstrate that a theory-based approach to evidence synthesis is feasible, and can prove beneficial in understanding intervention design, even where there is insufficient empirical evidence to reliably synthesise effects of specific intervention components.

Will the NHHRC recommendations drive quality performance?

General practice is the heart of the Australian health care system, addressing the health needs of people, in their communities and in diverse locations and contexts across Australia. With over 100 million items of service claimed by general practitioners each year, even small but incremental improvements in quality have the potential to translate into population level gains in the outcomes and safety of general practice care. In recent years, Australian general practice has undertaken significant work in quality improvement, with practice accreditation to The Royal Australian College of General Practitioners standards and the Australian Primary Care Collaboratives Program being examples. Will the recommendations of the National Health and Hospitals Reform Commission (NHRC) enhance this work?

Improving GP diabetes management - A PDSA audit cycle in Western Australia.

BACKGROUND: Tight glucose, blood pressure and lipid control in patients with diabetes can reduce morbidity and mortality from macro- and micro-vascular complications. However, treatment targets are not being met in a large proportion of patients. Clinical audit involves cycles of evaluation of current activity against standards. It allows problems to be identified and action to be taken to address them. METHODS: Annual retrospective audits over 3 years of random samples of up to 20 patient medical records from 13 general practitioners in the midwest region of Western Australia (n=807). Statistical tests compared the second and third audits with the first in regard to completeness of screening, health indicators, and the proportion of patients within The Royal Australian College of General Practitioners and Diabetes Australia guidelines targets. RESULTS: While there was a significant improvement in lipid monitoring over the study period (p<0.001), monitoring of HbA1c and blood pressure (BP) remained unchanged. Between the first and third audits, a reduction in mean HbA1c (p<0.001), mean total cholesterol (p=0.017), mean LDL cholesterol (p=0.014) and mean systolic BP (p=0.002) was seen. There was an improvement in the proportion of patients achieving cholesterol goals (measured by LDL and reaching a target of HbA1c <7%) between the first and third audits; however the proportion with BP within target declined. In the third audit, 11% of patients on diet alone, 36% on an oral hypoglycaemic agent, 90% on three oral hypoglycaemic agents and 84% of those on insulin were outside the target HbA1c. In the same audit, of those outside target BP, 53% were on no treatment and 65% were only on one type of medication. Eighty-seven percent of patients outside target cholesterol levels had not been prescribed a statin. DISCUSSION: Many of the audited GPs in our study undertreated BP, HbA1c and cholesterol. Improvement in some areas was seen over the study period, which may have been due to the quality assurance activities undertaken. These results reveal a therapeutic opportunity for reducing cardiovascular events in patients with diabetes. More aggressive management of BP and lipids by GPs may see rewards in terms of reducing cardiovascular events in patients with diabetes.

Quality assessment of cardiac surgery in Britain.

Data are available for every Cardiac Surgery unit in Britain and in 70 % are identifiable by surgeon. The data are linked to registration of deaths so survival for a range of operations, and associated patient or procedure related factors, can be evaluated. The choice of statistical triggers (outside 99.99 % confidence intervals) and the time frames of reported data (averaged over three years) (See P.285/353 of the report http://www.scts.org/documents/PDF/5thBlueBook2003.pdf) reduces its value as an early warning system but the rigour of data collecting systems and the level of scrutiny required probably ensure that poor performance will be detected.

RANZCR 2006 peer review audit instrument.

The Royal Australian and New Zealand College Radiologists (RANZCR) continuing professional development programme incorporates audit with feedback as one important activity. The 2004 audit tool improves radiation oncologist practice quality; however, the instrument is designed to be regularly refined. To refine the 2004 audit tool and present the new instrument we incorporated comments and suggestions from: (i) the auditor and radiation oncologist from the single machine unit trial; (ii) members of RANZCR Post-Fellowship Education Committee; (iii) New South Wales Department of Health mandatory prescription requirements; and (iv) the International Atomic Energy Agency audit tool. In July 2006, the revised instrument was designed then endorsed by Post-Fellowship Education Committee. Important changes include: (i) combining criteria which separately scored documentation and correctness for similar items; (ii) scoring treatment schedule more explicitly; (iii) separating target volume coverage and critical structure dose; (iv) altering performance criteria scoring to be sensitive to peer review when no consensus can be reached; and (v) strengthening instructions for use and notes to improve comprehension and acceptance. The refined 2006 instrument should be more user-friendly while increasing its usefulness.

Quantifying data quality for clinical trials using electronic data capture.

BACKGROUND: Historically, only partial assessments of data quality have been performed in clinical trials, for which the most common method of measuring database error rates has been to compare the case report form (CRF) to database entries and count discrepancies. Importantly, errors arising from medical record abstraction and transcription are rarely evaluated as part of such quality assessments. Electronic Data Capture (EDC) technology has had a further impact, as paper CRFs typically leveraged for quality measurement are not used in EDC processes. METHODS AND PRINCIPAL FINDINGS: The National Institute on Drug Abuse Treatment Clinical Trials Network has developed, implemented, and evaluated methodology for holistically assessing data quality on EDC trials. We characterize the average source-to-database error rate (14.3 errors per 10,000 fields) for the first year of use of the new evaluation method. This error rate was significantly lower than the average of published error rates for source-to-database audits, and was similar to CRF-to-database error rates reported in the published literature. We attribute this largely to an absence of medical record abstraction on the trials we examined, and to an outpatient setting characterized by less acute patient conditions. CONCLUSIONS: Historically, medical record abstraction is the most significant source of error by an order of magnitude, and should be measured and managed during the course of clinical trials. Source-to-database error rates are highly dependent on the amount of structured data collection in the clinical setting and on the complexity of the medical record, dependencies that should be considered when developing data quality benchmarks.

[Infrastructure of cancer clinical trial cooperative groups in western countries].

Efforts for international harmonization have made it clear that Japan is far behind in Western countries in all aspects of infrastructure for clinical trials. The introduction of ICH-GCP, 1998, has promoted the rapid growth of infrastructure for investigational new drug (IND) trials; however, the infrastructure for academic cancer trials has shown no remarkable progress. There is still no governmental regulation, no agency for quality control, and no quality assurance audit system even in government-sponsored trials. The author introduces the quality control systems in cooperative groups in Western countries, such as the Southwest Oncology Group (SWOG), National Surgical Adjuvant Breast and Bowel Project (NSABP) and European Organization for Research and Treatment of Cancer (EORTC), and the quality assurance systems by the National Cancer Institute-Cancer Therapy Evaluation Program (NCI-CTEP). Key activities for quality control in cooperative groups are in-house monitoring, site visit audits, institutional performance evaluations and case report form review by study coordinators. NCI-CTEP oversees cooperative group activities through protocol review, supervision of site visit audits and a monitoring committee. Cancer cooperative groups in Western countries have taken the initiatives in advancement of trial methodology and establishment of clinical trial infrastructure. In order to improve the quality of clinical trials, there is need to invest cancer cooperative groups, thus strengthening the activities for overall clinical trials.

Consejos de certificación de especialidades / Bord of specialitis certification

Se presenta una síntesis de los antecedentes históricos que dieron lugar al nacimiento de los consejos de certificación de especialidades en México. Se comentan las funciones que tiene la Academia Nacional de Medicina así como la relación que guarda con los consejos. Se presentan algunos datos estadísticos sobre los consejos avalados por la Academía, que hasta abril de 1992 agrupaban a 29.093 médicos en todo el país. Por último, se hace un breve comentario sobre el futuro de estas organizaciones en México y la influencia creciente que tendrá en la regulación del ejercicio de las especialidades