Improving patient safety governance and systems through learning from successes and failures: qualitative surveys and interviews with international experts.

Patient harm is a leading cause of global disease burden with considerable morbidity, mortality, and economic impacts for individuals, families, and wider society. Large bodies of evidence exist for strategies to improve safety and reduce harm. However, it is not clear which patient safety issues are being addressed globally, and which factors are the most (or least) important contributors to patient safety improvements. We aimed to explore the perspectives of international patient safety experts to identify: (1) the nature and range of patient safety issues being addressed, and (2) aspects of patient safety governance and systems that are perceived to provide value (or not) in improving patient outcomes. English-speaking Fellows and Experts of the International Society for Quality in Healthcare participated in a web-based survey and in-depth semistructured interview, discussing their experience in implementing interventions to improve patient safety. Data collection focused on understanding the elements of patient safety governance that influence outcomes. Demographic survey data were analysed descriptively. Qualitative data were coded, analysed thematically (inductive approach), and mapped deductively to the System-Theoretic Accident Model and Processes framework. Findings are presented as themes and a patient safety governance model. The study was approved by the University of South Australia Human Research Ethics Committee. Twenty-seven experts (59% female) participated. Most hailed from Africa (n = 6, 22%), Australasia, and the Middle East (n = 5, 19% each). The majority were employed in hospital settings (n = 23, 85%), and reported blended experience across healthcare improvement (89%), accreditation (76%), organizational operations (64%), and policy (60%). The number and range of patient safety issues within our sample varied widely with 14 topics being addressed. Thematically, 532 textual segments were grouped into 90 codes (n = 44 barriers, n = 46 facilitators) and used to identify and arrange key patient safety governance actors and factors as a 'system' within the System-Theoretic Accident Model and Processes framework. Four themes for improved patient safety governance were identified: (1) 'safety culture' in healthcare organizations, (2) 'policies and procedures' to investigate, implement, and demonstrate impact from patient safety initiatives, (3) 'supporting staff' to upskill and share learnings, and (4) 'patient engagement, experiences, and expectations'. For sustainable patient safety governance, experts highlighted the importance of safety culture in healthcare organizations, national patient safety policies and regulatory standards, continuing education for staff, and meaningful patient engagement approaches. Our proposed 'patient safety governance model' provides policymakers and researchers with a framework to develop data-driven patient safety policy.

Treating persistent pain after breast cancer: practice gaps and future directions.

This paper discusses the growing problem of persisting pain after successful treatment of breast cancer and presents recommendations for improving pain-related outcomes for this group. We discuss the dominant treatment approach for persisting pain post-breast cancer treatment and draw contrasts with contemporary treatment approaches to persistent pain in non-cancer-related populations. We discuss modern application of the biopsychosocial model of pain and the notion of variable sensitivity within the pain system, moment by moment and over time. We present the implications of increasing sensitivity over time for treatment selection and implementation. By drawing on transformative changes in treatment approaches to persistent non-cancer-related pain, we describe the potentially powerful role that an intervention called pain science education, which is now recommended in clinical guidelines for musculoskeletal pain, may play in improving pain and disability outcomes after successful breast cancer treatment. Finally, we present several research recommendations that centre around adaptation of the content and delivery models of contemporary pain science education, to the post-breast cancer context.

A Mixed-Methods Systematic Review of the Effectiveness and Experiences of Quality Improvement Interventions in Radiology.

OBJECTIVE: This study aimed to compile and synthesize evidence regarding the effectiveness of quality improvement interventions in radiology and the experiences and perspectives of staff and patients. METHODS: Databases searched for both published and unpublished studies were as follows: EMBASE, MEDLINE, CINAHL, Joanna Briggs Institute, Cochrane Central Register of Controlled Trials, PsycINFO, Scopus, Web of Science, Mednar, Trove, Google Gray, OCLC WorldCat, and Dissertations and Theses. This review included both qualitative and quantitative studies of patients undergoing radiological examinations and/or medical imaging health care professionals; a broad range of quality improvement interventions including introduction of health information technology, effects of training and education, improved reporting, safety programs, and medical devices; the experiences and perspectives of staff and patients; context of radiological setting; a broad range of outcomes including patient safety; and a result-based convergent synthesis design. RESULTS: Eighteen studies were selected from 4846 identified by a systematic literature search. Five groups of interventions were identified: health information technology (n = 6), training and education (n = 6), immediate and critical reporting (n = 3), safety programs (n = 2), and the introduction of mobile radiography (n = 1), with demonstrated improvements in outcomes, such as improved operational and workflow efficiency, report turnaround time, and teamwork and communication. CONCLUSIONS: The findings were constrained by the limited range of interventions and outcome measures. Further research should be conducted with study designs that might produce findings that are more generalizable, examine the other dimensions of quality, and address the issues of cost and risk versus benefit.

What Constitutes "Appropriate Care" for Low Back Pain?: Point-of-Care Clinical Indicators From Guideline Evidence and Experts (the STANDING Collaboration Project).

STUDY DESIGN: Multiround wiki-based Delphi expert panel survey. OBJECTIVE: To provide proof of concept for an alternative method for creating sets of nationally-agreed point-of-care clinical indicators, and obtain consensus among end-user groups on "appropriate care" for the assessment, diagnosis, acute, and ongoing care of people with low back pain (LBP). SUMMARY OF BACKGROUND DATA: The provision of inappropri ate and low value care for LBP is a significant healthcare and societal burden. Vague clinical practice guideline (CPG) recom mendations can be difficult to apply and measure in real world clinical practice, and a likely barrier to "appropriate care." METHODS: Draft "appropriate care" clinical indicators for LBP were derived from CPG recommendations published between 2011 and 2017. Included CPGs were independently appraised by two reviewers using the Appraisal of Guidelines for Research and Evaluation instrument. Headed by a Clinical Champion, a 20-member Expert Panel reviewed and commented on the draft indicators over a three-round modified e-Delphi process using a collaborative online wiki. At the conclusion of each review round, the research team and the Clinical Champion synthesized and responded to experts' comments and incorporated feedback into the next iteration of the draft indicators. RESULTS: From seven CPGs and six qualitative meta-syntheses, 299 recommendations and themes were used to draft 42 "appropriateness" indicators. In total, 17 experts reviewed these indicators over 18 months. A final set of 27 indicators compris ing screening and diagnostic processes (n = 8), assessment (n = 3), acute (n = 5), and ongoing care (n = 9), and two which crossed the acute-ongoing care continuum. Most indicators were geared toward recommended care (n = 21, 78%), with the remainder focused on care to be avoided. CONCLUSION: These 27 LBP clinical indicators can be used by healthcare consumers, clinicians, researchers, policy makers/ funders, and insurers to guide and monitor the provision of "appropriate care" for LBP.Level of Evidence: 4.

Does health information technology improve acknowledgement of radiology results for discharged Emergency Department patients? A before and after study.

BACKGROUND: The inadequate follow-up of test results is a key patient safety concern, carrying severe consequences for care outcomes. Patients discharged from the emergency department are at particular risk of having test results pending at discharge due to their short lengths of stay, with many hospitals acknowledging that they do not have reliable systems for managing such results. Health information technology hold the potential to reducing errors in the test result management process. This study aimed to measure changes in the proportion of acknowledged radiology reports pre and post introduction of an electronic result acknowledgement system and to determine the proportion of reports with abnormal results, including clinically significant abnormal results requiring follow-up action. METHODS: A before and after study was conducted in the emergency department of a 450-bed metropolitan teaching hospital in Australia. All radiology reports for discharged patients for a one-month period before and after implementation of the electronic result acknowledgement system were reviewed to determine; i) those that reported abnormal results; ii) evidence of test result acknowledgement. All unacknowledged radiology results with an abnormal finding were assessed by an independent panel of two senior emergency physicians for clinical significance. RESULTS: Of 1654 radiology reports in the pre-implementation period 70.6% (n = 1167) had documented evidence of acknowledgement by a clinician. For reports with abnormal results, 71.6% (n = 396) were acknowledged. Of 157 unacknowledged abnormal radiology reports reviewed by an independent emergency physician panel, 34.4% (n = 54) were identified as clinically significant and 50% of these (n = 27) were deemed to carry a moderate likelihood of patient morbidity if not followed up. Electronic acknowledgement occurred for all radiology reports in the post period (n = 1423), representing a 30.4% (95% CI: 28.1-32.6%) increase in acknowledgement rate, and an increase of 28.4% (95% CI: 24.6-32.2%) for abnormal radiology results. CONCLUSIONS: The findings of this study demonstrate the potential of health information technology to improve the safety and effectiveness of the diagnostic process by increasing the rate of follow up of results pending at hospital discharge.

A qualitative content analysis of retained surgical items: learning from root cause analysis investigations.

OBJECTIVE: To describe incidents of retained surgical items, including their characteristics and the circumstances in which they occur. DESIGN: A qualitative content analysis of root cause analysis investigation reports. SETTING: Public health services in Victoria, Australia, 2010-2015. PARTICIPANTS: Incidents of retained surgical items as described by 31 root cause analysis investigation reports. MAIN OUTCOME MEASURE(S): The type of retained surgical item, the length of time between the item being retained and detected and qualitative descriptors of the contributing factors and the circumstances in which the retained surgical items occurred. RESULTS: Surgical packs, drain tubes and vascular devices comprised 68% (21/31) of the retained surgical items. Nearly one-quarter of the retained surgical items were detected either immediately in the post-operative period or on the day of the procedure (7/31). However, about one-sixth (5/31) were only detected after 6 months, with the longest period being 18 months. Contributing factors included complex or multistage surgery; the use of packs not specific to the purpose of the surgery; and design features of the surgical items. CONCLUSION: Retained drains occurred in the post-operative phase where surgical counts are not applicable and clinician situational awareness may not be as great. Root cause analysis investigation reports can be a valuable means of characterizing infrequently occurring adverse events such as retained surgical items. They may detect incidents that are not detected by other data collections and can inform the design enhancements and development of technologies to reduce the impact of retained surgical items.

CareTrack Aged: the appropriateness of care delivered to Australians living in residential aged care facilities: a study protocol.

INTRODUCTION: The aged population is increasing rapidly across the world and this is expected to continue. People living in residential aged care facilities (RACFs) represent amongst the sickest and frailest cohort of the aged population, with a high prevalence of chronic conditions and complex comorbidities. Given the vulnerability of RACF residents and the demands on the system, there is a need to determine the extent that care is delivered in line with best practice ('appropriate care') in RACFs. There is also a recognition that systems should provide care that optimises quality of life (QoL), which includes support for physical and psychological well-being, independence, social relationships, personal beliefs and a caring external environment. The aims of CareTrack Aged are to develop sets of indicators for appropriate care and processes of care for commonly managed conditions, and then assess the appropriateness of care delivered and QoL of residents in RACFs in Australia. METHODS AND ANALYSIS: We will extract recommendations from clinical practice guidelines and, using expert review, convert these into sets of indicators for 15 common conditions and processes of care for people living in RACFs. We will recruit RACFs in three Australian states, and residents within these RACFs, using a stratified multistage sampling method. Experienced nurses, trained in the CareTrack Aged methods ('surveyors'), will review care records of recruited residents within a 1-month period in 2019 and 2020, and assess the care documented against the indicators of appropriate care. Surveyors will concurrently assess residents' QoL using validated questionnaires. ETHICS AND DISSEMINATION: The study has been reviewed and approved by the Human Research Ethics Committee of Macquarie University (5201800386). The research findings will be published in international and national journals and disseminated through conferences and presentations to interested stakeholder groups, including consumers, national agencies, healthcare professionals, policymakers and researchers.

Clinical indicators for common paediatric conditions: Processes, provenance and products of the CareTrack Kids study.

BACKGROUND: In order to determine the extent to which care delivered to children is appropriate (in line with evidence-based care and/or clinical practice guidelines (CPGs)) in Australia, we developed a set of clinical indicators for 21 common paediatric medical conditions for use across a range of primary, secondary and tertiary healthcare practice facilities. METHODS: Clinical indicators were extracted from recommendations found through systematic searches of national and international guidelines, and formatted with explicit criteria for inclusion, exclusion, time frame and setting. Experts reviewed the indicators using a multi-round modified Delphi process and collaborative online wiki to develop consensus on what constituted appropriate care. RESULTS: From 121 clinical practice guidelines, 1098 recommendations were used to draft 451 proposed appropriateness indicators. In total, 61 experts (n = 24 internal reviewers, n = 37 external reviewers) reviewed these indicators over 40 weeks. A final set of 234 indicators resulted, from which 597 indicator items were derived suitable for medical record audit. Most indicator items were geared towards capturing information about under-use in healthcare (n = 551, 92%) across emergency department (n = 457, 77%), hospital (n = 450, 75%) and general practice (n = 434, 73%) healthcare facilities, and based on consensus level recommendations (n = 451, 76%). The main reason for rejecting indicators was 'feasibility' (likely to be able to be used for determining compliance with 'appropriate care' from medical record audit). CONCLUSION: A set of indicators was developed for the appropriateness of care for 21 paediatric conditions. We describe the processes (methods), provenance (origins and evolution of indicators) and products (indicator characteristics) of creating clinical indicators within the context of Australian healthcare settings. Developing consensus on clinical appropriateness indicators using a Delphi approach and collaborative online wiki has methodological utility. The final indicator set can be used by clinicians and organisations to measure and reflect on their own practice.

What is the association between the presence of comorbidities and the appropriateness of care for low back pain? A population-based medical record review study.

BACKGROUND: Although "non-specific" in 90% of cases, low back pain (LBP) is often treated as an independent entity, even though comorbidities are commonly associated with it. There is evidence that some LBP may be related to chronic conditions or be a symptom of poor health. The purpose of this study was to clarify the extent of comorbidities amongst a cohort of Australian adults with LBP and examine if having concurrent conditions has any association with appropriateness of care for LBP. METHODS: A population-based sample of patients with one or more of 22 common conditions was recruited by telephone; consents were obtained to review their medical records. Trained surveyors extracted information from their medical records to examine the care patients received for their LBP with respect to ten indicators of appropriate care, ratified by LBP experts. Using LBP as the index condition, lists of self-reported comorbidities and those that were documented in medical records were compared. Medical records were reviewed and analysed with respect to appropriateness of care to identify any significant differences in care received between patients with LBP only and those with LBP plus comorbidities. RESULTS: One hundred and sixty four LBP patients were included in the analysis. Over 60% of adults with LBP in Australia had one of 17 comorbidities documented, with females being more likely than males to have comorbid conditions (63% vs 37%, p = 0.012). The more comorbidities, the poorer their reported health status (63% vs 30%, p = 0.006). Patients with comorbidities were significantly less likely to receive appropriate LBP care on nine of the ten LBP indicators (p < 0.05). CONCLUSIONS: This study established that the presence of comorbidities is associated with poorer care for LBP. Understanding why this is so is an important direction for future research. Further studies using a larger cohort are needed to explore the association between comorbidities and appropriateness of care for LBP, to better inform guidelines and practice in this area.

Quality of Health Care for Children in Australia, 2012-2013.

Importance: The quality of routine care for children is rarely assessed, and then usually in single settings or for single clinical conditions. Objective: To estimate the quality of health care for children in Australia in inpatient and ambulatory health care settings. Design, Setting, and Participants: Multistage stratified sample with medical record review to assess adherence with quality indicators extracted from clinical practice guidelines for 17 common, high-burden clinical conditions (noncommunicable [n = 5], mental health [n = 4], acute infection [n = 7], and injury [n = 1]), such as asthma, attention-deficit/hyperactivity disorder, tonsillitis, and head injury. For these 17 conditions, 479 quality indicators were identified, with the number varying by condition, ranging from 9 for eczema to 54 for head injury. Four hundred medical records were targeted for sampling for each of 15 conditions while 267 records were targeted for anxiety and 133 for depression. Within each selected medical record, all visits for the 17 targeted conditions were identified, and separate quality assessments made for each. Care was evaluated for 6689 children 15 years of age and younger who had 15 240 visits to emergency departments, for inpatient admissions, or to pediatricians and general practitioners in selected urban and rural locations in 3 Australian states. These visits generated 160 202 quality indicator assessments. Exposures: Quality indicators were identified through a systematic search of local and international guidelines. Individual indicators were extracted from guidelines and assessed using a 2-stage Delphi process. Main Outcomes and Measures: Quality of care for each clinical condition and overall. Results: Of 6689 children with surveyed medical records, 53.6% were aged 0 to 4 years and 55.5% were male. Adherence to quality of care indicators was estimated at 59.8% (95% CI, 57.5%-62.0%; n = 160 202) across the 17 conditions, ranging from a high of 88.8% (95% CI, 83.0%-93.1%; n = 2638) for autism to a low of 43.5% (95% CI, 36.8%-50.4%; n = 2354) for tonsillitis. The mean adherence by condition category was estimated as 60.5% (95% CI, 57.2%-63.8%; n = 41 265) for noncommunicable conditions (range, 52.8%-75.8%); 82.4% (95% CI, 79.0%-85.5%; n = 14 622) for mental health conditions (range, 71.5%-88.8%); 56.3% (95% CI, 53.2%-59.4%; n = 94 037) for acute infections (range, 43.5%-69.8%); and 78.3% (95% CI, 75.1%-81.2%; n = 10 278) for injury. Conclusions and Relevance: Among a sample of children receiving care in Australia in 2012-2013, the overall prevalence of adherence to quality of care indicators for important conditions was not high. For many of these conditions, the quality of care may be inadequate.

Are root cause analyses recommendations effective and sustainable? An observational study.

OBJECTIVE: To assess the strength of root cause analysis (RCA) recommendations and their perceived levels of effectiveness and sustainability. DESIGN: All RCAs related to sentinel events (SEs) undertaken between the years 2010 and 2015 in the public health system in Victoria, Australia were analysed. The type and strength of each recommendation in the RCA reports were coded by an expert patient safety classifier using the US Department of Veteran Affairs type and strength criteria. PARTICIPANTS AND SETTING: Thirty-six public health services. MAIN OUTCOME MEASURE(S): The proportion of RCA recommendations which were classified as 'strong' (more likely to be effective and sustainable), 'medium' (possibly effective and sustainable) or 'weak' (less likely to be effective and sustainable). RESULTS: There were 227 RCAs in the period of study. In these RCAs, 1137 recommendations were made. Of these 8% were 'strong', 44% 'medium' and 48% were 'weak'. In 31 RCAs, or nearly 15%, only weak recommendations were made. In 24 (11%) RCAs five or more weak recommendations were made. In 165 (72%) RCAs no strong recommendations were made. The most frequent recommendation types were reviewing or enhancing a policy/guideline/documentation, and training and education. CONCLUSIONS: Only a small proportion of recommendations arising from RCAs in Victoria are 'strong'. This suggests that insights from the majority of RCAs are not likely to inform practice or process improvements. Suggested improvements include more human factors expertise and independence in investigations, more extensive application of existing tools that assist teams to prioritize recommendations that are likely to be effective, and greater use of observational and simulation techniques to understand the underlying systems factors. Time spent in repeatedly investigating similar incidents may be better spent aggregating and thematically analysing existing sources of information about patient safety.

Lessons learnt from incidents involving the airway and breathing reported from Australasian emergency departments.

OBJECTIVE: To review incident reports relating to problems encountered during the ED management of patients with 'airway or breathing' problems, with the aim of finding and highlighting common themes within these rare events, and making recommendations to further improve patient safety in the areas in which deficiencies have been identified. METHODS: Thematic analysis of 36 incidents reported from Australasian EDs, which were related to problems with airway and breathing. RESULTS: In all, 51 problems were identified among the 36 incidents related to airway and/or breathing. Fourteen involved clinical decision-making, 11 equipment, nine communication, seven intubation, five surgical access and five pneumothorax. Eight incidents involved children and there were nine deaths within hours or days. CONCLUSIONS: Recommendations for improving preparedness of ED staff and facilities have been made for each of the problem areas identified with respect to clinical practice, equipment, communication and clinical process. Analysis of incidents from the Australasian Emergency Medicine Events Register allows clusters of like-events to be identified and characterised, providing the possibility of getting a better idea of how problems present and progress, with some information about contributing factors, characteristics and context. This will pave the way for earlier and better detection of life-threatening problems and the development and reinforcement of preventive and corrective strategies.

Mixed methods study on the use of and attitudes towards safety checklists in interventional radiology.

INTRODUCTION: The use of safety checklists in interventional radiology is an intervention aimed at reducing mortality and morbidity. Currently there is little known about their practical use in Australian radiology departments. The primary aim of this mixed methods study was to evaluate how safety checklists (SC) are used and completed in radiology departments within Australian hospitals, and attitudes towards their use as described by Australian radiologists. METHODS: A mixed methods approach employing both quantitative and qualitative techniques was used for this study. Direct observations of checklist use during radiological procedures were performed to determine compliance. Medical records were also audited to investigate whether there was any discrepancy between practice (actual care measured by direct observation) and documentation (documented care measured by an audit of records). A focus group with Australian radiologists was conducted to determine attitudes towards the use of checklists. RESULTS: Among the four participating radiology departments, overall observed mean completion of the components of the checklist was 38%. The checklist items most commonly observed to be addressed by the operating theatre staff as noted during observations were correct patient (80%) and procedure (60%). Findings from the direct observations conflicted with the medical record audit, where there was a higher percentage of completion (64% completion) in comparison to the 38% observed. The focus group participants spoke of barriers to the use of checklists, including the culture of radiology departments. CONCLUSION: This is the first study of safety checklist use in radiology within Australia. Overall completion was low across the sites included in this study. Compliance data collected from observations differed markedly from reported compliance in medical records. There remain significant barriers to the proper use of safety checklists in Australian radiology departments.

STANDING Collaboration: a study protocol for developing clinical standards.

INTRODUCTION: Despite widespread availability of clinical practice guidelines (CPGs), considerable gaps continue between the care that is recommended ('appropriate care') and the care provided. Problems with current CPGs are commonly cited as barriers to providing 'appropriate care'.Our study aims to develop and test an alternative method to keep CPGs accessible and up to date. This method aims to mitigate existing problems by using a single process to develop clinical standards (embodied in clinical indicators) collaboratively with researchers, healthcare professionals, patients and consumers. A transparent and inclusive online curated (purpose-designed, custom-built, wiki-type) system will use an ongoing and iterative documentation process to facilitate synthesis of up-to-date information and make available its provenance. All participants are required to declare conflicts of interest. This protocol describes three phases: engagement of relevant stakeholders; design of a process to develop clinical standards (embodied in indicators) for 'appropriate care' for common medical conditions; and evaluation of our processes, products and feasibility. METHODS AND ANALYSIS: A modified e-Delphi process will be used to gain consensus on 'appropriate care' for a range of common medical conditions. Clinical standards and indicators will be developed through searches of national and international guidelines, and formulated with explicit criteria for inclusion, exclusion, time frame and setting. Healthcare professionals and consumers will review the indicators via the wiki-based modified e-Delphi process. Reviewers will declare conflicts of interest which will be recorded and managed according to an established protocol. The provenance of all indicators and suggestions included or excluded will be logged from indicator inception to finalisation. A mixed-methods formative evaluation of our research methodology will be undertaken. ETHICS AND DISSEMINATION: Human Research Ethics Committee approval has been received from the University of South Australia. We will submit the results of the study to relevant journals and offer national and international presentations.

Effectiveness of an electronic patient-centred self-management tool for gout sufferers: a cluster randomised controlled trial protocol.

INTRODUCTION: Gout is increasing despite effective therapies to lower serum urate concentrations to 0.36 mmol/L or less, which, if sustained, significantly reduces acute attacks of gout. Adherence to urate-lowering therapy (ULT) is poor, with rates of less than 50% 1 year after initiation of ULT. Attempts to increase adherence in gout patients have been disappointing. We aim to evaluate the effectiveness of use of a personal, self-management, 'smartphone' application (app) to achieve target serum urate concentrations in people with gout. We hypothesise that personalised feedback of serum urate concentrations will improve adherence to ULT. METHODS AND ANALYSIS: Setting and designPrimary care. A prospective, cluster randomised (by general practitioner (GP) practices), controlled trial. PARTICIPANTS: GP practices will be randomised to either intervention or control clusters with their patients allocated to the same cluster. INTERVENTION: The intervention group will have access to the Healthy.me app tailored for the self-management of gout. The control group patients will have access to the same app modified to remove all functions except the Gout Attack Diary. PRIMARY AND SECONDARY OUTCOMES: The proportion of patients whose serum urate concentrations are less than or equal to 0.36 mmol/L after 6 months. Secondary outcomes will be proportions of patients achieving target urate concentrations at 12 months, ULT adherence rates, serum urate concentrations at 6 and 12 months, rates of attacks of gout, quality of life estimations and process and economic evaluations. The study is designed to detect a ≥30% improvement in the intervention group above the expected 50% achievement of target serum urate at 6 months in the control group: power 0.80, significance level 0.05, assumed 'dropout' rate 20%. ETHICS AND DISSEMINATION: This study has been approved by the University of New South Wales Human Research Ethics Committee. Study findings will be disseminated in international conferences and peer-reviewed journal. TRIAL REGISTRATION NUMBER: ACTRN12616000455460.

Linking clinical quality indicators to research evidence - a case study in asthma management for children.

BACKGROUND: Clinical quality indicators are used to monitor the performance of healthcare services and should wherever possible be based on research evidence. Little is known however about the extent to which indicators in common use are based on research. The objective of this study is to measure the extent to which clinical quality indicators used in asthma management in children with outcome measurements can be linked to results in randomised controlled clinical trial (RCT) reports. This work is part of a broader research program to trial methods that improve the efficiency and accuracy of indicator development. METHODS: National-level indicators for asthma management in children were extracted from the National Quality Measures Clearinghouse database and the National Institute for Health and Care Excellence quality standards by two independent appraisers. Outcome measures were extracted from all published English language RCT reports for asthma management in children below the age of 12 published between 2005 and 2014. The two sets were then linked by manually mapping both to a common set of Unified Medical Language System (UMLS) concepts. RESULTS: The analysis identified 39 indicators and 562 full text RCTs dealing with asthma management in children. About 95% (37/39) of the indicators could be linked to RCT outcome measures. CONCLUSIONS: It is possible to identify relevant RCT reports for the majority of indicators used to assess the quality of asthma management in childhood. The methods reported here could be automated to more generally support assessment of candidate indicators against the research evidence.

Using multiclass classification to automate the identification of patient safety incident reports by type and severity.

BACKGROUND: Approximately 10% of admissions to acute-care hospitals are associated with an adverse event. Analysis of incident reports helps to understand how and why incidents occur and can inform policy and practice for safer care. Unfortunately our capacity to monitor and respond to incident reports in a timely manner is limited by the sheer volumes of data collected. In this study, we aim to evaluate the feasibility of using multiclass classification to automate the identification of patient safety incidents in hospitals. METHODS: Text based classifiers were applied to identify 10 incident types and 4 severity levels. Using the one-versus-one (OvsO) and one-versus-all (OvsA) ensemble strategies, we evaluated regularized logistic regression, linear support vector machine (SVM) and SVM with a radial-basis function (RBF) kernel. Classifiers were trained and tested with "balanced" datasets (n_ Type = 2860, n_ SeverityLevel = 1160) from a state-wide incident reporting system. Testing was also undertaken with imbalanced "stratified" datasets (n_ Type = 6000, n_ SeverityLevel =5950) from the state-wide system and an independent hospital reporting system. Classifier performance was evaluated using a confusion matrix, as well as F-score, precision and recall. RESULTS: The most effective combination was a OvsO ensemble of binary SVM RBF classifiers with binary count feature extraction. For incident type, classifiers performed well on balanced and stratified datasets (F-score: 78.3, 73.9%), but were worse on independent datasets (68.5%). Reports about falls, medications, pressure injury, aggression and blood products were identified with high recall and precision. "Documentation" was the hardest type to identify. For severity level, F-score for severity assessment code (SAC) 1 (extreme risk) was 87.3 and 64% for SAC4 (low risk) on balanced data. With stratified data, high recall was achieved for SAC1 (82.8-84%) but precision was poor (6.8-11.2%). High risk incidents (SAC2) were confused with medium risk incidents (SAC3). CONCLUSIONS: Binary classifier ensembles appear to be a feasible method for identifying incidents by type and severity level. Automated identification should enable safety problems to be detected and addressed in a more timely manner. Multi-label classifiers may be necessary for reports that relate to more than one incident type.

Linking quality indicators to clinical trials: an automated approach.

OBJECTIVE: Quality improvement of health care requires robust measurable indicators to track performance. However identifying which indicators are supported by strong clinical evidence, typically from clinical trials, is often laborious. This study tests a novel method for automatically linking indicators to clinical trial registrations. DESIGN: A set of 522 quality of care indicators for 22 common conditions drawn from the CareTrack study were automatically mapped to outcome measures reported in 13 971 trials from ClinicalTrials.gov. INTERVENTION: Text mining methods extracted phrases mentioning indicators and outcome phrases, and these were compared using the Levenshtein edit distance ratio to measure similarity. MAIN OUTCOME MEASURE: Number of care indicators that mapped to outcome measures in clinical trials. RESULTS: While only 13% of the 522 CareTrack indicators were thought to have Level I or II evidence behind them, 353 (68%) could be directly linked to randomized controlled trials. Within these 522, 50 of 70 (71%) Level I and II evidence-based indicators, and 268 of 370 (72%) Level V (consensus-based) indicators could be linked to evidence. Of the indicators known to have evidence behind them, only 5.7% (4 of 70) were mentioned in the trial reports but were missed by our method. CONCLUSIONS: We automatically linked indicators to clinical trial registrations with high precision. Whilst the majority of quality indicators studied could be directly linked to research evidence, a small portion could not and these require closer scrutiny. It is feasible to support the process of indicator development using automated methods to identify research evidence.

What is the impact of an electronic test result acknowledgement system on Emergency Department physicians' work processes? A mixed-method pre-post observational study.

OBJECTIVE: To examine the impact of an electronic Results Acknowledgement (eRA) system on emergency physicians' test result management work processes and the time taken to acknowledge microbiology and radiology test results for patients discharged from an Emergency Department (ED). METHODS: The impact of the eRA system was assessed in an Australian ED using: a) semi-structured interviews with senior emergency physicians; and b) a time and motion direct observational study of senior emergency physicians completing test acknowledgment pre and post the implementation of the eRA system. RESULTS: The eRA system led to changes in the way results and actions were collated, stored, documented and communicated. Although there was a non-significant increase in the average time taken to acknowledge results in the post period, most types of acknowledgements (other than simple acknowledgements) took less time to complete. The number of acknowledgements where physicians sought additional information from the Electronic Medical Record (EMR) rose from 12% pre to 20% post implementation of eRA. CONCLUSIONS: Given that the type of results are unlikely to have changed significantly across the pre and post implementation periods, the increase in the time physicians spent accessing additional clinical information in the post period likely reflects the greater access to clinical information provided by the integrated electronic system. Easier access to clinical information may improve clinical decision making and enhance the quality of patient care. For instance, in situations where a senior clinician, not initially involved in the care process, is required to deal with the follow-up of non-normal results.

Automating the Identification of Patient Safety Incident Reports Using Multi-Label Classification.

Automated identification provides an efficient way to categorize patient safety incidents. Previous studies have focused on identifying single incident types relating to a specific patient safety problem, e.g., clinical handover. In reality, there are multiple types of incidents reflecting the breadth of patient safety problems and a single report may describe multiple problems, i.e., it can be assigned multiple type labels. This study evaluated the abilty of multi-label classification methods to identify multiple incident types in single reports. Three multi-label methods were evaluated: binary relevance, classifier chains and ensemble of classifier chains. We found that an ensemble of classifier chains was the most effective method using binary Support Vector Machines with radial basis function kernel and bag-of-words feature extraction, performing equally well on balanced and stratified datasets, (F-score: 73.7% vs. 74.7%). Classifiers were able to identify six common incident types: falls, medications, pressure injury, aggression, documentation problems and others.