Analysing aggregate clinical data to support evaluation of the Tackling Indigenous Smoking program, 2014-16.

Objective The aim of this study was to assess the change in recording of client population smoking attributes (smoking status recorded and smoking status) in Tackling Indigenous Smoking (TIS)-funded services compared with non-funded services for Aboriginal and Torres Strait Islander people, 2014-16. Methods The study included a cohort of 152 Aboriginal-community controlled services with aggregate client smoking data from 2014 to 2016. Negative binomial regression was used to assess change in smoking status recorded and smoking status between TIS and non-TIS funded organisations. The models controlled for size of client population, jurisdiction and remoteness. Results From 2014 to 2016, the overall reporting rate (change in recording of smoking status) of client smoking status was 1.58-fold higher (95% confidence interval (CI) 1.30-1.91; P<0.001) in TIS-funded than non-TIS-funded services after controlling for year, remoteness and their interaction. The highest change in reporting of client smoking status was for TIS-funded services in remote areas (reporting ratio 6.55; 95% CI 5.18-8.27; P<0.001). In 2016, TIS-funded services reported higher overall levels of recording client smoking status (current, ex- and non-smokers) than non-TIS funded services (RR 1.11; 95% CI 1.00-1.28; P<0.001). There was no significant change in the reporting of smokers, ex-smokers or non-smokers over the three reporting periods. Conclusion The analysis shows higher reporting of the proportion of the service client population for services funded under the TIS program compared with non-TIS-funded services. Existing evidence suggests that following-up smokers with targeted clinical interventions once they have had smoking status recorded could reduce smoking rates in the long term. The public health contribution of this study has defined one method for assessing smoking attributes when using aggregate health service data. This method could be applied to future tobacco control programs in health services. What is known about the topic? Aboriginal and Torres Strait Islander smoking prevalence is high. The Aboriginal and Torres Strait Islander primary healthcare providers' national key performance indicators (nKPIs) are one data source that can track changes over time in smoking in clients of these services. What does this paper add? This paper presents the first analytical study and evaluation of the nKPI dataset items on smoking. What are the implications for practitioners? There is value in analysing routinely collected data in program evaluations. The method used in this paper demonstrates one approach that could be used to assess smoking indicators and their changes over time in TIS program evaluation.

Exploring patterns of error in acute care using framework analysis.

BACKGROUND: Junior doctors are often the first responders to deteriorating patients in hospital. In the high-stakes and time-pressured context of acute care, the propensity for error is high. This study aimed to identify the main subject areas in which junior doctors' acute care errors occur, and cross-reference the errors with Reason's Generic Error Modelling System (GEMS). GEMS categorises errors according to the underlying cognitive processes, and thus provides insight into the causative factors. The overall aim of this study was to identify patterns in junior doctors' acute care errors in order to enhance understanding and guide the development of educational strategies. METHODS: This observational study utilised simulated acute care scenarios involving junior doctors dealing with a range of emergencies. Scenarios and the subsequent debriefs were video-recorded. Framework analysis was used to categorise the errors according to eight inductively-developed key subject areas. Subsequently, a multi-dimensional analysis was performed which cross-referenced the key subject areas with an earlier categorisation of the same errors using GEMS. The numbers of errors in each category were used to identify patterns of error. RESULTS: Eight key subject areas were identified; hospital systems, prioritisation, treatment, ethical principles, procedural skills, communication, situation awareness and infection control. There was a predominance of rule-based mistakes in relation to the key subject areas of hospital systems, prioritisation, treatment and ethical principles. In contrast, procedural skills, communication and situation awareness were more closely associated with skill-based slips and lapses. Knowledge-based mistakes were less frequent but occurred in relation to hospital systems and procedural skills. CONCLUSIONS: In order to improve the management of acutely unwell patients by junior doctors, medical educators must understand the causes of common errors. Adequate knowledge alone does not ensure prompt and appropriate management and referral. The teaching of acute care skills may be enhanced by encouraging medical educators to consider the range of potential error types, and their relationships to particular tasks and subjects. Rule-based mistakes may be amenable to simulation-based training, whereas skill-based slips and lapses may be reduced using strategies designed to raise awareness of the interplay between emotion, cognition and behaviour.

Exploring error in team-based acute care scenarios: an observational study from the United kingdom.

PURPOSE: To investigate the errors made by junior doctors (first year after primary medical qualification) in simulated acute care settings, using (and, for some purposes, amplifying) a previously published generic error-modeling system (GEMS). Possible error types were skill-based slips and lapses, rule-based mistakes, knowledge-based mistakes, and violations. METHOD: In August 2010, 38 junior doctors participated in high-fidelity simulated acute care scenarios in NHS Lothian, Scotland. Each video-recorded scenario was immediately followed by an audio-recorded debrief that encouraged articulation of underlying cognitive processes. Two researchers used evidence from the scenario, debrief, and field notes to determine which errors were attributable to a single underlying cause. In such cases, the errors were coded by template analysis into the GEMS framework. Errors for which a single cause could be identified but which did not fit the framework were coded inductively. RESULTS: A total of 243 errors were identified, with sufficient evidence available to identify a single cause in 190. Skill-based slips and lapses, rule-based mistakes, and knowledge-based mistakes were all clearly identified within the data. Two error types not originally included in the GEMS framework were identified: compound errors and submission errors. CONCLUSIONS: Amplification of GEMS provides a valid framework for categorization of the errors made by junior doctors in simulated acute care contexts. In addition, the amplified framework may be transferable to other, team-based contexts. An improved understanding of the knowledge and skills that are most vulnerable to each specific type of error will allow tailored educational strategies to be developed.

Leniency and halo effects in marking undergraduate short research projects.

BACKGROUND: Supervisors are often involved in the assessment of projects they have supervised themselves. Previous research suggests that detailed marking sheets may alleviate leniency and halo effects. We set out to determine if, despite using such a marking schedule, leniency and halo effects were evident in the supervisors' marking of undergraduate short research projects (special study modules (SSM)). METHODS: Review of grades awarded by supervisors, second markers and control markers to the written reports of 4th year medical students who had participated in an SSM during two full academic years (n = 399). Paired t-tests were used to compare mean marks, Pearson correlation to look at agreement between marks and multiple linear regression to test the prediction of one mark from several others adjusted for one another. RESULTS: There was a highly significant difference of approximately half a grade between supervisors and second markers with supervisors marking higher. (t = 3.12, p < 0.01, difference in grade score = 0.42, 95% CI for mean difference 0.18-0.80). There was a high correlation between the two marks awarded for performance of the project and the written report by the supervisor (r = 0.75), but a low-modest correlation between supervisor and second marker (r = 0.28). Linear regression analysis of the influence of the supervisors' mark for performance on their mark for the report gave a non-significant result. This suggests a leniency effect but no halo effect. CONCLUSIONS: This study shows that with the use of structured marking sheet for assessment of undergraduate medical students, supervisors marks are not associated with a halo effect, but leniency does occur. As supervisor assessment is becoming more common in both under graduate and postgraduate teaching new ways to improve objectivity in marking and to address the leniency of supervisors should be sought.