Integrating organizational impacts into health technology assessment (HTA): an analysis of the content and use of existing evaluation frameworks.

OBJECTIVE: The French health technology assessment (HTA) agency initiated a research between 2018 and 2019 with the aim of determining whether other HTA organizations (agencies, bodies, institutes, and expert networks) and researchers had developed an evaluation framework of organizational impacts (OIs). METHODS: Three types of investigation were carried out: (i) an analysis of documents published by selected HTA organizations, (ii) a rapid review on the OI issues, (iii) a questionnaire survey to experts of the International Network of Agencies for Health Technology Assessment. RESULTS: The analyses highlight six key points: (i) there is no explicit conceptual definition of OIs; (ii) OIs are often not included in a specific dimension of the evaluation or in the same dimensions; (iii) three recurring categories emerge from the assessment of OIs: processes, structure, and culture; (iv) despite its limitations, the European Network for Health Technology Assessment framework (Core Model) is the most mature assessment model to date; (v) the question of the scope of OIs to be considered is unresolved (micro-meso-macro); and (vi) the delineation between OI assessment and economic assessment must be addressed. CONCLUSIONS: Although the issue of considering OI in HTA has been raised for many years, it remains largely unresolved. Defining the concept of OI is a prerequisite for taking the next step toward an evaluation framework. As the question of the impact of innovation goes beyond the health sector, extensive research on how to define and take into account these OIs may be relevant.

Safety attitudes build safety culture: Nurse/midwife leaders improving health care using quantitative data.

AIM: We aim to determine safety attitudes of nurses and midwives across a Local Health District in Australia and compare results 1 year later following facilitated feedback of results. BACKGROUND: Positive safety cultures are imperative for positive patient and staff outcomes. Staff member's attitude contribute to an organisations safety culture but can differ between health professional groups and across different subcultures. METHOD: The Safety Attitudes Questionnaire (SAQ-Short version) was administered to all nurses and midwives within a Local Health District in NSW, Australia in 2019 and 2020. Results were facilitated back to nursing/midwifery leadership teams with an expectation of developing and enacting an action plan, based on results. RESULTS: Of the six domains in the SAQ-Short version, five domains scores increased significantly (p < .001) over the time period. CONCLUSIONS: Measures over time are important to establish differences in perceptions and feedback on impact of actions. Facilitated feedback of results shows meaning when nursing/midwifery leadership staff have data explained and an opportunity to discuss and plan. IMPLICATIONS FOR NURSING MANAGEMENT: This study shows that facilitated feedback of quantitative survey results brings improved results when a survey is replicated. Nurse managers should enact a contextualized action plan with teams based on survey results to influence improvement in safety attitudes of staff.

Review of Evaluation Metrics Used in Digital and Traditional Tobacco Control Campaigns.

BACKGROUND: Mass media campaigns for public health are increasingly using digital media platforms, such as web-based advertising and social media; however, there is a lack of evidence on how to best use these digital platforms for public health campaigns. To generate this evidence, appropriate campaign evaluations are needed, but with the proliferation of digital media-related metrics, there is no clear consensus on which evaluation metrics should be used. Public health campaigns are diverse in nature, so to facilitate analysis, this review has selected tobacco control campaigns as the scope of the study. OBJECTIVE: This literature review aimed to examine how tobacco control campaigns that use traditional and digital media platforms have been evaluated. METHODS: Medicine and science databases (Medical Literature Analysis and Retrieval System Online [MEDLINE], EMBASE, PsycINFO, Cumulative Index to Nursing and Allied Health Literature [CINAHL], and Scopus), and a marketing case study database (World Advertising Research Center) were searched for articles published between 2013 and 2018. Two authors established the eligibility criteria and reviewed articles for inclusion. Individual campaigns were identified from the articles, and information on campaigns and their evaluations were supplemented with searches on Google, Google Scholar, and social media platforms. Data about campaign evaluations were tabulated and mapped to a conceptual framework. RESULTS: In total, 17 campaigns were included in this review, with evaluations reported on by 51 articles, 17 marketing reports, and 4 grey literature reports. Most campaigns were from English-speaking countries, with behavioral change as the primary objective. In the process evaluations, a wide range of metrics were used to assess the reach of digital campaign activities, making comparison between campaigns difficult. Every campaign in the review, except one, reported some type of engagement impact measure, with website visits being the most commonly reported metric (11 of the 17 campaigns). Other commonly reported evaluation measures identified in this review include engagement on social media, changes in attitudes, and number of people contacting smoking cessation services. Of note, only 7 of the 17 campaigns attempted to measure media platform attribution, for example, by asking participants where they recalled seeing the campaign or using unique website tracking codes for ads on different media platforms. CONCLUSIONS: One of the key findings of this review is the numerous and diverse range of measures and metrics used in tobacco control campaign evaluations. To address this issue, we propose principles to guide the selection of digital media-related metrics for campaign evaluations, and also outline a conceptual framework to provide a coherent organization to the diverse range of metrics. Future research is needed to specifically investigate whether engagement metrics are associated with desired campaign outcomes, to determine whether reporting of engagement metrics is meaningful in campaign evaluations.

Effectiveness of an educational intervention to improve the safety culture in primary care: a randomized trial.

BACKGROUND: Fostering a culture of safety is an essential step in ensuring patient safety and quality in primary care. We aimed to evaluate the effectiveness of an educational intervention to improve the safety culture in the family and community medicine teaching units in an Atlantic European Region. METHODS: Randomized study conducted in family and community medicine teaching units in Galicia (Spain). Participants were all fourth-year residents and their tutors (N = 138). Those who agreed to participate were randomized into one of two groups (27 tutors/26 residents in the intervention group, 23 tutors/ 23 residents in the control one).All were sent the Survey on Patient Safety Culture. After that, the intervention group received specific training in safety; they also recorded incidents over 15 days, documented them following a structured approach, and had feedback on their performance. The control group did not receive any action. All participants completed the same survey four months later. Outcome measures were the changes in safety culture as quantified by the results variables of the Survey: Patient Safety Grade and Number of events reported. We conducted bivariate and adjusted analyses for the outcome measures. To explore the influence of participants' demographic characteristics and their evaluation of the 12 dimensions of the safety culture, we fitted a multivariate model for each outcome. RESULTS: Trial followed published protocol. There were 19 drop outs. The groups were comparable in outcome and independent variables at start. The experiment did not have any effect on Patient safety grade (- 0.040) in bivariate analysis. The odds of reporting one to two events increased by 1.14 (0.39-3.35), and by 13.75 (2.41-354.37) the odds of reporting 3 or more events. Different dimensions had significant independent effects on each outcome variable. CONCLUSION: A educational intervention in family and community medicine teaching units may improve the incidents reported. The associations observed among organizational dimensions and outcomes evidence the complexity of patient safety culture measurement and, also, show the paths for improvement. In the future, it would be worthwhile to replicate this study in teaching units from different settings and with different health professionals engaged. TRIAL REGISTRATION: It was retrospectively registered with ( ISRCTN41911128 , 31/12/2010).

Methods to Evaluate the Effects of Internet-Based Digital Health Interventions for Citizens: Systematic Review of Reviews.

BACKGROUND: Digital health can empower citizens to manage their health and address health care system problems including poor access, uncoordinated care and increasing costs. Digital health interventions are typically complex interventions. Therefore, evaluations present methodological challenges. OBJECTIVE: The objective of this study was to provide a systematic overview of the methods used to evaluate the effects of internet-based digital health interventions for citizens. Three research questions were addressed to explore methods regarding approaches (study design), effects and indicators. METHODS: We conducted a systematic review of reviews of the methods used to measure the effects of internet-based digital health interventions for citizens. The protocol was developed a priori according to Preferred Reporting Items for Systematic review and Meta-Analysis Protocols and the Cochrane Collaboration methodology for overviews of reviews. Qualitative, mixed-method, and quantitative reviews published in English or French from January 2010 to October 2016 were included. We searched for published reviews in PubMed, EMBASE, The Cochrane Database of Systematic Reviews, CINHAL and Epistemonikos. We categorized the findings based on a thematic analysis of the reviews structured around study designs, indicators, types of interventions, effects and perspectives. RESULTS: A total of 20 unique reviews were included. The most common digital health interventions for citizens were patient portals and patients' access to electronic health records, covered by 10/20 (50%) and 6/20 (30%) reviews, respectively. Quantitative approaches to study design included observational study (15/20 reviews, 75%), randomized controlled trial (13/20 reviews, 65%), quasi-experimental design (9/20 reviews, 45%), and pre-post studies (6/20 reviews, 30%). Qualitative studies or mixed methods were reported in 13/20 (65%) reviews. Five main categories of effects were identified: (1) health and clinical outcomes, (2) psychological and behavioral outcomes, (3) health care utilization, (4) system adoption and use, and (5) system attributes. Health and clinical outcomes were measured with both general indicators and disease-specific indicators and reported in 11/20 (55%) reviews. Patient-provider communication and patient satisfaction were the most investigated psychological and behavioral outcomes, reported in 13/20 (65%) and 12/20 (60%) reviews, respectively. Evaluation of health care utilization was included in 8/20 (40%) reviews, most of which focused on the economic effects on the health care system. CONCLUSIONS: Although observational studies and surveys have provided evidence of benefits and satisfaction for patients, there is still little reliable evidence from randomized controlled trials of improved health outcomes. Future evaluations of digital health interventions for citizens should focus on specific populations or chronic conditions which are more likely to achieve clinically meaningful benefits and use high-quality approaches such as randomized controlled trials. Implementation research methods should also be considered. We identified a wide range of effects and indicators, most of which focused on patients as main end users. Implications for providers and the health system should also be included in evaluations or monitoring of digital health interventions.

Digitised evaluation of speech intelligibility using vowels in maxillectomy patients.

Among the functional disabilities that patients face following maxillectomy, speech impairment is a major factor influencing quality of life. Proper rehabilitation of speech, which may include prosthodontic and surgical treatments and speech therapy, requires accurate evaluation of speech intelligibility (SI). A simple, less time-consuming yet accurate evaluation is desirable both for maxillectomy patients and the various clinicians providing maxillofacial treatment. This study sought to determine the utility of digital acoustic analysis of vowels for the prediction of SI in maxillectomy patients, based on a comprehensive understanding of speech production in the vocal tract of maxillectomy patients and its perception. Speech samples were collected from 33 male maxillectomy patients (mean age 57.4 years) in two conditions, without and with a maxillofacial prosthesis, and formant data for the vowels /a/,/e/,/i/,/o/, and /u/ were calculated based on linear predictive coding. The frequency range of formant 2 (F2) was determined by differences between the minimum and maximum frequency. An SI test was also conducted to reveal the relationship between SI score and F2 range. Statistical analyses were applied. F2 range and SI score were significantly different between the two conditions without and with a prosthesis (both P < .0001). F2 range was significantly correlated with SI score in both the conditions (Spearman's r = .843, P < .0001; r = .832, P < .0001, respectively). These findings indicate that calculating the F2 range from 5 vowels has clinical utility for the prediction of SI after maxillectomy.

Patient Reported Outcomes in Preoperative and Postoperative Patients with Hypospadias.

PURPOSE: Current outcome tools for hypospadias have limited focus on the caregiver or patient perspective of important patient centered outcomes. In this study we collaborated with patients, caregivers, and lay and medical experts to develop and pilot a patient reported outcome measure for hypospadias. MATERIALS AND METHODS: We developed a patient reported outcome measure based on systematic review of the literature and focus group input. The patient reported outcome measure was piloted in caregivers for boys younger than 8 years and in patients older than 8 years who presented for urology consultation before meeting with the surgeon. Patients were classified with uncorrected hypospadias, successful repair or failed repair based on the presence or absence of complications (fistula, diverticulum, meatal stenosis/stricture, greater than 30-degree recurrent curvature, glans dehiscence and/or skin reoperation). RESULTS: A patient reported outcome measure was developed and administered to 347 patients and/or caregivers-proxies, including 105 uncorrected cases, 162 successful repair cases and 80 failed cases. Satisfaction with appearance was highest in those with successful hypospadias repair compared to failed repair and uncorrected hypospadias (93% vs 77% and 67%, respectively). Voiding symptoms such as spraying or a deviated stream were highest in failed and uncorrected cases (39% and 37%, respectively). Overall dissatisfaction with voiding was highest for uncorrected hypospadias and failed repair compared to successful cases (54% and 47%, respectively, vs 15%). CONCLUSIONS: The evaluation of patient and caregiver-proxy reported outcomes in preoperative and postoperative patients with hypospadias allows for the quantification of benefits derived from hypospadias repair and may ultimately represent the gold standard outcome measure for hypospadias. This pilot study identified preliminary patient centered themes and demonstrated the feasibility of administering hypospadias patient reported outcome measures in clinical practice.

Vital signs monitoring on general wards: clinical staff perceptions of current practices and the planned introduction of continuous monitoring technology.

OBJECTIVE: Early detection of patient deterioration and prevention of adverse events are key challenges to patient safety. This study investigated clinical staff perceptions of current monitoring practices and the planned introduction of continuous monitoring devices on general wards. DESIGN: Multi-method study comprising structured surveys, in-depth interviews and device trial with log book feedback. SETTING: Two general wards in a large urban teaching hospital in Sydney, Australia. PARTICIPANTS: Respiratory and neurosurgery nursing staff and two doctors. RESULTS: Nurses were confident about their abilities to identify patients at risk of deterioration, using a combination of vital signs and visual assessment. There were concerns about the accuracy of current vital signs monitoring equipment and frequency of intermittent observation. Both the nurses and the doctors were enthusiastic about the prospect of continuous monitoring and perceived it would allow earlier identification of patient deterioration; provide reassurance to patients; and support interdisciplinary communication. There were also reservations about continuous monitoring, including potential decrease in bedside nurse-patient interactions; increase in inappropriate escalations of patient care; and discomfort to patients. CONCLUSIONS: While continuous monitoring devices were seen as a potentially positive tool to support the identification of patient deterioration, drawbacks, such as the potential for reduced patient contact, revealed key areas that will require close surveillance following the implementation of devices. Training and improved interdisciplinary communication were identified as key requisites for successful implementation.

Usability and Acceptance of the Librarian Infobutton Tailoring Environment: An Open Access Online Knowledge Capture, Management, and Configuration Tool for OpenInfobutton.

BACKGROUND: The Librarian Infobutton Tailoring Environment (LITE) is a Web-based knowledge capture, management, and configuration tool with which users can build profiles used by OpenInfobutton, an open source infobutton manager, to provide electronic health record users with context-relevant links to online knowledge resources. OBJECTIVE: We conducted a multipart evaluation study to explore users' attitudes and acceptance of LITE and to guide future development. METHODS: The evaluation consisted of an initial online survey to all LITE users, followed by an observational study of a subset of users in which evaluators' sessions were recorded while they conducted assigned tasks. The observational study was followed by administration of a modified System Usability Scale (SUS) survey. RESULTS: Fourteen users responded to the survey and indicated good acceptance of LITE with feedback that was mostly positive. Six users participated in the observational study, demonstrating average task completion time of less than 6 minutes and an average SUS score of 72, which is considered good compared with other SUS scores. CONCLUSIONS: LITE can be used to fulfill its designated tasks quickly and successfully. Evaluators proposed suggestions for improvements in LITE functionality and user interface.

HealthCheck: A method for evaluating persuasive mobile health applications.

Objectives: This paper introduces HealthCheck, a novel evaluation method for persuasive mobile health applications, aiming to fill the critical gap in quick and effective evaluation tools for this domain. Methods: Following Design Science Research, HealthCheck was developed through problem identification, solution design, implementation, evaluation, and iterative refinement. The implementation involved testing with seven experts to assess its applicability and effectiveness. Results: Feedback from the evaluators indicated that while a few heuristics in HealthCheck were considered irrelevant by some, the majority found the heuristics to be both pertinent and beneficial, especially within the caregiving context. This feedback highlights the practical value of HealthCheck and its potential to offer meaningful insights into improving the usability of persuasive eHealth applications. Conclusion: The study shows HealthCheck effectively evaluates persuasive mobile health applications, offering actionable insights to enhance usability. This validates the relevance and robustness of HealthCheck's heuristics, advancing information systems and human-computer interaction research.

Representation of Social Determinants of Health terminology in medical subject headings: impact of added terms.

OBJECTIVES: To enhance and evaluate the quality of PubMed search results for Social Determinants of Health (SDoH) through the addition of new SDoH terms to Medical Subject Headings (MeSH). MATERIALS AND METHODS: High priority SDoH terms and definitions were collated from authoritative sources, curated based on publication frequencies, and refined by subject matter experts. Descriptive analyses were used to investigate how PubMed search details and best match results were affected by the addition of SDoH concepts to MeSH. Three information retrieval metrics (Precision, Recall, and F measure) were used to quantitatively assess the accuracy of PubMed search results. Pre- and post-update documents were clustered into topic areas using a Natural Language Processing pipeline, and SDoH relevancy assessed. RESULTS: Addition of 35 SDoH terms to MeSH resulted in more accurate algorithmic translations of search terms and more reliable best match results. The Precision, Recall, and F measures of post-update results were significantly higher than those of pre-update results. The percentage of retrieved publications belonging to SDoH clusters was significantly greater in the post- than pre-update searches. DISCUSSION: This evaluation confirms that inclusion of new SDoH terms in MeSH can lead to qualitative and quantitative enhancements in PubMed search retrievals. It demonstrates the methodology for and impact of suggesting new terms for MeSH indexing. It provides a foundation for future efforts across behavioral and social science research (BSSR) domains. CONCLUSION: Improving the representation of BSSR terminology in MeSH can improve PubMed search results, thereby enhancing the ability of investigators and clinicians to build and utilize a cumulative BSSR knowledge base.

Engaging interested parties to optimize wildfire smoke communication in Canada: challenges with initiating change.

Background: In February 2022, an online Wildfire Smoke Communication Workshop series identified priorities and strategies to improve wildfire smoke communication in Canada. We evaluated the engagement methods, the workshop series and workshop summary report, to determine if participants/organizations initiated changes identified in the workshop to optimize wildfire smoke communication plans. Methods: Three evaluation surveys were developed using the RE-AIM (Reach, Effectiveness, Adoption, Implementation, and Maintenance) framework dimensions and PRISM (Practical, Robust, Implementation, and Sustainability Model) contextual domains to measure the engagement impact. Surveys 1, 2, and 3 were disseminated to workshop participants between February 2022 (post-workshop series), May 2022 (pre-wildfire season), and September 2022 (post-wildfire season). Likert survey responses were analyzed descriptively using means and standard deviations. Open-ended written responses were analyzed using deductive reasoning and response proportions. Results: Of 69 workshop participants, 28, 19, and 13 responded to surveys 1, 2, and 3, respectively. Workshop participation helped survey 1 respondents consider optimizing wildfire smoke communication (M = 3.93, SD = 0.88). Workshop participation and the summary report helped survey 2 respondents consider new actions to optimize wildfire smoke communication (M = 3.84, SD = 0.74). The most intended action in survey 2 (68%, n = 13) and the most common action taken in survey 3 (62%, n = 8) was to simplify message content. The primary limitation to optimization was capacity. Conclusion: The engagement methods, particularly the summary report, were beneficial for organizations to take action to optimize wildfire smoke communication in Canada. Future engagement methods should examine persisting system-level issues and capacity limitations as they undermine the ability to optimize wildfire smoke communication in Canada.

Orofacial Myofunctional Aspects of Nursing Infants and Preschoolers.

Introduction Clinical assessment in orofacial motricity is required for the speech therapist to diagnose and treat disorders involving the stomatognathic system. Validated tools can help establish a prognosis and outline intervention methods connected to human development. Objective The goal of the present study was to examine the domains of the oromyofunctional assessment of nursing infants and preschoolers according to sex and age group, as well as the application of the MMBGR Protocol - Nursing Infants and Preschool Children. Methods A quantitative technique was used to conduct an analytical and cross-sectional investigation. The present study included a total of 214 healthy breastfeeding infants and preschoolers of both sexes. The Mann-Whitney test was used to compare the medians. The Spearman correlation of each test domain was determined. R Core Team 2021 (R Foundation, Vienna, Austria) was used, and the significance threshold was set at 5%. Results In intraoral and extraoral examinations, there was a difference between sexes for tongue scores in nursing infants (d = - 0.428; p = 0.045), worse in males. When the orofacial functions were considered in nursing infants, there were differences between the sexes for the liquid/solid/semisolid deglutition scores (d = 0.479; p = 0.031), with females performing worse. There were sex differences in solid/semisolid deglutition (d = - 0.335; p = 0.043), and speech in preschoolers (d = - 0.478; p = 0.034), including the production of phones/phonemes (d = - 0.599; p = 0.007), which were always worse in males. Conclusion The research revealed sex disparities and related the domains of oromyofunctional assessment, according to scores, of the domains of myofunctional assessment, as recorded in a standardized oromyofunctional assessment protocol by age group.

Method validation for the quantification of fluconazole and its organic impurities in raw material using high-performance liquid chromatography / Validación de la metodología para cuantificar el fluconazol y sus impurezas orgánicas en materia prima por cromatografía líquida de alta resolución

Introduction. The real laboratory conditions of each country, including climate, can affect the method's efficiency in analyzing a pharmacological substance. Thus, it is necessary to validate the process according to the corresponding guidelines and optimize it to ensure success and confidence in the results. Objective. The objective was to validate a methodology for fluconazole and its organic impurities quantification in raw material using high-performance liquid chromatography, with a diode array detector, under tropical climate conditions, and complying with all regulatory requirements. Materials and methods. We performed pre-validation tests of the method consisting of system adequacy, filters study, quantification limit, absence of systematic error, forced degradation studies, and solutions stability. In addition, we validated the specificity, linearity, accuracy, precision, and robustness of the system. Results. Separation of the degradation products from the analyte peaks allowed the achievement of the method's spectral purity. The solution's stability was not affected during the evaluated time (24 hours) at room temperature and under refrigeration. Linearity resulted in correlation coefficients greater than or equal to 0.999 for the evaluation and greater than or equal to 0.997 for impurities. We obtained a fluconazole recovery varying from 98 to 102% with an accuracy between 80 to 120% for impurities detection. The repeatability and reproducibility factor did not exceed a relative standard deviation of 2.0% for the evaluation and of 5.0% for the impurities, demonstrating the adequate robustness of the method. In addition, a short analysis execution time allowed the quick determination of the raw material quality. Conclusion. We demonstrated that the fluconazole quantification method validated by high-performance liquid chromatography is sufficiently selective, precise, exact, linear, and robust to generate accurate analytical results under real conditions, including the tropical climate of Colombia.

Introducción. La eficiencia de una metodología para analizar una sustancia farmacológica puede verse afectada por las condiciones reales del laboratorio de cada país, incluyendo el clima. Por esta razón, se requiere validar el método con las pautas recomendadas para ello y optimizar el proceso, para asegurar el éxito y la confianza en los resultados. Objetivo. Validar una metodología para la cuantificación simultánea del fluconazol (materia prima) y sus impurezas orgánicas mediante cromatografía líquida de alta resolución con detector de arreglo de diodos en condiciones de clima tropical y con todos los requisitos normativos. Materiales y métodos. Se hicieron pruebas previas a la validación del método: idoneidad del sistema, estudio de filtros, límite de cuantificación, ausencia del error sistemático, estudios de degradación forzada y estabilidad de las soluciones. Además, se validaron: la especificidad, la linealidad, la exactitud, la precisión y la robustez. Resultados. La pureza espectral del método se logró al obtener la separación de los productos de degradación de los picos de los analitos. La estabilidad de las soluciones no se vio afectada, en la frecuencia evaluada de 24 horas, a temperatura ambiente y de refrigeración. Se obtuvo una linealidad con coeficientes de correlación mayores o iguales a 0,999 para la valoración y mayores o iguales a 0,997 para las impurezas. La recuperación estuvo en el rango de 98 a 102,0 % de fluconazol, con una exactitud entre el 80 y el 120 % para las impurezas. El factor de repetibilidad y reproducibilidad no superó la desviación estándar relativa del 2,0 % para la valoración y, la del 5,0 %, para las impurezas, lo cual mostró una solidez adecuada del método. Además, se obtuvo un tiempo corto de ejecución del análisis, lo que permitió la rápida determinación de la calidad de la materia prima. Conclusión. Se demostró que el método de cuantificación de fluconazol, validado por cromatografía líquida de alta resolución con detector de arreglo de diodos, es lo suficientemente selectivo, preciso, exacto, lineal y robusto; además, es capaz de generar resultados analíticos veraces en condiciones de uso reales, incluyendo el clima tropical de Colombia.

A comparison of prospective observations and chart audits for measuring quality of care of musculoskeletal injuries in the emergency department.

BACKGROUND: Accurate and efficient data collection is crucial for effective evaluation of quality of care. The objective of this study is to compare two methods of data collection used to score quality indicators for musculoskeletal injury management in Emergency departments: prospective observation, and chart audit. METHODS: An analysis was undertaken of data collected from 633 patients who presented with a musculoskeletal injury to eight emergency departments in Queensland, Australia in 2016-17. Twenty-two quality indicators were scored using both prospective observation and chart audit data for each occasion of service. Quality indicators were included if they were originally published with both collection methods. Analyses were performed to compare firstly, the quality indicator denominators, and secondly, the quality indicator trigger rates, scored using each collection method. Chi Square statistics were used to identify significant differences. RESULTS: Prospectively collected data scored quality indicator denominators significantly (p value<0.05) more often than chart audit data for five (22.7 %) of the 22 quality indicators. The remaining 17 quality indicators (77.3 %) showed no statistical differences. When comparing quality indicator trigger rates, 16 (72.7 %) had significantly different results between methods with 12 (54.5 %) scoring higher using prospective data and four (18.2 %) with chart audit data. The remaining six quality indicators (27.3 %) in this comparison showed no significant difference between chart and prospective data. CONCLUSION: Quality indicators including aspects of care associated with patient safety, and those relying on clinician written orders or forms were adequately scored using either prospective observation or chart audit data. Whereas quality indicators relying on time-sensitive information, elements of a social history, general physical exams and patient education and advice scored higher using prospective observation data collection.

Monthly Identification of High Frequency Emergency Presenters to Improve Care Delivery and Evaluation: A Unique Methodological Approach.

Background: Frequent presenters to emergency departments (EDs) pose many challenges around care delivery and health service management. The aim of this study was to investigate the presentation patterns of people with 5 or more ED visits in any calendar month (5+ frequent presenter [FP5+]) to develop a useful methodological framework on which the real impact of interventions may be assessed. Methods: This study is a retrospective analysis of de-identified frequent ED presentation data using segmented regression analysis of an interrupted time series (ITS). Results: A total of 82 FP5+ to this single ED were identified in a year. Of these presenters, 77% had 10 or more presentations in a year. The total FP5+ presentations in the 12 months preceding and after each participant's ≥5 presentations in 1 month (the trigger month for inclusion in the study) accounted for 1,064 and 1,606 visits, respectively. ITS analysis of frequent ED presentations did not show a significant level change or trend change during the data collection period. Monthly review of people who frequently present to a single ED showed that presentations typically occurred in bouts that may span calendar years. Presentation bouts then typically slow, potentially distorting evaluation of the effects of interventions. Conclusion: Rolling monthly examination of presentation data may facilitate timely case review and care delivery, as well as provide a holistic picture of the impacts of interventions targeting patient care needs. This unique analysis demands a reconsideration of the typical before-and-after analysis of interventions for this vulnerable and high-cost group of patients.

Impact of 'DEALTS2' education intervention on trainer dementia knowledge and confidence to utilise innovative training approaches: A national pre-test - post-test survey.

BACKGROUND: Gaps in acute care staff knowledge, skills, and attitudes towards dementia exist. Innovative training approaches that improve the delivery of care for people with dementia are needed. We were commissioned by Health Education England to develop and evaluate a new dementia education intervention 'Dementia Education And Learning Through Simulation 2' (DEALTS2), a simulation toolkit to enhance delivery of dementia training nationally across England. OBJECTIVES: Evaluate differences in trainer dementia knowledge scores pre and post training, satisfaction with DEALTS2 Train-The-Trainer (TTT) workshops and simulation toolkit, confidence to use training approaches, and spread of implementation. DESIGN: A questionnaire survey using a pre-test - post-test design with measures completed: before (pre-test); after (T1); and 12 months post training (T2). SETTING: Twelve one-day DEALTS2 TTT workshops delivered across England in 2017. PARTICIPANTS: National Health Service Trust staff employed in dementia training roles (n = 199 trainers). METHODS: Trainers attended DEALTS2 TTT workshops and received the simulation toolkit. Data were collected between 2017 and 2018 using a questionnaire capturing differences in dementia knowledge scores, Likert scales and closed-ended questions measured satisfaction, confidence and implementation. Data were analysed using quantitative methods. RESULTS: Response rate was 92% (n = 183) at pre-test/T1 and 26% (n = 51) at T2. Trainer dementia knowledge scores increased from pre-test to T1 (p < 0.001) and were retained after 12 months in 5 of the 6 areas measured (pre-test to T2, p < 0.002); largest gains in 'humanised approaches to dementia care'. 96% (n = 176/183) were satisfied with DEALTS2 TTT workshops and simulation toolkit; 66.7% (n = 34/51) felt confident to deliver dementia training informed by DEALTS2. Adherence rates were good with 45% (n = 23/51) using the innovative training approaches within twelve months. CONCLUSIONS: The results show DEALTS2 effectively increased trainer dementia knowledge and confidence to utilise innovative dementia training approaches. Implementation of DEALTS2 varied across organisations, therefore further research should explore factors determining successful implementation.

Scholarly Evaluation of Curricula and Educational Programs: Using a Systematic Approach to Produce Publishable Scholarship.

The goal of designing innovative curricula and programs in medical education is to create interventions that will change the attitudes, knowledge, skills, and behaviors of learners in order to prepare them to contribute to the health of patients and communities they serve. Systematic evaluation allows curriculum/program developers to optimize their curriculum/program and ensure that the goals of the curriculum/program are met. While curriculum/program evaluation is distinct from educational research, when approached in a rigorous manner, curriculum/program evaluation can be published as educational scholarship. The goal of our paper is to equip readers with tools to apply a scholarly approach to curriculum/program evaluation to produce findings that will advance the field. We start by defining curricula, educational programs, learner assessment, and evaluation. We then briefly describe examples of best practice models for curriculum/program development that incorporate the critical step of planning for evaluation. Building on published work, we distinguish the use of best practice models and conceptual frameworks to inform curriculum/program development and evaluation. More specifically, we outline steps to plan an evaluation that demonstrates WHAT was effective, describes HOW the curriculum/program contributed to the outcomes achieved, and points to WHY the curriculum/program led to the outcomes observed. We conclude with key considerations for publishing findings of an evaluation, including what to include in each section of a manuscript.

Evaluating a Novel Multifactorial Falls Prevention Activity Programme for Community-Dwelling Older People After Stroke: A Mixed-Method Feasibility Study.

PURPOSE: The overall purpose of this study was to explore participants' and physiotherapists' experiences regarding the acceptability, implementation, and practicality of a novel group-based multifactorial falls prevention activity programme for community-dwelling older people after stroke. Specifically, the purpose was to explore if and how participating could impact on the participants' health-related quality of life (HRQoL) in terms of their daily lived experience regarding physical, mental, emotional and social well-being. A secondary purpose was to explore whether participating in the programme could positively influence participants' balance, strength, falls efficacy, mobility and motor impairment of the trunk. MATERIALS AND METHODS: This was an exploratory mixed-method Phase I feasibility study. A convenience sample of five older community-dwelling people after stroke participated in a novel eight-week multifactorial activity programme which included falls education, a mix of individually tailored and group-based strength and balance exercises, exploring limits of stability and safe landing techniques and a social element. Qualitative data from post-intervention interview transcripts with the participants and the physiotherapists who delivered the programme were thematically analysed using both deductive and inductive approaches to explore the participants' and therapists' experiences with the programme. Quantitative outcomes included balance, strength, falls efficacy, mobility and motor impairment of the trunk. RESULTS: The programme was deemed feasible in terms of acceptability, implementation and practicality by the participants as well as the physiotherapists delivering the programme. The overarching theme regarding HRQoL identified that participating in the programme was perceived to empower the participants living with stroke and positively influenced their daily physical, mental, emotional and social well-being. Participant outcomes showed a change in the direction of improvements in balance, strength, mobility, motor impairment of the trunk and reduced concerns about falling. Subjectively, participants only reported perceived improvements in balance and strength. CONCLUSION: Running a novel multifactorial falls prevention activity programme for older community-dwelling people after stroke was feasible. Participating in the programme helped participants to perceive improved balance, strength and empower them to make meaningful changes, improving their daily lived experiences. A future fully powered study could build on these results to investigate physical improvements, prevention of falls and improvements to domains of HRQoOL.

The impacts of waterpipe (hookah, narghile) -use on quality of life: a special report.

Introduction While waterpipe-use induced deficiency and incapacity have already been extensively explored, its impacts on quality of life (QoL) was rarely raised. The main aim of this Special Report was to briefly highlight the key results of the studies raising the effects of waterpipe-use on smokers' QoL. Areas covered SCOPUS was consulted on July 3rd, 2020, using the combination of the following two medical subject headings (MeSH) words: 'Tobacco, Waterpipe' AND 'Quality of Life'. Only English original articles were retained. Merely five studies raised the effects of waterpipe-use on QoL. Studies were performed in Middle East (n = 2), USA (n = 1), Tunisia (n = 1) and 'Egypt, Morocco, Oman and Jordan' (n = 1). Conflicting results were reported related to the effects of waterpipe-use on QoL. While four studies concluded that waterpipe-smokers' have a worse QoL, one study concluded that waterpipe-smokers have a normal QoL. Expert opinion This Special Report is a call to encourage future research to identify the real effects of waterpipe-use on QoL.