Evaluation of intervention reporting in published emergency medicine clinical trials.

OBJECTIVE: The extent of intervention reporting in emergency medicine journals remains unclear. The primary objective is to assess overall completion of the Template for Intervention Description and Replication (TIDieR) checklist described in emergency medicine randomized clinical trials (RCTs). The secondary outcomes were to (1) compare reporting before and after TIDieR publication; (2) evaluate factors associated with intervention reporting. METHODS: Our cross-sectional study used Google Scholar's metrics to identify seven emergency medicine journals; of which, we randomly sampled 300 articles. Using two PubMed searches, we extracted 150 RCTs before and after publications of TIDieR. Two investigators independently extracted data. The primary analysis to measure overall completion included descriptive statistics for each checklist item. Our secondary analysis used an interrupted time series analysis and generalized estimating equations to determine the effect of TIDieR publication on intervention reporting. RESULTS: Our initial search yielded 635 articles; from which, we randomly sampled 300 articles. We excluded 67 articles, leaving 233 for analysis. The mean number of TIDieR items reported was 5.4 (standard deviation = 1.18). Of the 233 trials, 42.9% provided information about materials, 67% provided intervention procedures, and 99.1% provided intervention delivery. The least reported items were intervention modifications (2.6%), intervention adherence assessment methods (3.4%), and intervention adherence assessment outcomes (2.2%). CONCLUSIONS: The completeness of intervention reporting is suboptimal in emergency medicine journals, necessitating improvement. The current state of adherence could be improved through the combined efforts of journal editors, major editorial organizations, and authors.

Maintaining Intervention Fidelity When Using Technology Delivery Across Studies.

The current standard in healthcare research is to maintain scientific fidelity of any intervention being tested. Fidelity is defined as the consistent delivery of interventions that ensures that all participants are provided the same information, guidance, and/or materials. Notably, the methods for ensuring fidelity of intervention delivery must also be consistent. This article describes our Intervention and Technology Delivery Fidelity Checklists used to ensure consistency. These checklists were completed by trained nurse observers who rated the intervention implementation and the technology delivery. Across our clinical trials and pilot studies, the fidelity scores were tabulated and compared. Intervention information and materials were delivered by a variety of devices including telehealth monitors, videophones, and/or iPads. Each of the devices allows audiovisual connections between health professionals from their offices and patients and participants in their homes. Our checklists guide the monitoring of fidelity of technology delivery. Overall checklist ratings across our studies demonstrate consistent intervention, implementation, and technology delivery approaches. Uniquely, the fidelity checklist verifies the interventionist's correct use of the technology devices to ensure consistent audiovisual delivery. Checklist methods to ensure intervention fidelity and technology delivery are essential research procedures, which can be adapted for use by researchers across multiple disciplines.

The Effect of an Electronic Dynamic Cognitive Aid Versus a Static Cognitive Aid on the Management of a Simulated Crisis: A Randomized Controlled Trial.

The aim of this study was to assess the effect of a dynamic electronic cognitive aid with embedded clinical decision support (dCA) versus a static cognitive aid (sCA) tool. Anesthesia residents in clinical anesthesia years 2 and 3 were recruited to participate. Each subject was randomized to one of two groups and performed an identical simulated clinical scenario. The primary outcome was task checklist performance with a secondary outcome of performance using the Anesthesia Non-technical skills (ANTS) scoring system. 34 residents were recruited to participate in the study. 19 residents were randomized to the sCA group and 15 to the dCA group. Overall inter-rater agreement for total checklist, malignant hyperthermia, hyperkalemia and ventricular fibrillation was 98.9%, 97.8%, 99.5% and 99.5% respectively with similar Kappa coefficient. Inter-rater agreement for ANTS partial ratings, however, was only 53.5% with a similar Kappa of 0.15. Mean performance was statistically higher in the dCA group versus the sCA group for total check list performance (15.70 ± 1.93 vs 12.95 ± 2.16, p < 0.0001). The difference in performance between dCA and sCA is most notable in dose-dependent related checklist items (4.60 ± 1.3 vs 1.89 ± 1.23, p < 0.0001), while the performance score for dose-independent checklist items was similar between the two groups (p = 0.8908). ANTS ratings did not differ between groups. In conclusion, we evaluated the use of a sCA versus a dCA with embedded decision support in a simulated environment. The dCA group was found to perform more checklist items correctly.Clinical Trial Registration: Clinicaltrials.gov study #: NCT02440607.

Evaluating a smartphone application to improve child passenger safety and fire safety knowledge and behaviour.

BACKGROUND: Although proven measures for reducing injury due to motor vehicle collision and residential fires exist, the number of families properly and consistently using child passenger restraints and smoke alarms remains low. This paper describes the design of the Safety In Seconds (SIS) 2.0 study, which aims to evaluate the impact of a smartphone app on parents' use of child restraints and smoke alarms. METHODS: SIS is a multisite randomised controlled trial. Participants are parents of children aged 4-7 years who are visiting the Pediatric Emergency Department or Pediatric Trauma Service. Parents are randomised to receive tailored education about child passenger safety or about fire safety via the SIS smartphone app. A baseline and two follow-up surveys at 3 months and 6 months are conducted. Primary outcomes are: (1) having the correct child restraint for the child's age and size; (2) restraining the child in the back seat of the car; (3) buckling the child up for every ride; (4) having the restraint inspected by a child passenger safety technician; (5) having a working smoke alarm on every level of the home; (6) having hard-wired or lithium battery smoke alarms; (7) having and (8) practising a fire escape plan. DISCUSSION: Finding ways to communicate with parents about child passenger and fire safety continues to be a research priority. This study will contribute to the evidence about how to promote benefits of proper and consistent child restraint and smoke alarm use. TRIAL REGISTRATION NUMBER: NCT02345941; Pre-results.

Development and validation of the Efficacy Safety Score (ESS), a novel tool for postoperative patient management.

BACKGROUND: Several reports have shown that postoperative monitoring of general safety and quality issues, including pain treatment, after discharge from recovery is often non-systematic and inadequate. We suggest a new score with assessment of key recovery parameters, as a supportive tool for postoperative care and a call-out algorithm for need of extra help. The aim of this investigation was to validate the score. METHODS: After suggesting a prototype score from a pilot study in 182 postoperative patients, we performed a Delphi process by using international experts to create consensus on the final score contents and called the revised tool the Efficacy Safety Score (ESS). Then, we performed a prospective observational study with the ESS throughout the first 24 h postoperatively in 207 surgical in-patients. We compared ESS with Modified Early Warning Systems (MEWS), and postoperative journal information. We subsequently validated ESS by addressing recognized quality criteria for measurement of health status questionnaires. RESULTS: A call-out value of ESS ≥10 correlated with MEWS > 0 values and journal information about postoperative concerns with a sensitivity of 94% and 92%, respectively. All serious safety issues were identified with the ESS ≥ 10, and a higher number of quality issues were identified than with routine care or MEWS. We obtained positive ratings for six out of seven tested criteria of questionnaire quality; one criterion had an indeterminate rating. CONCLUSION: ESS fulfils suggested criteria for score quality validation and reflects the patient's postoperative status adequately and with high sensitivity. Further clinical trials are warranted to evaluate the usefulness of ESS as a simple tool for assessment of the postoperative safety and quality of patients.

Checklist for psychosocial risk evaluation: a prospective evaluation study.

BACKGROUND: Although psychosocial risk evaluation (PRE) is legally required, implementation is often suboptimal. Occupational health specialists face various challenges in implementing a systematic PRE process. AIMS: To evaluate if a checklist strengthens the competencies of occupational physicians in PRE design and implementation. METHODS: A prospective pilot survey with a baseline and follow-up assessment at 12 months was conducted with occupational physicians who applied the checklist. Three outcome criteria were assessed: (i) physicians' general competence in PRE, (ii) physicians' self-reported skills in PRE and (iii) feasibility of the checklist. RESULTS: Twenty-four occupational physicians participated. Checklist users reported increased competence in PRE at follow-up, increasing in competence twice as much as physicians who did not use the checklist during the study period although this was not statistically significant. With regard to specific skills over time, checklist users reported increased knowledge of the PRE procedure (P < 0.05), as well as increased strategic planning for PRE (P < 0.01). All feasibility criteria were rated positively. CONCLUSIONS: Our checklist may help occupational health clinicians to overcome the barriers that are responsible for the gap between official recommendations and occupational health and safety practice in PRE. However, our results are based on a preliminary study with a limited sample size.

Development and evaluation of two instruments for assessing screening, brief intervention, and referral to treatment (SBIRT) competency.

BACKGROUND: Screening, brief intervention, and referral to treatment (SBIRT) is shown to be effective in identifying, intervening with, and making appropriate referrals for patients with unhealthy alcohol use. SBIRT training consists of knowledge-based and skill-based components and has increased the use of screening and intervention skills in clinical settings. This article reports on the development and evaluation of 2 SBIRT proficiency checklists for use across institutions to assess SBIRT skills in both simulated and clinical encounters. METHODS: A national panel of 16 experts identified 137 discrete SBIRT skills items for the checklists. From this final list, 2 proficiency checklists were derived: the SBIRT Proficiency Checklist (SPC), composed of 22 questions for videotaped interviews, and the Clinical SBIRT Proficiency Checklist (CSPC), composed of 13 questions for direct clinical observation. An evaluation was conducted to test the reliability of the SPC and to assess the utility of the CSPC. RESULTS: Two checklists for assessing SBIRT proficiency were developed by a collaborative workgroup. Fleiss' kappa analyses indicated moderate agreement. In addition, faculty recorded satisfaction with the CSPC for assessing residents on their SBIRT performance during clinical encounters. CONCLUSIONS: The SPC and the CSPC are practical tools for assessing competence with SBIRT and are easily integrated as standard instruments in a wide range of training settings. Future advancements to the checklists and their evaluation include modification of the SPC rating scale to be consistent with the CSPC, developing a training program for using the checklists, and further testing to improve interrater reliability.

Sudden unexpected death in epilepsy: measures to reduce risk.

This review looks at the strategies that may help to reduce the risk of sudden unexpected death in epilepsy beyond that of trying to achieve seizure cessation, which is not possible for up to 30% of patients with epilepsy. These strategies include seizure safety checklists, mobile phone technology, telehealth and various devices currently available or in development. We highlight interventions where there is evidence of benefit, and draw attention for the need both to involve patients with epilepsy in risk reduction and to improve communication with those at risk.

Electronically Mediated Time-out Initiative to Reduce the Incidence of Wrong Surgery: An Interventional Observational Study.

BACKGROUND: "Wrong surgery" is defined as wrong site, wrong operation, or wrong patient, with estimated incidence up to 1 per 5,000 cases. Responding to national attention on wrong surgery, our objective was to create a care redesign intervention to minimize the rate of wrong surgery. METHODS: The authors created an electronic system using existing intraoperative electronic documentation to present a time-out checklist on large in-room displays. Time-out was dynamically interposed as a forced-function documentation step between "patient-in-operating room" and "incision." Time to complete documentation was obtained from audit logs. The authors measured the postimplementation wrong surgery rate and used Bayesian methods to compare the pre- and postimplementation rates at our institution. Previous probabilities were selected using wrong surgery rate estimates from the observed performance reported in the literature to generate previous probabilities (4.24 wrong surgeries per 100,000 cases). RESULTS: No documentation times exceeded 5 min; 97% of documentation tasks were completed within 2 min. The authors performed 243,939 operations over 5 yr using the system, with zero wrong surgeries, compared with 253,838 operations over 6 yr with two wrong surgeries before implementation. Bayesian analysis suggests an 84% probability that the postimplementation wrong rate is lower than baseline. However, given the rarity of wrong surgery in our sample, there is substantial uncertainty. The total system-development cost was $34,000, roughly half the published cost of one weighted median settlement for wrong surgery. CONCLUSION: Implementation of a forced-completion electronically mediated time-out process before incision is feasible, but it is unclear whether true performance improvements occur.

The Effect of an Electronic Checklist on Critical Care Provider Workload, Errors, and Performance.

PURPOSE: The strategy used to improve effective checklist use in intensive care unit (ICU) setting is essential for checklist success. This study aimed to test the hypothesis that an electronic checklist could reduce ICU provider workload, errors, and time to checklist completion, as compared to a paper checklist. METHODS: This was a simulation-based study conducted at an academic tertiary hospital. All participants completed checklists for 6 ICU patients: 3 using an electronic checklist and 3 using an identical paper checklist. In both scenarios, participants had full access to the existing electronic medical record system. The outcomes measured were workload (defined using the National Aeronautics and Space Association task load index [NASA-TLX]), the number of checklist errors, and time to checklist completion. Two independent clinician reviewers, blinded to participant results, served as the reference standard for checklist error calculation. RESULTS: Twenty-one ICU providers participated in this study. This resulted in the generation of 63 simulated electronic checklists and 63 simulated paper checklists. The median NASA-TLX score was 39 for the electronic checklist and 50 for the paper checklist (P = .005). The median number of checklist errors for the electronic checklist was 5, while the median number of checklist errors for the paper checklist was 8 (P = .003). The time to checklist completion was not significantly different between the 2 checklist formats (P = .76). CONCLUSION: The electronic checklist significantly reduced provider workload and errors without any measurable difference in the amount of time required for checklist completion. This demonstrates that electronic checklists are feasible and desirable in the ICU setting.

Developing a mobile electronic D2B checklist for treatment of ST elevation myocardial infarction patients who need a primary coronary intervention.

ST elevation myocardial infarction (STEMI), one main type of acute myocardial infarction with high mortality, requires percutaneous coronary intervention (PCI) with balloon inflation. Current guidelines recommend a door-to-balloon (D2B) interval (i.e., starts with the patient's arrival in the emergency department and ends when PCI with a catheter guidewire and balloon inflation crosses the culprit lesion) of no more than 90 min. However, promptly implementing PCI requires coordinating various medical teams. Checklists can be used to ensure consistency and operating sequences when executing complex tasks in a clinical routine. Developing an effective D2B checklist would enhance the care of STEMI patients who need PCI. Mobile information and communication technologies have the potential to greatly improve communication, facilitate access to information, and eliminate duplicated documentation without the limitations of space and time. In a research project by the Chi Mei Medical Center, "Developing a Mobile Electronic D2B Checklist for Managing the Treatment of STEMI Patients Who Need Primary Coronary Intervention," a prototype version of a mobile checklist was developed. This study describes the research project and the four phases of the system development life cycle, comprising system planning and selection, analysis, design, and implementation and operation. Face-to-face interviews with 16 potential users were conducted and revealed highly positive user perception and use intention toward the prototype. Discussion and directions for future research are also presented.

Adaptation of a communication interaction behavior instrument for use in mechanically ventilated, nonvocal older adults.

BACKGROUND: Valid and reliable instruments are needed to measure communication interaction behaviors between nurses and mechanically ventilated intensive care unit patients who are without oral speech. OBJECTIVES: The aim of this study was to refine and evaluate preliminary validity and reliability of a Communication Interaction Behavior Instrument (CIBI) adapted for use with mechanically ventilated, nonvocal patients in the intensive care unit. METHODS: Raters observed nurse-patient communication interactions using a checklist of nurse and patient behaviors, categorized as positive and negative behaviors. Three-minute video-recorded observations of five mechanically ventilated adults (<60 years old) in the intensive care unit and their nurses were used to establish preliminary interrater reliability and confirm appropriateness of definitions (four observations per dyad, n = 20). On the basis of expert input and reliability results, the behaviors and item definitions on the CIBI were revised. The revised tool was then tested in a larger sample of 38 mechanically ventilated intensive care patients (> 60 years old) and their nurses (four observations per dyad, n = 152) to determine interrater reliability. RESULTS: For preliminary testing, percent agreement for individual items ranged from 60% to 100% for nurse behaviors and 20% to 100% for patient behaviors across the five pilot cases. On the basis of these results, 11 definitions were modified and four items were dropped. Using the revised 29-item instrument, percent agreement improved for nurse behaviors (73%-100%) and patient behaviors (68%-100%). Kappa coefficients ranged from 0.13 to 1.00, with lower coefficients for patient behaviors. CONCLUSION: Preliminary results suggest that the revised CIBI has good face validity and shows good interrater reliability for many of the behaviors, but further refinement is needed. The use of dual raters with adjudication of discrepancies is the recommended method of administration for the revised CIBI.

Construction and validation of a checklist for the management of totally implanted catheters in children.

OBJECTIVES: to construct and validate the content of a checklist for the management of totally implanted catheters in hospitalized children and adolescents. METHODS: methodological research conducted from October 2021 to December 2022 in two stages: development of the instrument with care guidelines and content validation of the checklist. The instrument, containing 23 items presented in Likert format, was evaluated online by specialists in two rounds. The Content Validity Index was applied, considering indices above 0.8 as valid. RESULTS: the final checklist included four domains and 22 checklist items, validated with a Content Validity Index of 0.98. The overall evaluation of the instrument presented a global score of 9.9. CONCLUSIONS: the validation and application of instruments that standardize procedures, in addition to supporting professionals, promote autonomy and quality of care for children and adolescents using this device.

Impact of emergency preparedness exercise on performance.

The aim of this study was to investigate whether prior participation in preparedness exercises was predictive of better performance on objective measures of response. We conducted a statewide tabletop exercise that focused on a hazardous materials (HAZMAT) scenario and assessed performance using our developed evaluation tool. The evaluative score was analyzed in relation to the number of HAZMAT exercises in the past 3 years, participation in prior CHEMPACK-specific exercise, hospital size, teaching status of the hospital, preparedness training experience, and participants years of experience in preparedness activities. Hospitals that had participated in more exercises in the past 3 years performed significantly better than hospitals that had participated in fewer exercises. No significant differences were found between the performance of hospitals in relation to size, teaching status, preparedness training experience, and participants' years of experience in preparedness activities. Our results suggest that more frequent participation in exercises may result in improved overall response.

Does the Brief Symptom Inventory-18 case rule apply in adult survivors of childhood cancer? Comparison with the Symptom Checklist-90.

OBJECTIVE: Screening for psychological distress is an important tool for improving survivors' access to psychosocial care. The Brief Symptom Inventory-18 (BSI-18) has been widely used to identify psychological distress in cancer survivors, but few studies have reported on its validity. This study evaluated validity of the BSI-18 by comparing it to the Symptom Checklist-90 (SCL-90). METHODS: Concordance of cases identified by the BSI-18 and SCL-90 was examined in a sample of 193 adult survivors of childhood cancer. RESULTS: Receiver operating characteristics analysis showed strong diagnostic utility of the BSI-18 (area under curve = 0.922). However, the standard BSI-18 case-rule demonstrated low sensitivity (45.2%) against the SCL-90. An alternative case-rule showed better results; sensitivity (87.10%), specificity (83.33%). CONCLUSIONS: The BSI-18 is a useful measure for evaluating distress in adult survivors of childhood cancers; however, the standard BSI-18 case-rule has not been validated for this population, and an alternative case rule should be considered.

Initial development and validation of the checklist of risk behaviours for youth (CORBY).

The purpose of this study was to develop and begin to validate a brief self-report measure, the checklist of risk behaviours for youth (CORBY). This 15-item measure was designed to assess, over a 30-day period, multiple youth risk behaviours associated with mental health and well-being concerns. Early research provides initial support for the reliability and validity of the CORBY, but further research is warranted. The present study is the first to describe a brief survey addressing multiple youth risk behaviours, examining both the severity of health risk and number of behaviours in which youth are engaged. Such a measure may have important implications for both research and clinical applications.

Using community-based participatory research to design and initiate a study on immigrant worker health and safety in San Francisco's Chinatown restaurants.

BACKGROUND: Restaurant workers have among the highest rates of work-related illness and injury in the US, but little is known about the working conditions and occupational health status of Chinese immigrant restaurant workers. METHODS: Community-based participatory research (CBPR) was employed to study restaurant working conditions and worker health in San Francisco's Chinatown. A community/academic/health department collaborative was formed and 23 restaurant workers trained on research techniques and worker health and safety. A worker survey instrument and a restaurant observational checklist were collaboratively developed. The checklist was piloted in 71 Chinatown restaurants, and the questionnaire administered to 433 restaurant workers. RESULTS: Restaurant workers, together with other partners, made substantial contributions to construction of the survey and checklist tools and improved their cultural appropriateness. The utility of the checklist tool for restaurant-level data collection was demonstrated. CONCLUSIONS: CBPR holds promise for both studying worker health and safety among immigrant Chinese restaurant workers and developing culturally appropriate research tools. A new observational checklist also has potential for restaurant-level data collection on worker health and safety conditions.