Analysis of Oklahoma amputation trends and identification of risk factors to target areas for limb preservation interventions.

OBJECTIVE: Annual trends of lower extremity amputation due to end-stage chronic disease are on the rise in the United States. These amputations are leading to massive expenses for patients and the medical system. In Oklahoma, we have a high-risk population because access to care is low, the number of uninsured is high, cardiovascular health is poor, and our overall health care performance is ranked 50th in the country. But we know little about Oklahomans and their risk of limb loss. It is, therefore, imperative to look closely at this population to discover contemporary rates, trends, and state-specific risk factors for amputation due to diabetes and/or peripheral arterial disease (PAD). We hypothesize that state-specific groups will be identified as having the highest risk for limb loss and that contemporary trends in amputations are rising. To create implementable solutions to limb preservation, a baseline must be set. METHODS: We conducted a 12-consecutive-year observational study using Oklahoma's hospital discharge data. Discharges among patients 20 years or older with a primary or secondary diagnosis of diabetes and/or PAD were included. Diagnoses and amputation procedures were identified using International Classification of Disease-9 and -10 codes. Amputation rates were calculated per 1000 discharges. Trends in amputation rates were measured by annual percentage changes (APC). Prevalence ratios evaluated the differences in amputation rates across demographic groups. RESULTS: Over 5,000,000 discharges were identified from 2008 to 2019. Twenty-four percent had a diagnosis of diabetes and/or PAD. The overall amputation rate was 12 per 1000 discharges for those with diabetes and/or PAD. Diabetes and/or PAD-related amputation rates increased from 8.1 to 16.2 (APC, 6.0; 95% confidence interval [CI], 4.7-7.3). Most amputations were minor (59.5%), and although minor, increased at a faster rate compared with major amputations (minor amputation APC, 8.1; 95% CI, 6.7-9.6 vs major amputation APC, 3.1; 95% CI, 1.5-4.7); major amputations were notable in that they were significantly increasing. Amputation rates were the highest among males (16.7), American Indians (19.2), uninsured (21.2), non-married patients (12.7), and patients between 45 and 49 years of age (18.8), and calculated prevalence ratios for each were significant (P = .001) when compared within their respective category. CONCLUSIONS: Amputation rates in Oklahoma have nearly doubled in 12 years, with both major and minor amputations significantly increasing. This study describes a worsening trend, underscoring that amputations due to chronic disease is an urgent statewide health care problem. We also present imperative examples of amputation health care disparities. By defining these state-specific areas and populations at risk, we have identified areas to pursue and improve care. These distinctive risk factors will help to frame a statewide limb preservation intervention.

Usability Evaluation Ecological Validity: Is More Always Better?

BACKGROUND: The ecological validity associated with usability testing of health information technologies (HITs) can affect test results and the predictability of real-world performance. It is, therefore, necessary to identify conditions with the greatest effect on validity. METHOD: We conducted a comparative analysis of two usability testing conditions. We tested a HIT designed for anesthesiologists to detect pain signals and compared two fidelity levels of ecological validity. We measured the difference in the number and type of use errors identified between high and low-fidelity experimental conditions. RESULTS: We identified the same error types in both test conditions, although the number of errors varied as a function of the condition. The difference in total error counts was relatively modest and not consistent across levels of severity. CONCLUSIONS: Increasing ecological validity does not invariably increase the ability to detect use errors. Our findings suggest that low-fidelity tests are an efficient way to identify and mitigate usability issues affecting ease of use, effectiveness, and safety. We believe early low-fidelity testing is an efficient but underused way to maximize the value of usability testing.

Tabular, Annotated, Visual, or Trends + Contextual Information? Preferences for Online Laboratory Results Displays.

People are increasingly offered access to their personal health information (e.g., laboratory results, clinical notes, diagnostic imaging results). However, this information is the same as that used by health care providers with clinical expertise and training in medical terminology, which citizens typically do not have. In this study, we examined participants (N = 24) preferences for four different types of displays for online laboratory (lab) results: Tabular, Annotated, Visual, and Trends + Contextual Information. The Friedman test of difference comparing participants' ratings of the four displays was significant, χ2(3)=10.8, P=.013, and the Wilcoxon signed rank pairwise comparison tests revealed that participants rated the visual lab results display significantly more favourably than the traditional display (Z=-2.746, P=.006). These findings indicate that many people prefer lab results displayed using more visual cues and some perceived this format as easier to understand than the other display formats. Given the importance of people accessing, understanding, and using their own health information, it is crucial for displays and systems to provide a better user experience. Displaying data (e.g., lab results) visually is one possible way to improve interpretability of personal health information provided to the public.

Teledermatology: Simulating Hybrid Workflows for Telemedicine Education.

Given the importance of telemedicine in improving healthcare access for underserved patients, professional students need experience using virtual clinical workflows. We developed an educational workshop with (1) readings, (2) a knowledge assessment test, (3) dermatology and teledermatology lectures, (5) a telemedicine simulation with a standardized patient, and (6) a debriefing session. The simulation included a "hybrid" workflow with live videoconferencing and store-and-forward image review. We measured student performance using three American Association of Medical Colleges (AAMC) Telemedicine Competencies for medical education. Ninety-eight medical and physician assistant students completed this workshop between 2021 and 2022, and 80% were entrustable or approaching entrustment in each competency. Some students struggled with data collection and technology use. Our results suggest that this workshop offers a practical and generalizable way to teach about multiple virtual workflows and strengthen students' telemedicine competencies.

Simulating Telemedicine, Medication Reconciliation, and Social Determinants: A Novel Instructional Approach to Health Systems Competencies.

While medication reconciliation is necessary to reduce errors, it is often challenging to gather an accurate history in the clinic. Telemedicine offers a relative advantage over clinic and hospital-based interviews by enabling the clinician to inspect the home environment, review pill bottles, and identify social determinants affecting adherence, such as financial instability. To be effective, however, clinicians must be trained in best-practice interview methods and the proper use of telemedicine. There is very little information in the literature describing the best strategies for teaching students or measuring competencies in telemedicine. Therefore, we created an educational module with a telemedicine simulation and an evaluation rubric. We piloted this module with 48 medical and physician assistant students. Most students could complete a virtual interview and gather a medication history. However, only half identified an over-the-counter medication missing from the list. Most students were either entrustable or approaching entrustment in the six telemedicine competencies measured in this simulation. This simulation is valuable for teaching students about medication reconciliation, using telemedicine to close gaps in access to care, and identifying health-related social needs affecting medication adherence.

Consumers' Needs for Laboratory Results Portals: Questionnaire Study.

BACKGROUND: Over the last decade, there has been an increase in the number of health care consumers (ie, patients, citizens, and laypeople) with access to their laboratory results through portals. However, many portals are not designed with the consumer in mind, which can limit communication effectiveness and consumer empowerment. OBJECTIVE: We aimed to study design facilitators and barriers affecting consumer use of a laboratory results portal. We sought to identify modifiable design attributes to inform future interface specifications and improve patient safety. METHODS: A web-based questionnaire with open- and closed-ended items was distributed to consumers in British Columbia, Canada. Open-ended items with affinity diagramming and closed-ended questions with descriptive statistics were analyzed. RESULTS: Participants (N=30) preferred reviewing their laboratory results through portals rather than waiting to see their provider. However, respondents were critical of the interface design (ie, interface usability, information completeness, and display clarity). Scores suggest there are display issues impacting communication that require urgent attention. CONCLUSIONS: There are modifiable usability, content, and display issues associated with laboratory results portals that, if addressed, could arguably improve communication effectiveness, patient empowerment, and health care safety.

Automation vs. Innovation: Unexplored Strategies to Improve Virtual Care.

While there is a global desire to increase digital health capacity, digital health should transform health services delivery rather than simply automate - or worse - replicate existing practices. Failing to capitalize on this transformative potential misses an opportunity to engage patients and other users to provide a more person-centered experience. However, digital transformation done recklessly can disrupt workflow, alienate users, and jeopardize patient safety, as we have observed with implementation of many digital health tools. This paper uses a telemedicine example to provide insight into how digital health innovation can be a meaningful enabler of health system transformation. Examining different ways to leverage digital health technologies is crucial to best capitalize on their potential.

Differential Perceptions of What Constitutes a Medical Error Associated with Electronic Medical Records.

Perceptions of errors associated with healthcare information technology (HIT) often depend on the context and position of the viewer. HIT vendors posit very different causes of errors than clinicians, implementation teams, or IT staff. Even within the same hospital, members of departments and services often implicate other departments. Organizations may attribute errors to external care partners that refer patients, such as nursing homes or outside clinics. Also, the various clinical roles within an organization (e.g., physicians, nurses, pharmacists) can conceptualize errors and their root causes differently. Overarching all these perceptual factors, the definitions, mechanisms, and incidence of HIT-related errors are remarkably conflictual. There is neither a universal standard for defining or counting these errors. This paper attempts to enumerate and clarify the issues related to differential perceptions of medical errors associated with HIT. It then suggests solutions.

Identifying Failure Modes in Telemedicine: An Instructional Needs Assessment.

Technology failures in telehealth are common, and clinicians need the skills to diagnose and manage them at the point of care. However, there are issues beyond technology failures mediating the effective use of telehealth. We must teach best-practice procedures for conducting telemedicine visits and include in instructional simulations commonly encountered failure modes so students can build their skills. To this end, we recruited medical students to conduct a Healthcare Failure Modes and Effects Analysis (HFMEA) to predict failures in telemedicine, their potential causes, and the consequences to develop and teach prevention strategies. Sixteen students observed telehealth appointments independently. Based on their observations, we identified four categories of failures in telemedicine: technical issues, patient safety, communication, and social and structural determinants. We proposed a normalized workflow that included management and prevention strategies. Our findings can inform the creation of new curricula.

Assessing Telemedicine Competencies: Developing and Validating Learner Measures for Simulation-Based Telemedicine Training.

In 2021, the Association of American Medical Colleges published Telehealth Competencies Across the Learning Continuum, a roadmap for designing telemedicine curricula and evaluating learners. While this document advances educators' shared understanding of telemedicine's core content and performance expectations, it does not include turn-key-ready evaluation instruments. At the University of Oklahoma School of Community Medicine, we developed a year-long telemedicine curriculum for third-year medical and second-year physician assistant students. We used the AAMC framework to create program objectives and instructional simulations. We designed and piloted an assessment rubric for eight AAMC competencies to accompany the simulations. In this monograph, we describe the rubric development, scores for students participating in simulations, and results comparing inter-rater reliability between faculty and standardized patient evaluators. Our preliminary work suggests that our rubric provides a practical method for evaluating learners by faculty during telemedicine simulations. We also identified opportunities for additional reliability and validity testing.

Why Do People Use Online Lab Results and What Do They Look For: A Qualitative Study.

Laboratory (lab) test results are increasingly available online for patient review. However, there is a dearth of research with respect to users' information needs, goals, and information processing strategies. In this exploratory qualitative study, we interviewed a sample of (N = 25) online lab results users to understand their objectives and search targets. We transcribed their responses and used affinity diagramming to identify themes in their responses. Our analysis identified six reasons why people look at their online lab results (i.e., health status, reassurance, health education, speed, self-management, and patient safety) and two themes about what people look for (i.e., abnormal and normal values, trends). Knowing what drives users and what information they are looking for can inform the design of online lab reporting, improve usefulness, and better satisfy user needs.

Hidden in Plain Sight: Overlooked Results and Other Errors in Evaluating Online Laboratory Results.

People are increasingly accessing their own laboratory (lab) results online. However, Canadians may be expected to use different systems to access their results, depending upon where they are tested (e.g., community lab vs. hospital), and these results may be displayed differently. This study examined the extent to which participants without medical expertise (N = 25) made errors identifying lab results (i.e., missing or mis-identifying abnormal results) in a mock report. Six participants overlooked each of the flagged values, 20 participants missed an abnormal result that was not flagged, and 2 participants mis-identified a normal value as out of range. We describe potential causes of these errors and the implications for the design of consumer-facing lab results.

Interprofessional survey on medication reconciliation activities in the US Department of Veterans' Affairs: development and validation of an Implementation Readiness Questionnaire.

BACKGROUND: Medication reconciliation (MR) can detect medication history discrepancies at interfaces-in-care and help avoid downstream adverse drug events. However, organisations have struggled to implement high-quality MR programmes. The literature has identified systems barriers, including technology capabilities and data interoperability. However, organisational culture as a root cause has been underexplored. OBJECTIVES: Our objectives were to develop an implementation readiness questionnaire and measure staff attitudes towards MR across a healthcare enterprise. METHODS: We developed and distributed a questionnaire to 170 Veterans' Health Affairs (VHA) sites using Research Electronic Data Capture (REDCap) software. The questionnaire contained 21 Likert-scale items that measured three constructs, such as: (1) the extent that clinicians valued MR; (2) perceptions of workflow compatibility and (3) perceptions concerning organisational climate of implementation. RESULTS: 8704 clinicians and staff responded to our questionnaire (142 of 170 VHA facilities). Most staff believed reconciling medications can improve medication safety (approximately 90% agreed it was 'important'). However, most (approximately 90%) also expressed concerns about changes to their workflow. One-third of respondents prioritised other duties over MR and reported barriers associated with implementation climate. Only 47% of respondents agreed they had enough resources to address discrepancies when identified. INTERPRETATION: Our findings indicate that an MR readiness assessment can forecast challenges and inform development of a context-sensitive implementation bundle. Clinicians surveyed struggled with resources, technology challenges and implementation climate. A strong campaign should include clear leadership messaging, credible champions and resources to overcome technical challenges. CONCLUSIONS: This manuscript provides a method to conduct a readiness assessment and highlights the importance of organisational culture in an MR campaign. The data can help assess site or network readiness for an MR change management programme.

Using Simulations to Train Medical Students for Unanticipated Technology Failures in Telemedicine.

Simulations offer a safe environment for health professional training and the opportunity to predictably and consistently introduce events or variables that may be rare or dangerous in a live setting. Exposing trainees to unanticipated events during simulations can improve their ability to adapt and improvise. The COVID-19 pandemic accelerated the adoption of telehealth worldwide and highlighted the need for better training in health professional schools. In the United States, the Association of American Medical Colleges (AAMC) published new telehealth competency standards in 2021. The AAMC stated that health care providers should be aware of the risks of technology failures, capable of troubleshooting them, and lead systems interventions to improve safety. However, the AAMC does not provide guidance on the specific failures or solutions. In this study, we developed a set of technology failures that can be simulated in a telehealth curriculum. We incorporated one technology failure into a simulated telehealth encounter and gathered students' (N = 53) feedback on the exercise. Students' feedback was overwhelmingly positive. They agreed that integrating technology failures into telehealth simulations provides important practice managing these events during clinical encounters. While telehealth is an important healthcare delivery modality that can improve access-to-care, it is imperative to train medical students to navigate technology failures so that can adeptly manage these issues in clinical practice.

A Telepsychiatry Simulation for Suicide Assessment: Teaching Telemedicine Safety Competencies.

Educators must provide controlled scenarios for health professional students to develop patient safety competencies related to telemedicine, including when and how to escalate care. We developed a telepsychiatry workshop to give students experience with a high-stakes mental health condition. The workshop included (1) pre-session readings; (2) didactics on mood disorders and telepsychiatry; (3) a motivational interviewing exercise; (4) a simulated telemedicine encounter; and (5) a faculty-led group debrief. We evaluated teaching effectiveness using a competency assessment with three scales: (1) medical knowledge; (2) interpersonal and communication skills; and (3) telemedicine competencies. Between 0 and 59% of students were entrustable for each telemedicine competency. Our workshop demonstrates how to teach students about the safe use of telehealth technology and provides practice triaging mental health conditions commonly encountered in primary care and mental health telemedicine clinics.

Teaching Teledermatology: A Simulation Pilot for Health Professional Students.

We developed a teledermatology simulation to give medical and physician assistant students practice with live videoconferencing and store-and-forward workflows. The simulation included (1) pre-session reading; (2) a brief teledermatology didactic; (3) a simulated encounter with a standardized patient; and (4) faculty-led debriefs. The faculty observed students during the simulation and distributed a post-session learner satisfaction survey. Although students had mixed feelings about the simulation, 88% said the workshop met or exceeded expectations.

Overcoming Challenges to Inclusive User-based Testing of Health Information Technology with Vulnerable Older Adults: Recommendations from a Human Factors Engineering Expert Inquiry.

OBJECTIVES: Involving representative users in usability testing of health information technology (HIT) is central to user-centered design. However, (vulnerable) older adults as representative users have unique requirements. Aging processes may affect physical capabilities and cognitive skills, which can hamper testing with this demographic and may require special attention and revised protocols. This study was performed to provide expert-based recommendations for HIT user-testing with (vulnerable) older adults to support inclusive HIT design and evaluation. METHODS: First, we conducted a structured workshop with ten experts in HIT implementation and research, recruited through purposeful sampling, to generate insights into how characteristics of older adults may influence user-testing. Next, five Human Factor researchers experienced in HIT user-testing with (vulnerable) older adults validated the results and provided additional textual insights to gain consensus on the most important recommendations. A thematic analysis was performed on the resulting inquiries. Applied codes were based on the User-Centered Design framework. RESULTS: The analysis resulted in nine recommendations for user-testing of HIT with older adults, divided into three main themes: (1) empathetic approach and trust-building, (2) new requirements for testing and study design, and (3) adjustments to usability evaluation methods. For each theme a checklist of relevant items to follow-up on the recommendation is provided. CONCLUSIONS: The recommendations generated through expert inquiry contribute to more effective usability testing of HIT with older adults. This provides an important step towards improved accessibility of HIT amongst older adults through inclusive user-centered design.

Primary care provider perceptions and use of a novel medication reconciliation technology.

BACKGROUND: Although medication reconciliation (MR) can reduce medication discrepancies, it is challenging to operationalise. Consequently, we developed a health information technology (HIT) to collect a patient medication history and make it available to the primary care (PC) provider. We deployed a self-service kiosk in a PC clinic that permits patients to indicate a medication adherence history. Patient responses are immediately viewable in the legacy electronic health record. This paper describes a survey developed to assess PC provider perceptions of our HIT and HIT implementation effectiveness. METHODS: We developed and administered a survey to all PC providers to assess technology implementation effectiveness. The survey included scales measuring (1) user attitudes towards MR, (2) perceptions of our HIT and (3) the local organisational climate for implementation. We also assessed the consistency and quality of tool use. RESULTS: Nearly 90% of PC providers responded to the survey and 58% indicated that they were familiar with the technology and had seen the tool output. Most providers believed that MR represented an important safety intervention, although 43% did not believe that they had the necessary resources to manage discrepancies. Composite scale scores for the 58% of respondents familiar with the HIT indicate that the majority favoured our tool over usual care. However, composite scale scores suggest that the climate for implementation at our facility was suboptimal. Overall, the quality and consistency of tool use among providers was very heterogeneous. CONCLUSIONS: A patient self-service kiosk offers an efficient mechanism to collect a medication adherence history; provider survey responses indicate that they appreciated and used the MR kiosk output. Nonetheless, opportunities exist to improve data displays and embed decision support to facilitate discrepancy management.

A User Experience and eHealth Literacy Inspection of a Lab Test Interpretation Mobile App for Citizens.

People are increasingly able to access their laboratory (lab) results using patient-facing portals. However, lab reports for citizens are often identical to those for clinicians; without specialized training they can be near impossible to interpret. In this study, we inspected a mobile health application (app) that converts traditional lab results into a citizen-centred format. We used the Health Literacy Online (HLO) checklist to inspect the app. Our inspection revealed that most of the app's strengths were related to its Organization of Content and Simple Navigation and most of its weaknesses were related to Engage Users. We also identified several usability and user experience (UX) issues that were beyond the purview of the HLO checklist. Although this app represents an important step towards making lab results universally accessible, we identified several opportunities for improvements that could increase its value to citizens.