Technological Literacy as a Framework for Health Professions Education in the Digital Era.

With increasing use of information and communications technologies (ICTs) in health, and rapid technological changes, there is a pressing need to prepare current and future health professionals to use ICTs as an integral part of their practice. We propose the Technological Literacy Framework, which includes 3 interlinked elements-knowledge, capabilities, and critical thinking and decision making-as an overarching structure for organizing and designing competencies, learning objectives, and educational interventions for health professions education in the digital era. We provide examples of EHR and telehealth educational interventions and how they map to the framework.

Teaching and Evaluating the Virtual Physical Exam in Telemedicine.

With the rapid adoption of telemedicine since the COVID-19 pandemic, it has become imperative to teach and evaluate health professional trainees on skills important to conducting effective virtual visits. We developed a simulation-based workshop with (1) readings, (2) a lecture covering online communication and the virtual head and neck exam, (3) a telemedicine simulation with a standardized patient observed by faculty, (4) personalized feedback from faculty, and (5) a group debrief session. We created an evaluation rubric based on three of 20 Association of American Medical Colleges (AAMC) telemedicine competencies to assess learner performance during the simulations. Students (medical and physician assistant students; n = 50), and internal medicine residents (n = 20) completed this workshop in 2023. At least 90% of trainees across the two groups were rated as approaching entrustment or entrustable in each competency. This workshop is an example of a scalable telemedicine curriculum that can be used to teach and evaluate learners in the virtual physical exam across the training continuum.

What People Know vs. What They Should Know About Laboratory Reference Ranges.

Laboratory test results are increasingly available to health consumers. Almost all test results are accompanied by a reference range to aid in interpretation. This study asked 25 non-healthcare providers to explain the term reference range. The descriptions highlighted four principal themes: health consumers were unsure about their understanding of the term reference range; they equated a reference range with a normal range; few had some degree of awareness of the limitations of reference ranges; and few had limited awareness of the difference between a reference range and clinical cutoff. Further efforts should be made to educate health consumers on the utility and limitations of reference ranges. When providing results to health consumers, laboratories should consider including other means of contextualizing test results in order to support their understanding of potential clinical significance.

Towards Building Decision Support Tools for Older Adults at Home: A Qualitative Analysis.

Older adults (and caregivers) face important health-related decisions which can have important consequences on their well-being, independence, and outcomes (e.g., where to live, how to stay safe, where to get care). There is a critical need for tools to help them make informed decisions that reflects what is most important to them. We report on a qualitative analysis of survey data collected from home care providers to inform the design and development of digital decision support tools for older adults.

An Iterative Approach to Usability Evaluation: A Workplace Wellness Mobile Application Example.

Without a gold standard for mobile app usability testing, this paper outlines a case study of how a not-for-profit organization tested the usability of a public-facing mobile app designed to promote the health and wellbeing of workers in Canada by offering accessible resources conveniently through one's mobile phone. Although the app was developed by a third-party vendor, usability testing and refinement was left to the organization due to budgetary and time constraints. The app underwent three rounds of iterative usability testing. The first phase consisted of observing participants using the app followed by an interview. The second and third phases involved participants following task-based instructions and answering survey questions. All phases uncovered issues which helped to improve and refine the usability of the app before the launch to the public. This work offers a case study example of how low-cost in-house usability testing could be employed.

Usability Checklists for Health Technology: Case Study and Experts' Opinions.

Application of usability evaluations throughout the health technology lifecycle is necessary to improve the efficiency, safety, and effectiveness of health service delivery. Unfortunately, technology vendors and healthcare organizations may not have funding, time or expertise to conduct usability studies. In this paper, we describe how usability checklists can potentially fill this gap. First, we introduce a case study using a checklist to identify usability issues with a primary care dashboard. Then we provide an expert summary of the strengths and limitations of usability checklists. Findings suggest that checklists are efficient to identify important usability issues. They can be used effectively by project team members - including clinicians - without formal usability training. However, checklists should complement rather than replace usability evaluations with representative users.

A Comparison of Websites Offering Laboratory Test Information Targeted to Citizens.

Laboratory (lab) tests can assist diagnosis, treatment, and monitoring illness and health. Lab results are one of the most commonly accessible types of personal health information, yet they can be difficult for consumers (e.g., patients, laypeople, citizens) to understand. Consequently, many consumers turn to digital educational resources (e.g., websites, mobile applications) to make sense of their tests and results. In this study, we compared the understandability and readability of four different consumer targeted webpages with information about a commonly ordered blood test called the Complete Blood Count (CBC). The webpages varied in terms of understandability, and only one met the threshold. None of the web pages provided any information about how to respond to lab results. Although all four webpages were quite readable, some were much longer than others. The length of webpages may impact users' attention, ability to locate information, and determine what is most important. Future work is warranted to better understand users' information needs and the usability and user experience of these types of websites.

Usability Evaluation Ecological Validity: Is More Always Better?

BACKGROUND: The ecological validity associated with usability testing of health information technologies (HITs) can affect test results and the predictability of real-world performance. It is, therefore, necessary to identify conditions with the greatest effect on validity. METHOD: We conducted a comparative analysis of two usability testing conditions. We tested a HIT designed for anesthesiologists to detect pain signals and compared two fidelity levels of ecological validity. We measured the difference in the number and type of use errors identified between high and low-fidelity experimental conditions. RESULTS: We identified the same error types in both test conditions, although the number of errors varied as a function of the condition. The difference in total error counts was relatively modest and not consistent across levels of severity. CONCLUSIONS: Increasing ecological validity does not invariably increase the ability to detect use errors. Our findings suggest that low-fidelity tests are an efficient way to identify and mitigate usability issues affecting ease of use, effectiveness, and safety. We believe early low-fidelity testing is an efficient but underused way to maximize the value of usability testing.

Interpreting Laboratory Results with Complementary Health Information: A Human Factors Perspective.

The desire to access personal and high-quality health information electronically is increasing, not only in Canada, but globally. With the advent of the COVID - 19 pandemic the desire and demand for telemedicine and timely access to personal health data such as online laboratory (lab) results has increased substantially. This study examines citizens' perspectives of being provided with high-quality information about a specific lab test (i.e., potassium) in the same display as a trend graph. Therefore, the objective of this study is to test how participants managed this additional information about the context of the test, understood, and applied it. The researchers analyzed the responses of semi-structured interviews with Canadian participants (N=24) using conventional content analysis. This paper examined four themes related to providing complementary information concurrently with lab results in the same display: 1) Benefits of Collocated Information, 2) Information Overload, 3) Misinterpretation, 4) Confusion. This study provided examples of some of the difficulties that the participants faced accessing their lab values online, while navigating and discerning complimentary high-quality health information available in their patient portal.

Tabular, Annotated, Visual, or Trends + Contextual Information? Preferences for Online Laboratory Results Displays.

People are increasingly offered access to their personal health information (e.g., laboratory results, clinical notes, diagnostic imaging results). However, this information is the same as that used by health care providers with clinical expertise and training in medical terminology, which citizens typically do not have. In this study, we examined participants (N = 24) preferences for four different types of displays for online laboratory (lab) results: Tabular, Annotated, Visual, and Trends + Contextual Information. The Friedman test of difference comparing participants' ratings of the four displays was significant, χ2(3)=10.8, P=.013, and the Wilcoxon signed rank pairwise comparison tests revealed that participants rated the visual lab results display significantly more favourably than the traditional display (Z=-2.746, P=.006). These findings indicate that many people prefer lab results displayed using more visual cues and some perceived this format as easier to understand than the other display formats. Given the importance of people accessing, understanding, and using their own health information, it is crucial for displays and systems to provide a better user experience. Displaying data (e.g., lab results) visually is one possible way to improve interpretability of personal health information provided to the public.

Teledermatology: Simulating Hybrid Workflows for Telemedicine Education.

Given the importance of telemedicine in improving healthcare access for underserved patients, professional students need experience using virtual clinical workflows. We developed an educational workshop with (1) readings, (2) a knowledge assessment test, (3) dermatology and teledermatology lectures, (5) a telemedicine simulation with a standardized patient, and (6) a debriefing session. The simulation included a "hybrid" workflow with live videoconferencing and store-and-forward image review. We measured student performance using three American Association of Medical Colleges (AAMC) Telemedicine Competencies for medical education. Ninety-eight medical and physician assistant students completed this workshop between 2021 and 2022, and 80% were entrustable or approaching entrustment in each competency. Some students struggled with data collection and technology use. Our results suggest that this workshop offers a practical and generalizable way to teach about multiple virtual workflows and strengthen students' telemedicine competencies.

Simulating Telemedicine, Medication Reconciliation, and Social Determinants: A Novel Instructional Approach to Health Systems Competencies.

While medication reconciliation is necessary to reduce errors, it is often challenging to gather an accurate history in the clinic. Telemedicine offers a relative advantage over clinic and hospital-based interviews by enabling the clinician to inspect the home environment, review pill bottles, and identify social determinants affecting adherence, such as financial instability. To be effective, however, clinicians must be trained in best-practice interview methods and the proper use of telemedicine. There is very little information in the literature describing the best strategies for teaching students or measuring competencies in telemedicine. Therefore, we created an educational module with a telemedicine simulation and an evaluation rubric. We piloted this module with 48 medical and physician assistant students. Most students could complete a virtual interview and gather a medication history. However, only half identified an over-the-counter medication missing from the list. Most students were either entrustable or approaching entrustment in the six telemedicine competencies measured in this simulation. This simulation is valuable for teaching students about medication reconciliation, using telemedicine to close gaps in access to care, and identifying health-related social needs affecting medication adherence.

Canadian employers' perspectives on a new framework for health informatics competencies.

Competencies are the knowledge, skills, and abilities needed to operate and perform successfully in the workplace. Due to the evolving nature of health informatics, it is important continuously examine and refine competencies in this field. In this study, we administered a questionnaire to Canadian employers (N = 29) of health informatics cooperative education (co-op) students to garner their feedback on competencies within a New Health Informatics Professional Competencies Framework. Overall, the findings supported this new framework. An average of ratings within each of the four competency categories revealed that participants perceived Management Science to be the most important, followed by Information & Computer Science, then Health Science and finally Data Science. Further, at least 20 (69 %) respondents rated nine of the 12 competencies as important. Of the 12 competencies, Biological and Clinical Science was rated the lowest. Findings from this study can potentially be used to inform curricula, career progression, and hiring practices in health informatics. Future work includes refining the questionnaire to assess the competencies more comprehensively and potentially exploring the importance of more transferable skills or general competencies (e.g., communication, problem-solving). Additionally, we want to survey a broader sample of health informatics professionals and integrate recent national and international work on health informatics competencies. Future work is also recommended towards the development of a maturity model for competencies of more experienced health informatics professionals.

Citizens' Access to Online Health Information - An International Survey of IMIA Member Countries.

Citizens' access to their online health information is pivotal. Therefore, this study examines citizens' access to their online health information across countries and healthcare settings. The study is based on a survey design targeting the 98 IMIA representatives of the national societies. Results indicate that Test results and Medications are the two types of online information that citizens in most cases have access to. Ten countries provide citizens access to all the different types of information included in the study. That relatively few countries provide citizens access to all the included types of online health information underscores the importance of continuous emphasis on accessibility and research within this field.

Consumers' Needs for Laboratory Results Portals: Questionnaire Study.

BACKGROUND: Over the last decade, there has been an increase in the number of health care consumers (ie, patients, citizens, and laypeople) with access to their laboratory results through portals. However, many portals are not designed with the consumer in mind, which can limit communication effectiveness and consumer empowerment. OBJECTIVE: We aimed to study design facilitators and barriers affecting consumer use of a laboratory results portal. We sought to identify modifiable design attributes to inform future interface specifications and improve patient safety. METHODS: A web-based questionnaire with open- and closed-ended items was distributed to consumers in British Columbia, Canada. Open-ended items with affinity diagramming and closed-ended questions with descriptive statistics were analyzed. RESULTS: Participants (N=30) preferred reviewing their laboratory results through portals rather than waiting to see their provider. However, respondents were critical of the interface design (ie, interface usability, information completeness, and display clarity). Scores suggest there are display issues impacting communication that require urgent attention. CONCLUSIONS: There are modifiable usability, content, and display issues associated with laboratory results portals that, if addressed, could arguably improve communication effectiveness, patient empowerment, and health care safety.

Automation vs. Innovation: Unexplored Strategies to Improve Virtual Care.

While there is a global desire to increase digital health capacity, digital health should transform health services delivery rather than simply automate - or worse - replicate existing practices. Failing to capitalize on this transformative potential misses an opportunity to engage patients and other users to provide a more person-centered experience. However, digital transformation done recklessly can disrupt workflow, alienate users, and jeopardize patient safety, as we have observed with implementation of many digital health tools. This paper uses a telemedicine example to provide insight into how digital health innovation can be a meaningful enabler of health system transformation. Examining different ways to leverage digital health technologies is crucial to best capitalize on their potential.

Differential Perceptions of What Constitutes a Medical Error Associated with Electronic Medical Records.

Perceptions of errors associated with healthcare information technology (HIT) often depend on the context and position of the viewer. HIT vendors posit very different causes of errors than clinicians, implementation teams, or IT staff. Even within the same hospital, members of departments and services often implicate other departments. Organizations may attribute errors to external care partners that refer patients, such as nursing homes or outside clinics. Also, the various clinical roles within an organization (e.g., physicians, nurses, pharmacists) can conceptualize errors and their root causes differently. Overarching all these perceptual factors, the definitions, mechanisms, and incidence of HIT-related errors are remarkably conflictual. There is neither a universal standard for defining or counting these errors. This paper attempts to enumerate and clarify the issues related to differential perceptions of medical errors associated with HIT. It then suggests solutions.

Identifying Failure Modes in Telemedicine: An Instructional Needs Assessment.

Technology failures in telehealth are common, and clinicians need the skills to diagnose and manage them at the point of care. However, there are issues beyond technology failures mediating the effective use of telehealth. We must teach best-practice procedures for conducting telemedicine visits and include in instructional simulations commonly encountered failure modes so students can build their skills. To this end, we recruited medical students to conduct a Healthcare Failure Modes and Effects Analysis (HFMEA) to predict failures in telemedicine, their potential causes, and the consequences to develop and teach prevention strategies. Sixteen students observed telehealth appointments independently. Based on their observations, we identified four categories of failures in telemedicine: technical issues, patient safety, communication, and social and structural determinants. We proposed a normalized workflow that included management and prevention strategies. Our findings can inform the creation of new curricula.

Applying Logic to the Healthcare Journey.

Cross-disciplinary approaches to remediate complex healthcare service delivery issues may have merit. This study aims to establish the potential benefits of applying service design and evaluative research concepts in healthcare. Specifically, this study aims to demonstrate how a Customer Journey Map and a Logic Model could be used in unison to identify and remedy service delivery gaps to reduce barriers to care. This study provides systems thinking approach to solving operational issues in healthcare.

Humanizing Big Data and Detailing Social Determinants of Health via Information Visualizations.

The pandemic has had devastating impacts on humanity and the global healthcare sector. An analysis into the social determinants of health, in particular racial and ethnic disparities may explain why certain population groups have been disproportionately affected by COVID-19. The objective of this study is to humanize and personify numerical data. Additionally, COVID-19 population data will be stratified via three data visualization tools (i.e., a persona, a journey map, Sankey diagram) to create a Visualized Combined Experience (VCE) Diagram to illustrate the micro, and macro, perspectives of marginalized individuals across the continuum of care.