Integrating Explainable Machine Learning in Clinical Decision Support Systems: Study Involving a Modified Design Thinking Approach.

BACKGROUND: Though there has been considerable effort to implement machine learning (ML) methods for health care, clinical implementation has lagged. Incorporating explainable machine learning (XML) methods through the development of a decision support tool using a design thinking approach is expected to lead to greater uptake of such tools. OBJECTIVE: This work aimed to explore how constant engagement of clinician end users can address the lack of adoption of ML tools in clinical contexts due to their lack of transparency and address challenges related to presenting explainability in a decision support interface. METHODS: We used a design thinking approach augmented with additional theoretical frameworks to provide more robust approaches to different phases of design. In particular, in the problem definition phase, we incorporated the nonadoption, abandonment, scale-up, spread, and sustainability of technology in health care (NASSS) framework to assess these aspects in a health care network. This process helped focus on the development of a prognostic tool that predicted the likelihood of admission to an intensive care ward based on disease severity in chest x-ray images. In the ideate, prototype, and test phases, we incorporated a metric framework to assess physician trust in artificial intelligence (AI) tools. This allowed us to compare physicians' assessments of the domain representation, action ability, and consistency of the tool. RESULTS: Physicians found the design of the prototype elegant, and domain appropriate representation of data was displayed in the tool. They appreciated the simplified explainability overlay, which only displayed the most predictive patches that cumulatively explained 90% of the final admission risk score. Finally, in terms of consistency, physicians unanimously appreciated the capacity to compare multiple x-ray images in the same view. They also appreciated the ability to toggle the explainability overlay so that both options made it easier for them to assess how consistently the tool was identifying elements of the x-ray image they felt would contribute to overall disease severity. CONCLUSIONS: The adopted approach is situated in an evolving space concerned with incorporating XML or AI technologies into health care software. We addressed the alignment of AI as it relates to clinician trust, describing an approach to wire framing and prototyping, which incorporates the use of a theoretical framework for trust in the design process itself. Moreover, we proposed that alignment of AI is dependent upon integration of end users throughout the larger design process. Our work shows the importance and value of engaging end users prior to tool development. We believe that the described approach is a unique and valuable contribution that outlines a direction for ML experts, user experience designers, and clinician end users on how to collaborate in the creation of trustworthy and usable XML-based clinical decision support tools.

Telemedicine visits requiring follow-up in-person visits at an urban academic family medicine centre.

BACKGROUND: With the onset of the COVID-19 pandemic, telemedicine was rapidly implemented in care settings globally. To understand what factors affect the successful completion of telemedicine visits in our urban, academic family medicine clinic setting, we analysed telemedicine visits carried out during the pandemic. METHODS: We conducted a retrospective chart review of telemedicine visits from 2 clinical units within a family medicine centre. To investigate the association between incomplete visits and various factors (age, gender, presenting complaints, physician level of training [resident or staff] and patient-physician relational continuity), we performed a multivariable logistic regression on data from August 2020, February 2021, and May 2021. An incomplete visit is one that requires a follow-up in-person visit with a physician within 3 days. RESULTS: Of the 2,138 telemedicine patient visits we investigated, 9.6% were incomplete. Patients presenting with lumps and bumps (OR: 3.84, 95% CI: 1.44, 10.5), as well as those seen by resident physicians (OR: 1.77, 95% CI: 1.22, 2.56) had increased odds of incomplete visits. Telemedicine visits at the family medicine clinic (Site A) with registered patients had lower odds of incomplete visits (OR: 0.24, 95% CI: 0.15, 0.39) than those at the community clinic (Site B), which provides urgent/episodic care with no associated relational continuity between patients and physicians. CONCLUSION: In our urban clinical setting, only a small minority of telemedicine visits required an in-person follow-up visit. This information may be useful in guiding approaches to triaging patients to telemedicine or standard in-person care.

With the onset of the COVID-19 pandemic, telemedicine was rapidly implemented in care settings globally. To understand what factors affect the successful completion of telemedicine visits in our urban, academic family medicine clinic, we analysed telemedicine visits carried out during the pandemic. On the basis of patient charts, we investigated the association between incomplete visits (telemedicine visits requiring in-person follow-up within 3 days) and various factors (age, gender, presenting complaints, whether the treating physician was a resident or staff doctor, and whether the patient and physician had a prior clinical relationship). Patients presenting with lumps and bumps and those seen by resident physicians had higher odds of being asked to come in-person for further evaluation. Overall, though, these required in-person follow-ups were uncommon: less than 10% of telemedicine visits resulted in the patient physically coming to the clinic within 3 days. The findings of our study could help guide patients to appropriate care services.

Using incorpoRATE to examine clinician willingness to engage in shared decision making: A study of Family Medicine residents.

OBJECTIVES: We evaluated the willingness of Family Medicine residents to engage in SDM, before and after an educational intervention. METHODS: We delivered a lecture and a workshop for residents on implementing SDM in preventive health care. Before the lecture (T1), participants completed a measure of their willingness to engage in SDM. Six months later, participants completed the measure a second time (T2). RESULTS: At T1, 64 of 73 residents who attended the educational session completed incorpoRATE. Six months later, 44 of 64 participants completed the measure a second time (T2). The range of incorpoRATE sum scores at T1 was from 4.9 to 9.1 out of 10. Among the 44 participants who completed incorpoRATE at both time points, the mean scores were 7.0 ± 1.0 at T1 and 7.4 ± 1.0 at T2 (t = -2.833, p = 0.007, Cohen's D = 0.43). CONCLUSION: Among Family Medicine residents, the willingness to engage in SDM is highly variable. This suggests a lack of consensus in the mind of these residents about SDM. Although mean scores at T2 were significantly higher, we question the educational importance of this change. PRACTICE IMPLICATIONS: incorpoRATE is a promising measure for educators. Understanding how willing a particular physician audience is to undertake SDM, and which elements require attention, could be helpful in designing more targeted curricula. Further research is needed to understand how the perceived stakes of a clinical situation influence physician willingness to engage in SDM.

Uninsured Pregnant Patients: Where Do We Begin?

Pregnant immigrant women without medical insurance often receive inadequate prenatal care. They are more likely to present late in their pregnancy for care, to receive less prenatal testing, and to receive inadequate prenatal follow-up. There is a documented association between inadequate prenatal care and poor birth outcomes, including preterm delivery and low birth weight. Caring for uninsured women causes stress for physicians and health care teams. A standardized approach to caring for uninsured pregnant women has the potential to improve access to care while providing a framework to healthcare providers that may decrease the tensions that arise within health care teams caring for these patients. We believe that giving uninsured women and the physicians who care for them a voice in constructing a system to address barriers to care is essential.

Effectiveness of a 3-hour breastfeeding course for family physicians.

OBJECTIVE: To test the effectiveness of a 3-hour course on breastfeeding for family physicians. DESIGN: A previously validated questionnaire testing confidence in, attitudes toward, and knowledge of breastfeeding issues was administered to participants 1 week before and 2 months after taking the Breastfeeding Basics for the Practicing Physician course. SETTING: Canada. PARTICIPANTS: Family physicians and residents who attended the Breastfeeding Basics for the Practicing Physician course between July 2008 and December 2010. INTERVENTIONS: Participants' scores on the questionnaires that were filled out before and after the course were collected and analyzed.

Perspectives of family physicians on the care of uninsured pregnant women.

OBJECTIVE: No official provisions are made for the medically uninsured under provincial public health programs in Canada. Studies have shown that uninsured pregnant women have inadequate access to prenatal and obstetrical services that favour healthy maternal and child outcomes. This qualitative study aimed to explore the perspectives of family physicians who provided care to uninsured pregnant women. METHODS: Eight family physicians affiliated with two Montreal-based primary-care clinics and one tertiary care hospital between 2004 and 2007 were interviewed using a semi-structured interview guide. Data were assessed using thematic analysis. RESULTS: Uninsured pregnant patients were characterized by physicians as socially vulnerable, with precarious immigration status that limited their access to health services. Uninsured patients were thought not to benefit from the same standard of perinatal care as their insured counterparts. Care of uninsured women was generally thought to be a professional obligation, regardless of the woman's ability to pay. Caring for this population was considered by family physicians to be challenging, engendering psychological stress, increased workload, and occasional tensions with other health care providers. CONCLUSION: In the present context, family physicians are left to negotiate the health care system in an attempt to provide adequate perinatal care for uninsured pregnant patients. This situation has repercussions for physicians, for patients and, ultimately, for infants. Leadership is required to ensure that all pregnant women in Canada have access to appropriate health care during the perinatal period.

Objectif : Il n'existe aucune disposition officielle en ce qui concerne les personnes qui ne sont pas couvertes par les régimes publics d'assurance-maladie provinciaux au Canada. Des études ont démontré que les femmes enceintes non assurées ne disposent pas d'un accès adéquat aux services prénataux et obstétricaux qui favorisent l'obtention de résultats maternels et infantiles sains. Cette étude qualitative avait pour but d'explorer les points de vue de médecins de famille ayant offert des soins à des femmes enceintes non assurées. Méthodes : Des entrevues semi-structurées ont été menées auprès de huit médecins de famille affiliés à deux cliniques montréalaises de soins primaires et à un hôpital de soins tertiaires de la même région entre 2004 et 2007. Les données ont été évaluées au moyen d'une analyse thématique. Résultats : Les patientes enceintes non assurées ont été caractérisées, par ces médecins, comme étant des personnes vulnérables sur le plan social dont le statut précaire en matière d'immigration limitait leur accès aux services de santé. Ces médecins estimaient que les patientes non assurées ne bénéficiaient pas du même standard de soins périnataux que leurs homologues assurées. D'ordre général, ils estimaient que l'offre de soins aux femmes non assurées constituait une obligation professionnelle, sans égard à la capacité de payer. Les médecins de famille considéraient que l'offre de soins à cette population était une activité complexe, qu'elle engendrait du stress psychologique, qu'elle entraînait une augmentation de la charge de travail et qu'elle donnait occasionnellement lieu à des tensions dans leurs relations avec d'autres fournisseurs de soins de santé. Conclusion : Dans le contexte actuel, les médecins de famille sont laissés à eux-mêmes dans leurs efforts visant à utiliser le système de santé pour tenter d'offrir des soins prénataux adéquats aux patientes enceintes non assurées. Cette situation a des répercussions pour les médecins, les patientes et, en bout de ligne, les enfants. Les décideurs doivent faire preuve de leadership pour s'assurer que, au Canada, toutes les femmes enceintes obtiennent accès à des soins de santé appropriés au cours de la période périnatale.

Undocumented pregnant women: what does the literature tell us?

The present literature review was conducted to determine what information has been published on the topic of undocumented pregnant migrants. Scientific databases and gray literature sources were searched for articles published between January 1967 and September 2010. Eighty-seven articles met the inclusion criteria and were reviewed. A final sample of 23 articles was included in the review. Existing evidence suggests that pregnant undocumented migrants living in Western societies tend to be younger, unmarried, and more likely to be employed in the domestic sector than documented migrants and permanent residents. They have less access to prenatal care and consult later in pregnancy than controls. Findings concerning delivery and birth outcomes are conflicting and subject to several biases. Little has been published on programs to address the needs of undocumented pregnant women living in Western countries. More research on the particular health and social issues faced by these women is needed.

Retrospective review of prenatal care and perinatal outcomes in a group of uninsured pregnant women.

OBJECTIVE: To assess the adequacy of prenatal care and perinatal outcomes for uninsured pregnant women at two primary care centres in Canada. METHODS: We conducted a retrospective case comparison study of uninsured women presenting for prenatal care between 2004 and 2007 (n = 71). Control subjects (n = 72) were chosen from provincially insured women presenting for prenatal care during the same period. A modified Kotelchuck Index was used to assess adequacy of care. Frequency of routine prenatal testing (blood tests, ultrasound, cervical swabs, Pap testing, and genetic screening) was compared. Perinatal outcomes assessed included gestational age and birth weight. RESULTS: Uninsured pregnant women presented for initial care 13.6 weeks later than insured women (at 25.6 weeks vs. 12.0 weeks, P < 0.001). Uninsured women had fewer blood tests (93.7% vs. 100%, P = 0.045), ultrasound screenings (82.5% vs. 98.4%, P = 0.003), cervical swabs (69.8% vs. 85.2%, P = 0.04), Pap tests (38.1% vs. 75.4%, P < 0.001), genetic screenings (12.7% vs. 44.3%, P < 0.001), and visits with health care providers (6.6 vs. 10.7, P = 0.05). Using a modified Kotelchuck Adequacy of Prenatal Care Utilization Index, uninsured women were more likely to be categorized as receiving "inadequate care" (uninsured 61.9% vs. insured 11.7%, P < 0.001). CONCLUSION: This study begins to document the care of uninsured pregnant women in Canada. Women in this category presented late for prenatal care, were less likely to have adequate screening tests, and were more likely to receive "inadequate care" as defined by the modified Kotelchuck Index. This information may be valuable in helping to plan programs to improve access to timely and adequate medical care for uninsured pregnant women.

A participatory approach to understanding and measuring patient satisfaction in a primary care teaching setting.

BACKGROUND: Patient satisfaction is a complex, multidimensional concept that is difficult to measure. However, there is agreement that understanding the expectations of a patient community or "what is important to them" is an essential consideration. We chose a participatory approach to address patient satisfaction in the context of a primary care teaching clinic. OBJECTIVES: The objectives of this project were to use a participatory research team of patients staff and researchers to (1) adapt an existing patient satisfaction questionnaire (PSQ) to the specific cultural and organizational elements ofa primary care teaching clinic, (2) administer the revised questionnaire and use the findings as a tool for organizational improvement, with the ultimate goal of increasing patient satisfaction, and (3) ensure that all decision making involved patients and staff to empower them in the process of organizational change. METHODS: We used an iterative, mixed methods approach to conduct this project. An interdisciplinary committee composed of members of the patient community, clinical and administrative staff, and researchers worked together as the primary decision making body. RESULTS: We modified a preexisting questionnaire to address the unique care delivery model of the clinic, issues of cultural sensitivity, and the need for simplified language and response format. Patient dissatisfaction was found to center on continuity and access to care. CONCLUSIONS: The participatory approach was critical to our success in understanding and measuring patient satisfaction from the patients' perspective. The involvement of the interdisciplinary committee and the high level of joint decision making in this project represents a unique contribution to assessing primary care patient satisfaction.