A method for structuring complex clinical knowledge and its representational formalisms to support composite knowledge interoperability in healthcare.

INTRODUCTION: The use and interoperability of clinical knowledge starts with the quality of the formalism utilized to express medical expertise. However, a crucial challenge is that existing formalisms are often suboptimal, lacking the fidelity to represent complex knowledge thoroughly and concisely. Often this leads to difficulties when seeking to unambiguously capture, share, and implement the knowledge for care improvement in clinical information systems used by providers and patients. OBJECTIVES: To provide a systematic method to address some of the complexities of knowledge composition and interoperability related to standards-based representational formalisms of medical knowledge. METHODS: Several cross-industry (Healthcare, Linguistics, System Engineering, Standards Development, and Knowledge Engineering) frameworks were synthesized into a proposed reference knowledge framework. The framework utilizes IEEE 42010, the MetaObject Facility, the Semantic Triangle, an Ontology Framework, and the Domain and Comprehensibility Appropriateness criteria. The steps taken were: 1) identify foundational cross-industry frameworks, 2) select architecture description method, 3) define life cycle viewpoints, 4) define representation and knowledge viewpoints, 5) define relationships between neighboring viewpoints, and 6) establish characteristic definitions of the relationships between components. System engineering principles applied included separation of concerns, cohesion, and loose coupling. RESULTS: A "Multilayer Metamodel for Representation and Knowledge" (M*R/K) reference framework was defined. It provides a standard vocabulary for organizing and articulating medical knowledge curation perspectives, concepts, and relationships across the artifacts created during the life cycle of language creation, authoring medical knowledge, and knowledge implementation in clinical information systems such as electronic health records (EHR). CONCLUSION: M*R/K provides a systematic means to address some of the complexities of knowledge composition and interoperability related to medical knowledge representations used in diverse standards. The framework may be used to guide the development, assessment, and coordinated use of knowledge representation formalisms. M*R/K could promote the alignment and aggregated use of distinct domain-specific languages in composite knowledge artifacts such as clinical practice guidelines (CPGs).

Clinical decision support models and frameworks: Seeking to address research issues underlying implementation successes and failures.

Computer-based clinical decision support (CDS) has been pursued for more than five decades. Despite notable accomplishments and successes, wide adoption and broad use of CDS in clinical practice has not been achieved. Many issues have been identified as being partially responsible for the relatively slow adoption and lack of impact, including deficiencies in leadership, recognition of purpose, understanding of human interaction and workflow implications of CDS, cognitive models of the role of CDS, and proprietary implementations with limited interoperability and sharing. To address limitations, many approaches have been proposed and evaluated, drawing on theoretical frameworks, as well as management, technical and other disciplines and experiences. It seems clear, because of the multiple perspectives involved, that no single model or framework is adequate to encompass these challenges. This Viewpoint paper seeks to review the various foci of CDS and to identify aspects in which theoretical models and frameworks for CDS have been explored or could be explored and where they might be expected to be most useful.

Multiple perspectives on clinical decision support: a qualitative study of fifteen clinical and vendor organizations.

BACKGROUND: Computerized clinical decision support (CDS) can help hospitals to improve healthcare. However, CDS can be problematic. The purpose of this study was to discover how the views of clinical stakeholders, CDS content vendors, and EHR vendors are alike or different with respect to challenges in the development, management, and use of CDS. METHODS: We conducted ethnographic fieldwork using a Rapid Assessment Process within ten clinical and five health information technology (HIT) vendor organizations. Using an inductive analytical approach, we generated themes from the clinical, content vendor, and electronic health record vendor perspectives and compared them. RESULTS: The groups share views on the importance of appropriate manpower, careful knowledge management, CDS that fits user workflow, the need for communication among the groups, and for mutual strategizing about the future of CDS. However, views of usability, training, metrics, interoperability, product use, and legal issues differed. Recommendations for improvement include increased collaboration to address legal, manpower, and CDS sharing issues. CONCLUSIONS: The three groups share thinking about many aspects of CDS, but views differ in a number of important respects as well. Until these three groups can reach a mutual understanding of the views of the other stakeholders, and work together, CDS will not reach its potential.

Measuring agreement between decision support reminders: the cloud vs. the local expert.

BACKGROUND: A cloud-based clinical decision support system (CDSS) was implemented to remotely provide evidence-based guideline reminders in support of preventative health. Following implementation, we measured the agreement between preventive care reminders generated by an existing, local CDSS and the new, cloud-based CDSS operating on the same patient visit data. METHODS: Electronic health record data for the same set of patients seen in primary care were sent to both the cloud-based web service and local CDSS. The clinical reminders returned by both services were captured for analysis. Cohen's Kappa coefficient was calculated to compare the two sets of reminders. Kappa statistics were further adjusted for prevalence and bias due to the potential effects of bias in the CDS logic and prevalence in the relative small sample of patients. RESULTS: The cloud-based CDSS generated 965 clinical reminders for 405 patient visits over 3 months. The local CDSS returned 889 reminders for the same patient visit data. When adjusted for prevalence and bias, observed agreement varied by reminder from 0.33 (95% CI 0.24 - 0.42) to 0.99 (95% CI 0.97 - 1.00) and demonstrated almost perfect agreement for 7 of the 11 reminders. CONCLUSIONS: Preventive care reminders delivered by two disparate CDS systems show substantial agreement. Subtle differences in rule logic and terminology mapping appear to account for much of the discordance. Cloud-based CDSS therefore show promise, opening the door for future development and implementation in support of health care providers with limited resources for knowledge management of complex logic and rules.

Criteria for assessing high-priority drug-drug interactions for clinical decision support in electronic health records.

BACKGROUND: High override rates for drug-drug interaction (DDI) alerts in electronic health records (EHRs) result in the potentially dangerous consequence of providers ignoring clinically significant alerts. Lack of uniformity of criteria for determining the severity or validity of these interactions often results in discrepancies in how these are evaluated. The purpose of this study was to identify a set of criteria for assessing DDIs that should be used for the generation of clinical decision support (CDS) alerts in EHRs. METHODS: We conducted a 20-year systematic literature review of MEDLINE and EMBASE to identify characteristics of high-priority DDIs. These criteria were validated by an expert panel consisting of medication knowledge base vendors, EHR vendors, in-house knowledge base developers from academic medical centers, and both federal and private agencies involved in the regulation of medication use. RESULTS: Forty-four articles met the inclusion criteria for assessing characteristics of high-priority DDIs. The panel considered five criteria to be most important when assessing an interaction- Severity, Probability, Clinical Implications of the interaction, Patient characteristics, and the Evidence supporting the interaction. In addition, the panel identified barriers and considerations for being able to utilize these criteria in medication knowledge bases used by EHRs. CONCLUSIONS: A multi-dimensional approach is needed to understanding the importance of an interaction for inclusion in medication knowledge bases for the purpose of CDS alerting. The criteria identified in this study can serve as a first step towards a uniform approach in assessing which interactions are critical and warrant interruption of a provider's workflow.

A survey of computable biomedical knowledge repositories.

Introduction: While data repositories are well-established in clinical and research enterprises, knowledge repositories with shareable computable biomedical knowledge (CBK) are relatively new entities to the digital health ecosystem. Trustworthy knowledge repositories are necessary for learning health systems, but the policies, standards, and practices to promote trustworthy CBK artifacts and methods to share, and safely and effectively use them are not well studied. Methods: We conducted an online survey of 24 organizations in the United States known to be involved in the development or deployment of CBK. The aim of the survey was to assess the current policies and practices governing these repositories and to identify best practices. Descriptive statistics methods were applied to data from 13 responding organizations, to identify common practices and policies instantiating the TRUST principles of Transparency, Responsibility, User Focus, Sustainability, and Technology. Results: All 13 respondents indicated to different degrees adherence to policies that convey TRUST. Transparency is conveyed by having policies pertaining to provenance, credentialed contributors, and provision of metadata. Repositories provide knowledge in machine-readable formats, include implementation guidelines, and adhere to standards to convey Responsibility. Repositories report having Technology functions that enable end-users to verify, search, and filter for knowledge products. Less common TRUST practices are User Focused procedures that enable consumers to know about user licensing requirements or query the use of knowledge artifacts. Related to Sustainability, less than a majority post describe their sustainability plans. Few organizations publicly describe whether patients play any role in their decision-making. Conclusion: It is essential that knowledge repositories identify and apply a baseline set of criteria to lay a robust foundation for their trustworthiness leading to optimum uptake, and safe, reliable, and effective use to promote sharing of CBK. Identifying current practices suggests a set of desiderata for the CBK ecosystem in its continued evolution.

Randomized controlled trial of health maintenance reminders provided directly to patients through an electronic PHR.

BACKGROUND: Provider and patient reminders can be effective in increasing rates of preventive screenings and vaccinations. However, the effect of patient-directed electronic reminders is understudied. OBJECTIVE: To determine whether providing reminders directly to patients via an electronic Personal Health Record (PHR) improved adherence to care recommendations. DESIGN: We conducted a cluster randomized trial without blinding from 2005 to 2007 at 11 primary care practices in the Partners HealthCare system. PARTICIPANTS: A total of 21,533 patients with access to a PHR were invited to the study, and 3,979 (18.5%) consented to enroll. INTERVENTIONS: Patients in the intervention arm received health maintenance (HM) reminders via a secure PHR "eJournal," which allowed them to review and update HM and family history information. Patients in the active control arm received access to an eJournal that allowed them to input and review information related to medications, allergies and diabetes management. MAIN MEASURES: The primary outcome measure was adherence to guideline-based care recommendations. KEY RESULTS: Intention-to-treat analysis showed that patients in the intervention arm were significantly more likely to receive mammography (48.6% vs 29.5%, p = 0.006) and influenza vaccinations (22.0% vs 14.0%, p = 0.018). No significant improvement was observed in rates of other screenings. Although Pap smear completion rates were higher in the intervention arm (41.0% vs 10.4%, p < 0.001), this finding was no longer significant after excluding women's health clinics. Additional on-treatment analysis showed significant increases in mammography (p = 0.019) and influenza vaccination (p = 0.015) for intervention arm patients who opened an eJournal compared to control arm patients, but no differences for any measure among patients who did not open an eJournal. CONCLUSIONS: Providing patients with HM reminders via a PHR may be effective in improving some elements of preventive care.

Interface design principles for usable decision support: a targeted review of best practices for clinical prescribing interventions.

Developing effective clinical decision support (CDS) systems for the highly complex and dynamic domain of clinical medicine is a serious challenge for designers. Poor usability is one of the core barriers to adoption and a deterrent to its routine use. We reviewed reports describing system implementation efforts and collected best available design conventions, procedures, practices and lessons learned in order to provide developers a short compendium of design goals and recommended principles. This targeted review is focused on CDS related to medication prescribing. Published reports suggest that important principles include consistency of design concepts across networked systems, use of appropriate visual representation of clinical data, use of controlled terminology, presenting advice at the time and place of decision making and matching the most appropriate CDS interventions to clinical goals. Specificity and contextual relevance can be increased by periodic review of trigger rules, analysis of performance logs and maintenance of accurate allergy, problem and medication lists in health records in order to help avoid excessive alerting. Developers need to adopt design practices that include user-centered, iterative design and common standards based on human-computer interaction (HCI) research methods rooted in ethnography and cognitive science. Suggestions outlined in this report may help clarify the goals of optimal CDS design but larger national initiatives are needed for systematic application of human factors in health information technology (HIT) development. Appropriate design strategies are essential for developing meaningful decision support systems that meet the grand challenges of high-quality healthcare.

A study of diverse clinical decision support rule authoring environments and requirements for integration.

BACKGROUND: Efficient rule authoring tools are critical to allow clinical Knowledge Engineers (KEs), Software Engineers (SEs), and Subject Matter Experts (SMEs) to convert medical knowledge into machine executable clinical decision support rules. The goal of this analysis was to identify the critical success factors and challenges of a fully functioning Rule Authoring Environment (RAE) in order to define requirements for a scalable, comprehensive tool to manage enterprise level rules. METHODS: The authors evaluated RAEs in active use across Partners Healthcare, including enterprise wide, ambulatory only, and system specific tools, with a focus on rule editors for reminder and medication rules. We conducted meetings with users of these RAEs to discuss their general experience and perceived advantages and limitations of these tools. RESULTS: While the overall rule authoring process is similar across the 10 separate RAEs, the system capabilities and architecture vary widely. Most current RAEs limit the ability of the clinical decision support (CDS) interventions to be standardized, sharable, interoperable, and extensible. No existing system meets all requirements defined by knowledge management users. CONCLUSIONS: A successful, scalable, integrated rule authoring environment will need to support a number of key requirements and functions in the areas of knowledge representation, metadata, terminology, authoring collaboration, user interface, integration with electronic health record (EHR) systems, testing, and reporting.

Recommended practices for computerized clinical decision support and knowledge management in community settings: a qualitative study.

BACKGROUND: The purpose of this study was to identify recommended practices for computerized clinical decision support (CDS) development and implementation and for knowledge management (KM) processes in ambulatory clinics and community hospitals using commercial or locally developed systems in the U.S. METHODS: Guided by the Multiple Perspectives Framework, the authors conducted ethnographic field studies at two community hospitals and five ambulatory clinic organizations across the U.S. Using a Rapid Assessment Process, a multidisciplinary research team: gathered preliminary assessment data; conducted on-site interviews, observations, and field surveys; analyzed data using both template and grounded methods; and developed universal themes. A panel of experts produced recommended practices. RESULTS: The team identified ten themes related to CDS and KM. These include: 1) workflow; 2) knowledge management; 3) data as a foundation for CDS; 4) user computer interaction; 5) measurement and metrics; 6) governance; 7) translation for collaboration; 8) the meaning of CDS; 9) roles of special, essential people; and 10) communication, training, and support. Experts developed recommendations about each theme. The original Multiple Perspectives framework was modified to make explicit a new theoretical construct, that of Translational Interaction. CONCLUSIONS: These ten themes represent areas that need attention if a clinic or community hospital plans to implement and successfully utilize CDS. In addition, they have implications for workforce education, research, and national-level policy development. The Translational Interaction construct could guide future applied informatics research endeavors.

Summary of fourth annual MCBK public meeting: Mobilizing computable biomedical knowledge-metadata and trust.

The exponential growth of biomedical knowledge in computable formats challenges organizations to consider mobilizing artifacts in findable, accessible, interoperable, reusable, and trustable (FAIR+T) ways1. There is a growing need to apply biomedical knowledge artifacts to improve health in Learning Health Systems, health delivery organizations, and other settings. However, most organizations lack the infrastructure required to consume and apply computable knowledge, and national policies and standards adoption are insufficient to ensure that it is discoverable and used safely and fairly, nor is there widespread experience in the process of knowledge implementation as clinical decision support. The Mobilizing Computable Biomedical Knowledge (MCBK) community formed in 2016 to address these needs. This report summarizes the main outputs of the Fourth Annual MCBK public meeting, which was held virtually July 20 to July 21, 2021 and convened over 100 participants spanning diverse domains to frame and address important dimensions for mobilizing CBK.

Comparison of clinical knowledge management capabilities of commercially-available and leading internally-developed electronic health records.

BACKGROUND: We have carried out an extensive qualitative research program focused on the barriers and facilitators to successful adoption and use of various features of advanced, state-of-the-art electronic health records (EHRs) within large, academic, teaching facilities with long-standing EHR research and development programs. We have recently begun investigating smaller, community hospitals and out-patient clinics that rely on commercially-available EHRs. We sought to assess whether the current generation of commercially-available EHRs are capable of providing the clinical knowledge management features, functions, tools, and techniques required to deliver and maintain the clinical decision support (CDS) interventions required to support the recently defined "meaningful use" criteria. METHODS: We developed and fielded a 17-question survey to representatives from nine commercially available EHR vendors and four leading internally developed EHRs. The first part of the survey asked basic questions about the vendor's EHR. The second part asked specifically about the CDS-related system tools and capabilities that each vendor provides. The final section asked about clinical content. RESULTS: All of the vendors and institutions have multiple modules capable of providing clinical decision support interventions to clinicians. The majority of the systems were capable of performing almost all of the key knowledge management functions we identified. CONCLUSION: If these well-designed commercially-available systems are coupled with the other key socio-technical concepts required for safe and effective EHR implementation and use, and organizations have access to implementable clinical knowledge, we expect that the transformation of the healthcare enterprise that so many have predicted, is achievable using commercially-available, state-of-the-art EHRs.

Scales for assessing self-efficacy of nurses and assistants for preventing falls.

AIM: This paper is a report of the development and testing of the Self-Efficacy for Preventing Falls Nurse and Assistant scales. BACKGROUND: Patient falls and fall-related injuries are traumatic ordeals for patients, family members and providers, and carry a toll for hospitals. Self-efficacy is an important factor in determining actions persons take and levels of performance they achieve. Performance of individual caregivers is linked to the overall performance of hospitals. Scales to assess nurses and certified nursing assistants' self-efficacy to prevent patients from falling would allow for targeting resources to increase SE, resulting in improved individual performance and ultimately decreased numbers of patient falls. METHOD: Four phases of instrument development were carried out to (1) generate individual items from eight focus groups (four each nurse and assistant conducted in October 2007), (2) develop prototype scales, (3) determine content validity during a second series of four nurse and assistant focus groups (January 2008) and (4) conduct item analysis, paired t-tests, Student's t-tests and internal consistency reliability to refine and confirm the scales. Data were collected during February-December, 2008. RESULTS: The 11-item Self-Efficacy for Preventing Falls Nurse had an alpha of 0·89 with all items in the range criterion of 0·3-0·7 for item total correlation. The 8-item Self-Efficacy for Preventing Falls Assistant had an alpha of 0·74 and all items had item total correlations in the 0·3-0·7 range. CONCLUSIONS: The Self-Efficacy for Preventing Falls Nurse and Self-Efficacy for Preventing Falls Assistant scales demonstrated psychometric adequacy and are recommended to measure bedside staff's self-efficacy beliefs in preventing patient falls.

Policy needs and options for a common transatlantic approach towards measuring adoption, usage and benefits of eHealth.

The European Union (EU) sponsored ARGOS project analysed current eHealth policy thinking in both the EU and the USA, compared strategic challenges and outcomes in selected fields, and drafted roadmaps towards developing advanced global approaches for these issues. This policy brief focuses on better understanding the benefits and costs of eHealth investments, assessing their overall socio-economic impact, identifying challenges and success factors, as well as measuring and globally benchmarking the concrete usage of eHealth solutions. These are by now key policy priorities not only of national governments and the European Commission, but also of international institutions like WHO or OECD. There is a strong felt transatlantic need for stocktaking, identifying lessons learned, sharing of experience, and working together to advance these issues for the benefit of health systems. A growing number of national and international activities can be taken advantage of. Recommendations on how to proceed with such transatlantic activities are proposed.

Summary of third annual MCBK public meeting: Mobilizing computable biomedical knowledge-Accelerating the second knowledge revolution.

The volume of biomedical knowledge is growing exponentially and much of this knowledge is represented in computer executable formats, such as models, algorithms, and programmatic code. There is a growing need to apply this knowledge to improve health in Learning Health Systems, health delivery organizations, and other settings. However, most organizations do not yet have the infrastructure required to consume and apply computable knowledge, and national policies and standards adoption are not sufficient to ensure that it is discoverable and used safely and fairly, nor is there widespread experience in the process of knowledge implementation as clinical decision support. The Mobilizing Computable Biomedical Knowledge (MCBK) community was formed in 2016 to address these needs. This report summarizes the main outputs of the third annual MCBK public meeting, which was held virtually from June 30 to July 1, 2020 and brought together over 200 participants from various domains to frame and address important dimensions for mobilizing CBK.

Complementary methods of system usability evaluation: surveys and observations during software design and development cycles.

Poor usability of clinical information systems delays their adoption by clinicians and limits potential improvements to the efficiency and safety of care. Recurring usability evaluations are therefore, integral to the system design process. We compared four methods employed during the development of outpatient clinical documentation software: clinician email response, online survey, observations and interviews. Results suggest that no single method identifies all or most problems. Rather, each approach is optimal for evaluations at a different stage of design and characterizes different usability aspect. Email responses elicited from clinicians and surveys report mostly technical, biomedical, terminology and control problems and are most effective when a working prototype has been completed. Observations of clinical work and interviews inform conceptual and workflow-related problems and are best performed early in the cycle. Appropriate use of these methods consistently during development may significantly improve system usability and contribute to higher adoption rates among clinicians and to improved quality of care.

Self-reported familiarity with acute respiratory infection guidelines and antibiotic prescribing in primary care.

OBJECTIVE: Familiarity with guidelines is generally thought to be associated with guideline implementation, adherence and improved quality of care. We sought to determine if self-reported familiarity with acute respiratory infection (ARI) antibiotic treatment guidelines was associated with reduced or more appropriate antibiotic prescribing for ARIs in primary care. PARTICIPANTS: and MAIN OUTCOME MEASURES: We surveyed primary care clinicians about their familiarity with ARI antibiotic treatment guidelines and linked responses to administrative diagnostic and prescribing data for non-pneumonia ARI visits. RESULTS: Sixty-five percent of clinicians responded to the survey question about guideline familiarity. There were 208 survey respondents who had ARI patient visits during the study period. Respondents reported being 'not at all' (7%), 'somewhat' (30%), 'moderately' (45%) or 'extremely' (18%) familiar with the guidelines. After dichotomizing responses, compared with clinicians who reported being less familiar with the guidelines, clinicians who reported being more familiar with the guidelines had higher rates of antibiotic prescribing for all ARIs combined (46% versus 38%; n = 11 164; P < 0.0001), for antibiotic-appropriate diagnoses (69% versus 59%; n = 3213; P < 0.0001) and for non-antibiotic appropriate diagnoses (38% versus 28%; n = 7951; P < 0.0001). After adjusting for potential confounders, self-reported guideline familiarity was an independent predictor of increased antibiotic prescribing (odds ratio, 1.36; 95% confidence interval, 1.25-1.48). CONCLUSIONS: Self-reported familiarity with an ARI antibiotic treatment guideline was, seemingly paradoxically, associated with increased antibiotic prescribing. Self-reported familiarity with guidelines should not be assumed to be associated with consistent guideline adherence or higher quality of care.

Fall prevention in acute care hospitals: a randomized trial.

CONTEXT: Falls cause injury and death for persons of all ages, but risk of falls increases markedly with age. Hospitalization further increases risk, yet no evidence exists to support short-stay hospital-based fall prevention strategies to reduce patient falls. OBJECTIVE: To investigate whether a fall prevention tool kit (FPTK) using health information technology (HIT) decreases patient falls in hospitals. DESIGN, SETTING, AND PATIENTS: Cluster randomized study conducted January 1, 2009, through June 30, 2009, comparing patient fall rates in 4 urban US hospitals in units that received usual care (4 units and 5104 patients) or the intervention (4 units and 5160 patients). INTERVENTION: The FPTK integrated existing communication and workflow patterns into the HIT application. Based on a valid fall risk assessment scale completed by a nurse, the FPTK software tailored fall prevention interventions to address patients' specific determinants of fall risk. The FPTK produced bed posters composed of brief text with an accompanying icon, patient education handouts, and plans of care, all communicating patient-specific alerts to key stakeholders. MAIN OUTCOME MEASURES: The primary outcome was patient falls per 1000 patient-days adjusted for site and patient care unit. A secondary outcome was fall-related injuries. RESULTS: During the 6-month intervention period, the number of patients with falls differed between control (n = 87) and intervention (n = 67) units (P=.02). Site-adjusted fall rates were significantly higher in control units (4.18 [95% confidence interval {CI}, 3.45-5.06] per 1000 patient-days) than in intervention units (3.15 [95% CI, 2.54-3.90] per 1000 patient-days; P = .04). The FPTK was found to be particularly effective with patients aged 65 years or older (adjusted rate difference, 2.08 [95% CI, 0.61-3.56] per 1000 patient-days; P = .003). No significant effect was noted in fall-related injuries. CONCLUSION: The use of a fall prevention tool kit in hospital units compared with usual care significantly reduced rate of falls. TRIAL REGISTRATION: clinicaltrials.gov Identifier: NCT00675935.

Identifying best practices for clinical decision support and knowledge management in the field.

To investigate best practices for implementing and managing clinical decision support (CDS) in community hospitals and ambulatory settings, we carried out a series of ethnographic studies to gather information from nine diverse organizations. Using the Rapid Assessment Process methodology, we conducted surveys, interviews, and observations over a period of two years in eight different geographic regions of the U.S.A. We first utilized a template organizing method for an expedited analysis of the data, followed by a deeper and more time consuming interpretive approach. We identified five major categories of best practices that require careful consideration while carrying out the planning, implementation, and knowledge management processes related to CDS. As more health care organizations implement clinical systems such as computerized provider order entry with CDS, descriptions of lessons learned by CDS pioneers can provide valuable guidance so that CDS can have optimal impact on health care quality.

Building and maintaining trust in clinical decision support: Recommendations from the Patient-Centered CDS Learning Network.

Knowledge artifacts in digital repositories for clinical decision support (CDS) can promote the use of CDS in clinical practice. However, stakeholders will benefit from knowing which they can trust before adopting artifacts from knowledge repositories. We discuss our investigation into trust for knowledge artifacts and repositories by the Patient-Centered CDS Learning Network's Trust Framework Working Group (TFWG). The TFWG identified 12 actors (eg, vendors, clinicians, and policy makers) within a CDS ecosystem who each may play a meaningful role in prioritizing, authoring, implementing, or evaluating CDS and developed 33 recommendations distributed across nine "trust attributes." The trust attributes and recommendations represent a range of considerations such as the "Competency" of knowledge artifact engineers and the "Organizational Capacity" of institutions that develop and implement CDS. The TFWG findings highlight an initial effort to make trust explicit and embedded within CDS knowledge artifacts and repositories and thus more broadly accepted and used.