Using a Natural Language Processing Approach to Support Rapid Knowledge Acquisition.

Implementing artificial intelligence to extract insights from large, real-world clinical data sets can supplement and enhance knowledge management efforts for health sciences research and clinical care. At Vanderbilt University Medical Center (VUMC), the in-house developed Word Cloud natural language processing system extracts coded concepts from patient records in VUMC's electronic health record repository using the Unified Medical Language System terminology. Through this process, the Word Cloud extracts the most prominent concepts found in the clinical documentation of a specific patient or population. The Word Cloud provides added value for clinical care decision-making and research. This viewpoint paper describes a use case for how the VUMC Center for Knowledge Management leverages the condition-disease associations represented by the Word Cloud to aid in the knowledge generation needed to inform the interpretation of phenome-wide association studies.

Knowledgebase strategies to aid interpretation of clinical correlation research.

OBJECTIVE: Knowledgebases are needed to clarify correlations observed in real-world electronic health record (EHR) data. We posit design principles, present a unifying framework, and report a test of concept. MATERIALS AND METHODS: We structured a knowledge framework along 3 axes: condition of interest, knowledge source, and taxonomy. In our test of concept, we used hypertension as our condition of interest, literature and VanderbiltDDx knowledgebase as sources, and phecodes as our taxonomy. In a cohort of 832 566 deidentified EHRs, we modeled blood pressure and heart rate by sex and age, classified individuals by hypertensive status, and ran a Phenome-wide Association Study (PheWAS) for hypertension. We compared the correlations from PheWAS to the associations in our knowledgebase. RESULTS: We produced PhecodeKbHtn: a knowledgebase comprising 167 hypertension-associated diseases, 15 of which were also negatively associated with blood pressure (pos+neg). Our hypertension PheWAS included 1914 phecodes, 129 of which were in the PhecodeKbHtn. Among the PheWAS association results, phecodes that were in PhecodeKbHtn had larger effect sizes compared with those phecodes not in the knowledgebase. DISCUSSION: Each source contributed unique and additive associations. Models of blood pressure and heart rate by age and sex were consistent with prior cohort studies. All but 4 PheWAS positive and negative correlations for phecodes in PhecodeKbHtn may be explained by knowledgebase associations, hypertensive cardiac complications, or causes of hypertension independently associated with hypotension. CONCLUSION: It is feasible to assemble a knowledgebase that is compatible with EHR data to aid interpretation of clinical correlation research.

Systematic review of international studies evaluating MDRD and CKD-EPI estimated glomerular filtration rate (eGFR) equations in Black adults.

Use of race adjustment in estimating glomerular filtration rate (eGFR) has been challenged given concerns that it may negatively impact the clinical care of Black patients, as it results in Black patients being systematically assigned higher eGFR values than non-Black patients. We conducted a systematic review to assess how well eGFR, with and without race adjustment, estimates measured GFR (mGFR) in Black adults globally. A search across multiple databases for articles published from 1999 to May 2021 that compared eGFR to mGFR and reported outcomes by Black race was performed. We included studies that assessed eGFR using the Modification of Diet in Renal Disease (MDRD) and Chronic Kidney Disease Epidemiology Collaboration (CKD-EPICr) creatinine equations. Risk of study bias and applicability were assessed with the QUality Assessment of Diagnostic Accuracy Studies-2. Of 13,167 citations identified, 12 met the data synthesis criteria (unique patient cohorts in which eGFR was compared to mGFR with and without race adjustment). The studies included patients with and without kidney disease from Africa (n = 6), the United States (n = 3), Europe (n = 2), and Brazil (n = 1). Of 11 CKD-EPI equation studies, all assessed bias, 8 assessed accuracy, 6 assessed precision, and 5 assessed correlation/concordance. Of 7 MDRD equation studies, all assessed bias, 6 assessed accuracy, 5 assessed precision, and 3 assessed correlation/concordance. The majority of studies found that removal of race adjustment improved bias, accuracy, and precision of eGFR equations for Black adults. Risk of study bias was often unclear, but applicability concerns were low. Our systematic review supports the need for future studies to be conducted in diverse populations to assess the possibility of alternative approaches for estimating GFR. This study additionally provides systematic-level evidence for the American Society of Nephrology-National Kidney Foundation Task Force efforts to pursue other options for GFR estimation.

Design and implementation of a massive open online course on enhancing the recruitment of minorities in clinical trials - Faster Together.

BACKGROUND: Racial and ethnic minorities are often underrepresented in clinical trials, threatening the generalizability of trial results. Several factors may contribute to underrepresentation of minorities in clinical trials, including lack of training for researchers and staff on the importance of diversity in clinical trials and effective strategies for recruiting and retaining minority populations. METHODS: Applying community engaged research principles, we developed a massive open online course (MOOC) to help research team members develop knowledge and skills to enhance the recruitment of minorities in clinical trials. A transdisciplinary working group, consisting of clinical researchers, community engagement specialists, minority clinical trial recruitment and retention educators and specialists, and knowledge management information scientists, was formed to develop an evidence-based curriculum. Feedback from the Recruitment Innovation Center Community Advisory Board was incorporated to help finalize the curriculum. The course was implemented in Coursera, an online learning platform offering MOOCs. A bootstrap paired sample t-test was used to compare pre- and post-assessments of knowledge, attitudes, and intentions as it relates to minority recruitment. RESULTS: The final course, entitled Faster Together, was divided into eight 1-h modules. Each module included video presentations, reading assignments, and quizzes. After 10 months, 382 individuals enrolled in the course, 105 participants completed the pre-test, and 14 participants completed the post-test. Participants' knowledge scores were higher with an increase in the mean number of correct answers from 15.4 (95% CI:12.1-18.7) on the pre-test to 18.7 (95% CI:17.42-20.2) on the post-test. All post-test respondents (n = 14) indicated that the course improved their professional knowledge, and 71.4% of respondents indicated that they were very likely to make changes to their recruitment practices. CONCLUSIONS: Faster Together, a massive open online course, is an acceptable, accessible approach to educating research teams on minority recruitment in clinical trials. Preliminary evidence indicates the course increased knowledge on how to recruit minorities into clinical trials and could promote change in their recruitment practices.

Collection of social determinants of health in the community clinic setting: a cross-sectional study.

BACKGROUND: Addressing social and behavioral determinants of health (SBDs) may help improve health outcomes of community clinic patients. This cross-sectional study explored how assessing SBDs can be used to complement health data collection strategies and provide clinicians with a more in-depth understanding of their patients. METHODS: Adult patients, ages 18 and older, at an urban community health care clinic in Tennessee, U.S.A., were asked to complete a questionnaire regarding health status, health history and SBDs while waiting for their clinic appointment. The SBD component included items from the National Academy of Medicine, the Protocol for Responding to and Assessing Patient Assets, Risks, and Experiences instrument, and the Survey of Household Economics and Decisionmaking. Data collection and analysis occurred in 2017. RESULTS: One hundred participants completed the study. The questionnaire took approximately 11 min to complete, and the response rate was 90% or higher for all items except annual household income (unanswered by 40 participants). The median number of negative SBDs was 4 (IQR 2.75-7.0), 96 participants had at least one unmet need, and the most common negative SBD was physical activity (75%; 75/100). CONCLUSIONS: The hybrid questionnaire provided insight into a community clinic population's SBDs and allowed for a more complete understanding than a single questionnaire alone. The brief questionnaire administration time and low non-response rate support the questionnaire's feasibility in the community clinic setting, and results can be used by clinicians to further the personalization goals of precision medicine. Next steps include evaluating how to connect patients with appropriate resources for addressing their SBDs.

Internet-Based Assessment of Oncology Health Care Professional Learning Style and Optimization of Materials for Web-Based Learning: Controlled Trial With Concealed Allocation.

BACKGROUND: Precision medicine has resulted in increasing complexity in the treatment of cancer. Web-based educational materials can help address the needs of oncology health care professionals seeking to understand up-to-date treatment strategies. OBJECTIVE: This study aimed to assess learning styles of oncology health care professionals and to determine whether learning style-tailored educational materials lead to enhanced learning. METHODS: In all, 21,465 oncology health care professionals were invited by email to participate in the fully automated, parallel group study. Enrollment and follow-up occurred between July 13 and September 7, 2015. Self-enrolled participants took a learning style survey and were assigned to the intervention or control arm using concealed alternating allocation. Participants in the intervention group viewed educational materials consistent with their preferences for learning (reading, listening, and/or watching); participants in the control group viewed educational materials typical of the My Cancer Genome website. Educational materials covered the topic of treatment of metastatic estrogen receptor-positive (ER+) breast cancer using cyclin-dependent kinases 4/6 (CDK4/6) inhibitors. Participant knowledge was assessed immediately before (pretest), immediately after (posttest), and 2 weeks after (follow-up test) review of the educational materials. Study statisticians were blinded to group assignment. RESULTS: A total of 751 participants enrolled in the study. Of these, 367 (48.9%) were allocated to the intervention arm and 384 (51.1%) were allocated to the control arm. Of those allocated to the intervention arm, 256 (69.8%) completed all assessments. Of those allocated to the control arm, 296 (77.1%) completed all assessments. An additional 12 participants were deemed ineligible and one withdrew. Of the 552 participants, 438 (79.3%) self-identified as multimodal learners. The intervention arm showed greater improvement in posttest score compared to the control group (0.4 points or 4.0% more improvement on average; P=.004) and a higher follow-up test score than the control group (0.3 points or 3.3% more improvement on average; P=.02). CONCLUSIONS: Although the study demonstrated more learning with learning style-tailored educational materials, the magnitude of increased learning and the largely multimodal learning styles preferred by the study participants lead us to conclude that future content-creation efforts should focus on multimodal educational materials rather than learning style-tailored content.

National Academy of Medicine Social and Behavioral Measures: Associations With Self-Reported Health.

INTRODUCTION: Social and behavioral factors play important roles in physical and mental health; however, they are not routinely assessed in the healthcare system. A brief panel of measures of social and behavioral determinants of health (SBDs) were recommended in a National Academy of Medicine report for use in electronic health records. Initial testing of the panel established feasibility of use and robustness of the measures. This study evaluates their convergent and divergent validity in relation to self-reported physical and mental health and social desirability bias. METHODS: Adults, aged ≥18 years, were recruited through Qualtrics online panel survey platform in 2015 (data analyzed in 2015-2016). Participants completed the (1) panel of SBD measures; (2) 12-Item Short Form Health Survey to assess associations with global physical and mental health; and (3) Marlowe-Crowne Social Desirability Scale short form to assess whether social desirability influenced associations between SBD measures and self-reported health. RESULTS: The sample included 513 participants (mean age, 47.9 [SD=14.2] years; 65.5% female). Several SBD domain measures were associated with physical and mental health. Adjusting for age, poorer physical and mental health were observed among participants reporting higher levels of financial resource strain, stress, depression, physical inactivity, current tobacco use, and a positive score for intimate partner violence. These associations remained significant after adjustment for social desirability bias. CONCLUSIONS: SBD domains were associated with global measures of physical and mental health and were not impacted by social desirability bias. The panel of SBD measures should now be tested in clinical settings.

Institute of Medicine Measures of Social and Behavioral Determinants of Health: A Feasibility Study.

INTRODUCTION: Social and behavioral factors are known to affect health but are not routinely assessed in medical practice. To date, no studies have assessed a parsimonious panel of measures of social and behavioral determinants of health (SBDs). This study evaluated the panel of SBD measures recommended by the Institute of Medicine and examined the effect of question order. METHODS: Adults, aged ≥18 years, were recruited using ResearchMatch.org for this randomized, parallel design study conducted in 2015 (data analyzed in 2015-2016). Three versions of the SBD measures, sharing the same items but in different orders of presentation (Versions 1-3), were developed. Randomized to six groups, participants completed each version at least 1 week apart (Weeks 1-3). Version order was counterbalanced across each administration and randomization was stratified by gender, race, and age. Main outcomes were effect of question order, completion time, and non-response rates. RESULTS: Of 781 participants, 624 (80%) completed the Week 1 questionnaire; median completion time for answering all SBD questions was 5 minutes, 583/624 participants answered all items, and no statistically significant differences associated with question order were observed when comparing responses across all versions. No significant differences in responses within assignment groups over time were found, with the exception of the stress measure for Group 5 (p=0.036). CONCLUSION: Question order did not significantly impact participant responses. Time to complete the questionnaire was brief, and non-response rate was low. Findings support the feasibility of using the Institute of Medicine-recommended questionnaire to capture SBDs.

My Cancer Genome: Evaluating an Educational Model to Introduce Patients and Caregivers to Precision Medicine Information.

This study tested an innovative model for creating consumer-level content about precision medicine based on health literacy and learning style principles. "Knowledge pearl" videos, incorporating multiple learning modalities, were created to explain genetic and cancer medicine concepts. Cancer patients and caregivers (n=117) were randomized to view professional-level content directly from the My Cancer Genome (MCG) website (Group A; control), content from MCG with knowledge pearls embedded (Group B), or a consumer translation, targeted at the sixth grade level, with knowledge pearls embedded (Group C). A multivariate analysis showed that Group C, but not Group B, showed greater knowledge gains immediately after viewing the educational material than Group A. Statistically significant group differences in test performance were no longer observed three weeks later. These findings suggest that adherence to health literacy and learning style principles facilitates comprehension of precision medicine concepts and that ongoing review of the educational information is necessary.

A personalized approach to deliver health care information to diabetic patients in community care clinics.

OBJECTIVE: The study's purpose was to test the generalizability of an individualized information prescription model, which has been previously validated for educating patients about hypertension in emergency department and community health center settings. Study investigators assessed the effects of educational materials targeted to health literacy levels and learning styles on patients' diabetes knowledge in a community clinic setting. METHODS: From May to August 2012, 160 patients were recruited and randomized into intervention (n = 81) and control (n = 79) groups. Inclusion criteria included 18 years or older, English or Spanish speaker, and a type 2 diabetes diagnosis. Measures included modified versions of the Diabetes Knowledge Test and Subjective Numeracy Scale, along with brief health literacy and learning style assessments. Study team members contacted both groups after 2 and 6 weeks to reassess diabetes knowledge. RESULTS: The control group showed no significant change in diabetes knowledge at both follow-ups. In contrast, the mean number of diabetes knowledge questions answered correctly by the intervention group increased significantly after 2 weeks (Δ = 2.66, P = 0.000), which persisted at 6 weeks (Δ = 2.46, P = 0.00). CONCLUSIONS: This study showed that patients' knowledge about diabetes increased significantly after exposure to educational materials targeted to their health literacy levels and learning style preferences and that the model is transferrable among health conditions.

Strategically aligning a mandala of competencies to advance a transformative vision.

OBJECTIVE: This paper offers insight into the processes that have shaped the Eskind Biomedical Library's (EBL's) strategic direction and its alignment to the institution's transformative vision. SETTING: The academic biomedical library has a notable track record for developing and pioneering roles for information professionals focused on a sophisticated level of information provision that draws from and fuels practice evolutions. STRATEGY: The medical center's overall transformative vision informs the creation of a fully aligned library strategic plan designed to effectively contribute to the execution of key organizational goals. Annual goals reflect organizational priorities and contain quantifiable and measurable deliverables. Two strategic themes, facilitating genetic literacy and preserving community history, are described in detail to illustrate the concept of goal setting. CONCLUSION: The strategic planning model reflects EBL's adaptation to the ever-changing needs of its organization. The paper provides a characterization of a workable model that can be replicated by other institutions.

Using health literacy and learning style preferences to optimize the delivery of health information.

Limited patient understanding of hypertension contributes to poor health outcomes. In 2 sequential randomized studies, the authors determined the impact of administering information tailored to health literacy level alone or in combination with preferred learning style on patients' understanding of hypertension. Patients with high blood pressure were recruited in an academic emergency department. In Experiment 1 (N = 85), the control group received only the routine discharge instructions; the intervention group received discharge instructions combined with information consistent with their health literacy level as determined by the Short Test of Functional Health Literacy. In Experiment 2 (N = 87), the information provided to the intervention group was tailored to both health literacy and learning style, as indicated by the VARK™ Questionnaire. To measure learning, the authors compared scores on a hypertension assessment administered during the emergency department visit and 2 weeks after discharge. Participants who received materials tailored to both health literacy level and learning style preference showed greater gains in knowledge than did those receiving information customized for health literacy level only. This study demonstrates that personalizing health information to learning style preferences and literacy level improves patient understanding of hypertension.

A pilot study to evaluate learning style-tailored information prescriptions for hypertensive emergency department patients.

OBJECTIVE: This pilot study explored whether learning style-tailored education materials, "information prescriptions," are effective in increasing hypertension knowledge in emergency room patients. METHODS: In a randomized trial, hypertensive emergency medicine patients received either standard care discharge instructions or discharge instructions in combination with an information prescription individualized to each patient's learning-style preference. Two weeks post-visit, the study team assessed changes in hypertension knowledge via a survey. RESULTS: No significant difference was observed for changes in quiz scores on the hypertension knowledge assessment, though patients receiving the tailored information prescriptions reported higher levels of satisfaction with intervention materials. CONCLUSION: The study demonstrated the workflow feasibility of implementing a learning-style approach to patient education in the emergency department setting. Further research is needed to develop more robust measures of high blood pressure knowledge among the emergency department patient population. This work will contribute to establishing a framework for developing customized information prescriptions that can be broadly adapted for use in varied settings and with varied health care conditions.

Integrating best evidence into patient care: a process facilitated by a seamless integration with informatics tools.

The Vanderbilt University paper discusses how the Eskind Biomedical Library at Vanderbilt University Medical Center transitioned from a simplistic approach that linked resources to the institutional electronic medical record system, StarPanel, to a value-added service that is designed to deliver highly relevant information. Clinical teams formulate complex patient-specific questions via an evidence-based medicine literature request basket linked to individual patient records. The paper transitions into discussing how the StarPanel approach acted as a springboard for two additional projects that use highly trained knowledge management librarians with informatics expertise to integrate evidence into both order sets and a patient portal, MyHealth@Vanderbilt.

A randomized effectiveness trial of a clinical informatics consult service: impact on evidence-based decision-making and knowledge implementation.

OBJECTIVE: To determine the effectiveness of providing synthesized research evidence to inform patient care practices via an evidence based informatics program, the Clinical Informatics Consult Service (CICS). DESIGN: Consults were randomly assigned to one of two conditions: CICS Provided, in which clinicians received synthesized information from the biomedical literature addressing the consult question or No CICS Provided, in which no information was provided. MEASUREMENT: Outcomes were measured via online post-consult forms that assessed consult purpose, actual and potential impact, satisfaction, time spent searching, and other variables. RESULTS: Two hundred twenty six consults were made during the 19-month study period. Clinicians primarily made requests in order to update themselves (65.0%, 147/226) and were satisfied with the service results (Mean 4.52 of possible 5.0, SD 0.94). Intention to treat (ITT) analyses showed that consults in the CICS Provided condition had a greater actual and potential impact on clinical actions and clinician satisfaction than No CICS consults. Evidence provided by the service primarily impacted the use of a new or different treatment (OR 8.19 95% CI 1.04-64.00). Reasons for no or little impact included a lack of evidence addressing the issue or that the clinician was already implementing the practices indicated by the evidence. CONCLUSIONS: Clinical decision-making, particularly regarding treatment issues, was statistically significantly impacted by the service. Programs such as the CICS may provide an effective tool for facilitating the integration of research evidence into the management of complex patient care and may foster clinicians' engagement with the biomedical literature.

Evolution of a mature clinical informationist model.

Achieving evidence-based practice will require new approaches to providing information during health care delivery and to integrating evidence and informatics at the point of care. To support evidence-based practice, Vanderbilt University Medical Center's Eskind Biomedical Library (EBL) introduced the role of clinical informationist, an information specialist with sufficient knowledge and insight to function as a true partner in the health care team. To further disseminate evidence-based knowledge, the Vanderbilt University Medical Center's (VUMC) electronic medical record system and pathway development processes integrate advanced information synthesis capabilities provided by clinical informationists. Combining clinical informationist expertise with informatics tools is an effective strategy for delivering the evidence needed to support patient care decisions.

Benchmarking information needs and use in the Tennessee public health community.

OBJECTIVE: The objective is to provide insight to understanding public health officials' needs and promote access to data repositories and communication tools. METHODS: Survey questions were identified by a focus group with members drawn from the fields of librarianship, public health, and informatics. The resulting comprehensive information needs survey, organized in five distinct broad categories, was distributed to 775 Tennessee public health workers from ninety-five counties in 1999 as part of the National Library of Medicine-funded Partners in Information Access contract. RESULTS: The assessment pooled responses from 571 public health workers (73% return rate) representing seventy-two of ninety-five counties (53.4% urban and 46.6% rural) about their information-seeking behaviors, frequency of resources used, computer skills, and level of Internet access. Sixty-four percent of urban and 43% of rural respondents had email access at work and more than 50% of both urban and rural respondents had email at home (N = 289). Approximately 70% of urban and 78% of rural public health officials never or seldom used or needed the Centers for Disease Control (CDC) Website. Frequency data pooled from eleven job categories representing a subgroup of 232 health care professionals showed 72% never or seldom used or needed MEDLINE. Electronic resources used daily or weekly were email, Internet search engines, internal databases and mailing lists, and the Tennessee Department of Health Website. CONCLUSIONS: While, due to the small sample size, data cannot be generalized to the larger population, a clear trend of significant barriers to computer and Internet access can be identified across the public health community. This contributes to an overall limited use of existing electronic resources that inhibits evidence-based practice.

Print versus electronic journals: a preliminary investigation into the effect of journal format on research processes.

PURPOSE: To begin investigating the impact of electronic journals on research processes such as information seeking, the authors conducted a pilot journal-use study to test the hypothesis that patrons use print and electronic journals differently. METHODOLOGY: We placed fifteen high-use print titles also available in electronic format behind the circulation desk; patrons were asked to complete a survey upon requesting a journal. We also conducted a parallel survey of patrons using library computers. Both surveys asked patrons to identify themselves by user category and queried them about their journal use. RESULTS: During the month-long study, patrons completed sixty-nine surveys of electronic and ninety surveys of print journal use. Results analysis indicated that fellows, students, and residents preferred electronic journals, and faculty preferred print journals. Patrons used print journals for reading articles and scanning contents; they employed electronic journals for printing articles and checking references. Users considered electronic journals easier to access and search than print journals; however, they reported that print journals had higher quality text and figures. DISCUSSION/CONCLUSION: This study is an introductory step in examining how electronic journals affect research processes. Our data revealed that there were distinct preferences in format among categories. In addition to collection management implications for libraries, these data also have implications for publishers and educators; current electronic formats do not facilitate all types of uses and thus may be changing learning patterns as well.