Data Science Curriculum in the iField.

Many disciplines, including the broad Field of Information (iField), have been offering Data Science (DS) programs. There have been significant efforts exploring an individual discipline's identity and unique contributions to the broader DS education landscape. To advance DS education in the iField, the iSchool Data Science Curriculum Committee (iDSCC) was formed and charged with building and recommending a DS education framework for iSchools. This paper reports on the research process and findings of a series of studies to address important questions: What is the iField identity in the multidisciplinary DS education landscape? What is the status of DS education in iField schools? What knowledge and skills should be included in the core curriculum for iField DS education? What are the jobs available for DS graduates from the iField? What are the differences between graduate-level and undergraduate-level DS education? Answers to these questions will not only distinguish an iField approach to DS education but also define critical components of DS curriculum. The results will inform individual DS programs in the iField to develop curriculum to support undergraduate and graduate DS education in their local context.

Health Recommender Systems Development, Usage, and Evaluation from 2010 to 2022: A Scoping Review.

A health recommender system (HRS) provides a user with personalized medical information based on the user's health profile. This scoping review aims to identify and summarize the HRS development in the most recent decade by focusing on five key aspects: health domain, user, recommended item, recommendation technology, and system evaluation. We searched PubMed, ACM Digital Library, IEEE Xplore, Web of Science, and Scopus databases for English literature published between 2010 and 2022. Our study selection and data extraction followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews. The following are the primary results: sixty-three studies met the eligibility criteria and were included in the data analysis. These studies involved twenty-four health domains, with both patients and the general public as target users and ten major recommended items. The most adopted algorithm of recommendation technologies was the knowledge-based approach. In addition, fifty-nine studies reported system evaluations, in which two types of evaluation methods and three categories of metrics were applied. However, despite existing research progress on HRSs, the health domains, recommended items, and sample size of system evaluation have been limited. In the future, HRS research shall focus on dynamic user modelling, utilizing open-source knowledge bases, and evaluating the efficacy of HRSs using a large sample size. In conclusion, this study summarized the research activities and evidence pertinent to HRSs in the most recent ten years and identified gaps in the existing research landscape. Further work shall address the gaps and continue improving the performance of HRSs to empower users in terms of healthcare decision making and self-management.

Cohort Identification from Free-Text Clinical Notes Using SNOMED CT's Hierarchical Semantic Relations.

In this paper, a new cohort identification system that exploits the semantic hierarchy of SNOMED CT is proposed to overcome the limitations of supervised machine learning-based approaches. Eligibility criteria descriptions and free-text clinical notes from the 2018 National NLP Clinical Challenge (n2c2) were processed to map to relevant SNOMED CT concepts and to measure semantic similarity between the eligibility criteria and patients. The eligibility of a patient was determined if the patient had a similarity score higher than a threshold cut-off value. The performance of the proposed system was evaluated for three eligibility criteria. The performance of the current system exceeded the previously reported results of the 2018 n2c2, achieving the average F1 score of 0.933. This study demonstrated that SNOMED CT alone can be leveraged for cohort identification tasks without referring to external textual sources for training.

A systematic review of automatic text summarization for biomedical literature and EHRs.

OBJECTIVE: Biomedical text summarization helps biomedical information seekers avoid information overload by reducing the length of a document while preserving the contents' essence. Our systematic review investigates the most recent biomedical text summarization researches on biomedical literature and electronic health records by analyzing their techniques, areas of application, and evaluation methods. We identify gaps and propose potential directions for future research. MATERIALS AND METHODS: This review followed the PRISMA methodology and replicated the approaches adopted by the previous systematic review published on the same topic. We searched 4 databases (PubMed, ACM Digital Library, Scopus, and Web of Science) from January 1, 2013 to April 8, 2021. Two reviewers independently screened title, abstract, and full-text for all retrieved articles. The conflicts were resolved by the third reviewer. The data extraction of the included articles was in 5 dimensions: input, purpose, output, method, and evaluation. RESULTS: Fifty-eight out of 7235 retrieved articles met the inclusion criteria. Thirty-nine systems used single-document biomedical research literature as their input, 17 systems were explicitly designed for clinical support, 47 systems generated extractive summaries, and 53 systems adopted hybrid methods combining computational linguistics, machine learning, and statistical approaches. As for the assessment, 51 studies conducted an intrinsic evaluation using predefined metrics. DISCUSSION AND CONCLUSION: This study found that current biomedical text summarization systems have achieved good performance using hybrid methods. Studies on electronic health records summarization have been increasing compared to a previous survey. However, the majority of the works still focus on summarizing literature.

The use of SNOMED CT, 2013-2020: a literature review.

OBJECTIVE: This article reviews recent literature on the use of SNOMED CT as an extension of Lee et al's 2014 review on the same topic. The Lee et al's article covered literature published from 2001-2012, and the scope of this review was 2013-2020. MATERIALS AND METHODS: In line with Lee et al's methods, we searched the PubMed and Embase databases and identified 1002 articles for review, including studies from January 2013 to September 2020. The retrieved articles were categorized and analyzed according to SNOMED CT focus categories (ie, indeterminate, theoretical, pre-development, implementation, and evaluation/commodity), usage categories (eg, illustrate terminology systems theory, prospective content coverage, used to classify or code in a study, retrieve or analyze patient data, etc.), medical domains, and countries. RESULTS: After applying inclusion and exclusion criteria, 622 articles were selected for final review. Compared to the papers published between 2001 and 2012, papers published between 2013 and 2020 revealed an increase in more mature usage of SNOMED CT, and the number of papers classified in the "implementation" and "evaluation/commodity" focus categories expanded. When analyzed by decade, papers in the "pre-development," "implementation," and "evaluation/commodity" categories were much more numerous in 2011-2020 than in 2001-2010, increasing from 169 to 293, 30 to 138, and 3 to 65, respectively. CONCLUSION: Published papers in more mature usage categories have substantially increased since 2012. From 2013 to present, SNOMED CT has been increasingly implemented in more practical settings. Future research should concentrate on addressing whether SNOMED CT influences improvement in patient care.

Machine Learning and Medication Adherence: Scoping Review.

BACKGROUND: This is the first scoping review to focus broadly on the topics of machine learning and medication adherence. OBJECTIVE: This review aims to categorize, summarize, and analyze literature focused on using machine learning for actions related to medication adherence. METHODS: PubMed, Scopus, ACM Digital Library, IEEE, and Web of Science were searched to find works that meet the inclusion criteria. After full-text review, 43 works were included in the final analysis. Information of interest was systematically charted before inclusion in the final draft. Studies were placed into natural categories for additional analysis dependent upon the combination of actions related to medication adherence. The protocol for this scoping review was created using the PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses Extension for Scoping Reviews) guidelines. RESULTS: Publications focused on predicting medication adherence have uncovered 20 strong predictors that were significant in two or more studies. A total of 13 studies that predicted medication adherence used either self-reported questionnaires or pharmacy claims data to determine medication adherence status. In addition, 13 studies that predicted medication adherence did so using either logistic regression, artificial neural networks, random forest, or support vector machines. Of the 15 studies that predicted medication adherence, 6 reported predictor accuracy, the lowest of which was 77.6%. Of 13 monitoring systems, 12 determined medication administration using medication container sensors or sensors in consumer electronics, like smartwatches or smartphones. A total of 11 monitoring systems used logistic regression, artificial neural networks, support vector machines, or random forest algorithms to determine medication administration. The 4 systems that monitored inhaler administration reported a classification accuracy of 93.75% or higher. The 2 systems that monitored medication status in patients with Parkinson disease reported a classification accuracy of 78% or higher. A total of 3 studies monitored medication administration using only smartwatch sensors and reported a classification accuracy of 78.6% or higher. Two systems that provided context-aware medication reminders helped patients to achieve an adherence level of 92% or higher. Two conversational artificial intelligence reminder systems significantly improved adherence rates when compared against traditional reminder systems. CONCLUSIONS: Creation of systems that accurately predict medication adherence across multiple data sets may be possible due to predictors remaining strong across multiple studies. Higher quality measures of adherence should be adopted when possible so that prediction algorithms are based on accurate information. Currently, medication adherence can be predicted with a good level of accuracy, potentially allowing for the development of interventions aimed at preventing nonadherence. Monitoring systems that track inhaler use currently classify inhaler-related actions with an excellent level of accuracy, allowing for tracking of adherence and potentially proper inhaler technique. Systems that monitor medication states in patients with Parkinson disease can currently achieve a good level of classification accuracy and have the potential to inform medication therapy changes in the future. Medication administration monitoring systems that only use motion sensors in smartwatches can currently achieve a good level of classification accuracy but only when differentiating between a small number of possible activities. Context-aware reminder systems can help patients achieve high levels of medication adherence but are also intrusive, which may not be acceptable to users. Conversational artificial intelligence reminder systems can significantly improve adherence.

Research evidence on strategies enabling integration of electronic health records in the health care systems of low- and middle-income countries: A literature review.

Integration of electronic health records (EHRs) in the national health care systems of low- and middle-income countries (LMICs) is vital for achieving the United Nations Sustainable Development Goal of ensuring healthy lives and promoting well-being for all people of all ages. National EHR systems are increasing, but mostly in developed countries. Besides, there is limited research evidence on successful strategies for ensuring integration of national EHRs in the health care systems of LMICs. To fill this evidence gap, a comprehensive survey of literature was conducted using scientific electronic databases-PubMed, SCOPUS, Web of Science, and Global Health-and consultations with international experts. The review highlights the lack of evidence on strategies for integrating EHR systems, although there was ample evidence on implementation challenges and relevance of EHRs to vertical disease programs such as HIV. The findings describe the narrow focus of EHR implementation, the prominence of vertical disease programs in EHR adoption, testing of theoretical and conceptual models for EHR implementation and success, and strategies for EHR implementation. The review findings are further amplified through examples of EHR implementation in Sierra Leone, Malawi, and India. Unless evidence-based strategies are identified and applied, integration of national EHRs in the health care systems of LMICs is difficult.

Federated health information architecture: Enabling healthcare providers and policymakers to use data for decision-making.

Health information systems (HIS) in India, as in most other developing countries, support public health management but fail to enable healthcare providers to use data for delivering quality services. Such a failure is surprising, given that the population healthcare data that the system collects are aggregated from patient records. An important reason for this failure is that the health information architecture (HIA) of the HIS is designed primarily to serve the information needs of policymakers and program managers. India has recognised the architectural gaps in its HIS and proposes to develop an integrated HIA. An enabling HIA that attempts to balance the autonomy of local systems with the requirements of a centralised monitoring agency could meet the diverse information needs of various stakeholders. Given the lack of in-country knowledge and experience in designing such an HIA, this case study was undertaken to analyse HIS in the Bihar state of India and to understand whether it would enable healthcare providers, program managers and policymakers to use data for decision-making. Based on a literature review and data collected from interviews with key informants, this article proposes a federated HIA, which has the potential to improve HIS efficiency; provide flexibility for local innovation; cater to the diverse information needs of healthcare providers, program managers and policymakers; and encourage data-based decision-making.

Physicians' perception of alternative displays of clinical research evidence for clinical decision support - A study with case vignettes.

OBJECTIVE: To design alternate information displays that present summaries of clinical trial results to clinicians to support decision-making; and to compare the displays according to efficacy and acceptability. METHODS: A 6-between (information display presentation order) by 3-within (display type) factorial design. Two alternate displays were designed based on Information Foraging theory: a narrative summary that reduces the content to a few sentences; and a table format that structures the display according to the PICO (Population, Intervention, Comparison, Outcome) framework. The designs were compared with the summary display format available in PubMed. Physicians were asked to review five clinical studies retrieved for a case vignette; and were presented with the three display formats. Participants were asked to rate their experience with each of the information displays according to a Likert scale questionnaire. RESULTS: Twenty physicians completed the study. Overall, participants rated the table display more highly than either the text summary or PubMed's summary format (5.9vs. 5.4vs. 3.9 on a scale between 1 [strongly disagree] and 7 [strongly agree]). Usefulness ratings of seven pieces of information, i.e. patient population, patient age range, sample size, study arm, primary outcome, results of primary outcome, and conclusion, were high (average across all items=4.71 on a 1 to 5 scale, with 1=not at all useful and 5=very useful). Study arm, primary outcome, and conclusion scored the highest (4.9, 4.85, and 4.85 respectively). Participants suggested additional details such as rate of adverse effects. CONCLUSION: The table format reduced physicians' perceived cognitive effort when quickly reviewing clinical trial information and was more favorably received by physicians than the narrative summary or PubMed's summary format display.

Formative evaluation of a patient-specific clinical knowledge summarization tool.

OBJECTIVE: To iteratively design a prototype of a computerized clinical knowledge summarization (CKS) tool aimed at helping clinicians finding answers to their clinical questions; and to conduct a formative assessment of the usability, usefulness, efficiency, and impact of the CKS prototype on physicians' perceived decision quality compared with standard search of UpToDate and PubMed. MATERIALS AND METHODS: Mixed-methods observations of the interactions of 10 physicians with the CKS prototype vs. standard search in an effort to solve clinical problems posed as case vignettes. RESULTS: The CKS tool automatically summarizes patient-specific and actionable clinical recommendations from PubMed (high quality randomized controlled trials and systematic reviews) and UpToDate. Two thirds of the study participants completed 15 out of 17 usability tasks. The median time to task completion was less than 10s for 12 of the 17 tasks. The difference in search time between the CKS and standard search was not significant (median=4.9 vs. 4.5m in). Physician's perceived decision quality was significantly higher with the CKS than with manual search (mean=16.6 vs. 14.4; p=0.036). CONCLUSIONS: The CKS prototype was well-accepted by physicians both in terms of usability and usefulness. Physicians perceived better decision quality with the CKS prototype compared to standard search of PubMed and UpToDate within a similar search time. Due to the formative nature of this study and a small sample size, conclusions regarding efficiency and efficacy are exploratory.

Feasibility of Extracting Key Elements from ClinicalTrials.gov to Support Clinicians' Patient Care Decisions.

Motivation: Clinicians need up-to-date evidence from high quality clinical trials to support clinical decisions. However, applying evidence from the primary literature requires significant effort. Objective: To examine the feasibility of automatically extracting key clinical trial information from ClinicalTrials.gov. Methods: We assessed the coverage of ClinicalTrials.gov for high quality clinical studies that are indexed in PubMed. Using 140 random ClinicalTrials.gov records, we developed and tested rules for the automatic extraction of key information. Results: The rate of high quality clinical trial registration in ClinicalTrials.gov increased from 0.2% in 2005 to 17% in 2015. Trials reporting results increased from 3% in 2005 to 19% in 2015. The accuracy of the automatic extraction algorithm for 10 trial attributes was 90% on average. Future research is needed to improve the algorithm accuracy and to design information displays to optimally present trial information to clinicians.

Emergency Medical Text Classifier: New system improves processing and classification of triage notes.

OBJECTIVE: Automated syndrome classification aims to aid near real-time syndromic surveillance to serve as an early warning system for disease outbreaks, using Emergency Department (ED) data. We present a system that improves the automatic classification of an ED record with triage note into one or more syndrome categories using the vector space model coupled with a 'learning' module that employs a pseudo-relevance feedback mechanism. MATERIALS AND METHODS: Terms from standard syndrome definitions are used to construct an initial reference dictionary for generating the syndrome and triage note vectors. Based on cosine similarity between the vectors, each record is classified into a syndrome category. We then take terms from the top-ranked records that belong to the syndrome of interest as feedback. These terms are added to the reference dictionary and the process is repeated to determine the final classification. The system was tested on two different datasets for each of three syndromes: Gastro-Intestinal (GI), Respiratory (Resp) and Fever-Rash (FR). Performance was measured in terms of sensitivity (Se) and specificity (Sp). RESULTS: The use of relevance feedback produced high values of sensitivity and specificity for all three syndromes in both test sets: GI: 90% and 71%, Resp: 97% and 73%, FR: 100% and 87%, respectively, in test set 1, and GI: 88% and 69%, Resp: 87% and 61%, FR: 97% and 71%, respectively, in test set 2. CONCLUSIONS: The new system for pre-processing and syndromic classification of ED records with triage notes achieved improvements in Se and Sp. Our results also demonstrate that the system can be tuned to achieve different levels of performance based on user requirements.

Text summarization in the biomedical domain: a systematic review of recent research.

OBJECTIVE: The amount of information for clinicians and clinical researchers is growing exponentially. Text summarization reduces information as an attempt to enable users to find and understand relevant source texts more quickly and effortlessly. In recent years, substantial research has been conducted to develop and evaluate various summarization techniques in the biomedical domain. The goal of this study was to systematically review recent published research on summarization of textual documents in the biomedical domain. MATERIALS AND METHODS: MEDLINE (2000 to October 2013), IEEE Digital Library, and the ACM digital library were searched. Investigators independently screened and abstracted studies that examined text summarization techniques in the biomedical domain. Information is derived from selected articles on five dimensions: input, purpose, output, method and evaluation. RESULTS: Of 10,786 studies retrieved, 34 (0.3%) met the inclusion criteria. Natural language processing (17; 50%) and a hybrid technique comprising of statistical, Natural language processing and machine learning (15; 44%) were the most common summarization approaches. Most studies (28; 82%) conducted an intrinsic evaluation. DISCUSSION: This is the first systematic review of text summarization in the biomedical domain. The study identified research gaps and provides recommendations for guiding future research on biomedical text summarization. CONCLUSION: Recent research has focused on a hybrid technique comprising statistical, language processing and machine learning techniques. Further research is needed on the application and evaluation of text summarization in real research or patient care settings.

Anonymous Record Linkage Between EPR and CDW-H: Toward Development of a Federated Genotype-Phenotype System.

Environmental Polymorphisms Registry (EPR) is a large-scale phenotype-by-genotype registry developed by National Institute of Environmental Health Sciences to facilitate translational research. The link between personal identity and collected genomic data was preserved in EPR which creates opportunities for EPR to be linked to phenotype-rich databases, such as the Carolina Data Warehouse for Health (CDW-H) located at the University of North Carolina hospital system. CDW-H contains clinically-relevant data for patients who have been admitted to UNC healthcare system. To validate the feasibility of linking EPR with CDWH, the number of matching records between the two databases had to be established. To that end, combinations of subjects' demographic identifiers from both databases were converted to anonymized hash codes, which were then matched to determine the number of overlapping records. Preliminary results showed that combination of last name, gender, data of birth and zip code would generate over 2,700 matches between the two databases.

Automatic identification of comparative effectiveness research from medline citations to support clinicians' treatment information needs.

UNLABELLED: Online knowledge resources such as Medline can address most clinicians' patient care information needs. Yet, significant barriers, notably lack of time, limit the use of these sources at the point of care. The most common information needs raised by clinicians are treatment-related. Comparative effectiveness studies allow clinicians to consider multiple treatment alternatives for a particular problem. Still, solutions are needed to enable efficient and effective consumption of comparative effectiveness research at the point of care. OBJECTIVE: Design and assess an algorithm for automatically identifying comparative effectiveness studies and extracting the interventions investigated in these studies. METHODS: The algorithm combines semantic natural language processing, Medline citation metadata, and machine learning techniques. We assessed the algorithm in a case study of treatment alternatives for depression. RESULTS: Both precision and recall for identifying comparative studies was 0.83. A total of 86% of the interventions extracted perfectly or partially matched the gold standard. CONCLUSION: Overall, the algorithm achieved reasonable performance. The method provides building blocks for the automatic summarization of comparative effectiveness research to inform point of care decision-making.

The secure medical research workspace: an IT infrastructure to enable secure research on clinical data.

Clinical data have tremendous value for translational research, but only if security and privacy concerns can be addressed satisfactorily. A collaboration of clinical and informatics teams, including RENCI, NC TraCS, UNC's School of Information and Library Science, Information Technology Service's Research Computing and other partners at the University of North Carolina at Chapel Hill have developed a system called the Secure Medical Research Workspace (SMRW) that enables researchers to use clinical data securely for research. SMRW significantly minimizes the risk presented when using identified clinical data, thereby protecting patients, researchers, and institutions associated with the data. The SMRW is built on a novel combination of virtualization and data leakage protection and can be combined with other protection methodologies and scaled to production levels.

Automatically extracting sentences from Medline citations to support clinicians' information needs.

OBJECTIVE: Online health knowledge resources contain answers to most of the information needs raised by clinicians in the course of care. However, significant barriers limit the use of these resources for decision-making, especially clinicians' lack of time. In this study we assessed the feasibility of automatically generating knowledge summaries for a particular clinical topic composed of relevant sentences extracted from Medline citations. METHODS: The proposed approach combines information retrieval and semantic information extraction techniques to identify relevant sentences from Medline abstracts. We assessed this approach in two case studies on the treatment alternatives for depression and Alzheimer's disease. RESULTS: A total of 515 of 564 (91.3%) sentences retrieved in the two case studies were relevant to the topic of interest. About one-third of the relevant sentences described factual knowledge or a study conclusion that can be used for supporting information needs at the point of care. CONCLUSIONS: The high rate of relevant sentences is desirable, given that clinicians' lack of time is one of the main barriers to using knowledge resources at the point of care. Sentence rank was not significantly associated with relevancy, possibly due to most sentences being highly relevant. Sentences located closer to the end of the abstract and sentences with treatment and comparative predications were likely to be conclusive sentences. Our proposed technical approach to helping clinicians meet their information needs is promising. The approach can be extended for other knowledge resources and information need types.

Implementation of Emergency Medical Text Classifier for syndromic surveillance.

Public health officials use syndromic surveillance systems to facilitate early detection and response to infectious disease outbreaks. Emergency department clinical notes are becoming more available for surveillance but present the challenge of accurately extracting concepts from these text data. The purpose of this study was to implement a new system, Emergency Medical Text Classifier (EMT-C), into daily production for syndromic surveillance and evaluate system performance and user satisfaction. The system was designed to meet user preferences for a syndromic classifier that maximized positive predictive value and minimized false positives in order to provide a manageable workload. EMT-C performed better than the baseline system on all metrics and users were slightly more satisfied with it. It is vital to obtain user input and test new systems in the production environment.

Patient electronic health data-driven approach to clinical decision support.

This article presents a novel visual analytics (VA)-based clinical decision support (CDS) tool prototype that was designed as a collaborative work between Renaissance Computing Institute and Duke University. Using Major Depressive Disorder data from MindLinc electronic health record system at Duke, the CDS tool shows an approach to leverage data from comparative population (patients with similar medical profile) to enhance a clinicians' decision making process at the point of care. The initial work is being extended in collaboration with the University of North Carolina CTSA to address the key challenges of CDS, as well as to show the use of VA to derive insight from large volumes of Electronic Health Record patient data.