Artificial intelligence in nursing: Priorities and opportunities from an international invitational think-tank of the Nursing and Artificial Intelligence Leadership Collaborative.

AIM: To develop a consensus paper on the central points of an international invitational think-tank on nursing and artificial intelligence (AI). METHODS: We established the Nursing and Artificial Intelligence Leadership (NAIL) Collaborative, comprising interdisciplinary experts in AI development, biomedical ethics, AI in primary care, AI legal aspects, philosophy of AI in health, nursing practice, implementation science, leaders in health informatics practice and international health informatics groups, a representative of patients and the public, and the Chair of the ITU/WHO Focus Group on Artificial Intelligence for Health. The NAIL Collaborative convened at a 3-day invitational think tank in autumn 2019. Activities included a pre-event survey, expert presentations and working sessions to identify priority areas for action, opportunities and recommendations to address these. In this paper, we summarize the key discussion points and notes from the aforementioned activities. IMPLICATIONS FOR NURSING: Nursing's limited current engagement with discourses on AI and health posts a risk that the profession is not part of the conversations that have potentially significant impacts on nursing practice. CONCLUSION: There are numerous gaps and a timely need for the nursing profession to be among the leaders and drivers of conversations around AI in health systems. IMPACT: We outline crucial gaps where focused effort is required for nursing to take a leadership role in shaping AI use in health systems. Three priorities were identified that need to be addressed in the near future: (a) Nurses must understand the relationship between the data they collect and AI technologies they use; (b) Nurses need to be meaningfully involved in all stages of AI: from development to implementation; and (c) There is a substantial untapped and an unexplored potential for nursing to contribute to the development of AI technologies for global health and humanitarian efforts.

Pain process of patients with cardiac surgery-Semantic annotation of electronic patient record data.

AIMS AND OBJECTIVES: To describe and compare the pain process of the patients' with cardiac surgery through nurses' and physicians' documentations in the electronic patient records. BACKGROUND: Postoperative pain assessment and management should be documented regularly, to ensure optimal pain care process for patients. Despite availability of evidence-based guidelines, pain assessment and documentation remain inadequate. DESIGN: A retrospective patients' record review. METHODS: The original data consisted of the electronic patient records of 26,922 patients with a diagnosed heart disease. A total of 1,818 care episodes of patients with cardiac surgery were selected from the data. We used random sampling to obtain 280 care episodes for annotation. These 280 care episodes contained 2,156 physician reports and 1,327 days of nursing notes. We developed an annotation manual and schema, and then, we manually conducted semantic annotation on care episodes, using the Brat annotation tool. We analysed the annotation units using thematic analysis. Consolidated criteria for reporting qualitative research guideline was followed in reporting where appropriate in this study design. RESULTS: We discovered expressions of six different aspects of pain process: (a) cause, (b) situation, (c) features, (d) consequences, (e) actions and (f) outcomes. We determined that five of the aspects existed chronologically. However, the features of pain were simultaneously existing. They indicated the location, quality, intensity, and temporality of the pain and they were present in every phase of the patient's pain process. Cardiac and postoperative pain documentations differed from each other in used expressions and in the quantity and quality of descriptions. CONCLUSION: We could construct a comprehensive pain process of the patients with cardiac surgery from several electronic patient records. The challenge remains how to support systematic documentation in each patient. RELEVANCE TO CLINICAL PRACTICE: The study provides knowledge and guidance of pain process aspects that can be used to achieve an effective pain assessment and more comprehensive documentation.

Ontology Development for Patient Education Documents Using a Professional- and Patient-Oriented Delphi Method.

Written patient education materials are essential to motivate and help patients to participate in their own care, but the production and management of a large collection of high-quality and easily accessible patient education documents can be challenging. Ontologies can aid in these tasks, but the existing resources are not directly applicable to patient education. An ontology that models patient education documents and their readers was constructed. The Delphi method was used to identify a compact but sufficient set of entities with which the topics of documents may be described. The preferred terms of the entities were also considered to ensure their understandability. In the ontology, readers may be characterized by gender, age group, language, and role (patient or professional), whereas documents may be characterized by audience, topic(s), and content, as well as the time and place of use. The Delphi method yielded 265 unique document topics that are organized into seven hierarchies. Advantages and disadvantages of the ontology design, as well as possibilities for improvements, were identified. The patient education material ontology can enhance many applications, but further development is needed to reach its full potential.

TEES 2.2: Biomedical Event Extraction for Diverse Corpora.

BACKGROUND: The Turku Event Extraction System (TEES) is a text mining program developed for the extraction of events, complex biomedical relationships, from scientific literature. Based on a graph-generation approach, the system detects events with the use of a rich feature set built via dependency parsing. The TEES system has achieved record performance in several of the shared tasks of its domain, and continues to be used in a variety of biomedical text mining tasks. RESULTS: The TEES system was quickly adapted to the BioNLP'13 Shared Task in order to provide a public baseline for derived systems. An automated approach was developed for learning the underlying annotation rules of event type, allowing immediate adaptation to the various subtasks, and leading to a first place in four out of eight tasks. The system for the automated learning of annotation rules is further enhanced in this paper to the point of requiring no manual adaptation to any of the BioNLP'13 tasks. Further, the scikit-learn machine learning library is integrated into the system, bringing a wide variety of machine learning methods usable with TEES in addition to the default SVM. A scikit-learn ensemble method is also used to analyze the importances of the features in the TEES feature sets. CONCLUSIONS: The TEES system was introduced for the BioNLP'09 Shared Task and has since then demonstrated good performance in several other shared tasks. By applying the current TEES 2.2 system to multiple corpora from these past shared tasks an overarching analysis of the most promising methods and possible pitfalls in the evolving field of biomedical event extraction are presented.

Application of the EVEX resource to event extraction and network construction: Shared Task entry and result analysis.

BACKGROUND: Modern methods for mining biomolecular interactions from literature typically make predictions based solely on the immediate textual context, in effect a single sentence. No prior work has been published on extending this context to the information automatically gathered from the whole biomedical literature. Thus, our motivation for this study is to explore whether mutually supporting evidence, aggregated across several documents can be utilized to improve the performance of the state-of-the-art event extraction systems. RESULTS: In the GE task, our re-ranking approach led to a modest performance increase and resulted in the first rank of the official Shared Task results with 50.97% F-score. Additionally, in this paper we explore and evaluate the usage of distributed vector representations for this challenge. CONCLUSIONS: For the GRN task, we were able to produce a gene regulatory network from the EVEX data, warranting the use of such generic large-scale text mining data in network biology settings. A detailed performance and error analysis provides more insight into the relatively low recall rates.

Care episode retrieval: distributional semantic models for information retrieval in the clinical domain.

Patients' health related information is stored in electronic health records (EHRs) by health service providers. These records include sequential documentation of care episodes in the form of clinical notes. EHRs are used throughout the health care sector by professionals, administrators and patients, primarily for clinical purposes, but also for secondary purposes such as decision support and research. The vast amounts of information in EHR systems complicate information management and increase the risk of information overload. Therefore, clinicians and researchers need new tools to manage the information stored in the EHRs. A common use case is, given a--possibly unfinished--care episode, to retrieve the most similar care episodes among the records. This paper presents several methods for information retrieval, focusing on care episode retrieval, based on textual similarity, where similarity is measured through domain-specific modelling of the distributional semantics of words. Models include variants of random indexing and the semantic neural network model word2vec. Two novel methods are introduced that utilize the ICD-10 codes attached to care episodes to better induce domain-specificity in the semantic model. We report on experimental evaluation of care episode retrieval that circumvents the lack of human judgements regarding episode relevance. Results suggest that several of the methods proposed outperform a state-of-the art search engine (Lucene) on the retrieval task.

Predicting patient acuity from electronic patient records.

BACKGROUND: The ability to predict acuity (patients' care needs), would provide a powerful tool for health care managers to allocate resources. Such estimations and predictions for the care process can be produced from the vast amounts of healthcare data using information technology and computational intelligence techniques. Tactical decision-making and resource allocation may also be supported with different mathematical optimization models. METHODS: This study was conducted with a data set comprising electronic nursing narratives and the associated Oulu Patient Classification (OPCq) acuity. A mathematical model for the automated assignment of patient acuity scores was utilized and evaluated with the pre-processed data from 23,528 electronic patient records. The methods to predict patient's acuity were based on linguistic pre-processing, vector-space text modeling, and regularized least-squares regression. RESULTS: The experimental results show that it is possible to obtain accurate predictions about patient acuity scores for the coming day based on the assigned scores and nursing notes from the previous day. Making same-day predictions leads to even better results, as access to the nursing notes for the same day boosts the predictive performance. Furthermore, textual nursing notes allow for more accurate predictions than previous acuity scores. The best results are achieved by combining both of these information sources. The developed model achieves a concordance index of 0.821 when predicting the patient acuity scores for the following day, given the scores and text recorded on the previous day. CONCLUSIONS: By applying language technology to electronic patient documents it is possible to accurately predict the value of the acuity scores of the coming day based on the previous daýs assigned scores and nursing notes.

University of Turku in the BioNLP'11 Shared Task.

BACKGROUND: We present a system for extracting biomedical events (detailed descriptions of biomolecular interactions) from research articles, developed for the BioNLP'11 Shared Task. Our goal is to develop a system easily adaptable to different event schemes, following the theme of the BioNLP'11 Shared Task: generalization, the extension of event extraction to varied biomedical domains. Our system extends our BioNLP'09 Shared Task winning Turku Event Extraction System, which uses support vector machines to first detect event-defining words, followed by detection of their relationships. RESULTS: Our current system successfully predicts events for every domain case introduced in the BioNLP'11 Shared Task, being the only system to participate in all eight tasks and all of their subtasks, with best performance in four tasks. Following the Shared Task, we improve the system on the Infectious Diseases task from 42.57% to 53.87% F-score, bringing performance into line with the similar GENIA Event Extraction and Epigenetics and Post-translational Modifications tasks. We evaluate the machine learning performance of the system by calculating learning curves for all tasks, detecting areas where additional annotated data could be used to improve performance. Finally, we evaluate the use of system output on external articles as additional training data in a form of self-training. CONCLUSIONS: We show that the updated Turku Event Extraction System can easily be adapted to all presently available event extraction targets, with competitive performance in most tasks. The scope of the performance gains between the 2009 and 2011 BioNLP Shared Tasks indicates event extraction is still a new field requiring more work. We provide several analyses of event extraction methods and performance, highlighting potential future directions for continued development.

Semantically linking molecular entities in literature through entity relationships.

BACKGROUND: Text mining tools have gained popularity to process the vast amount of available research articles in the biomedical literature. It is crucial that such tools extract information with a sufficient level of detail to be applicable in real life scenarios. Studies of mining non-causal molecular relations attribute to this goal by formally identifying the relations between genes, promoters, complexes and various other molecular entities found in text. More importantly, these studies help to enhance integration of text mining results with database facts. RESULTS: We describe, compare and evaluate two frameworks developed for the prediction of non-causal or 'entity' relations (REL) between gene symbols and domain terms. For the corresponding REL challenge of the BioNLP Shared Task of 2011, these systems ranked first (57.7% F-score) and second (41.6% F-score). In this paper, we investigate the performance discrepancy of 16 percentage points by benchmarking on a related and more extensive dataset, analysing the contribution of both the term detection and relation extraction modules. We further construct a hybrid system combining the two frameworks and experiment with intersection and union combinations, achieving respectively high-precision and high-recall results. Finally, we highlight extremely high-performance results (F-score > 90%) obtained for the specific subclass of embedded entity relations that are essential for integrating text mining predictions with database facts. CONCLUSIONS: The results from this study will enable us in the near future to annotate semantic relations between molecular entities in the entire scientific literature available through PubMed. The recent release of the EVEX dataset, containing biomolecular event predictions for millions of PubMed articles, is an interesting and exciting opportunity to overlay these entity relations with event predictions on a literature-wide scale.

Clustering Nursing Sentences - Comparing Three Sentence Embedding Methods.

In health sciences, high-quality text embeddings may augment qualitative data analysis of large amounts of text by enabling, e.g., searching and clustering of health information. This study aimed to evaluate three different sentence-level embedding methods in clustering sentences in nursing narratives from individual patients' hospital care episodes. Two of these embeddings are generated from language models based on the BERT framework, and the third on the Sent2Vec method. These embedding methods were used to cluster sentences from 20 patient care episodes and the results were manually evaluated. Findings suggest that the best clusters were produced by the embeddings from a BERT model fine-tuned for the proxy task of predicting subject headings for nursing text.

Neural Network and Random Forest Models in Protein Function Prediction.

Over the past decade, the demand for automated protein function prediction has increased due to the volume of newly sequenced proteins. In this paper, we address the function prediction task by developing an ensemble system automatically assigning Gene Ontology (GO) terms to the given input protein sequence. We develop an ensemble system which combines the GO predictions made by random forest (RF) and neural network (NN) classifiers. Both RF and NN models rely on features derived from BLAST sequence alignments, taxonomy and protein signature analysis tools. In addition, we report on experiments with a NN model that directly analyzes the amino acid sequence as its sole input, using a convolutional layer. The Swiss-Prot database is used as the training and evaluation data. In the CAFA3 evaluation, which relies on experimental verification of the functional predictions, our submitted ensemble model demonstrates competitive performance ranking among top-10 best-performing systems out of over 100 submitted systems. In this paper, we evaluate and further improve the CAFA3-submitted system. Our machine learning models together with the data pre-processing and feature generation tools are publicly available as an open source software at https://github.com/TurkuNLP/CAFA3.

Text Classification Model Explainability for Keyword Extraction - Towards Keyword-Based Summarization of Nursing Care Episodes.

Tools to automate the summarization of nursing entries in electronic health records (EHR) have the potential to support healthcare professionals to obtain a rapid overview of a patient's situation when time is limited. This study explores a keyword-based text summarization method for the nursing text that is based on machine learning model explainability for text classification models. This study aims to extract keywords and phrases that provide an intuitive overview of the content in multiple nursing entries in EHRs written during individual patients' care episodes. The proposed keyword extraction method is used to generate keyword summaries from 40 patients' care episodes and its performance is compared to a baseline method based on word embeddings combined with the PageRank method. The two methods were assessed with manual evaluation by three domain experts. The results indicate that it is possible to generate representative keyword summaries from nursing entries in EHRs and our method outperformed the baseline method.

U-Compare bio-event meta-service: compatible BioNLP event extraction services.

BACKGROUND: Bio-molecular event extraction from literature is recognized as an important task of bio text mining and, as such, many relevant systems have been developed and made available during the last decade. While such systems provide useful services individually, there is a need for a meta-service to enable comparison and ensemble of such services, offering optimal solutions for various purposes. RESULTS: We have integrated nine event extraction systems in the U-Compare framework, making them intercompatible and interoperable with other U-Compare components. The U-Compare event meta-service provides various meta-level features for comparison and ensemble of multiple event extraction systems. Experimental results show that the performance improvements achieved by the ensemble are significant. CONCLUSIONS: While individual event extraction systems themselves provide useful features for bio text mining, the U-Compare meta-service is expected to improve the accessibility to the individual systems, and to enable meta-level uses over multiple event extraction systems such as comparison and ensemble.

Complex event extraction at PubMed scale.

MOTIVATION: There has recently been a notable shift in biomedical information extraction (IE) from relation models toward the more expressive event model, facilitated by the maturation of basic tools for biomedical text analysis and the availability of manually annotated resources. The event model allows detailed representation of complex natural language statements and can support a number of advanced text mining applications ranging from semantic search to pathway extraction. A recent collaborative evaluation demonstrated the potential of event extraction systems, yet there have so far been no studies of the generalization ability of the systems nor the feasibility of large-scale extraction. RESULTS: This study considers event-based IE at PubMed scale. We introduce a system combining publicly available, state-of-the-art methods for domain parsing, named entity recognition and event extraction, and test the system on a representative 1% sample of all PubMed citations. We present the first evaluation of the generalization performance of event extraction systems to this scale and show that despite its computational complexity, event extraction from the entire PubMed is feasible. We further illustrate the value of the extraction approach through a number of analyses of the extracted information. AVAILABILITY: The event detection system and extracted data are open source licensed and available at http://bionlp.utu.fi/.

Supporting the use of standardized nursing terminologies with automatic subject heading prediction: a comparison of sentence-level text classification methods.

OBJECTIVE: This study focuses on the task of automatically assigning standardized (topical) subject headings to free-text sentences in clinical nursing notes. The underlying motivation is to support nurses when they document patient care by developing a computer system that can assist in incorporating suitable subject headings that reflect the documented topics. Central in this study is performance evaluation of several text classification methods to assess the feasibility of developing such a system. MATERIALS AND METHODS: Seven text classification methods are evaluated using a corpus of approximately 0.5 million nursing notes (5.5 million sentences) with 676 unique headings extracted from a Finnish university hospital. Several of these methods are based on artificial neural networks. Evaluation is first done in an automatic manner for all methods, then a manual error analysis is done on a sample. RESULTS: We find that a method based on a bidirectional long short-term memory network performs best with an average recall of 0.5435 when allowed to suggest 1 subject heading per sentence and 0.8954 when allowed to suggest 10 subject headings per sentence. However, other methods achieve comparable results. The manual analysis indicates that the predictions are better than what the automatic evaluation suggests. CONCLUSIONS: The results indicate that several of the tested methods perform well in suggesting the most appropriate subject headings on sentence level. Thus, we find it feasible to develop a text classification system that can support the use of standardized terminologies and save nurses time and effort on care documentation.

Assisting nurses in care documentation: from automated sentence classification to coherent document structures with subject headings.

BACKGROUND: Up to 35% of nurses' working time is spent on care documentation. We describe the evaluation of a system aimed at assisting nurses in documenting patient care and potentially reducing the documentation workload. Our goal is to enable nurses to write or dictate nursing notes in a narrative manner without having to manually structure their text under subject headings. In the current care classification standard used in the targeted hospital, there are more than 500 subject headings to choose from, making it challenging and time consuming for nurses to use. METHODS: The task of the presented system is to automatically group sentences into paragraphs and assign subject headings. For classification the system relies on a neural network-based text classification model. The nursing notes are initially classified on sentence level. Subsequently coherent paragraphs are constructed from related sentences. RESULTS: Based on a manual evaluation conducted by a group of three domain experts, we find that in about 69% of the paragraphs formed by the system the topics of the sentences are coherent and the assigned paragraph headings correctly describe the topics. We also show that the use of a paragraph merging step reduces the number of paragraphs produced by 23% without affecting the performance of the system. CONCLUSIONS: The study shows that the presented system produces a coherent and logical structure for freely written nursing narratives and has the potential to reduce the time and effort nurses are currently spending on documenting care in hospitals.

Evaluating pain in intensive care.

Optimal pain management is essential for good care outcomes, but assessing pain is particularly complex in intensive care, as patients are often unable to communicate. We hypothesize that the task could be supported through human language technology. To evaluate the feasibility of such tools, we study how pain is documented in electronic Finnish free-text intensive care nursing notes by statistically comparing annotations of ten nursing professionals on a set of 1548 documents. The aspects considered include the amount and writing style of pain-related notes, pain intensity, and given pain care. More than half of the documents contained information relevant for patients' pain status but it was expressed usually indirectly. Also pain medication was commented as free-text. Although annotators' pain intensity evaluations diverged, the substantial amount of pain-related notes encourages developing computational tools for pain assessment.

Unsupervised Phrase-Level Query Rewriting for Assisting Search in Clinical Free Text.

We report on the pilot evaluation of an experimental query-based search functionality that enables phrase-level query rewriting in an unsupervised way. It is intended for supporting search in clinical text. Qualitative evaluation is done by three clinicans using a prototype search tool. They report that they find the tested search functionality to be beneficial for making query-based searching in clinical text more efficient.

Secondary use of electronic health records: Availability aspects in two Nordic countries.

BACKGROUND: The potential for the secondary use of electronic health records (EHRs) is underused due to restrictions in national legislation. For privacy purposes, legislative restrictions limit the availability and content of EHR data provided to secondary users. These limitations do not encourage healthcare organisations to develop procedures to promote the secondary use of EHRs. OBJECTIVE: The objective of this study is to identify factors that restrict the secondary use of unstructured EHRs in academic research in Finland and Sweden. METHOD: A study was conducted to identify these availability-restricting issues that pertain to the academic secondary use of unstructured EHRs. Using semi-structured interviews, 14 domain experts in science, hospital management and business were interviewed to evaluate the efficiency of procedures and technologies that are implemented in secondary use processes. RESULTS: The results demonstrate three aspects that restrict the availability of unstructured EHRs for secondary purposes: (i) the management and (ii) privacy preservation of such data as well as (iii) potential secondary users. CONCLUSION: Based on these categories, two approaches for the secondary use of unstructured EHRs are identified: the protected processing environment and altered data. IMPLICATIONS: The protected processing environment ensures patient privacy by providing unstructured EHRs for exclusive user groups that have preferred use intentions. Compared to the use of such processing environments, data alteration enables the secondary use of unstructured EHRs for a larger user group with various use intentions but that yield less valuable content.