Design and Perceived Value of a Novel Solution for Asynchronous Communication in Radiology.

RATIONALE AND OBJECTIVES: Communication with and within the Radiology Department is typically initiated over phone, face-to-face or general-purpose chat, causing frequent interruptions, additional mental workload, workflow inefficiencies and diagnostic errors. We developed and evaluated a new communication solution that aims to reduce avoidable interruptions caused by technologist-radiologist communication. MATERIALS AND METHODS: Following an iterative design process with future end users, a scalable web-based software solution, RadConnect, was developed enabling a chat-based communication workflow between a technologist and a radiologist. As a first experimental implementation, technologists can send categorized tickets to a radiology section account. Radiologists receive the tickets in a worklist that is prioritized by urgency. Consented radiologists and technologists performed scripted tasks in 2 hr sessions and completed a structured questionnaire on perceived value and comparison to standard communication modes. RESULTS: Of 17 participants from three academic European institutes, 65% (11/17) believed they would use RadConnect frequently; 53% (9/17) believed that it reduces phone calls >80%; and 88% (15/17) believed it adds value compared to general-purpose enterprise chat applications. DISCUSSION: Participants recognized the value of RadConnect especially its categorized tickets, prioritized worklist and role-based interaction model. Inter-institute differences in perceived value of RadConnect may have been caused by technologist-radiologist proximity and communication alternatives in the institutions. CONCLUSION: Chat-based role-based communication might be a viable mode of communication between technologists and radiologists to reduce avoidable interruptions. Tailoring the chat solution to the needs of and tightly integrated with the radiology workflow is valued by future end users after exposure to the tool in a simulated environment.

Improved efficiency in clinical workflow of reporting measured oncology lesions via PACS-integrated lesion tracking tool.

OBJECTIVE. Imaging provides evidence for the response to oncology treatment by the serial measurement of reference lesions. Unfortunately, the identification, comparison, measurement, and documentation of several reference lesions can be an inefficient process. We tested the hypothesis that optimized workflow orchestration and tight integration of a lesion tracking tool into the PACS and speech recognition system can result in improvements in oncologic lesion measurement efficiency. SUBJECTS AND METHODS. A lesion management tool tightly integrated into the PACS workflow was developed. We evaluated the effect of the use of the tool on measurement reporting time by means of a prospective time-motion study on 86 body CT examinations with 241 measureable oncologic lesions with four radiologists. RESULTS. Aggregated measurement reporting time per lesion was 11.64 seconds in standard workflow, 16.67 seconds if readers had to register measurements de novo, and 6.36 seconds for each subsequent follow-up study. Differences were statistically significant (p < 0.05) for each reader, except for one difference for one reader. CONCLUSION. Measurement reporting time can be reduced by using a PACS workflow-integrated lesion management tool, especially for patients with multiple follow-up examinations, reversing the onetime efficiency penalty at baseline registration.

A natural language processing pipeline for pairing measurements uniquely across free-text CT reports.

OBJECTIVE: To standardize and objectivize treatment response assessment in oncology, guidelines have been proposed that are driven by radiological measurements, which are typically communicated in free-text reports defying automated processing. We study through inter-annotator agreement and natural language processing (NLP) algorithm development the task of pairing measurements that quantify the same finding across consecutive radiology reports, such that each measurement is paired with at most one other ("partial uniqueness"). METHODS AND MATERIALS: Ground truth is created based on 283 abdomen and 311 chest CT reports of 50 patients each. A pre-processing engine segments reports and extracts measurements. Thirteen features are developed based on volumetric similarity between measurements, semantic similarity between their respective narrative contexts and structural properties of their report positions. A Random Forest classifier (RF) integrates all features. A "mutual best match" (MBM) post-processor ensures partial uniqueness. RESULTS: In an end-to-end evaluation, RF has precision 0.841, recall 0.807, F-measure 0.824 and AUC 0.971; with MBM, which performs above chance level (P<0.001), it has precision 0.899, recall 0.776, F-measure 0.833 and AUC 0.935. RF (RF+MBM) has error-free performance on 52.7% (57.4%) of report pairs. DISCUSSION: Inter-annotator agreement of three domain specialists with the ground truth (κ>0.960) indicates that the task is well defined. Domain properties and inter-section differences are discussed to explain superior performance in abdomen. Enforcing partial uniqueness has mixed but minor effects on performance. CONCLUSION: A combined machine learning-filtering approach is proposed for pairing measurements, which can support prospective (supporting treatment response assessment) and retrospective purposes (data mining).

Identifying errors and inconsistencies in real time while using facilitated echocardiographic reporting.

BACKGROUND: Facilitated reporting using a discrete set of finding codes (FCs) is a common method of generating echocardiographic reports. METHODS: The investigators developed a tool that allows echocardiographic reports to be evaluated in real time for errors, omissions, and inconsistencies on the basis of a defined group of "rules" applied to the FCs present in the report. At the time of report finalization, conflicts were displayed for the interpreting physicians, and their responses to each rule conflict were logged. RESULTS: Over the course of 1 year, 7,986 transthoracic echocardiographic reports were analyzed prospectively during study interpretation. Overall, 30 ± 4.7 FCs were used to generate finalized reports. An average of 2.4 ± 2.0 conflicts were detected per finalized study. Eighty-three percent of studies had at least one conflict identified. There was no significant correlation between physician experience and conflict rates, but time of day (earlier) and rate at which studies were being finalized (faster) were both correlated with increased conflict rate. Overall, physicians ignored identified conflicts 52% of the time and altered their readings to eliminate the conflicts 48% of the time. Overall, at least one change was made in 54% of all finalized studies. There was a small but significant trend for physicians to produce more conflicts over time as the tool was used. CONCLUSIONS: This study demonstrates that facilitated reporting of echocardiographic studies, using a discrete set of FCs, allows the generation of rules that can be used to identify discrepancies in echocardiographic reports before finalization. Conflicts are common in clinical practice, and the identification of these conflicts in real time allowed readers to review their interpretations and frequently resulted in alterations to echocardiographic reports.

Automated mutual exclusion rules discovery for structured observational codes in echocardiography reporting.

Structured reporting in medicine has been argued to support and enhance machine-assisted processing and communication of pertinent information. Retrospective studies showed that structured echocardiography reports, constructed through point-and-click selection of finding codes (FCs), contain pair-wise contradictory FCs (e.g., "No tricuspid regurgitation" and "Severe regurgitation") downgrading report quality and reliability thereof. In a prospective study, contradictions were detected automatically using an extensive rule set that encodes mutual exclusion patterns between FCs. Rules creation is a labor and knowledge-intensive task that could benefit from automation. We propose a machine-learning approach to discover mutual exclusion rules in a corpus of 101,211 structured echocardiography reports through semantic and statistical analysis. Ground truth is derived from the extensive prospectively evaluated rule set. On the unseen test set, F-measure (0.439) and above-chance level AUC (0.885) show that our approach can potentially support the manual rules creation process. Our methods discovered previously unknown rules per expert review.

Evaluating the Referring Physician's Clinical History and Indication as a Means for Communicating Chronic Conditions That Are Pertinent at the Point of Radiologic Interpretation.

The clinical history and indication (CHI) provided with a radiological examination are critical components of a quality interpretation by the radiologist. A patient's chronic conditions offer the context in which acute symptoms and findings can be interpreted more accurately. Seven pertinent (potentially diagnosis altering) chronic conditions, which are fairly prevalent at our institution, were selected. We analyze if and how in 140 CHIs there was mention of a patient's previously reported chronic condition and if and how the condition was subsequently described in the radiology report using a four-item scheme (Mention/Specialization, Generalization, Common comorbidity, No mention). In 40.7% of CHIs, the condition was rated Mention/Specialization. Therefore, we reject our first hypothesis that the CHI is a reliable source for obtaining pertinent chronic conditions (≥ 90.0%). Non-oncological conditions were significantly more likely rated No mention in the CHI than oncological conditions (58.7 versus 8.3%, P < 0.0001). Stat cases were significantly more frequently No mention than non-stat cases (60.0 versus 31.3%, P = 0.0134). We accept our second hypothesis that the condition's rating in the CHI is significantly correlated with its rating of the final radiology report (χ(2) test, P < 0.00001). Our study demonstrates an alarming lack of communication of pertinent medical information to the radiologist, which may negatively impact interpretation quality. Presenting automatically aggregated patient information to the radiologist may be a potential avenue for improving interpretation and adding value of the radiology department to the care chain.

A context-sensitive image annotation recommendation engine for radiology.

In the typical radiology reading workflow, a radiologist would go through an imaging study and annotate specific regions of interest. The radiologist has the option to select a suitable description (e.g., "calcification") from a list of predefined descriptions, or input the description directly as free-text. However, this process is time-consuming and the descriptions are not standardized over time, even for the same patient or the same general finding. In this paper, we describe an approach that presents finding descriptions based on textual information extracted from a patient's prior reports. Using 133 finding descriptions obtained in routine oncology workflow, we demonstrate how the system can be used to reduce keystrokes by up to 86% in about 38% of the instances. We have integrated our solution into a PACS and discuss how the system can be used in a clinical setting to improve the image annotation workflow efficiency and promote standardization of finding descriptions.

Preferences for structured reporting of measurement data: an institutional survey of medical oncologists, oncology registrars, and radiologists.

RATIONALE AND OBJECTIVES: The aim of this study was to determine whether key radiology report "consumers" in our institution prefer structured measurement reporting in a dedicated report section over the current practice of embedding measurements throughout the "Findings" section, given the availability of new tools for quantitative imaging interpretation that enable automated structured reporting of measurement data. MATERIALS AND METHODS: Oncologic clinicians and radiologists at our institution were surveyed regarding their preferences for a standard report versus three reports each having uniquely formatted dedicated "Measurements" sections and regarding their impressions of various characteristics of report quality demonstrated by these reports. The online survey was completed by 25 radiologists, 16 oncologists, and 17 oncology nurses and research assistants (registrars). RESULTS: Aggregation of respondents' preferences by group into single orderings using the Kemeny-Young method revealed that both oncology groups preferred all proposed reports to the standard report but that radiologists only preferred two of the proposed reports to the standard report. All preferences for proposed reports in the two oncology groups were statistically significant based on Wilcoxon tests, but the preference for only one of the proposed reports was significant for radiologists. Additional results suggest that these preferences are driven by respondent favor for the readability of and confidence conveyed by the proposed reports compared to the standard report. CONCLUSIONS: Oncologic clinicians responding to our survey preferred communication of lesion measurements in a separate report section to the current practice of embedding measurements throughout the "Findings" section, based on their assessments of reports containing simulated measurement sections assembled from a single sample report using standardized formatting.

Cross-sectional relatedness between sentences in breast radiology reports: development of an SVM classifier and evaluation against annotations of five breast radiologists.

Introduce the notion of cross-sectional relatedness as an informational dependence relation between sentences in the conclusion section of a breast radiology report and sentences in the findings section of the same report. Assess inter-rater agreement of breast radiologists. Develop and evaluate a support vector machine (SVM) classifier for automatically detecting cross-sectional relatedness. A standard reference is manually created from 444 breast radiology reports by the first author. A subset of 37 reports is annotated by five breast radiologists. Inter-rater agreement is computed among their annotations and standard reference. Thirteen numerical features are developed to characterize pairs of sentences; the optimal feature set is sought through forward selection. Inter-rater agreement is F-measure 0.623. SVM classifier has F-measure of 0.699 in the 12-fold cross-validation protocol against standard reference. Report length does not correlate with the classifier's performance (correlation coefficient = -0.073). SVM classifier has average F-measure of 0.505 against annotations by breast radiologists. Mediocre inter-rater agreement is possibly caused by: (1) definition is insufficiently actionable, (2) fine-grained nature of cross-sectional relatedness on sentence level, instead of, for instance, on paragraph level, and (3) higher-than-average complexity of 37-report sample. SVM classifier performs better against standard reference than against breast radiologists's annotations. This is supportive of (3). SVM's performance on standard reference is satisfactory. Since optimal feature set is not breast specific, results may transfer to non-breast anatomies. Applications include a smart report viewing environment and data mining.

Using image references in radiology reports to support enhanced report-to-image navigation.

Radiology reports frequently contain references to image slices that are illustrative of described findings, for instance, "Neurofibroma in superior right extraconal space (series 5, image 104)". In the current workflow, if a report consumer wants to view a referenced image, he or she needs to (1) open prior study, (2) open the series of interest (series 5 in this example), and (3) navigate to the corresponding image slice (image 104). This research aims to improve this report-to-image navigation process by providing hyperlinks to images. We develop and evaluate a regular expressions-based algorithm that recognizes image references at a sentence level. Validation on 314 image references from general radiology reports shows precision of 99.35%, recall of 98.08% and F-measure of 98.71%, suggesting this is a viable approach for image reference extraction. We demonstrate how recognized image references can be hyperlinked in a PACS report viewer allowing one-click access to the images.

Automatically pairing measured findings across narrative abdomen CT reports.

Radiological measurements are one of the key variables in widely adopted guidelines (WHO, RECIST) that standardize and objectivize response assessment in oncology care. Measurements are typically described in free-text, narrative radiology reports. We present a natural language processing pipeline that extracts measurements from radiology reports and pairs them with extracted measurements from prior reports of the same clinical finding, e.g., lymph node or mass. A ground truth was created by manually pairing measurements in the abdomen CT reports of 50 patients. A Random Forest classifier trained on 15 features achieved superior results in an end-to-end evaluation of the pipeline on the extraction and pairing task: precision 0.910, recall 0.878, F-measure 0.894, AUC 0.988. Representing the narrative content in terms of UMLS concepts did not improve results. Applications of the proposed technology include data mining, advanced search and workflow support for healthcare professionals managing radiological measurements.

Algorithmic and user study of an autocompletion algorithm on a large medical vocabulary.

INTRODUCTION: Autocompletion supports human-computer interaction in software applications that let users enter textual data. We will be inspired by the use case in which medical professionals enter ontology concepts, catering the ongoing demand for structured and standardized data in medicine. OBJECTIVES: Goal is to give an algorithmic analysis of one particular autocompletion algorithm, called multi-prefix matching algorithm, which suggests terms whose words' prefixes contain all words in the string typed by the user, e.g., in this sense, opt ner me matches optic nerve meningioma. Second we aim to investigate how well it supports users entering concepts from a large and comprehensive medical vocabulary (snomed ct). METHODS: We give a concise description of the multi-prefix algorithm, and sketch how it can be optimized to meet required response time. Performance will be compared to a baseline algorithm, which gives suggestions that extend the string typed by the user to the right, e.g. optic nerve m gives optic nerve meningioma, but opt ner me does not. We conduct a user experiment in which 12 participants are invited to complete 40 snomed ct terms with the baseline algorithm and another set of 40 snomed ct terms with the multi-prefix algorithm. RESULTS: Our results show that users need significantly fewer keystrokes when supported by the multi-prefix algorithm than when supported by the baseline algorithm. CONCLUSIONS: The proposed algorithm is a competitive candidate for searching and retrieving terms from a large medical ontology.

Automatically correlating clinical findings and body locations in radiology reports using MedLEE.

In this paper, we describe and evaluate a system that extracts clinical findings and body locations from radiology reports and correlates them. The system uses Medical Language Extraction and Encoding System (MedLEE) to map the reports' free text to structured semantic representations of their content. A lightweight reasoning engine extracts the clinical findings and body locations from MedLEE's semantic representation and correlates them. Our study is illustrative for research in which existing natural language processing software is embedded in a larger system. We manually created a standard reference based on a corpus of neuro and breast radiology reports. The standard reference was used to evaluate the precision and recall of the proposed system and its modules. Our results indicate that the precision of our system is considerably better than its recall (82.32-91.37% vs. 35.67-45.91%). We conducted an error analysis and discuss here the practical usability of the system given its recall and precision performance.

Bridging the text-image gap: a decision support tool for real-time PACS browsing.

In this paper, we introduce an ontology-based technology that bridges the gap between MR images on the one hand and knowledge sources on the other hand. The proposed technology allows the user to express interest in a body region by selecting this region on the MR image he or she is viewing with a mouse device. The proposed technology infers the intended body structure from the manual selection and searches the external knowledge source for pertinent information. This technology can be used to bridge the gap between image data in the clinical workflow and (external) knowledge sources that help to assess the case with increased certainty, accuracy, and efficiency. We evaluate an instance of the proposed technology in the neurodomain by means of a user study in which three neuroradiologists participated. The user study shows that the technology has high recall (>95%) when it comes to inferring the intended brain region from the participant's manual selection. We are confident that this helps to increase the experience of browsing external knowledge sources.

SNOMED CT Saves Keystrokes: Quantifying Semantic Autocompletion.

In many applications autocompletion functionality saves keystrokes, increases user experience, and helps the user to comply with standardized terminology. Intuitively, the more context information we have about the user, the more accurate autocompletion suggestions we can give. In this paper we research the added value of contextual information for autocompletion algorithms, measured as the average number of saved keystrokes. In our experiments, a context is represented as a set of SNOMED CT terms. Using the structure of SNOMED CT we determine the semantic distance of each SNOMED CT term to the context terms. The resulting distance function is injected in the autocompletion algorithms to reward terms that are semantically close to the context. Our results show that semantic enhancement saves up to 18% of keystrokes, in addition to the percentage of keystrokes saved for the non-semantic base algorithm.