Improved efficiency in clinical workflow of reporting measured oncology lesions via PACS-integrated lesion tracking tool.

OBJECTIVE. Imaging provides evidence for the response to oncology treatment by the serial measurement of reference lesions. Unfortunately, the identification, comparison, measurement, and documentation of several reference lesions can be an inefficient process. We tested the hypothesis that optimized workflow orchestration and tight integration of a lesion tracking tool into the PACS and speech recognition system can result in improvements in oncologic lesion measurement efficiency. SUBJECTS AND METHODS. A lesion management tool tightly integrated into the PACS workflow was developed. We evaluated the effect of the use of the tool on measurement reporting time by means of a prospective time-motion study on 86 body CT examinations with 241 measureable oncologic lesions with four radiologists. RESULTS. Aggregated measurement reporting time per lesion was 11.64 seconds in standard workflow, 16.67 seconds if readers had to register measurements de novo, and 6.36 seconds for each subsequent follow-up study. Differences were statistically significant (p < 0.05) for each reader, except for one difference for one reader. CONCLUSION. Measurement reporting time can be reduced by using a PACS workflow-integrated lesion management tool, especially for patients with multiple follow-up examinations, reversing the onetime efficiency penalty at baseline registration.

Evaluating the Referring Physician's Clinical History and Indication as a Means for Communicating Chronic Conditions That Are Pertinent at the Point of Radiologic Interpretation.

The clinical history and indication (CHI) provided with a radiological examination are critical components of a quality interpretation by the radiologist. A patient's chronic conditions offer the context in which acute symptoms and findings can be interpreted more accurately. Seven pertinent (potentially diagnosis altering) chronic conditions, which are fairly prevalent at our institution, were selected. We analyze if and how in 140 CHIs there was mention of a patient's previously reported chronic condition and if and how the condition was subsequently described in the radiology report using a four-item scheme (Mention/Specialization, Generalization, Common comorbidity, No mention). In 40.7% of CHIs, the condition was rated Mention/Specialization. Therefore, we reject our first hypothesis that the CHI is a reliable source for obtaining pertinent chronic conditions (≥ 90.0%). Non-oncological conditions were significantly more likely rated No mention in the CHI than oncological conditions (58.7 versus 8.3%, P < 0.0001). Stat cases were significantly more frequently No mention than non-stat cases (60.0 versus 31.3%, P = 0.0134). We accept our second hypothesis that the condition's rating in the CHI is significantly correlated with its rating of the final radiology report (χ(2) test, P < 0.00001). Our study demonstrates an alarming lack of communication of pertinent medical information to the radiologist, which may negatively impact interpretation quality. Presenting automatically aggregated patient information to the radiologist may be a potential avenue for improving interpretation and adding value of the radiology department to the care chain.

A natural language processing pipeline for pairing measurements uniquely across free-text CT reports.

OBJECTIVE: To standardize and objectivize treatment response assessment in oncology, guidelines have been proposed that are driven by radiological measurements, which are typically communicated in free-text reports defying automated processing. We study through inter-annotator agreement and natural language processing (NLP) algorithm development the task of pairing measurements that quantify the same finding across consecutive radiology reports, such that each measurement is paired with at most one other ("partial uniqueness"). METHODS AND MATERIALS: Ground truth is created based on 283 abdomen and 311 chest CT reports of 50 patients each. A pre-processing engine segments reports and extracts measurements. Thirteen features are developed based on volumetric similarity between measurements, semantic similarity between their respective narrative contexts and structural properties of their report positions. A Random Forest classifier (RF) integrates all features. A "mutual best match" (MBM) post-processor ensures partial uniqueness. RESULTS: In an end-to-end evaluation, RF has precision 0.841, recall 0.807, F-measure 0.824 and AUC 0.971; with MBM, which performs above chance level (P<0.001), it has precision 0.899, recall 0.776, F-measure 0.833 and AUC 0.935. RF (RF+MBM) has error-free performance on 52.7% (57.4%) of report pairs. DISCUSSION: Inter-annotator agreement of three domain specialists with the ground truth (κ>0.960) indicates that the task is well defined. Domain properties and inter-section differences are discussed to explain superior performance in abdomen. Enforcing partial uniqueness has mixed but minor effects on performance. CONCLUSION: A combined machine learning-filtering approach is proposed for pairing measurements, which can support prospective (supporting treatment response assessment) and retrospective purposes (data mining).

A context-sensitive image annotation recommendation engine for radiology.

In the typical radiology reading workflow, a radiologist would go through an imaging study and annotate specific regions of interest. The radiologist has the option to select a suitable description (e.g., "calcification") from a list of predefined descriptions, or input the description directly as free-text. However, this process is time-consuming and the descriptions are not standardized over time, even for the same patient or the same general finding. In this paper, we describe an approach that presents finding descriptions based on textual information extracted from a patient's prior reports. Using 133 finding descriptions obtained in routine oncology workflow, we demonstrate how the system can be used to reduce keystrokes by up to 86% in about 38% of the instances. We have integrated our solution into a PACS and discuss how the system can be used in a clinical setting to improve the image annotation workflow efficiency and promote standardization of finding descriptions.

Preferences for structured reporting of measurement data: an institutional survey of medical oncologists, oncology registrars, and radiologists.

RATIONALE AND OBJECTIVES: The aim of this study was to determine whether key radiology report "consumers" in our institution prefer structured measurement reporting in a dedicated report section over the current practice of embedding measurements throughout the "Findings" section, given the availability of new tools for quantitative imaging interpretation that enable automated structured reporting of measurement data. MATERIALS AND METHODS: Oncologic clinicians and radiologists at our institution were surveyed regarding their preferences for a standard report versus three reports each having uniquely formatted dedicated "Measurements" sections and regarding their impressions of various characteristics of report quality demonstrated by these reports. The online survey was completed by 25 radiologists, 16 oncologists, and 17 oncology nurses and research assistants (registrars). RESULTS: Aggregation of respondents' preferences by group into single orderings using the Kemeny-Young method revealed that both oncology groups preferred all proposed reports to the standard report but that radiologists only preferred two of the proposed reports to the standard report. All preferences for proposed reports in the two oncology groups were statistically significant based on Wilcoxon tests, but the preference for only one of the proposed reports was significant for radiologists. Additional results suggest that these preferences are driven by respondent favor for the readability of and confidence conveyed by the proposed reports compared to the standard report. CONCLUSIONS: Oncologic clinicians responding to our survey preferred communication of lesion measurements in a separate report section to the current practice of embedding measurements throughout the "Findings" section, based on their assessments of reports containing simulated measurement sections assembled from a single sample report using standardized formatting.

Cross-sectional relatedness between sentences in breast radiology reports: development of an SVM classifier and evaluation against annotations of five breast radiologists.

Introduce the notion of cross-sectional relatedness as an informational dependence relation between sentences in the conclusion section of a breast radiology report and sentences in the findings section of the same report. Assess inter-rater agreement of breast radiologists. Develop and evaluate a support vector machine (SVM) classifier for automatically detecting cross-sectional relatedness. A standard reference is manually created from 444 breast radiology reports by the first author. A subset of 37 reports is annotated by five breast radiologists. Inter-rater agreement is computed among their annotations and standard reference. Thirteen numerical features are developed to characterize pairs of sentences; the optimal feature set is sought through forward selection. Inter-rater agreement is F-measure 0.623. SVM classifier has F-measure of 0.699 in the 12-fold cross-validation protocol against standard reference. Report length does not correlate with the classifier's performance (correlation coefficient = -0.073). SVM classifier has average F-measure of 0.505 against annotations by breast radiologists. Mediocre inter-rater agreement is possibly caused by: (1) definition is insufficiently actionable, (2) fine-grained nature of cross-sectional relatedness on sentence level, instead of, for instance, on paragraph level, and (3) higher-than-average complexity of 37-report sample. SVM classifier performs better against standard reference than against breast radiologists's annotations. This is supportive of (3). SVM's performance on standard reference is satisfactory. Since optimal feature set is not breast specific, results may transfer to non-breast anatomies. Applications include a smart report viewing environment and data mining.

Using image references in radiology reports to support enhanced report-to-image navigation.

Radiology reports frequently contain references to image slices that are illustrative of described findings, for instance, "Neurofibroma in superior right extraconal space (series 5, image 104)". In the current workflow, if a report consumer wants to view a referenced image, he or she needs to (1) open prior study, (2) open the series of interest (series 5 in this example), and (3) navigate to the corresponding image slice (image 104). This research aims to improve this report-to-image navigation process by providing hyperlinks to images. We develop and evaluate a regular expressions-based algorithm that recognizes image references at a sentence level. Validation on 314 image references from general radiology reports shows precision of 99.35%, recall of 98.08% and F-measure of 98.71%, suggesting this is a viable approach for image reference extraction. We demonstrate how recognized image references can be hyperlinked in a PACS report viewer allowing one-click access to the images.

Automatically pairing measured findings across narrative abdomen CT reports.

Radiological measurements are one of the key variables in widely adopted guidelines (WHO, RECIST) that standardize and objectivize response assessment in oncology care. Measurements are typically described in free-text, narrative radiology reports. We present a natural language processing pipeline that extracts measurements from radiology reports and pairs them with extracted measurements from prior reports of the same clinical finding, e.g., lymph node or mass. A ground truth was created by manually pairing measurements in the abdomen CT reports of 50 patients. A Random Forest classifier trained on 15 features achieved superior results in an end-to-end evaluation of the pipeline on the extraction and pairing task: precision 0.910, recall 0.878, F-measure 0.894, AUC 0.988. Representing the narrative content in terms of UMLS concepts did not improve results. Applications of the proposed technology include data mining, advanced search and workflow support for healthcare professionals managing radiological measurements.

Algorithmic and user study of an autocompletion algorithm on a large medical vocabulary.

INTRODUCTION: Autocompletion supports human-computer interaction in software applications that let users enter textual data. We will be inspired by the use case in which medical professionals enter ontology concepts, catering the ongoing demand for structured and standardized data in medicine. OBJECTIVES: Goal is to give an algorithmic analysis of one particular autocompletion algorithm, called multi-prefix matching algorithm, which suggests terms whose words' prefixes contain all words in the string typed by the user, e.g., in this sense, opt ner me matches optic nerve meningioma. Second we aim to investigate how well it supports users entering concepts from a large and comprehensive medical vocabulary (snomed ct). METHODS: We give a concise description of the multi-prefix algorithm, and sketch how it can be optimized to meet required response time. Performance will be compared to a baseline algorithm, which gives suggestions that extend the string typed by the user to the right, e.g. optic nerve m gives optic nerve meningioma, but opt ner me does not. We conduct a user experiment in which 12 participants are invited to complete 40 snomed ct terms with the baseline algorithm and another set of 40 snomed ct terms with the multi-prefix algorithm. RESULTS: Our results show that users need significantly fewer keystrokes when supported by the multi-prefix algorithm than when supported by the baseline algorithm. CONCLUSIONS: The proposed algorithm is a competitive candidate for searching and retrieving terms from a large medical ontology.