Association Between Medicare's Sepsis Reporting Policy (SEP-1) and the Documentation of a Sepsis Diagnosis in the Clinical Record.

STUDY DESIGN: Interrupted time series analysis of a retrospective, electronic health record cohort. OBJECTIVE: To determine the association between the implementation of Medicare's sepsis reporting measure (SEP-1) and sepsis diagnosis rates as assessed in clinical documentation. BACKGROUND: The role of health policy in the effort to improve sepsis diagnosis remains unclear. PATIENTS AND METHODS: Adult patients hospitalized with suspected infection and organ dysfunction within 6 hours of presentation to the emergency department, admitted to one of 11 hospitals in a multi-hospital health system from January 2013 to December 2017. Clinician-diagnosed sepsis, as reflected by the inclusion of the terms "sepsis" or "septic" in the text of clinical notes in the first two calendar days following presentation. RESULTS: Among 44,074 adult patients with sepsis admitted to 11 hospitals over 5 years, the proportion with sepsis documentation was 32.2% just before the implementation of SEP-1 in the third quarter of 2015 and increased to 37.3% by the fourth quarter of 2017. Of the 9 post-SEP-1 quarters, 8 had odds ratios for a sepsis diagnosis >1 (overall range: 0.98-1.26; P value for a joint test of statistical significance = 0.005). The effects were clinically modest, with a maximum effect of an absolute increase of 4.2% (95% CI: 0.9-7.8) at the end of the study period. The effect was greater in patients who did not require vasopressors compared with patients who required vasopressors ( P value for test of interaction = 0.02). CONCLUSIONS: SEP-1 implementation was associated with modest increases in sepsis diagnosis rates, primarily among patients who did not require vasoactive medications.

NLPReViz: an interactive tool for natural language processing on clinical text.

The gap between domain experts and natural language processing expertise is a barrier to extracting understanding from clinical text. We describe a prototype tool for interactive review and revision of natural language processing models of binary concepts extracted from clinical notes. We evaluated our prototype in a user study involving 9 physicians, who used our tool to build and revise models for 2 colonoscopy quality variables. We report changes in performance relative to the quantity of feedback. Using initial training sets as small as 10 documents, expert review led to final F1scores for the "appendiceal-orifice" variable between 0.78 and 0.91 (with improvements ranging from 13.26% to 29.90%). F1for "biopsy" ranged between 0.88 and 0.94 (-1.52% to 11.74% improvements). The average System Usability Scale score was 70.56. Subjective feedback also suggests possible design improvements.

Heuristic sample selection to minimize reference standard training set for a part-of-speech tagger.

Part-of-speech tagging represents an important first step for most medical natural language processing (NLP) systems. The majority of current statistically-based POS taggers are trained using a general English corpus. Consequently, these systems perform poorly on medical text. Annotated medical corpora are difficult to develop because of the time and labor required. We investigated a heuristic-based sample selection method to minimize annotated corpus size for retraining a Maximum Entropy (ME) POS tagger. We developed a manually annotated domain specific corpus (DSC) of surgical pathology reports and a domain specific lexicon (DL). We sampled the DSC using two heuristics to produce smaller training sets and compared the retrained performance against (1) the original ME modeled tagger trained on general English, (2) the ME tagger retrained on the DL, and (3) the MedPost tagger trained on MEDLINE abstracts. RESULTS showed that the ME tagger retrained with a DSC was superior to the tagger retrained with the DL, and also superior to MedPost. Heuristic methods for sample selection produced performance equivalent to use of the entire training set, but with many fewer sentences. Learning curve analysis showed that sample selection would enable an 84% decrease in the size of the training set without a decrement in performance. We conclude that heuristic sample selection can be used to markedly reduce human annotation requirements for training of medical NLP systems.