Establishing a carotid artery stenosis disease cohort for comparative effectiveness research using natural language processing.

OBJECTIVE: Investigation of asymptomatic carotid stenosis treatment is hindered by the lack of a contemporary population-based disease cohort. We describe the use of natural language processing (NLP) to identify stenosis in patients undergoing carotid imaging. METHODS: Adult patients with carotid imaging between 2008 and 2012 in a large integrated health care system were identified and followed through 2017. An NLP process was developed to characterize carotid stenosis according to the Society of Radiologists in Ultrasound (for ultrasounds) and North American Symptomatic Carotid Endarterectomy Trial (NASCET) (for axial imaging) guidelines. The resulting algorithm assessed text descriptors to categorize normal/non-hemodynamically significant stenosis, moderate or severe stenosis as well as occlusion in both carotid ultrasound (US) and axial imaging (computed tomography and magnetic resonance angiography [CTA/MRA]). For US reports, internal carotid artery systolic and diastolic velocities and velocity ratios were assessed and matched for laterality to supplement accuracy. To validate the NLP algorithm, positive predictive value (PPV or precision) and sensitivity (recall) were calculated from simple random samples from the population of all imaging studies. Lastly, all non-normal studies were manually reviewed for confirmation for prevalence estimates and disease cohort assembly. RESULTS: A total of 95,896 qualifying index studies (76,276 US and 19,620 CTA/MRA) were identified among 94,822 patients including 1059 patients who underwent multiple studies on the same day. For studies of normal/non-hemodynamically significant stenosis arteries, the NLP algorithm showed excellent performance with a PPV of 99% for US and 96.5% for CTA/MRA. PPV/sensitivity to identify a non-normal artery with correct laterality in the CTA/MRA and US samples were 76.9% (95% confidence interval [CI], 74.1%-79.5%)/93.1% (95% CI, 91.1%-94.8%) and 74.7% (95% CI, 69.3%-79.5%)/94% (95% CI, 90.2%-96.7%), respectively. Regarding cohort assembly, 15,522 patients were identified with diseased carotid artery, including 2674 exhibiting equal bilateral disease. This resulted in a laterality-specific cohort with 12,828 moderate, 5283 severe, and 1895 occluded arteries and 326 diseased arteries with unknown stenosis. During follow-up, 30.1% of these patients underwent 61,107 additional studies. CONCLUSIONS: Use of NLP to detect carotid stenosis or occlusion can result in accurate exclusion of normal/non-hemodynamically significant stenosis disease states with more moderate precision with lesion identification, which can substantially reduce the need for manual review. The resulting cohort allows for efficient research and holds promise for similar reporting in other vascular diseases.

Validation of natural language processing to determine the presence and size of abdominal aortic aneurysms in a large integrated health system.

OBJECTIVE: Previous studies of the natural history of abdominal aortic aneurysms (AAAs) have been limited by small cohort sizes or heterogeneous analyses of pooled data. By quickly and efficiently extracting imaging data from the health records, natural language processing (NLP) has the potential to substantially improve how we study and care for patients with AAAs. The aim of the present study was to test the ability of an NLP tool to accurately identify the presence or absence of AAAs and detect the maximal abdominal aortic diameter in a large dataset of imaging study reports. METHODS: Relevant imaging study reports (n = 230,660) from 2003 to 2017 were obtained for 32,778 patients followed up in a prospective aneurysm surveillance registry within a large, diverse, integrated healthcare system. A commercially available NLP algorithm was used to assess the presence of AAAs, confirm the absence of AAAs, and extract the maximal diameter of the abdominal aorta, if stated. A blinded expert manual review of 18,000 randomly selected imaging reports was used as the reference standard. The positive predictive value (PPV or precision), sensitivity (recall), and the kappa statistics were calculated. RESULTS: Of the randomly selected 18,000 studies that underwent expert manual review, 48.7% were positive for AAAs. In confirming the presence of an AAA, the interrater reliability of the NLP compared with the expert review showed a kappa value of 0.84 (95% confidence interval [CI], 0.83-0.85), with a PPV of 95% and sensitivity of 88.5%. The NLP algorithm showed similar results for confirming the absence of an AAA, with a kappa of 0.79 (95% CI, 0.799-0.80), PPV of 77.7%, and sensitivity of 91.9%. The kappa, PPV, and sensitivity of the NLP for correctly identifying the maximal aortic diameter was 0.88 (95% CI, 0.87-0.89), 88.8%, and 88.2% respectively. CONCLUSIONS: The use of NLP software can accurately analyze large volumes of radiology report data to detect AAA disease and assemble a contemporary aortic diameter-based cohort of patients for longitudinal analysis to guide surveillance, medical management, and operative decision making. It can also potentially be used to identify from the electronic medical records pre- and postoperative AAA patients "lost to follow-up," leverage human resources engaged in the ongoing surveillance of patients with AAAs, and facilitate the construction and implementation of AAA screening programs.

Qualitative factors in patients who die shortly after emergency department discharge.

OBJECTIVES: Early death after emergency department (ED) discharge may signal opportunities to improve care. Prior studies are limited by incomplete mortality ascertainment and lack of clinically important information in administrative data. The goal in this hypothesis-generating study was to identify patient and process of care themes that may provide possible explanations for early postdischarge mortality. METHODS: This was a qualitative analysis of medical records of adult patients who visited the ED of any of six hospitals in an integrated health system (Kaiser Permanente Southern California [KPSC]) and died within 7 days of discharge in 2007 and 2008. Nonmembers, visits to non-health plan hospitals, patients receiving or referred to hospice care, and patients with do not attempt resuscitation or do not intubate orders (DNAR/DNI) were excluded. Under the guidance of two qualitative research scientists, a team of three emergency physicians used grounded theory techniques to identify patient clinical presentations and processes of care that serve as potential explanations for poor outcome after discharge. RESULTS: The source population consisted of a total of 290,092 members with 446,120 discharges from six KPSC EDs in 2007 and 2008. A total of 203 deaths occurred within 7 days of ED discharge (0.05%). Sixty-one randomly chosen cases were reviewed. Patient-level themes that emerged included an unexplained persistent acute change in mental status, recent fall, abnormal vital signs, ill-appearing presentation, malfunctioning indwelling device, and presenting symptoms remaining at discharge. Process-of-care factors included a discrepancy in history of present illness, incomplete physical examination, and change of discharge plan by a third party, such as a consulting or admitting physician. CONCLUSIONS: In this hypothesis-generating study, qualitative research techniques were used to identify clinical and process-of-care factors in patients who died within days after discharge from an ED. These potential predictors will be formally tested in a future quantitative study.

Patterns and predictors of short-term death after emergency department discharge.

STUDY OBJECTIVE: The emergency department (ED) is an inherently high-risk setting. Early death after an ED evaluation is a rare and devastating outcome; understanding it can potentially help improve patient care and outcomes. Using administrative data from an integrated health system, we describe characteristics and predictors of patients who experienced 7-day death after ED discharge. METHODS: Administrative data from 12 hospitals were used to identify death after discharge in adults aged 18 year or older within 7 days of ED presentation from January 1, 2007, to December 31, 2008. Patients who were nonmembers of the health system, in hospice care, or treated at out-of-network EDs were excluded. Predictors of 7-day postdischarge death were identified with multivariable logistic regression. RESULTS: The study cohort contained a total of 475,829 members, with 728,312 discharges from Kaiser Permanente Southern California EDs in 2007 and 2008. Death within 7 days of discharge occurred in 357 cases (0.05%). Increasing age, male sex, and number of preexisting comorbidities were associated with increased risk of death. The top 3 primary discharge diagnoses predictive of 7-day death after discharge included noninfectious lung disease (odds ratio [OR] 7.1; 95% confidence interval [CI] 2.9 to 17.4), renal disease (OR 5.6; 95% CI 2.2 to 14.2), and ischemic heart disease (OR 3.8; 95% CI 1.0 to 13.6). CONCLUSION: Our study suggests that 50 in 100,000 patients in the United States die within 7 days of discharge from an ED. To our knowledge, our study is the first to identify potentially "high-risk" discharge diagnoses in patients who experience a short-term death after discharge.