Automated identification of surveillance colonoscopy in inflammatory bowel disease using natural language processing.

BACKGROUND: Differentiating surveillance from non-surveillance colonoscopy for colorectal cancer in patients with inflammatory bowel disease (IBD) using electronic medical records (EMR) is important for practice improvement and research purposes, but diagnosis code algorithms are lacking. The automated retrieval console (ARC) is natural language processing (NLP)-based software that allows text-based document-level classification. AIMS: The purpose of this study was to test the feasibility and accuracy of ARC in identifying surveillance and non-surveillance colonoscopy in IBD using EMR. METHODS: We performed a split validation study of electronic reports of colonoscopy pathology for patients with IBD from the Michael E. DeBakey VA Medical Center. A gastroenterologist manually classified pathology reports as either derived from surveillance or non-surveillance colonoscopy. Pathology reports were randomly split into two sets: 70 % for algorithm derivation and 30 % for validation. An ARC generated classification model was applied to the validation set of pathology reports. The performance of the model was compared with manual classification for surveillance and non-surveillance colonoscopy. RESULTS: A total of 575 colonoscopy pathology reports were available on 195 IBD patients, of which 400 reports were designated as training and 175 as testing sets. Within the testing set, a total of 69 pathology reports were classified as surveillance by manual review, whereas the ARC model classified 66 reports as surveillance for a recall of 0.77, precision of 0.80, and specificity of 0.88. CONCLUSIONS: ARC was able to identify surveillance colonoscopy for IBD without customized software programming. NLP-based document-level classification may be used to differentiate surveillance from non-surveillance colonoscopy in IBD.

Measuring use of evidence based psychotherapy for posttraumatic stress disorder.

To improve methods of estimating use of evidence-based psychotherapy for posttraumatic stress disorder in the Veteran's health administration, we evaluated administrative data and note text for patients newly enrolling in six VHA outpatient PTSD clinics in New England during the 2010 fiscal year (n = 1,924). Using natural language processing, we developed machine learning algorithms that mimic human raters in classifying note text. We met our targets for algorithm performance as measured by precision, recall, and F-measure. We found that 6.3 % of our study population received at least one session of evidence-based psychotherapy during the initial 6 months of treatment. Evidence-based psychotherapies appear to be infrequently utilized in VHA outpatient PTSD clinics in New England. Our method could support efforts to improve use of these treatments.

Blue Circle Health: A Novel Patient-Centered Model of Health Care Delivery for Low-Income Patients With Type 1 Diabetes.

Analog insulins, insulin pumps, and continuous glucose monitors (CGM) have revolutionized type 1 diabetes (T1D) treatment over the last 50 years. Nevertheless, less than 20% of patients in the United States reach guideline-based HbA1c targets. The dysfunctional delivery of U.S. health care has further worsened glycemic outcomes among structurally disadvantaged groups such as non-Hispanic Black and low-income populations. Administrative complexities resulting from mixed insurance coverage and delivery systems, incongruity between effective policies and reimbursement, structural racism, and implicit biases have led to high diabetes care-related costs, provider scarcity and burnout, and patient diabetes distress. The Extension for Community Healthcare Outcomes (ECHO) Diabetes tele-education outreach model was created to increase self-efficacy among primary care providers through a combination of weekly didactic sessions led by a team of diabetes experts and access to community-based peer coaches. As an evolution of ECHO Diabetes, Blue Circle Health has been established as a philanthropically funded health care delivery system, using a whole-person, individualized approach to T1D care for adults living in underserved communities. The program will provide direct-to-patient telehealth services, including diabetes education, management, and related psychological care regardless of ability to pay. Community-based diabetes support coaches will serve as the primary point of contact, or guide on the "Blue Circle Health Member Journey." Access to needed insulins, supplies, and CGMs will be provided at no cost to the individual. Through a continuous learning and improvement model, a person-centered, equitable, accessible, and effective health care delivery model will be built for people living with T1D.

The Clinical Outcomes Assessment Toolkit: a framework to support automated clinical records-based outcomes assessment and performance measurement research.

The Clinical Outcomes Assessment Toolkit (COAT) was created through a collaboration between the University of California, Los Angeles and Brigham and Women's Hospital to address the challenge of gathering, formatting, and abstracting data for clinical outcomes and performance measurement research. COAT provides a framework for the development of information pipelines to transform clinical data from its original structured, semi-structured, and unstructured forms to a standardized format amenable to statistical analysis. This system includes a collection of clinical data structures, reusable utilities for information analysis and transformation, and a graphical user interface through which pipelines can be controlled and their results audited by nontechnical users. The COAT architecture is presented, as well as two case studies of current implementations in the domain of prostate cancer outcomes assessment.

Facilitating Clinical Outcomes Assessment through the automated identification of quality measures for prostate cancer surgery.

OBJECTIVES: The College of American Pathologists (CAP) Category 1 quality measures, tumor stage, Gleason score, and surgical margin status, are used by physicians and cancer registrars to categorize patients into groups for clinical trials and treatment planning. This study was conducted to evaluate the effectiveness of an application designed to automatically extract these quality measures from the postoperative pathology reports of patients having undergone prostatectomies for treatment of prostate cancer. DESIGN: An application was developed with the Clinical Outcomes Assessment Toolkit that uses an information pipeline of regular expressions and support vector machines to extract CAP Category 1 quality measures. System performance was evaluated against a gold standard of 676 pathology reports from the University of California at Los Angeles Medical Center and Brigham and Women's Hospital. To evaluate the feasibility of clinical implementation, all pathology reports were gathered using administrative codes with no manual preprocessing of the data performed. MEASUREMENTS: The sensitivity, specificity, and overall accuracy of system performance were measured for all three quality measures. Performance at both hospitals was compared, and a detailed failure analysis was conducted to identify errors caused by poor data quality versus system shortcomings. RESULTS: Accuracies for Gleason score were 99.7%, tumor stage 99.1%, and margin status 97.2%, for an overall accuracy of 98.67%. System performance on data from both hospitals was comparable. Poor clinical data quality led to a decrease in overall accuracy of only 0.3% but accounted for 25.9% of the total errors. CONCLUSION: Despite differences in document format and pathologists' reporting styles, strong system performance indicates the potential of using a combination of regular expressions and support vector machines to automatically extract CAP Category 1 quality measures from postoperative prostate cancer pathology reports.

Feasibility of Automating Patient Acuity Measurement Using a Machine Learning Algorithm.

BACKGROUND AND PURPOSE: One method of determining nurse staffing is to match patient demand for nursing care (patient acuity) with available nursing staff. This pilot study explored the feasibility of automating acuity measurement using a machine learning algorithm. METHODS: Natural language processing combined with a machine learning algorithm was used to predict acuity levels based on electronic health record data. RESULTS: The algorithm was able to predict acuity relatively well. A main challenge was discordance among nurse raters of acuity in generating a gold standard of acuity before applying the machine learning algorithm. CONCLUSIONS: This pilot study tested applying machine learning techniques to acuity measurement and yielded a moderate level of performance. Higher agreement among the gold standard may yield higher performance in future studies.

Using natural language processing on the free text of clinical documents to screen for evidence of homelessness among US veterans.

Information retrieval algorithms based on natural language processing (NLP) of the free text of medical records have been used to find documents of interest from databases. Homelessness is a high priority non-medical diagnosis that is noted in electronic medical records of Veterans in Veterans Affairs (VA) facilities. Using a human-reviewed reference standard corpus of clinical documents of Veterans with evidence of homelessness and those without, an open-source NLP tool (Automated Retrieval Console v2.0, ARC) was trained to classify documents. The best performing model based on document level work-flow performed well on a test set (Precision 94%, Recall 97%, F-Measure 96). Processing of a naïve set of 10,000 randomly selected documents from the VA using this best performing model yielded 463 documents flagged as positive, indicating a 4.7% prevalence of homelessness. Human review noted a precision of 70% for these flags resulting in an adjusted prevalence of homelessness of 3.3% which matches current VA estimates. Further refinements are underway to improve the performance. We demonstrate an effective and rapid lifecycle of using an off-the-shelf NLP tool for screening targets of interest from medical records.

Automated concept-level information extraction to reduce the need for custom software and rules development.

OBJECTIVE: Despite at least 40 years of promising empirical performance, very few clinical natural language processing (NLP) or information extraction systems currently contribute to medical science or care. The authors address this gap by reducing the need for custom software and rules development with a graphical user interface-driven, highly generalizable approach to concept-level retrieval. MATERIALS AND METHODS: A 'learn by example' approach combines features derived from open-source NLP pipelines with open-source machine learning classifiers to automatically and iteratively evaluate top-performing configurations. The Fourth i2b2/VA Shared Task Challenge's concept extraction task provided the data sets and metrics used to evaluate performance. RESULTS: Top F-measure scores for each of the tasks were medical problems (0.83), treatments (0.82), and tests (0.83). Recall lagged precision in all experiments. Precision was near or above 0.90 in all tasks. Discussion With no customization for the tasks and less than 5 min of end-user time to configure and launch each experiment, the average F-measure was 0.83, one point behind the mean F-measure of the 22 entrants in the competition. Strong precision scores indicate the potential of applying the approach for more specific clinical information extraction tasks. There was not one best configuration, supporting an iterative approach to model creation. CONCLUSION: Acceptable levels of performance can be achieved using fully automated and generalizable approaches to concept-level information extraction. The described implementation and related documentation is available for download.

Statins and prostate cancer diagnosis and grade in a veterans population.

BACKGROUND: Although prostate cancer is commonly diagnosed, few risk factors for high-grade prostate cancer are known and few prevention strategies exist. Statins have been proposed as a possible treatment to prevent prostate cancer. METHODS: Using electronic and administrative files from the Veterans Affairs New England Healthcare System, we identified 55,875 men taking either a statin or antihypertensive medication. We used age- and multivariable-adjusted Cox proportional hazard models to calculate hazard ratios (HRs) and 95% confidence intervals (CIs) for prostate cancer incidence among patients taking statins (n = 41,078) compared with patients taking antihypertensive medications (n = 14,797). We performed similar analyses for all lipid parameters including total cholesterol examining each lipid parameter as a continuous variable and by quartiles. All statistical tests were two-sided. RESULTS: Compared with men taking an antihypertensive medication, statin users were 31% less likely (HR = 0.69, 95% CI = 0.52 to 0.90) to be diagnosed with prostate cancer. Furthermore, statin users were 14% less likely (HR = 0.86, 95% CI = 0.62 to 1.20) to be diagnosed with low-grade prostate cancer and 60% less likely (HR = 0.40, 95% CI = 0.24 to 0.65) to be diagnosed with high-grade prostate cancer compared with antihypertensive medication users. Increased levels of total cholesterol were also associated with both total (HR = 1.02, 95% CI = 1.00 to 1.05) and high-grade (HR = 1.06, 95% CI = 1.02 to 1.10) prostate cancer incidence but not with low-grade prostate cancer incidence (HR = 1.01, 95% CI = 0.98 to 1.04). CONCLUSIONS: Statin use is associated with statistically significantly reduced risk for total and high-grade prostate cancer, and increased levels of serum cholesterol are associated with higher risk for total and high-grade prostate cancer. These findings indicate that clinical trials of statins for prostate cancer prevention are warranted.

Comparative effectiveness research and medical informatics.

As is the case for environmental, ecological, astronomical, and other sciences, medical practice and research finds itself in a tsunami of data. This data deluge, due primarily to the introduction of digitalization in routine medical care and medical research, affords the opportunity for improved patient care and scientific discovery. Medical informatics is the subdiscipline of medicine created to make greater use of information in order to improve healthcare. The 4 areas of medical informatics research (information access, structure, analysis, and interaction) are used as a framework to discuss the overlap in information needs of comparative effectiveness research and potential contributions of medical informatics. Examples of progress from the medical informatics literature and the Veterans Affairs Healthcare System are provided.

Evaluation of a generalizable approach to clinical information retrieval using the automated retrieval console (ARC).

Reducing custom software development effort is an important goal in information retrieval (IR). This study evaluated a generalizable approach involving with no custom software or rules development. The study used documents "consistent with cancer" to evaluate system performance in the domains of colorectal (CRC), prostate (PC), and lung (LC) cancer. Using an end-user-supplied reference set, the automated retrieval console (ARC) iteratively calculated performance of combinations of natural language processing-derived features and supervised classification algorithms. Training and testing involved 10-fold cross-validation for three sets of 500 documents each. Performance metrics included recall, precision, and F-measure. Annotation time for five physicians was also measured. Top performing algorithms had recall, precision, and F-measure values as follows: for CRC, 0.90, 0.92, and 0.89, respectively; for PC, 0.97, 0.95, and 0.94; and for LC, 0.76, 0.80, and 0.75. In all but one case, conditional random fields outperformed maximum entropy-based classifiers. Algorithms had good performance without custom code or rules development, but performance varied by specific application.

Automatic identification and classification of surgical margin status from pathology reports following prostate cancer surgery.

Prostate cancer removal surgeries result in tumor found at the surgical margin, otherwise known as a positive surgical margin, have a significantly higher chance of biochemical recurrence and clinical progression. To support clinical outcomes assessment a system was designed to automatically identify, extract, and classify key phrases from pathology reports describing this outcome. Heuristics and boundary detection were used to extract phrases. Phrases were then classified using support vector machines into one of three classes: 'positive (involved) margins,' 'negative (uninvolved) margins,' and 'not-applicable or definitive.' A total of 851 key phrases were extracted from a sample of 782 reports produced between 1996 and 2006 from two major hospitals. Despite differences in reporting style, at least 1 sentence containing a diagnosis was extracted from 780 of the 782 reports (99.74%). Of the 851 sentences extracted, 97.3% contained diagnoses. Overall accuracy of automated classification of extracted sentences into the three categories was 97.18%.

Generating models of surgical procedures using UMLS concepts and multiple sequence alignment.

Surgical procedures can be viewed as a process composed of a sequence of steps performed on, by, or with the patient's anatomy. This sequence is typically the pattern followed by surgeons when generating surgical report narratives for documenting surgical procedures. This paper describes a methodology for semi-automatically deriving a model of conducted surgeries, utilizing a sequence of derived Unified Medical Language System (UMLS) concepts for representing surgical procedures. A multiple sequence alignment was computed from a collection of such sequences and was used for generating the model. These models have the potential of being useful in a variety of informatics applications such as information retrieval and automatic document generation.