Defining the Subtypes of Long COVID and Risk Factors for Prolonged Disease: Population-Based Case-Crossover Study.

BACKGROUND: There have been over 772 million confirmed cases of COVID-19 worldwide. A significant portion of these infections will lead to long COVID (post-COVID-19 condition) and its attendant morbidities and costs. Numerous life-altering complications have already been associated with the development of long COVID, including chronic fatigue, brain fog, and dangerous heart rhythms. OBJECTIVE: We aim to derive an actionable long COVID case definition consisting of significantly increased signs, symptoms, and diagnoses to support pandemic-related clinical, public health, research, and policy initiatives. METHODS: This research employs a case-crossover population-based study using International Classification of Diseases, 10th Revision, Clinical Modification (ICD-10-CM) data generated at Veterans Affairs medical centers nationwide between January 1, 2020, and August 18, 2022. In total, 367,148 individuals with ICD-10-CM data both before and after a positive COVID-19 test were selected for analysis. We compared ICD-10-CM codes assigned 1 to 7 months following each patient's positive test with those assigned up to 6 months prior. Further, 350,315 patients had novel codes assigned during this window of time. We defined signs, symptoms, and diagnoses as being associated with long COVID if they had a novel case frequency of ≥1:1000, and they significantly increased in our entire cohort after a positive test. We present odds ratios with CIs for long COVID signs, symptoms, and diagnoses, organized by ICD-10-CM functional groups and medical specialty. We used our definition to assess long COVID risk based on a patient's demographics, Elixhauser score, vaccination status, and COVID-19 disease severity. RESULTS: We developed a long COVID definition consisting of 323 ICD-10-CM diagnosis codes grouped into 143 ICD-10-CM functional groups that were significantly increased in our 367,148 patient post-COVID-19 population. We defined 17 medical-specialty long COVID subtypes such as cardiology long COVID. Patients who were COVID-19-positive developed signs, symptoms, or diagnoses included in our long COVID definition at a proportion of at least 59.7% (268,320/449,450, based on a denominator of all patients who were COVID-19-positive). The long COVID cohort was 8 years older with more comorbidities (2-year Elixhauser score 7.97 in the patients with long COVID vs 4.21 in the patients with non-long COVID). Patients who had a more severe bout of COVID-19, as judged by their minimum oxygen saturation level, were also more likely to develop long COVID. CONCLUSIONS: An actionable, data-driven definition of long COVID can help clinicians screen for and diagnose long COVID, allowing identified patients to be admitted into appropriate monitoring and treatment programs. This long COVID definition can also support public health, research, and policy initiatives. Patients with COVID-19 who are older or have low oxygen saturation levels during their bout of COVID-19, or those who have multiple comorbidities should be preferentially watched for the development of long COVID.

ACORN SDOH survey: Terminological representation for use with NLP and CDS.

Objective: Social Determinants of Health (SDOH) greatly influence health outcomes. SDOH surveys, such as the Assessing Circumstances & Offering Resources for Needs (ACORN) survey, have been developed to screen for SDOH in Veterans. The purpose of this study is to determine the terminological representation of the ACORN survey, to aid in natural language processing (NLP). Methods: Each ACORN survey question was read to determine its concepts. Next, Solor was searched for each of the concepts and for the appropriate attributes. If no attributes or concepts existed, they were proposed. Then, each question's concepts and attributes were arranged into subject-relation-object triples. Results: Eleven unique attributes and 18 unique concepts were proposed. These results demonstrate a gap in representing SDOH with terminologies. We believe that using these new concepts and relations will improve NLP, and thus, the care provided to Veterans.

COVID-19 vaccination and venous thromboembolism risk in older veterans.

Introduction: It is important for SARS-CoV-2 vaccine providers, vaccine recipients, and those not yet vaccinated to be well informed about vaccine side effects. We sought to estimate the risk of post-vaccination venous thromboembolism (VTE) to meet this need. Methods: We conducted a retrospective cohort study to quantify excess VTE risk associated with SARS-CoV-2 vaccination in US veterans age 45 and older using data from the Department of Veterans Affairs (VA) National Surveillance Tool. The vaccinated cohort received at least one dose of a SARS-CoV-2 vaccine at least 60 days prior to 3/06/22 (N = 855,686). The control group was those not vaccinated (N = 321,676). All patients were COVID-19 tested at least once before vaccination with a negative test. The main outcome was VTE documented by ICD10-CM codes. Results: Vaccinated persons had a VTE rate of 1.3755 (CI: 1.3752-1.3758) per thousand, which was 0.1 percent over the baseline rate of 1.3741 (CI: 1.3738-1.3744) per thousand in the unvaccinated patients, or 1.4 excess cases per 1,000,000. All vaccine types showed a minimal increased rate of VTE (rate of VTE per 1000 was 1.3761 (CI: 1.3754-1.3768) for Janssen; 1.3757 (CI: 1.3754-1.3761) for Pfizer, and for Moderna, the rate was 1.3757 (CI: 1.3748-1.3877)). The tiny differences in rates comparing either Janssen or Pfizer vaccine to Moderna were statistically significant (p < 0.001). Adjusting for age, sex, BMI, 2-year Elixhauser score, and race, the vaccinated group had a minimally higher relative risk of VTE as compared to controls (1.0009927 CI: 1.007673-1.0012181; p < 0.001). Conclusion: The results provide reassurance that there is only a trivial increased risk of VTE with the current US SARS-CoV-2 vaccines used in veterans older than age 45. This risk is significantly less than VTE risk among hospitalized COVID-19 patients. The risk-benefit ratio favors vaccination, given the VTE rate, mortality, and morbidity associated with COVID-19 infection.

Leukotriene inhibitors with dexamethasone show promise in the prevention of death in COVID-19 patients with low oxygen saturations.

Introduction: COVID-19 is a major health threat around the world causing hundreds of millions of infections and millions of deaths. There is a pressing global need for effective therapies. We hypothesized that leukotriene inhibitors (LTIs), that have been shown to lower IL6 and IL8 levels, may have a protective effect in patients with COVID-19. Methods: In this retrospective controlled cohort study, we compared death rates in COVID-19 patients who were taking a LTI with those who were not taking an LTI. We used the Department of Veterans Affairs (VA) Corporate Data Warehouse (CDW) to create a cohort of COVID-19-positive patients and tracked their use of LTIs between November 1, 2019 and November 11, 2021. Results: Of the 1,677,595 cohort of patients tested for COVID-19, 189,195 patients tested positive for COVID-19. Forty thousand seven hundred one were admitted. 38,184 had an oxygen requirement and 1214 were taking an LTI. The use of dexamethasone plus a LTI in hospital showed a survival advantage of 13.5% (CI: 0.23%-26.7%; p < 0.01) in patients presenting with a minimal O2Sat of 50% or less. For patients with an O2Sat of <60 and <50% if they were on LTIs as outpatients, continuing the LTI led to a 14.4% and 22.25 survival advantage if they were continued on the medication as inpatients. Conclusions: When combined dexamethasone and LTIs provided a mortality benefit in COVID-19 patients presenting with an O2 saturations <50%. The LTI cohort had lower markers of inflammation and cytokine storm.

CDS-Compare: A Web Application for Machine Learning Assisted Curation of Clinical Order Sets.

Order sets that adhere to disease-specific guidelines have been shown to increase clinician efficiency and patient safety but curating these order sets, particularly for consistency across multiple sites, is difficult and time consuming. We created software called CDS-Compare to alleviate the burden on expert reviewers in rapidly and effectively curating large databases of order sets. We applied our clustering-based software to a database of NLP-processed order sets extracted from VA's Electronic Health Record, then had subject-matter experts review the web application version of our software for clustering validity.

Establishing Trust in Pharmaceutical Data with an Independent Verification and Validation Methodology.

Our aim is to demonstrate a general-purpose data and knowledge validation approach that enables reproducible metrics for data and knowledge quality and safety. We researched widely accepted statistical process control methods from high-quality, high-safety industries and applied them to pharmacy prescription data being migrated between EHRs. Natural language medication instructions from prescriptions were independently categorized by two terminologists as a first step toward encoding those medication instructions using standardized terminology. Overall, the weighted average of medication instructions that were matched by reviewers was 43%, with strong agreement between reviewers for short instructions (K=0.82) and long instructions (K=0.85), and moderate agreement for medium instructions (K=0.61). Category definitions will be refined in future work to mitigate discrepancies. We recommend incorporating appropriate statistical tests, such as evaluating inter-rater and intra-rater reliability and bivariate comparison of reviewer agreement over an adequate statistical sample, when developing benchmarks for health data and knowledge quality and safety.

Automated Modeling of Clinical Narrative with High Definition Natural Language Processing Using Solor and Analysis Normal Form.

OBJECTIVE: One important concept in informatics is data which meets the principles of Findability, Accessibility, Interoperability and Reusability (FAIR). Standards, such as terminologies (findability), assist with important tasks like interoperability, Natural Language Processing (NLP) (accessibility) and decision support (reusability). One terminology, Solor, integrates SNOMED CT, LOINC and RxNorm. We describe Solor, HL7 Analysis Normal Form (ANF), and their use with the high definition natural language processing (HD-NLP) program. METHODS: We used HD-NLP to process 694 clinical narratives prior modeled by human experts into Solor and ANF. We compared HD-NLP output to the expert gold standard for 20% of the sample. Each clinical statement was judged "correct" if HD-NLP output matched ANF structure and Solor concepts, or "incorrect" if any ANF structure or Solor concepts were missing or incorrect. Judgements were summed to give totals for "correct" and "incorrect". RESULTS: 113 (80.7%) correct, 26 (18.6%) incorrect, and 1 error. Inter-rater reliability was 97.5% with Cohen's kappa of 0.948. CONCLUSION: The HD-NLP software provides useable complex standards-based representations for important clinical statements designed to drive CDS.

Relationship Between Terrestrial Background and Remedial Criteria for Naturally Occurring Radioactive Material in the United States.

ABSTRACT: In the United States (US), Federal and State agencies have established radiological public exposure limits and remedial action ("clean up") criteria for naturally occurring radionuclides (NORM-primarily for uranium and thorium series radionuclides). Often, these criteria are intended to control human exposure to what is referred to in the US as technologically enhanced naturally occurring radioactive material (TENORM). This can be any naturally occurring radioactive material for which the potential for human exposure has been enhanced due to anthropogenic (human activities), e.g., removal from its "place in nature," and/or processed in some way resulting in concentration. In some cases, the values of these regulatory criteria can be similar to or even less than those levels of exposure and those concentrations of NORM that exists in nature independent of any previous human activity. The potential variability of NORM radionuclides in the soil and rocks can be significant, even over relatively short distances or depths due to factors such as geology, hydrology, and geochemistry. Given this, it is important to recognize that defining "the radiation background" for purposes of establishing and/or comparing remedial action criteria and/or exposure limits requires recognition of the specificity at the location(s) of interest, not in other geological and/or mineralogical regimes several miles away. The purpose of this paper is to demonstrate this variability for comparison to exposure levels and concentrations being defined in the US as levels above which require regulatory control and / or above which are being defined as an "unacceptable risk." The primary background exposure component of specific interest here is the annual dose contribution from terrestrial radiation exposure, i.e., from uranium and thorium series radionuclides in the ground, excluding radon inhalation. The exposure sources being controlled by some US regulatory limits are primarily associated with the primordial radionuclides in soil. The average annual terrestrial component of background can vary by upwards of a few tenths of a mSv across the US that can be several times higher than the applicable exposure limits. This can result in "unacceptable risk" or "remedial action" concentration criteria statistically equivalent to or less than the background concentrations of these same primordial nuclides. The statistical and analytical uncertainties of distinguishing naturally occurring radionuclides (i.e., NORM) from those resulting from anthropogenic (human caused) activities (i.e., TENORM) can be quite challenging and in some cases may be technically impossible. Consideration must be given to the relationship of the amount of actual total risk avoidance achieved if any, relative to the traditional health and safety risks of construction and associated construction and waste management costs for remedial activities, so that a practical and scientifically based approach for development of these criteria can be achieved.

Occupational Radiation Protection Aspects of Alkaline Leach Uranium in Situ Recovery (ISR) Facilities in the United States.

In situ recovery or in situ leach (ISR/ISL) uranium facilities, also referred to in the past as "uranium solution mining" have operated since the late 1960s in the US and in recent years have accounted for over 70% of US production and, internationally, approximately half of worldwide uranium supplies. Note that throughout this paper, the uranium in situ recovery process, also known as in situ leach, will be abbreviated as "ISR." This paper presents a summary of the occupational radiation protection aspects of typical ISR processes being employed in the United States today that have traditionally used alkaline-based uranium recovery solutions known as lixiviants. The paper describes the health physics and associated monitoring programs necessary to adequately measure and control radiological doses to workers based on the radiological character of these processes. Although many radiological characteristics are similar to that of conventional mills, conventional-type tailings as such are not generated. However, liquid and solid by-product materials may be generated and impounded, which can result in sources of occupational exposure. Some special monitoring considerations are required due to the manner in which Rn gas is involved in the process. The major aspects of the health physics and radiation protection programs that have been developed at these facilities over many years are discussed and listed in the Conclusion section of the paper.

One-Way and Round-Trip Analysis Demonstrates Surprising Limitations of Standards-Based Terminology Maps.

The informatics community has a long-standing vision of freely flowing and highly re-usable patient-specific clinical data that improves care quality and safety. We sought to evaluate the extent to which a standards-based mapping approach is sufficient to support semantic interoperability. We simulated large-scale clinical data transmission and measured semantic success between VA and DoD systems via one-way testing (OWT) and round-trip testing (RTT). Simulations were accomplished via SQL queries and production standards-based maps for medications, allergens, document titles, vitals and payers. Success rates for mapping local codes to national standards varied from 62.5% for DoD document titles and medications, to 100% for VA and DoD vital signs. Successful, one-way testing was considerably lower, ranging from 8.52% to 62.7%. Round-trip success rates were lower still, ranging from 1.7% to 76.3%. We present an error framework, lessons learned, and proposed mitigating steps to enhance standards-based semantic interoperability.

Establishing Radiological Screening Levels for Defense-related Uranium Mine (DRUM) Sites on BLM Land Using a Recreational Future-use Scenario.

Thousands of former uranium mining sites in the United States, primarily in the southwestern states of Colorado, Arizona, New Mexico, Arizona, and Utah, are being identified and evaluated to assess their potential for causing public and environmental impacts. The common radiological contaminant of concern that characterizes these sites is naturally occurring uranium ore and associated wastes that may have been left behind postmining. The majority of these sites were abandoned and in general, are referred to as abandoned uranium mines, regardless of the government authority currently managing the land or in some cases, assigned responsibility for the oversight of assessment and remediation. The U.S. Department of Energy has identified over 4,000 defense-related uranium mine sites from which uranium ore was purchased by the U.S. government for nuclear defense programs prior to 1970. U.S. Department of Energy has established a program to inventory and perform environmental screening on defense-related uranium mine sites. The focus of this paper is the approximately 2,400 defense-related uranium mine sites located on federal land managed by the Bureau of Land Management and the U.S. Forest Service. This paper presents the results of an analysis to develop radiological screening criteria for U.S. Department of Energy's defense-related uranium mine sites that can be used as input to the overall ranking of these sites for prioritization of additional assessment, reclamation, or remedial actions. For these sites managed by Bureau of Land Management, public access is typically limited to short-term use, primarily for recreational purposes. This is a broad category that can cover a range of possible activities, including camping, hiking, hunting, biking, all-terrain vehicle use, and horseback riding. The radiological screening levels were developed by calculating the radiological dose to future recreational users of defense-related uranium mine sites assuming a future camper spends two weeks per year at the site engaged in recreational activities. Although a number of possible exposure pathways were included in this analysis (inhalation and ingestion of dust and soil, radon and progeny inhalation, and gamma radiation exposure from the soil), it is desirable as a practical matter to determine what gamma exposure rate would ensure that the annual acceptable exposure as determined by the regulatory authority will not be exceeded in the future. Because these sites are generally remote and located in semiarid environments, traditional exposure scenarios often applied in these types of analyses (e.g., subsistent farmers and ranchers), including exposure pathways for the ingestion of locally grown food products and water, were not considered relevant to short-term recreational use.

Importance of Uranium Recovery Facility Product Characteristics for Dose Assessment and Assignment.

Two traditional methods are used, often in combination, for assessing the intake and resulting dose from the inhalation of radioactive aerosols. The first calculates the intake based on air sampling programs and assessing and assigning dose using published dose conversion factors. The second approach assigns dose from the results of bioassay programs using measurements of radionuclides in human excreta (ex vivo, sometimes referred to as "in vitro") or via direct measurements of radionuclides in the body (in vivo) in combination with metabolic models. This paper describes standard practices using each of these methods to assess and assign worker dose from inhalation of uranium products produced at natural uranium processing facilities, namely uranium mills and in-situ uranium recovery facilities (ISRs). Chemical speciation is an important consideration, which relates directly to solubility in body fluids and associated metabolic behavior. The concepts are illustrated by specific examples applicable to the products to which workers can be exposed at natural uranium processing facilities.

Uranium Mining and Norm in North America-Some Perspectives on Occupational Radiation Exposure.

All soils and rocks contain naturally occurring radioactive materials (NORM). Many ores and raw materials contain relatively elevated levels of natural radionuclides, and processing such materials can further increase the concentrations of naturally occurring radionuclides. In the U.S., these materials are sometimes referred to as technologically-enhanced naturally occurring radioactive materials (TENORM). Examples of NORM minerals include uranium ores, monazite (a source of rare earth minerals), and phosphate rock used to produce phosphate fertilizer. The processing of these materials has the potential to result in above-background radiation exposure to workers. Following a brief review of the sources and potential for worker exposure from NORM in these varied industries, this paper will then present an overview of uranium mining and recovery in North America, including discussion on the mining methods currently being used for both conventional (underground, open pit) and in situ leach (ISL), also referred to as In Situ Recovery (ISR), and the production of NORM materials and wastes associated with these uranium recovery methods. The radiological composition of the NORM products and wastes produced and recent data on radiological exposures received by workers in the North American uranium recovery industry are then described. The paper also identifies the responsible government agencies in the U.S. and Canada assigned the authority to regulate and control occupational exposure from these NORM materials.

ICD9-CM Claims Data are Insufficient for Influenza Surveillance.

BACKGROUND: Influenza and Influenza like illness are representative of a class of epidemic infectious diseases that have important public health implications. Early detection via biosurveillance can speed lifesaving public heath responses. In the United States, biosurveillance is typically conducted using ICD9 coded visit diagnoses and uncoded chief complaint data. OBJECTIVE: To determine the accuracy of ICD9 diagnoses using laboratory confirmed cases as the gold standard. DESIGN: A six-year retrospective cohort study. SETTING: A tertiary referral center. PATIENTS: All 3,825 patients with an ICD9-CM diagnosis of Influenza and all 1455 patients with laboratory confirmed Influenza. RESULTS: Of the 3,828 patients assigned ICD9-CM visit codes indicating a diagnosis of Influenza, 2,825 were not confirmed by laboratory testing and 1,003 patients under went laboratory testing. Only 664 (66.2%) tested positive for Influenza. Of the 1,455 patients who tested positive for Influenza 45.6% were identified by ICD9-CM code. CONCLUSION: ICD9-CM had a low 66.2% Positive Predictive Value (precision) for Influenza and a low 45.6% Sensitivity (recall) for Influenza in patients tested for Influenza. ICD9 coded visit diagnoses/claims data are insufficient alone to serve as the basis for Influenza Surveillance. PRIMARY FUNDING SOURCE: CDC grants PH00022 and HK00014.

Worker protection implications of the solubility and human metabolism of modern uranium mill products in the U.S.

This paper presents an analysis of the implications of some recent studies performed to characterize uranium products from modern uranium recovery facilities important for worker protection. Assumptions about the solubility (related to the molecular species being produced) of these materials in humans are critical to properly assess radiation dose from intakes, understand chemotoxic implications, and establish protective exposure standards (airborne concentrations, limits on intake, etc.). Recent studies, as well as information in the historical professional literature, were reviewed that address the issue of solubility and related characteristics. These data are important for the design of programs for assessment of both chemical and radiological aspects of worker exposure to the products of modern uranium recovery plants (conventional uranium mills and in situ recovery plants; i.e., ISRs). The data suggest strongly that the oxide form produced by these facilities (and therefore, product solubility) is related to precipitation chemistry and thermal exposure (dryer temperature). Given the peroxide precipitation and low temperature drying methods being used at many modern uranium recovery facilities in the U.S. today, very soluble products are being produced. The dosimetric impacts of these products to the pulmonary system (except perhaps in case of an extreme acute insult) would be small, and any residual pulmonary retention beyond a month or two would most likely be too small to measure by traditional urinalysis sampling or the current state-of-the-art of natural uranium in vivo lung counting techniques. Uranium recovery plants should revisit the adequacy of current bioassay programs in the context of their process and product specifics. Workers potentially exposed to these very soluble yellowcake concentrates should have urine specimens submitted for uranium analysis on an approximately weekly basis, including analysis for the biomarkers associated with potential renal injury [e.g., glucose, lactate dehydrogenase (LDH) and protein albumen]. Additionally, implications for compliance with current U.S. Nuclear Regulatory Commission (NRC) regulations (e.g., 10 CFR20) are discussed. NRC, the applicable Agreement State agencies, and licensees need to recognize the importance of the uranium chemotoxicity versus dose relationship in the interest of worker protection.

Assisted annotation of medical free text using RapTAT.

OBJECTIVE: To determine whether assisted annotation using interactive training can reduce the time required to annotate a clinical document corpus without introducing bias. MATERIALS AND METHODS: A tool, RapTAT, was designed to assist annotation by iteratively pre-annotating probable phrases of interest within a document, presenting the annotations to a reviewer for correction, and then using the corrected annotations for further machine learning-based training before pre-annotating subsequent documents. Annotators reviewed 404 clinical notes either manually or using RapTAT assistance for concepts related to quality of care during heart failure treatment. Notes were divided into 20 batches of 19-21 documents for iterative annotation and training. RESULTS: The number of correct RapTAT pre-annotations increased significantly and annotation time per batch decreased by ~50% over the course of annotation. Annotation rate increased from batch to batch for assisted but not manual reviewers. Pre-annotation F-measure increased from 0.5 to 0.6 to >0.80 (relative to both assisted reviewer and reference annotations) over the first three batches and more slowly thereafter. Overall inter-annotator agreement was significantly higher between RapTAT-assisted reviewers (0.89) than between manual reviewers (0.85). DISCUSSION: The tool reduced workload by decreasing the number of annotations needing to be added and helping reviewers to annotate at an increased rate. Agreement between the pre-annotations and reference standard, and agreement between the pre-annotations and assisted annotations, were similar throughout the annotation process, which suggests that pre-annotation did not introduce bias. CONCLUSIONS: Pre-annotations generated by a tool capable of interactive training can reduce the time required to create an annotated document corpus by up to 50%.

Development and evaluation of RapTAT: a machine learning system for concept mapping of phrases from medical narratives.

Rapid, automated determination of the mapping of free text phrases to pre-defined concepts could assist in the annotation of clinical notes and increase the speed of natural language processing systems. The aim of this study was to design and evaluate a token-order-specific naïve Bayes-based machine learning system (RapTAT) to predict associations between phrases and concepts. Performance was assessed using a reference standard generated from 2860 VA discharge summaries containing 567,520 phrases that had been mapped to 12,056 distinct Systematized Nomenclature of Medicine - Clinical Terms (SNOMED CT) concepts by the MCVS natural language processing system. It was also assessed on the manually annotated, 2010 i2b2 challenge data. Performance was established with regard to precision, recall, and F-measure for each of the concepts within the VA documents using bootstrapping. Within that corpus, concepts identified by MCVS were broadly distributed throughout SNOMED CT, and the token-order-specific language model achieved better performance based on precision, recall, and F-measure (0.95±0.15, 0.96±0.16, and 0.95±0.16, respectively; mean±SD) than the bag-of-words based, naïve Bayes model (0.64±0.45, 0.61±0.46, and 0.60±0.45, respectively) that has previously been used for concept mapping. Precision, recall, and F-measure on the i2b2 test set were 92.9%, 85.9%, and 89.2% respectively, using the token-order-specific model. RapTAT required just 7.2ms to map all phrases within a single discharge summary, and mapping rate did not decrease as the number of processed documents increased. The high performance attained by the tool in terms of both accuracy and speed was encouraging, and the mapping rate should be sufficient to support near-real-time, interactive annotation of medical narratives. These results demonstrate the feasibility of rapidly and accurately mapping phrases to a wide range of medical concepts based on a token-order-specific naïve Bayes model and machine learning.

ICD90CM claims data are insufficient for influenza surveillance.

Influenza and Influenza like illness are representative of a class of epidemic infectious diseases that have important public health implications. Early detection via Biosurveillance can speed life saving public heath responses. In the United States Biosurveillance is typically conducted using ICD9 coded visit diagnoses and uncoded chief complaint data. To determine the accuracy of ICD9 diagnoses using laboratory confirmed cases as the gold standard. We determined the sensitivity and specificity of ICD9 in detecting laboratory confirmed vs unconfirmed Influenza. ICD9-CM had a low 66.2% Positive Predictive Value (precision) for Influenza and a low 45.6% Sensitivity (recall) for Influenza. ICD9-CM proved insufficient alone for use in biosurveillance.

Exploring the frontier of electronic health record surveillance: the case of postoperative complications.

BACKGROUND: The aim of this study was to build electronic algorithms using a combination of structured data and natural language processing (NLP) of text notes for potential safety surveillance of 9 postoperative complications. METHODS: Postoperative complications from 6 medical centers in the Southeastern United States were obtained from the Veterans Affairs Surgical Quality Improvement Program (VASQIP) registry. Development and test datasets were constructed using stratification by facility and date of procedure for patients with and without complications. Algorithms were developed from VASQIP outcome definitions using NLP-coded concepts, regular expressions, and structured data. The VASQIP nurse reviewer served as the reference standard for evaluating sensitivity and specificity. The algorithms were designed in the development and evaluated in the test dataset. RESULTS: Sensitivity and specificity in the test set were 85% and 92% for acute renal failure, 80% and 93% for sepsis, 56% and 94% for deep vein thrombosis, 80% and 97% for pulmonary embolism, 88% and 89% for acute myocardial infarction, 88% and 92% for cardiac arrest, 80% and 90% for pneumonia, 95% and 80% for urinary tract infection, and 77% and 63% for wound infection, respectively. A third of the complications occurred outside of the hospital setting. CONCLUSIONS: Computer algorithms on data extracted from the electronic health record produced respectable sensitivity and specificity across a large sample of patients seen in 6 different medical centers. This study demonstrates the utility of combining NLP with structured data for mining the information contained within the electronic health record.

Detecting temporal expressions in medical narratives.

BACKGROUND: Clinical practice and epidemiological information aggregation require knowing when, how long, and in what sequence medically relevant events occur. The Temporal Awareness and Reasoning Systems for Question Interpretation (TARSQI) Toolkit (TTK) is a complete, open source software package for the temporal ordering of events within narrative text documents. TTK was developed on newspaper articles. We extended TTK to support medical notes using veterans' affairs (VA) clinical notes and compared it to TTK. METHODS: We used a development set consisting of 200 VA clinical notes to modify and append rules to TTK's time tagger, creating Med-TTK. We then evaluated the performances of TTK and Med-TTK on an independent random selection of 100 clinical notes. Evaluation tasks were to identify and classify time-referring expressions as one of four temporal classes (DATE, TIME, DURATION, and SET). The reference standard for this test set was generated by dual human manual review with disagreements resolved by a third reviewer. Outcome measures included recall and precision for each class, and inter-rater agreement scores. RESULTS: There were 3146 temporal expressions in the reference standard. TTK identified 1595 temporal expressions. Recall was 0.15 (95% confidence interval [CI] 0.12-0.15) and precision was 0.27 (95% CI 0.25-0.29) for TTK. Med-TTK identified 3174 expressions. Recall was 0.86 (95% CI 0.84-0.87) and precision was 0.85 (95% CI 0.84-0.86) for Med-TTK. CONCLUSION: The algorithms for identifying and classifying temporal expressions in medical narratives developed within Med-TTK significantly improved performance compared to TTK. Natural language processing applications such as Med-TTK provide a foundation for meaningful longitudinal mapping of patient history events among electronic health records. The tool can be accessed at the following site: http://code.google.com/p/med-ttk/.