Validation of Claims-Based Algorithm for Lyme Disease, Massachusetts, USA.

Compared with notifiable disease surveillance, claims-based algorithms estimate higher Lyme disease incidence, but their accuracy is unknown. We applied a previously developed Lyme disease algorithm (diagnosis code plus antimicrobial drug prescription dispensing within 30 days) to an administrative claims database in Massachusetts, USA, to identify a Lyme disease cohort during July 2000-June 2019. Clinicians reviewed and adjudicated medical charts from a cohort subset by using national surveillance case definitions. We calculated positive predictive values (PPVs). We identified 12,229 Lyme disease episodes in the claims database and reviewed and adjudicated 128 medical charts. The algorithm's PPV for confirmed, probable, or suspected cases was 93.8% (95% CI 88.1%-97.3%); the PPV was 66.4% (95% CI 57.5%-74.5%) for confirmed and probable cases only. In a high incidence setting, a claims-based algorithm identified cases with a high PPV, suggesting it can be used to assess Lyme disease burden and supplement traditional surveillance data.

Validation of acute myocardial infarction in the Food and Drug Administration's Mini-Sentinel program.

PURPOSE: To validate an algorithm based upon International Classification of Diseases, 9(th) revision, Clinical Modification (ICD-9-CM) codes for acute myocardial infarction (AMI) documented within the Mini-Sentinel Distributed Database (MSDD). METHODS: Using an ICD-9-CM-based algorithm (hospitalized patients with 410.x0 or 410.x1 in primary position), we identified a random sample of potential cases of AMI in 2009 from four Data Partners participating in the Mini-Sentinel Program. Cardiologist reviewers used information abstracted from hospital records to assess the likelihood of an AMI diagnosis based on criteria from the Joint European Society of Cardiology and American College of Cardiology Global Task Force. Positive predictive values (PPVs) of the ICD-9-based algorithm were calculated. RESULTS: Of the 153 potential cases of AMI identified, hospital records for 143 (93%) were retrieved and abstracted. Overall, the PPV was 86.0% (95% confidence interval; 79.2%, 91.2%). PPVs ranged from 76.3% to 94.3% across the four Data Partners. CONCLUSIONS: The overall PPV of potential AMI cases, as identified using an ICD-9-CM-based algorithm, may be acceptable for safety surveillance; however, PPVs do vary across Data Partners. This validation effort provides a contemporary estimate of the reliability of this algorithm for use in future surveillance efforts conducted using the Food and Drug Administration's MSDD.

Validation of anaphylaxis in the Food and Drug Administration's Mini-Sentinel.

PURPOSE: We aim to develop and validate the positive predictive value (PPV) of an algorithm to identify anaphylaxis using health plan administrative and claims data. Previously published PPVs for anaphylaxis using International Classification of Diseases, ninth revision, Clinical Modification (ICD-9-CM) codes range from 52% to 57%. METHODS: We conducted a retrospective study using administrative and claims data from eight health plans. Using diagnosis and procedure codes, we developed an algorithm to identify potential cases of anaphylaxis from the Mini-Sentinel Distributed Database between January 2009 and December 2010. A random sample of medical charts (n = 150) was identified for chart abstraction. Two physician adjudicators reviewed each potential case. Using physician adjudicator judgments on whether the case met diagnostic criteria for anaphylaxis, we calculated a PPV for the algorithm. RESULTS: Of the 122 patients for whom complete charts were received, 77 were judged by physician adjudicators to have anaphylaxis. The PPV for the algorithm was 63.1% (95%CI: 53.9-71.7%), using the clinical criteria by Sampson as the gold standard. The PPV was highest for inpatient encounters with ICD-9-CM codes of 995.0 or 999.4. By combining only the top performing ICD-9-CM codes, we identified an algorithm with a PPV of 75.0%, but only 66% of cases of anaphylaxis were identified using this modified algorithm. CONCLUSIONS: The PPV for the ICD-9-CM-based algorithm for anaphylaxis was slightly higher than PPV estimates reported in prior studies, but remained low. We were able to identify an algorithm that optimized the PPV but demonstrated lower sensitivity for anaphylactic events.

The organizational structure and governing principles of the Food and Drug Administration's Mini-Sentinel pilot program.

The US Food and Drug Administration's Mini-Sentinel pilot program is developing an organizational structure as well as principles and policies to govern its operations. These will inform the structure and function of the eventual Sentinel System. Mini-Sentinel is a collaboration that includes 25 participating institutions. We describe the program's current organizational structure and its major principles and policies. The organization includes a coordinating center with program leadership provided by a principal investigator; a planning board and subcommittees; an operations center; and data, methods, and protocol cores. Ad hoc workgroups are created as needed. A privacy panel advises about protection of individual health information. Principles and policies are intended to ensure that Mini-Sentinel conforms to the principles of fair information practices, protects the privacy of individual health information, maintains the security and integrity of data, assures the confidentiality of proprietary information, provides accurate and timely communications, prevents or manages conflicts of interest, and preserves respect for intellectual property rights.

Design for validation of acute myocardial infarction cases in Mini-Sentinel.

PURPOSE: To describe the acute myocardial infarction (AMI) validation project, a test case for health outcome validation within the US Food and Drug Administration-funded Mini-Sentinel pilot program. METHODS: The project consisted of four parts: (i) case identification-developing an algorithm based on the International Classification of Diseases, Ninth Revision, to identify hospitalized AMI patients within the Mini-Sentinel Distributed Database; (ii) chart retrieval-establishing procedures that ensured patient privacy (collection and transfer of minimum necessary amount of information, and redaction of direct identifiers to validate potential cases of AMI); (iii) abstraction and adjudication-trained nurse abstractors gathered key data using a standardized form with cardiologist adjudication; and (iv) calculation of the positive predictive value of the constructed algorithm. RESULTS: Key decision points included (i) breadth of the AMI algorithm, (ii) centralized versus distributed abstraction, and (iii) approaches to maintaining patient privacy and to obtaining charts for public health purposes. We used an algorithm limited to International Classification of Diseases, Ninth Revision, codes 410.x0-410.x1. Centralized data abstraction was performed because of the modest number of charts requested (<155). The project's public health status accelerated chart retrieval in most instances. CONCLUSIONS: We have established a process to validate AMI within Mini-Sentinel, which may be used for other health outcomes. Challenges include the following: (i) ensuring that only minimum necessary data are transmitted by Data Partners for centralized chart review, (ii) establishing procedures to maintain data privacy while still allowing for timely access to medical charts, and (iii) securing access to charts for public health uses that do not require approval from an institutional review board while maintaining patient privacy.

Healthcare claims-based Lyme disease case-finding algorithms in the United States: A systematic literature review.

BACKGROUND AND OBJECTIVE: Lyme disease (LD) is the fifth most commonly reported notifiable infectious disease in the United States (US) with approximately 35,000 cases reported in 2019 via public health surveillance. However, healthcare claims-based studies estimate that the number of LD cases is >10 times larger than reported through surveillance. To assess the burden of LD using healthcare claims data and the effectiveness of interventions for LD prevention and treatment, it is important to use validated well-performing LD case-finding algorithms ("LD algorithms"). We conducted a systematic literature review to identify LD algorithms used with US healthcare claims data and their validation status. METHODS: We searched PubMed and Embase for articles published in English since January 1, 2000 (search date: February 20, 2021), using the following search terms: (1) "Lyme disease"; and (2) "claim*" or "administrative* data"; and (3) "United States" or "the US*". We then reviewed the titles, abstracts, full texts, and bibliographies of the articles to select eligible articles, i.e., those describing LD algorithms used with US healthcare claims data. RESULTS: We identified 15 eligible articles. Of these, seven studies used LD algorithms with LD diagnosis codes only, four studies used LD diagnosis codes and antibiotic dispensing records, and the remaining four studies used serologic test order codes in combination with LD diagnosis codes and antibiotics records. Only one of the studies that provided data on algorithm performance: sensitivity 50% and positive predictive value 5%, and this was based on Lyme disease diagnosis code only. CONCLUSIONS: US claims-based LD case-finding algorithms have used diverse strategies. Only one algorithm was validated, and its performance was poor. Further studies are warranted to assess performance for different algorithm designs and inform efforts to better assess the true burden of LD.