Seek COVER: using a disease proxy to rapidly develop and validate a personalized risk calculator for COVID-19 outcomes in an international network.

BACKGROUND: We investigated whether we could use influenza data to develop prediction models for COVID-19 to increase the speed at which prediction models can reliably be developed and validated early in a pandemic. We developed COVID-19 Estimated Risk (COVER) scores that quantify a patient's risk of hospital admission with pneumonia (COVER-H), hospitalization with pneumonia requiring intensive services or death (COVER-I), or fatality (COVER-F) in the 30-days following COVID-19 diagnosis using historical data from patients with influenza or flu-like symptoms and tested this in COVID-19 patients. METHODS: We analyzed a federated network of electronic medical records and administrative claims data from 14 data sources and 6 countries containing data collected on or before 4/27/2020. We used a 2-step process to develop 3 scores using historical data from patients with influenza or flu-like symptoms any time prior to 2020. The first step was to create a data-driven model using LASSO regularized logistic regression, the covariates of which were used to develop aggregate covariates for the second step where the COVER scores were developed using a smaller set of features. These 3 COVER scores were then externally validated on patients with 1) influenza or flu-like symptoms and 2) confirmed or suspected COVID-19 diagnosis across 5 databases from South Korea, Spain, and the United States. Outcomes included i) hospitalization with pneumonia, ii) hospitalization with pneumonia requiring intensive services or death, and iii) death in the 30 days after index date. RESULTS: Overall, 44,507 COVID-19 patients were included for model validation. We identified 7 predictors (history of cancer, chronic obstructive pulmonary disease, diabetes, heart disease, hypertension, hyperlipidemia, kidney disease) which combined with age and sex discriminated which patients would experience any of our three outcomes. The models achieved good performance in influenza and COVID-19 cohorts. For COVID-19 the AUC ranges were, COVER-H: 0.69-0.81, COVER-I: 0.73-0.91, and COVER-F: 0.72-0.90. Calibration varied across the validations with some of the COVID-19 validations being less well calibrated than the influenza validations. CONCLUSIONS: This research demonstrated the utility of using a proxy disease to develop a prediction model. The 3 COVER models with 9-predictors that were developed using influenza data perform well for COVID-19 patients for predicting hospitalization, intensive services, and fatality. The scores showed good discriminatory performance which transferred well to the COVID-19 population. There was some miscalibration in the COVID-19 validations, which is potentially due to the difference in symptom severity between the two diseases. A possible solution for this is to recalibrate the models in each location before use.

Implementation of the COVID-19 Vulnerability Index Across an International Network of Health Care Data Sets: Collaborative External Validation Study.

BACKGROUND: SARS-CoV-2 is straining health care systems globally. The burden on hospitals during the pandemic could be reduced by implementing prediction models that can discriminate patients who require hospitalization from those who do not. The COVID-19 vulnerability (C-19) index, a model that predicts which patients will be admitted to hospital for treatment of pneumonia or pneumonia proxies, has been developed and proposed as a valuable tool for decision-making during the pandemic. However, the model is at high risk of bias according to the "prediction model risk of bias assessment" criteria, and it has not been externally validated. OBJECTIVE: The aim of this study was to externally validate the C-19 index across a range of health care settings to determine how well it broadly predicts hospitalization due to pneumonia in COVID-19 cases. METHODS: We followed the Observational Health Data Sciences and Informatics (OHDSI) framework for external validation to assess the reliability of the C-19 index. We evaluated the model on two different target populations, 41,381 patients who presented with SARS-CoV-2 at an outpatient or emergency department visit and 9,429,285 patients who presented with influenza or related symptoms during an outpatient or emergency department visit, to predict their risk of hospitalization with pneumonia during the following 0-30 days. In total, we validated the model across a network of 14 databases spanning the United States, Europe, Australia, and Asia. RESULTS: The internal validation performance of the C-19 index had a C statistic of 0.73, and the calibration was not reported by the authors. When we externally validated it by transporting it to SARS-CoV-2 data, the model obtained C statistics of 0.36, 0.53 (0.473-0.584) and 0.56 (0.488-0.636) on Spanish, US, and South Korean data sets, respectively. The calibration was poor, with the model underestimating risk. When validated on 12 data sets containing influenza patients across the OHDSI network, the C statistics ranged between 0.40 and 0.68. CONCLUSIONS: Our results show that the discriminative performance of the C-19 index model is low for influenza cohorts and even worse among patients with COVID-19 in the United States, Spain, and South Korea. These results suggest that C-19 should not be used to aid decision-making during the COVID-19 pandemic. Our findings highlight the importance of performing external validation across a range of settings, especially when a prediction model is being extrapolated to a different population. In the field of prediction, extensive validation is required to create appropriate trust in a model.

Baseline phenotype and 30-day outcomes of people tested for COVID-19: an international network cohort including >3.32 million people tested with real-time PCR and >219,000 tested positive for SARS-CoV-2 in South Korea, Spain and the United States.

Early identification of symptoms and comorbidities most predictive of COVID-19 is critical to identify infection, guide policies to effectively contain the pandemic, and improve health systems' response. Here, we characterised socio-demographics and comorbidity in 3,316,107persons tested and 219,072 persons tested positive for SARS-CoV-2 since January 2020, and their key health outcomes in the month following the first positive test. Routine care data from primary care electronic health records (EHR) from Spain, hospital EHR from the United States (US), and claims data from South Korea and the US were used. The majority of study participants were women aged 18-65 years old. Positive/tested ratio varied greatly geographically (2.2:100 to 31.2:100) and over time (from 50:100 in February-April to 6.8:100 in May-June). Fever, cough and dyspnoea were the most common symptoms at presentation. Between 4%-38% required admission and 1-10.5% died within a month from their first positive test. Observed disparity in testing practices led to variable baseline characteristics and outcomes, both nationally (US) and internationally. Our findings highlight the importance of large scale characterization of COVID-19 international cohorts to inform planning and resource allocation including testing as countries face a second wave.

A method for cohort selection of cardiovascular disease records from an electronic health record system.

INTRODUCTION: An electronic healthcare record (EHR) system, when used by healthcare providers, improves the quality of care for patients and helps to lower costs. Information collected from manual or electronic health records can also be used for purposes not directly related to patient care delivery, in which case it is termed secondary use. EHR systems facilitate the collection of this secondary use data, which can be used for research purposes like observational studies, taking advantage of improvement in the structuring and retrieval of patient information. However, some of the following problems are common when conducting a research using this kind of data: (i) Over time, systems and data storage methods become obsolete; (ii) Data concerns arise since the data is being used in a context removed from its original intention; (iii) There are privacy concerns when sharing data about individual subjects; (iv) The partial availability of standard medical vocabularies and natural language processing tools for non-English language limits information extraction from structured and unstructured data in the EHR systems. A systematic approach is therefore needed to overcome these, where local data processing is performed prior to data sharing. METHOD: The proposed study describes a local processing method to extract cohorts of patients for observational studies in four steps: (1) data reorganization from an existing local logical schema into a common external schema over which information can be extracted; (2) cleaning of data, generation of the database profile and retrieval of indicators; (3) computation of derived variables from original variables; (4) application of study design parameters to transform longitudinal data into anonymized data sets ready for statistical analysis and sharing. Mapping from the local logical schema into a common external schema must be performed differently for each EHR and is not subject of this work, but step 2, 3 and 4 are common to all EHRs. The external schema accepts parameters that facilitate the extraction of different cohorts for different studies without having to change the extraction algorithms, and ensures that, given an immutable data set, can be done by the idempotent process. Statistical analysis is part of the process to generate the results necessary for inclusion in reports. The generation of indicators to describe the database allows description of its characteristics, highlighting study results. The set extraction/statistical processing is available in a version controlled repository and can be used at any time to reproduce results, allowing the verification of alterations and error corrections. This methodology promotes the development of reproducible studies and allows potential research problems to be tracked upon extraction algorithms and statistical methods RESULTS: This method was applied to an admissions database, SI3, from the InCor-HCFMUSP, a tertiary referral hospital for cardiovascular disease in the city of São Paulo, as a source of secondary data with 1116848 patients records from 1999 up to 2013. The cleaning process resulted in 313894 patients records and 27698 patients in the cohort selection, with the following criteria: study period: 2003-2013, gender: Male, Female, age:≥18years old, at least 2 outpatient encounters, diagnosis of cardiovascular disease (ICD-10 codes: I20-I25, I64-I70 and G45). An R script provided descriptive statistics of the extracted cohort. CONCLUSION: This method guarantees a reproducible cohort extraction for use of secondary data in observational studies with enough parameterization to support different study designs and can be used on diverse data sources. Moreover it allows observational electronic health record cohort research to be performed in a non-English language with limited international recognized medical vocabulary.

Método de extração de coortes em bases de dados assistenciais para estudos da doença cardiovascular / A method for the cohort selection of cardiovascular disease records from an electronic health record system

A informação coletada de prontuários manuais ou eletrônicos, quando usada para propósitos não diretamente relacionados ao atendimento do paciente, é chamado de uso secundário de dados. A adoção de um sistema de registro eletrônico em saúde (RES) pode facilitar a coleta de dados para uso secundário em pesquisa, aproveitando as melhorias na estruturação e recuperação da informação do paciente, recursos não disponíveis nos tradicionais prontuários em papel. Estudos observacionais baseados no uso secundário de dados têm o potencial de prover evidências para a construção de políticas em saúde. No entanto, a pesquisa através desses dados apresenta problemas característicos a essa fonte de dados. Ao longo do tempo, os sistemas e seus métodos de armazenar dados se tornam obsoletos ou são reestruturados, existem questões de privacidade para o compartilhamento dos dados dos indivíduos e questões relacionadas ao uso desses dados em um contexto diferente do seu propósito original. É necessária uma abordagem sistemática para contornar esses problemas, onde o processamento dos dados é efetuado antes do seu compartilhamento. O objetivo desta Tese é propor um método de extração de coortes de pacientes para estudos observacionais contemplando quatro etapas: (1) mapeamento: a reorganização de dados a partir de um esquema lógico existente em um esquema externo comum sobre o qual é aplicado o método; (2) limpeza: preparação dos dados, levantamento do perfil da base de dados e cálculo dos indicadores de qualidade; (3) seleção da coorte: aplicação dos parâmetros do estudo para seleção de dados longitudinais dos pacientes para a formação da coorte; (4) transformação: derivação de variáveis de estudo que não estão presentes nos dados originais e transformação dos dados longitudinais em dados anonimizados prontos para análise estatística e compartilhamento. O mapeamento é uma etapa específica para cada RES e não é objeto desse trabalho, mas foi realizada para a aplicação do método....

Information collected from manual or electronic health records can also be used for purposes not directly related to patient care delivery, in which case it is termed secondary use. The adoption of electronic health record (EHR) systems can facilitate the collection of this secondary use data, which can be used for research purposes such as observational studies. These studies have the power to provide necessary evidence for the formation of healthcare policies. However, several problems arise when conducting research using this kind of data. For example, over time, systems and their methods of storing data become obsolete, data concerns arise since the data is being used in a different context to where it originated and privacy concerns arise when sharing data about individual subjects. To overcome these problems a systematic approach is required where local data processing is performed prior to data sharing. The objective of this thesis is to propose a method to extract patient cohorts for observational studies in four steps: (1) data mapping from an existing local logical schema into a common external schema over which information can be extracted; (2) cleaning of data, generation of the database profile and retrieval of indicators; (3) computation of derived variables from original variables; (4) application of study design parameters to transform longitudinal data into anonymized data sets ready for statistical analysis and sharing. Mapping is a specific stage for each EHR and although it is not the focus of this work, a detail of the mapping is included. The stages of cleaning, selection of cohort and transformation are common to all EHRs and form the main objective. The use of an external schema allows the use of parameters that facilitate the extraction of different cohorts for different studies without the need for changes to the extraction algorithms. This ensures that, given an immutable dataset, the extraction can be done by...