RESUMEN
The propensity score-integrated composite likelihood (PSCL) method is one method that can be utilized to design and analyze an application when real-world data (RWD) are leveraged to augment a prospectively designed clinical study. In the PSCL, strata are formed based on propensity scores (PS) such that similar subjects in terms of the baseline covariates from both the current study and RWD sources are placed in the same stratum, and then composite likelihood method is applied to down-weight the information from the RWD. While PSCL was originally proposed for a fixed design, it can be extended to be applied under an adaptive design framework with the purpose to either potentially claim an early success or to re-estimate the sample size. In this paper, a general strategy is proposed due to the feature of PSCL. For the possibility of claiming early success, Fisher's combination test is utilized. When the purpose is to re-estimate the sample size, the proposed procedure is based on the test proposed by Cui, Hung, and Wang. The implementation of these two procedures is demonstrated via an example.
Asunto(s)
Proyectos de Investigación , Humanos , Puntaje de Propensión , Tamaño de la MuestraRESUMEN
In a randomized controlled trial with time-to-event endpoint, some commonly used statistical tests to test for various aspects of survival differences, such as survival probability at a fixed time point, survival function up to a specific time point, and restricted mean survival time, may not be directly applicable when external data are leveraged to augment an arm (or both arms) of an RCT. In this paper, we propose a propensity score-integrated approach to extend such tests when external data are leveraged. Simulation studies are conducted to evaluate the operating characteristics of three propensity score-integrated statistical tests, and an illustrative example is given to demonstrate how these proposed procedures can be implemented.
RESUMEN
In a randomized controlled trial (RCT), it is possible to improve precision and power and reduce sample size by appropriately adjusting for baseline covariates. There are multiple statistical methods to adjust for prognostic baseline covariates, such as an ANCOVA method. In this paper, we propose a clustering-based stratification method for adjusting for the prognostic baseline covariates. Clusters (strata) are formed only based on prognostic baseline covariates, not outcome data nor treatment assignment. Therefore, the clustering procedure can be completed prior to the availability of outcome data. The treatment effect is estimated in each cluster, and the overall treatment effect is derived by combining all cluster-specific treatment effect estimates. The proposed implementation of the procedure is described. Simulations studies and an example are presented.
Asunto(s)
Proyectos de Investigación , Humanos , Interpretación Estadística de Datos , Ensayos Clínicos Controlados Aleatorios como Asunto , Tamaño de la Muestra , Análisis por ConglomeradosRESUMEN
In the area of diagnostics, it is common practice to leverage external data to augment a traditional study of diagnostic accuracy consisting of prospectively enrolled subjects to potentially reduce the time and/or cost needed for the performance evaluation of an investigational diagnostic device. However, the statistical methods currently being used for such leveraging may not clearly separate study design and outcome data analysis, and they may not adequately address possible bias due to differences in clinically relevant characteristics between the subjects constituting the traditional study and those constituting the external data. This paper is intended to draw attention in the field of diagnostics to the recently developed propensity score-integrated composite likelihood approach, which originally focused on therapeutic medical products. This approach applies the outcome-free principle to separate study design and outcome data analysis and can mitigate bias due to imbalance in covariates, thereby increasing the interpretability of study results. While this approach was conceived as a statistical tool for the design and analysis of clinical studies for therapeutic medical products, here, we will show how it can also be applied to the evaluation of sensitivity and specificity of an investigational diagnostic device leveraging external data. We consider two common scenarios for the design of a traditional diagnostic device study consisting of prospectively enrolled subjects, which is to be augmented by external data. The reader will be taken through the process of implementing this approach step-by-step following the outcome-free principle that preserves study integrity.
Asunto(s)
Funciones de Verosimilitud , Humanos , Puntaje de Propensión , Sensibilidad y EspecificidadRESUMEN
The interest in utilizing real-world data (RWD) has been considerably increasing in medical product development and evaluation. With proper usage and analysis of high-quality real-world data, real-world evidence (RWE) can be generated to inform regulatory and healthcare decision-making. This paper proposes a study design and data analysis approach for a prospective, single-arm clinical study that is supplemented with patients from multiple real-world data sources containing patient-level covariate and outcome data. After the amount of information to be borrowed from each real-world data source is determined, the propensity score-integrated composite likelihood method is applied to obtain an estimate of the parameter of interest based on data from the prospective clinical study and this real-world data source. This method is applied to each real-world data source. The final estimate of the parameter of interest is then obtained by taking a weighted average of all these estimates. The performance of the proposed approach is evaluated via a simulation study. A hypothetical example is presented to illustrate how to implement the proposed approach.
Asunto(s)
Almacenamiento y Recuperación de la Información , Proyectos de Investigación , Simulación por Computador , Humanos , Puntaje de Propensión , Estudios ProspectivosRESUMEN
In this paper, a propensity score-integrated power prior approach is developed to augment the control arm of a two-arm randomized controlled trial (RCT) with subjects from multiple external data sources such as real-world data (RWD) and historical clinical studies containing subject-level outcomes and covariates. The propensity scores for the subjects in the external data sources versus the subjects in the RCT are first estimated, and then subjects are placed in different strata based on their estimated propensity scores. Within each propensity score stratum, a power prior is formulated with the information contributed by the external data sources, and Bayesian inference on the treatment effect is obtained. The proposed approach is implemented under the two-stage study design framework utilizing the outcome-free principle to ensure the integrity of a study. An illustrative example is provided to demonstrate the implementation of the proposed approach.
Asunto(s)
Almacenamiento y Recuperación de la Información , Proyectos de Investigación , Humanos , Puntaje de PropensiónRESUMEN
External data, referred to as data external to the traditional clinical study being planned, include but are not limited to real-world data (RWD) and data collected from clinical studies being conducted in the past or in other countries. The external data are sometimes leveraged to augment a single-arm, prospectively designed study when appropriate. In such an application, recently developed propensity score-integrated approaches including PSPP and PSCL can be used for study design and data analysis when the clinical outcomes are binary or continuous. In this paper, the propensity score-integrated Kaplan-Meier (PSKM) method is proposed for a similar situation but the outcome of interest is time-to-event. The propensity score methodology is used to select external subjects that are similar to those in the current study in terms of baseline covariates and to stratify the selected subjects from both data sources into more homogeneous strata. The stratum-specific PSKM estimators are obtained based on all subjects in the stratum with the external data being down-weighted, and then these estimators are combined to obtain an overall PSKM estimator. A simulation study is conducted to assess the performance of the PSKM method, and an illustrative example is presented to demonstrate how to implement the proposed method.
Asunto(s)
Análisis de Datos , Proyectos de Investigación , Simulación por Computador , Humanos , Puntaje de Propensión , Análisis de SupervivenciaRESUMEN
The document ICH E9 (R1) has brought much attention to the concept of estimand in the clinical trials community. ICH stands for International Conference for Harmonization. In this article, we draw attention to one facet of estimand that is not discussed in that document but is crucial in the context of observational studies, namely weighting for covariate balance. How weighting schemes are connected to estimand, or more specifically to one of its five attributes identified in ICH E9 (R1), the attribute of population, is illustrated using the Rubin Causal Model. Three estimands are examined from both theoretical and practical perspectives. Factors that may be considered in choosing among these estimands are discussed.
Asunto(s)
Modelos Estadísticos , Proyectos de Investigación , Interpretación Estadística de Datos , Humanos , Estudios Observacionales como AsuntoRESUMEN
Performance goals are numerical target values pertaining to effectiveness or safety endpoints in single-arm medical device clinical studies. Typically, performance goals are determined at the planning stage of the investigational study under consideration based on summarized outcome information from existing relevant clinical trials. In recent years, there is a growing interest in leveraging real-world evidence in medical product development. In this article, we introduce a new method for proposing performance goals by leveraging real-world evidence. The method applies entropy balancing to address possible patient dissimilarities between the study's target patient population and existing real-world patients, and can take into account operation differences between clinical studies and real-world clinical practice. An illustrative example is provided to demonstrate how to implement the proposed method for performance goal determination while leveraging real-world evidence.
Asunto(s)
Objetivos , Proyectos de Investigación , HumanosRESUMEN
The evaluation of diagnostic tests usually involves statistical inference for its sensitivity. As sensitivity is defined as the probability that the test result will be positive when the target condition is present, the key study design consideration of sample size is the determination of the number of subjects with the target condition such that the estimation has adequate precision, or the hypothesis testing has adequate power. Traditionally, one may rely on prospective screening of subjects to obtain the required sample size, which means that if the prevalence of the disease is very low, a large number of subjects would need to be screened, increasing the study duration and cost. In this paper, we consider the possibility of substantially reducing the length and cost of a clinical study by leveraging subjects from a real-world data (RWD) source, focusing specifically on the diagnostic test for the cancer of interest. Using the propensity score methodology, we developed a procedure which ensures that the real-world subjects being leveraged are similar to their prospectively enrolled counterparts, thereby making the leveraging more justified. The procedure allows the down-weighting of the real-world subjects, which can be achieved by either using a Frequentist's method based on the composite likelihood or a Bayesian method based on the power prior. The proposed approach can be applied to the evaluation of any diagnostic test and it is not limited to the current clinical study regarding a cancer diagnostic test. Notably, this paper is in close alignment with a recently released draft framework by the Medical Device Innovation Consortium (MDIC) on real-world clinical evidence and in vitro diagnostics, being a showcase of appropriately leveraging real-world data in diagnostic test evaluation for diseases with low prevalence to support regulatory decision-making.
Asunto(s)
Pruebas Diagnósticas de Rutina , Teorema de Bayes , Humanos , Prevalencia , Puntaje de Propensión , Estudios ProspectivosRESUMEN
In this paper, a propensity score-integrated composite likelihood (PSCL) approach is developed for cases in which the control arm of a two-arm randomized controlled trial (RCT) (treated vs control) is augmented with patients from real-world data (RWD) containing both clinical outcomes and covariates at the patient-level. RWD patients who were treated with the same therapy as the control arm of the RCT are considered for the augmentation. The PSCL approach first estimates the propensity score for every patient as the probability of the patient being in the RCT rather than the RWD, and then stratifies all patients into strata based on the estimated propensity scores. Within each propensity score stratum, a composite likelihood function is specified and utilized to down-weight the information contributed by the RWD source. Estimates of the stratum-specific parameters are obtained by maximizing the composite likelihood function. These stratum-specific estimates are then combined to obtain an overall population-level estimate of the parameter of interest. The performance of the proposed approach is evaluated via a simulation study. A hypothetical two-arm RCT and a hypothetical RWD source are used to illustrate the implementation of the proposed approach.
Asunto(s)
Simulación por Computador/estadística & datos numéricos , Ensayos Clínicos Pragmáticos como Asunto/estadística & datos numéricos , Puntaje de Propensión , Ensayos Clínicos Controlados Aleatorios como Asunto/estadística & datos numéricos , Sistema de Registros/estadística & datos numéricos , Insuficiencia Cardíaca/epidemiología , Insuficiencia Cardíaca/terapia , Humanos , Funciones de Verosimilitud , Ensayos Clínicos Pragmáticos como Asunto/métodos , Ensayos Clínicos Controlados Aleatorios como Asunto/métodosRESUMEN
In medical product development, there has been an increased interest in utilizing real-world data which have become abundant with recent advances in biomedical science, information technology, and engineering. High-quality real-world data may be analyzed to generate real-world evidence that can be utilized in the regulatory and healthcare decision-making. In this paper, we consider the case in which a single-arm clinical study, viewed as the primary data source, is supplemented with patients from a real-world data source containing both clinical outcome and covariate data at the patient-level. Propensity score methodology is used to identify real-world data patients that are similar to those in the single-arm study in terms of the baseline characteristics, and to stratify these patients into strata based on the proximity of the propensity scores. In each stratum, a composite likelihood function of a parameter of interest is constructed by down-weighting the information from the real-world data source, and an estimate of the stratum-specific parameter is obtained by maximizing the composite likelihood function. These stratum-specific estimates are then combined to obtain an overall population-level estimate of the parameter of interest. The performance of the proposed approach is evaluated via a simulation study. A hypothetical example based on our experience is provided to illustrate the implementation of the proposed approach.
Asunto(s)
Simulación por Computador/estadística & datos numéricos , Interpretación Estadística de Datos , Ensayos Clínicos Pragmáticos como Asunto/estadística & datos numéricos , Puntaje de Propensión , Humanos , Funciones de Verosimilitud , Ensayos Clínicos Pragmáticos como Asunto/métodosRESUMEN
Observational (non-randomized) comparative studies have been adopted in the pre-market safety/effectiveness evaluation of medical devices. There has been an increased interest in utilizing this design with the growing available real-world data. However, in such studies, biases that are introduced in every stage and aspect of study need to be addressed. Otherwise, the objectivity of study design and validity of study results will be compromised. In this paper, challenges and opportunities are discussed from the regulatory perspective. Considerations and good statistical practice to mitigate the potential bias are presented.
Asunto(s)
Interpretación Estadística de Datos , Aprobación de Recursos , Ensayos Clínicos Controlados Aleatorios como Asunto , Sesgo , Exactitud de los Datos , Humanos , Puntaje de Propensión , Proyectos de InvestigaciónRESUMEN
Noninferiority trials are commonly utilized to evaluate the safety and effectiveness of medical devices. It could happen that the noninferiority hypothesis is rejected while the performance of the active control is clinically not satisfactory. This may pose a great challenge when making a regulatory decision. To avoid such a difficult situation, we propose to conduct a companion test to assess the performance of the active control when testing the main noninferiority hypothesis and to incorporate such a test into the study design. Under our proposal, the noninferiority of the investigational device to the active control can only be claimed when both hypotheses are rejected. The operating characteristics of the proposed study design based on these two tests can be fully evaluated at the design stage. This proposed approach is aimed to facilitate regulatory decision making in a more transparent manner.
Asunto(s)
Bioestadística/métodos , Aprobación de Recursos , Estudios de Equivalencia como Asunto , Proyectos de Investigación/estadística & datos numéricos , Arritmias Cardíacas/cirugía , Catéteres Cardíacos/efectos adversos , Catéteres Cardíacos/estadística & datos numéricos , Ablación por Catéter/efectos adversos , Ablación por Catéter/instrumentación , Ablación por Catéter/estadística & datos numéricos , Interpretación Estadística de Datos , Humanos , Medición de Riesgo , Resultado del TratamientoRESUMEN
We are now at an amazing time for medical product development in drugs, biological products and medical devices. As a result of dramatic recent advances in biomedical science, information technology and engineering, ``big data'' from health care in the real-world have become available. Although big data may not necessarily be attuned to provide the preponderance of evidence to a clinical study, high-quality real-world data can be transformed into scientific evidence for regulatory and healthcare decision-making using proven analytical methods and techniques, such as propensity score methodology and Bayesian inference. In this paper, we extend the Bayesian power prior approach for a single-arm study (the current study) to leverage external real-world data. We use propensity score methodology to pre-select a subset of real-world data containing patients that are similar to those in the current study in terms of covariates, and to stratify the selected patients together with those in the current study into more homogeneous strata. The power prior approach is then applied in each stratum to obtain stratum-specific posterior distributions, which are combined to complete the Bayesian inference for the parameters of interest. We evaluate the performance of the proposed method as compared to that of the ordinary power prior approach by simulation and illustrate its implementation using a hypothetical example, based on our regulatory review experience.
Asunto(s)
Productos Biológicos , Ensayos Clínicos Pragmáticos como Asunto/métodos , Ensayos Clínicos Pragmáticos como Asunto/estadística & datos numéricos , Puntaje de Propensión , Teorema de Bayes , Productos Biológicos/química , Productos Biológicos/uso terapéutico , HumanosRESUMEN
A question that routinely arises in medical device clinical studies is the homogeneity across demographic subgroups, geographical regions, or investigational sites of the enrolled patients in terms of treatment effects or outcome variables. The main objective of this paper is to discuss statistical concepts and methods for the assessment of such homogeneity and to provide the practitioner a statistical framework and points to consider in conducting homogeneity assessment. Demographic subgroups, geographical regions, and investigational sites are discussed separately as each has its unique issues. Specific considerations are also given to randomized controlled trials, non-randomized comparative studies, and single-arm studies. We point out that judicious use of statistical methods, in conjunction with sound clinical judgment, is essential in handling the issue of homogeneity of treatment effect in medical device clinical studies.
Asunto(s)
Equipos y Suministros/estadística & datos numéricos , Ensayos Clínicos Controlados Aleatorios como Asunto/métodos , Ensayos Clínicos Controlados Aleatorios como Asunto/estadística & datos numéricos , Ensayos Clínicos como Asunto/métodos , Femenino , Humanos , MasculinoRESUMEN
Regulatory decisions are made based on the assessment of risk and benefit of medical devices at the time of pre-market approval and subsequently, when post-market risk-benefit balance needs reevaluation. Such assessments depend on scientific evidence obtained from pre-market studies, post-approval studies, post-market surveillance studies, patient perspective information, as well as other real world data such as national and international registries. Such registries provide real world evidence and are playing a more and more important role in enhancing the safety and effectiveness evaluation of medical devices. While these registries provide large quantities of data reflecting real world practice and can potentially reduce the cost of clinical trials, challenges arise concerning (1) data quality adequate for regulatory decision-making, (2) bias introduced at every stage and aspect of study, (3) scientific validity of study designs, and (4) reliability and interpretability of study results. This article will discuss related statistical and regulatory challenges and opportunities with examples encountered in medical device regulatory reviews.
Asunto(s)
Aprobación de Recursos , Regulación Gubernamental , Sistema de Registros , Sesgo , Exactitud de los Datos , Toma de Decisiones , Humanos , Reproducibilidad de los Resultados , Proyectos de Investigación , Medición de RiesgoAsunto(s)
Cardiopatías Congénitas/diagnóstico por imagen , Cardiopatías Congénitas/terapia , Accidente Cerebrovascular/diagnóstico por imagen , Accidente Cerebrovascular/terapia , Adulto , Anticoagulantes/administración & dosificación , Cardiopatías Congénitas/complicaciones , Humanos , Masculino , Accidente Cerebrovascular/complicaciones , Terapia Trombolítica/métodosRESUMEN
Due to the special nature of medical device clinical studies, observational (nonrandomized) comparative studies play important roles in the premarket safety/effectiveness evaluation of medical devices. While historical data collected in earlier investigational device exemption studies of a previously approved medical device have been used to form control groups in comparative studies, high-quality registry data are emerging to provide opportunities for the premarket evaluation of new devices. However, in such studies, various biases could be introduced in every stage and aspect of study and may compromise the objectivity of study design and validity of study results. In this article, challenges and opportunities in the design of such studies using propensity score methodology are discussed from regulatory perspectives.