Rigorous and rapid evidence assessment in digital health with the evidence DEFINED framework.

Dozens of frameworks have been proposed to assess evidence for digital health interventions (DHIs), but existing frameworks may not facilitate DHI evidence reviews that meet the needs of stakeholder organizations including payers, health systems, trade organizations, and others. These organizations may benefit from a DHI assessment framework that is both rigorous and rapid. Here we propose a framework to assess Evidence in Digital health for EFfectiveness of INterventions with Evaluative Depth (Evidence DEFINED). Designed for real-world use, the Evidence DEFINED Quick Start Guide may help streamline DHI assessment. A checklist is provided summarizing high-priority evidence considerations in digital health. Evidence-to-recommendation guidelines are proposed, specifying degrees of adoption that may be appropriate for a range of evidence quality levels. Evidence DEFINED differs from prior frameworks in its inclusion of unique elements designed for rigor and speed. Rigor is increased by addressing three gaps in prior frameworks. First, prior frameworks are not adapted adequately to address evidence considerations that are unique to digital health. Second, prior frameworks do not specify evidence quality criteria requiring increased vigilance for DHIs in the current regulatory context. Third, extant frameworks rarely leverage established, robust methodologies that were developed for non-digital interventions. Speed is achieved in the Evidence DEFINED Framework through screening optimization and deprioritization of steps that may have limited value. The primary goals of Evidence DEFINED are to a) facilitate standardized, rapid, rigorous DHI evidence assessment in organizations and b) guide digital health solutions providers who wish to generate evidence that drives DHI adoption.

Considerations for Analyzing and Interpreting Data from Biometric Monitoring Technologies in Clinical Trials.

Background: The proliferation and increasing maturity of biometric monitoring technologies allow clinical investigators to measure the health status of trial participants in a more holistic manner, especially outside of traditional clinical settings. This includes capturing meaningful aspects of health in daily living and a more granular and objective manner compared to traditional tools in clinical settings. Summary: Within multidisciplinary teams, statisticians and data scientists are increasingly involved in clinical trials that incorporate digital clinical measures. They are called upon to provide input into trial planning, generation of evidence on the clinical validity of novel clinical measures, and evaluation of the adequacy of existing evidence. Analysis objectives related to demonstrating clinical validity of novel clinical measures differ from typical objectives related to demonstrating safety and efficacy of therapeutic interventions using established measures which statisticians are most familiar with. Key Messages: This paper discusses key considerations for generating evidence for clinical validity through the lens of the type and intended use of a clinical measure. This paper also briefly discusses the regulatory pathways through which clinical validity evidence may be reviewed and highlights challenges that investigators may encounter while dealing with data from biometric monitoring technologies.

Recommendations for Defining and Reporting Adherence Measured by Biometric Monitoring Technologies: Systematic Review.

BACKGROUND: Suboptimal adherence to data collection procedures or a study intervention is often the cause of a failed clinical trial. Data from connected sensors, including wearables, referred to here as biometric monitoring technologies (BioMeTs), are capable of capturing adherence to both digital therapeutics and digital data collection procedures, thereby providing the opportunity to identify the determinants of adherence and thereafter, methods to maximize adherence. OBJECTIVE: We aim to describe the methods and definitions by which adherence has been captured and reported using BioMeTs in recent years. Identifying key gaps allowed us to make recommendations regarding minimum reporting requirements and consistency of definitions for BioMeT-based adherence data. METHODS: We conducted a systematic review of studies published between 2014 and 2019, which deployed a BioMeT outside the clinical or laboratory setting for which a quantitative, nonsurrogate, sensor-based measurement of adherence was reported. After systematically screening the manuscripts for eligibility, we extracted details regarding study design, participants, the BioMeT or BioMeTs used, and the definition and units of adherence. The primary definitions of adherence were categorized as a continuous variable based on duration (highest resolution), a continuous variable based on the number of measurements completed, or a categorical variable (lowest resolution). RESULTS: Our PubMed search terms identified 940 manuscripts; 100 (10.6%) met our eligibility criteria and contained descriptions of 110 BioMeTs. During literature screening, we found that 30% (53/177) of the studies that used a BioMeT outside of the clinical or laboratory setting failed to report a sensor-based, nonsurrogate, quantitative measurement of adherence. We identified 37 unique definitions of adherence reported for the 110 BioMeTs and observed that uniformity of adherence definitions was associated with the resolution of the data reported. When adherence was reported as a continuous time-based variable, the same definition of adherence was adopted for 92% (46/50) of the tools. However, when adherence data were simplified to a categorical variable, we observed 25 unique definitions of adherence reported for 37 tools. CONCLUSIONS: We recommend that quantitative, nonsurrogate, sensor-based adherence data be reported for all BioMeTs when feasible; a clear description of the sensor or sensors used to capture adherence data, the algorithm or algorithms that convert sample-level measurements to a metric of adherence, and the analytic validation data demonstrating that BioMeT-generated adherence is an accurate and reliable measurement of actual use be provided when available; and primary adherence data be reported as a continuous variable followed by categorical definitions if needed, and that the categories adopted are supported by clinical validation data and/or consistent with previous reports.

Verification, analytical validation and clinical validation (V3) of wearable dosimeters and light loggers.

Background: Light exposure is an important driver and modulator of human physiology, behavior and overall health, including the biological clock, sleep-wake cycles, mood and alertness. Light can also be used as a directed intervention, e.g., in the form of light therapy in seasonal affective disorder (SAD), jetlag prevention and treatment, or to treat circadian disorders. Recently, a system of quantities and units related to the physiological effects of light was standardized by the International Commission on Illumination (CIE S 026/E:2018). At the same time, biometric monitoring technologies (BioMeTs) to capture personalized light exposure were developed. However, because there are currently no standard approaches to evaluate the digital dosimeters, the need to provide a firm framework for the characterization, calibration, and reporting for these digital sensors is urgent. Objective: This article provides such a framework by applying the principles of verification, analytic validation and clinical validation (V3) as a state-of-the-art approach for tools and standards in digital medicine to light dosimetry. Results: This article describes opportunities for the use of digital dosimeters for basic research, for monitoring light exposure, and for measuring adherence in both clinical and non-clinical populations to light-based interventions in clinical trials.

Performance of a redesigned HIV Selectest enzyme-linked immunosorbent assay optimized to minimize vaccine-induced seropositivity in HIV vaccine trial participants.

Vaccine-induced seropositivity (VISP) or seroreactivity (VISR), defined as the reaction of antibodies elicited by HIV vaccines with antigens used in HIV diagnostic immunoassays, can result in reactive assay results for vaccinated but uninfected individuals, with subsequent misclassification of their infection status. The eventual licensure of a vaccine will magnify this issue and calls for the development of mitigating solutions in advance. An immunoassay that discriminates between antibodies elicited by vaccine antigens and those elicited by infection has been developed to address this laboratory testing need. The HIV Selectest is based on consensus and clade-specific HIV peptides that are omitted in many HIV vaccine constructs. The assay was redesigned to enhance performance across worldwide clades and to simplify routine use via a standard kit format. The redesigned assay was evaluated with sera from vaccine trial participants, HIV-infected and uninfected individuals, and healthy controls. The HIV Selectest exhibited specificities of 99.5% with sera from uninfected recipients of 6 different HIV vaccines and 100% with sera from normal donors, while detecting HIV-1 infections, including intercurrent infections, with 95 to 100% sensitivity depending on the clade, with the highest sensitivities for clades A and C. HIV Selectest sensitivity decreased in very early seroconversion specimens, which possibly explains the slightly lower sensitivity observed for asymptomatic blood donors than for clinical HIV cases. Thus, the HIV Selectest provides a new laboratory tool for use in vaccine settings to distinguish the immune response to HIV vaccine antigens from that due to true infection.

Magnitude and breadth of a nonprotective neutralizing antibody response in an efficacy trial of a candidate HIV-1 gp120 vaccine.

BACKGROUND: A candidate vaccine consisting of human immunodeficiency virus type 1 (HIV-1) subunit gp120 protein was found previously to be nonprotective in an efficacy trial (Vax004) despite strong antibody responses against the vaccine antigens. Here we assessed the magnitude and breadth of neutralizing antibody responses in Vax004. METHODS: Neutralizing antibodies were measured against highly sensitive (tier 1) and moderately sensitive (tier 2) strains of HIV-1 subtype B in 2 independent assays. Vaccine recipients were stratified by sex, race, and high versus low behavioral risk of HIV-1 acquisition. RESULTS: Most vaccine recipients mounted potent neutralizing antibody responses against HIV-1(MN) and other tier 1 viruses. Occasional weak neutralizing activity was detected against tier 2 viruses. The response against tier 1 and tier 2 viruses was significantly stronger in women than in men. Race and behavioral risk of HIV-1 acquisition had no significant effect on the response. Prior vaccination had little effect on the neutralizing antibody response that arose after infection. CONCLUSIONS: Weak overall neutralizing antibody responses against tier 2 viruses is consistent with a lack of protection in this trial. The magnitude and breadth of neutralization reported here should be useful for identifying improved vaccines.

HIV-Selectest enzyme immunoassay and rapid test: ability to detect seroconversion following HIV-1 infection.

HIV-Selectest is a serodiagnostic enzyme immunoassay (EIA), containing p6 and gp41 peptides, designed to differentiate between vaccine-induced antibodies and true infections. A rapid test version of the HIV-Selectest was developed. Both assays detected HIV antibodies in men and women within 2 to 4 weeks of infection, with sensitivity similar to third-generation EIAs.

Treatment of AIDS-related Kaposi's sarcoma with interleukin-12: rationale and preliminary evidence of clinical activity.

In this article, we review the preliminary evidence for the activity of interleukin-12 (IL-12) against Kaposi's sarcoma (KS) and discuss these results in the context of the biology of IL-12 and KS. IL-12 is a cytokine that enhances type 1 immunity, induces production of interferon gamma (IFN-gamma), and mediates antiangiogenic effects. In addition, it can downregulate a constitutively active G protein coupled receptor that is encoded by Kaposi's sarcoma-associated herpesvirus, the causative agent of KS. These factors suggested that IL-12 might be worth exploring as a potential anti-KS agent. In an initial phase I pilot study, IL-12 was found to have anti-KS activity when used alone in patients with AIDS-associated KS who were on a stable regimen of antiretroviral therapy. In preliminary results from a subsequent study of the combination of IL-12 plus liposomal doxorubicin along with highly active antiretroviral therapy, remissions were obtained in a substantial percentage of patients with advanced AIDS-associated KS. IL-12 has also been found active in patients with certain lymphomas. These results suggest that IL-12 may be worth exploring further as a potential antitumor agent in selected tumors.

Activity of subcutaneous interleukin-12 in AIDS-related Kaposi sarcoma.

Interleukin-12 (IL-12) enhances Th1-type T-cell responses and exerts antiangiogenic effects. We initiated a phase 1 pilot study of IL-12 in 32 patients with acquired immunodeficiency syndrome (AIDS)-related Kaposi sarcoma (KS) whose KS was progressing while on antiretroviral therapy. Fifteen patients had poor prognosis T(1)S(1) disease. IL-12 was administered subcutaneously twice weekly at doses from 100 to 625 ng/kg. The maximum tolerated dose was 500 ng/kg, and the principal toxicities were flulike symptoms, transaminase or bilirubin elevations, neutropenia, hemolytic anemia, and depression. No tumor responses were seen at the lowest dose (100 ng/kg), but 17 of 24 evaluable patients at the higher doses had partial or complete responses (response rate, 71%; 95% confidence interval, 48%-89%). Only 3 of 17 patients had a change in antiretroviral therapy before responding, and there were no significant differences between responders and nonresponders with regard to changes in CD4 counts or viral loads. Patients had increases in their serum IL-12, interferon-gamma, and inducible protein-10 (IP-10) after the first dose, and increases above baseline persisted after week 4. These results provide preliminary evidence that IL-12 has substantial activity against AIDS-related KS with acceptable toxicity and warrants further investigation for this indication.

Current advances and challenges in HIV-1 vaccines.

Recent advances in science, which have aided HIV-1 vaccine development, include an improved understanding of HIV-1 envelope structure and function, expansion of the pipeline with innovative vaccine strategies, promising multi-gene and multi-clade vaccines that elicit cellular immunity, conduct of clinical trials in a global network, and development of validated techniques that enable simultaneous measurement of multiple T cell vaccine-induced immune responses in humans. A common feature of several preventive vaccine strategies now in early clinical trials is their ability in nonhuman primates to attenuate clinical disease rather than completely prevent HIV-1 infection. One vaccine concept has been tested in large-scale clinical trials, two are currently in efficacy trials, and one more is poised to enter efficacy trial in the next few years. Simultaneously, expanded efforts continue to identify new designs that induce mucosal immunity as well as broadly neutralizing antibodies.

Human immunodeficiency virus (HIV) vaccine trials: a novel assay for differential diagnosis of HIV infections in the face of vaccine-generated antibodies.

All current human immunodeficiency virus (HIV) vaccine candidates contain multiple viral components and elicit antibodies that react positively in licensed HIV diagnostic tests, which contain similar viral products. Thus, vaccine trial participants could be falsely diagnosed as infected with HIV. Additionally, uninfected, seropositive vaccinees may encounter long-term social and economic harms. Moreover, this also interferes with early detection of true HIV infections during preventive HIV vaccine trials. An HIV-seropositive test result among uninfected vaccine trial participants is a major public health concern for volunteers who want to participate in future HIV vaccine trials. Based on the increased number of HIV vaccines being tested globally, it is essential to differentiate vaccine- from virus-induced antibodies. Using a whole-HIV-genome phage display library, we identified conserved sequences in Env-gp41 and Gag-p6 which are recognized soon after infection, do not contain protective epitopes, and are not part of most current HIV vaccines. We established a new HIV serodetection assay based on these peptides. To date, this assay, termed HIV-SELECTEST, demonstrates >99% specificity and sensitivity. Importantly, in testing of plasma samples from multiple HIV vaccine trials, uninfected trial participants scored negative, while all intercurrent infections were detected within 1 to 3 months of HIV infection. The new HIV-SELECTEST is a simple but robust diagnostic tool for easy implementation in HIV vaccine trials and blood banks worldwide.

A pilot study of cidofovir in patients with kaposi sarcoma.

A clinical trial was conducted to test the activity of cidofovir (CDV), a drug with in vitro activity against Kaposi sarcoma (KS)-associated herpesvirus (KSHV), in KS. Five patients with human immunodeficiency virus-associated KS (4 receiving antiretroviral therapy) and 2 patients with classical KS were administered CDV (5 mg/kg/dose) weekly for 2 weeks and then every other week. All 7 patients had progression of their KS at a median of 8.1 weeks (range, 5-27 weeks). Skin biopsy specimens of KS lesions showed no change in expression of latent or early lytic genes, but, in the 1 assessable patient, there was decreased expression of a late lytic gene. There was no decrease in the virus load of KSHV in peripheral blood mononuclear cells. This study does not provide proof of principle for the treatment of KS with CDV. However, it remains possible that antiherpesvirus therapy can be developed for herpes-induced tumors.

Characterization of the antigenic, immunogenic, and pathogenic variation of infectious bursal disease virus due to propagation in different host systems (bursa, embryo, and cell culture). I. Antigenicity and immunogenicity.

In vitro and in ovo virus neutralization assays were conducted to assess the role of different host systems in infectious bursal disease virus (IBDV) antigenic and immunogenic variation. Four different strains, two variant (1084 E and GLS) and two standard (Edgar and STC), were propagated separately in the bursa of Fabricius and embryos, and were compared with cell culture-adapted preparations of the homologous strains. Chicken polyclonal antisera were prepared against each IBDV and neutralizing antibody titres were determined. Normalized IBDV antibody concentrations were used in neutralization assays against homologous and heterologous IBDVs in 10-day-old specific pathogen free embryos. Both antigenic and immunogenic changes occurred in IBDVs evaluated, as evidenced by differences in the ability of normalized antibody to neutralize IBDV propagated in different host systems. Antibody induced by bursal-derived IBDV neutralized all isolates equally well, whereas antibody induced by cell culture-derived virus neutralized bursal-derived IBDV much less effectively.

Characterization of the antigenic, immunogenic, and pathogenic variation of infectious bursal disease virus due to propagation in different host systems (bursa, embryo, and cell culture). II. Antigenicity at the epitope level.

Antigenic variation of infectious bursal disease virus (IBDV) due to propagation in different host systems (bursa of Fabricius, embryos, or cell cultures) was determined by enzyme-linked immunosorbent assay (indirect and antigen capture) and western blot analysis. To conduct this study, we used 27 non-neutralizing anti-VP(2) monoclonal antibodies, a reference panel of nine neutralizing monoclonal antibodies, and 13 neutralizing anti-IBDV chicken polyclonal antibodies. Changes occurred in neutralizing, cross-reactive, conformation-dependent epitopes on the VP(2) protein of IBDV. Interestingly, non-neutralizing, cross-reactive, conformation-dependent and confirmation-independent epitopes also changed on VP(2). These epitope changes were directly associated with the method used to propagate IBDV. These results demonstrate that different host systems may play an important role in the antigenicity of IBDV.

Characterization of the antigenic, immunogenic, and pathogenic variation of infectious bursal disease virus due to propagation in different host systems (bursa, embryo, and cell culture). III. Pathogenicity.

Differences in the relative pathogenicity of variant (1084 E and GLS) and standard (Edgar and STC) infectious bursal disease virus (IBDV) strains were observed after propagation in the bursa of Fabricius, embryos, or cell cultures. Bursa-derived IBDV induced the most severe lesions in the bursa of Fabricius when compared with strains propagated in embryos or cell cultures. Embryo-derived IBDV induced moderate gross bursal lesions, whereas cell culture-derived IBDV did not damage the bursa grossly. A high frequency of virus re-isolations was obtained from bursal, spleen, and thymic samples collected from birds inoculated with bursa-derived or embryo-derived IBDV. Virus re-isolation occurred much less frequently from birds inoculated with cell culture-adapted IBDV. Serological evaluations demonstrated that bursa-derived IBDV strains induced a higher neutralizing antibody response than did embryo-derived or cell culture-derived strains. These results document that the relative pathogenicity and immunogenicity of IBDV is reduced following propagation in embryos or cell cultures.