Value of Engagement in Digital Health Technology Research: Evidence Across 6 Unique Cohort Studies.

BACKGROUND: Wearable digital health technologies and mobile apps (personal digital health technologies [DHTs]) hold great promise for transforming health research and care. However, engagement in personal DHT research is poor. OBJECTIVE: The objective of this paper is to describe how participant engagement techniques and different study designs affect participant adherence, retention, and overall engagement in research involving personal DHTs. METHODS: Quantitative and qualitative analysis of engagement factors are reported across 6 unique personal DHT research studies that adopted aspects of a participant-centric design. Study populations included (1) frontline health care workers; (2) a conception, pregnant, and postpartum population; (3) individuals with Crohn disease; (4) individuals with pancreatic cancer; (5) individuals with central nervous system tumors; and (6) families with a Li-Fraumeni syndrome affected member. All included studies involved the use of a study smartphone app that collected both daily and intermittent passive and active tasks, as well as using multiple wearable devices including smartwatches, smart rings, and smart scales. All studies included a variety of participant-centric engagement strategies centered on working with participants as co-designers and regular check-in phone calls to provide support over study participation. Overall retention, probability of staying in the study, and median adherence to study activities are reported. RESULTS: The median proportion of participants retained in the study across the 6 studies was 77.2% (IQR 72.6%-88%). The probability of staying in the study stayed above 80% for all studies during the first month of study participation and stayed above 50% for the entire active study period across all studies. Median adherence to study activities varied by study population. Severely ill cancer populations and postpartum mothers showed the lowest adherence to personal DHT research tasks, largely the result of physical, mental, and situational barriers. Except for the cancer and postpartum populations, median adherences for the Oura smart ring, Garmin, and Apple smartwatches were over 80% and 90%, respectively. Median adherence to the scheduled check-in calls was high across all but one cohort (50%, IQR 20%-75%: low-engagement cohort). Median adherence to study-related activities in this low-engagement cohort was lower than in all other included studies. CONCLUSIONS: Participant-centric engagement strategies aid in participant retention and maintain good adherence in some populations. Primary barriers to engagement were participant burden (task fatigue and inconvenience), physical, mental, and situational barriers (unable to complete tasks), and low perceived benefit (lack of understanding of the value of personal DHTs). More population-specific tailoring of personal DHT designs is needed so that these new tools can be perceived as personally valuable to the end user.

The digital redesign of mental health: leveraging connected digital technologies for agency-driven patient-focused care.

Digital psychiatry could empower individuals to navigate their context-specific experiences outside healthcare visits. This editorial discusses how leveraging digital health technologies could dramatically transform how we conceptualise mental health and the mental health professional's day-day practice, and how patients could be enabled to navigate their mental health with greater agency.

Crowdsourcing biomedical research: leveraging communities as innovation engines.

The generation of large-scale biomedical data is creating unprecedented opportunities for basic and translational science. Typically, the data producers perform initial analyses, but it is very likely that the most informative methods may reside with other groups. Crowdsourcing the analysis of complex and massive data has emerged as a framework to find robust methodologies. When the crowdsourcing is done in the form of collaborative scientific competitions, known as Challenges, the validation of the methods is inherently addressed. Challenges also encourage open innovation, create collaborative communities to solve diverse and important biomedical problems, and foster the creation and dissemination of well-curated data repositories.

Combining tumor genome simulation with crowdsourcing to benchmark somatic single-nucleotide-variant detection.

The detection of somatic mutations from cancer genome sequences is key to understanding the genetic basis of disease progression, patient survival and response to therapy. Benchmarking is needed for tool assessment and improvement but is complicated by a lack of gold standards, by extensive resource requirements and by difficulties in sharing personal genomic information. To resolve these issues, we launched the ICGC-TCGA DREAM Somatic Mutation Calling Challenge, a crowdsourced benchmark of somatic mutation detection algorithms. Here we report the BAMSurgeon tool for simulating cancer genomes and the results of 248 analyses of three in silico tumors created with it. Different algorithms exhibit characteristic error profiles, and, intriguingly, false positives show a trinucleotide profile very similar to one found in human tumors. Although the three simulated tumors differ in sequence contamination (deviation from normal cell sequence) and in subclonality, an ensemble of pipelines outperforms the best individual pipeline in all cases. BAMSurgeon is available at https://github.com/adamewing/bamsurgeon/.

A Crowdsourcing Approach to Developing and Assessing Prediction Algorithms for AML Prognosis.

Acute Myeloid Leukemia (AML) is a fatal hematological cancer. The genetic abnormalities underlying AML are extremely heterogeneous among patients, making prognosis and treatment selection very difficult. While clinical proteomics data has the potential to improve prognosis accuracy, thus far, the quantitative means to do so have yet to be developed. Here we report the results and insights gained from the DREAM 9 Acute Myeloid Prediction Outcome Prediction Challenge (AML-OPC), a crowdsourcing effort designed to promote the development of quantitative methods for AML prognosis prediction. We identify the most accurate and robust models in predicting patient response to therapy, remission duration, and overall survival. We further investigate patient response to therapy, a clinically actionable prediction, and find that patients that are classified as resistant to therapy are harder to predict than responsive patients across the 31 models submitted to the challenge. The top two performing models, which held a high sensitivity to these patients, substantially utilized the proteomics data to make predictions. Using these models, we also identify which signaling proteins were useful in predicting patient therapeutic response.

Crowdsourced estimation of cognitive decline and resilience in Alzheimer's disease.

Identifying accurate biomarkers of cognitive decline is essential for advancing early diagnosis and prevention therapies in Alzheimer's disease. The Alzheimer's disease DREAM Challenge was designed as a computational crowdsourced project to benchmark the current state-of-the-art in predicting cognitive outcomes in Alzheimer's disease based on high dimensional, publicly available genetic and structural imaging data. This meta-analysis failed to identify a meaningful predictor developed from either data modality, suggesting that alternate approaches should be considered for prediction of cognitive performance.

Improving breast cancer survival analysis through competition-based multidimensional modeling.

Breast cancer is the most common malignancy in women and is responsible for hundreds of thousands of deaths annually. As with most cancers, it is a heterogeneous disease and different breast cancer subtypes are treated differently. Understanding the difference in prognosis for breast cancer based on its molecular and phenotypic features is one avenue for improving treatment by matching the proper treatment with molecular subtypes of the disease. In this work, we employed a competition-based approach to modeling breast cancer prognosis using large datasets containing genomic and clinical information and an online real-time leaderboard program used to speed feedback to the modeling team and to encourage each modeler to work towards achieving a higher ranked submission. We find that machine learning methods combined with molecular features selected based on expert prior knowledge can improve survival predictions compared to current best-in-class methodologies and that ensemble models trained across multiple user submissions systematically outperform individual models within the ensemble. We also find that model scores are highly consistent across multiple independent evaluations. This study serves as the pilot phase of a much larger competition open to the whole research community, with the goal of understanding general strategies for model optimization using clinical and molecular profiling data and providing an objective, transparent system for assessing prognostic models.

Accelerating drug discovery: Open source cancer cell biology?

The possibility that experimental data from diverse cell biology experiments might shed light on other experiments has been generally outside the realm of cancer biologists. Recent experiments suggest that core RNA expression profiles distilled from experiments using a set of known members with related attributes may be used as query tools to probe expression profiles from other unrelated experiments. The potential benefit arises from the possibility to share findings without fully reconstructing the exact initial conditions. The limitations will be framed by the robustness of the hypotheses so generated.

Effect of self-managed lifestyle treatment on glycemic control in patients with type 2 diabetes.

The lack of effective, scalable solutions for lifestyle treatment is a global clinical problem, causing severe morbidity and mortality. We developed a method for lifestyle treatment that promotes self-reflection and iterative behavioral change, provided as a digital tool, and evaluated its effect in 370 patients with type 2 diabetes (ClinicalTrials.gov identifier: NCT04691973). Users of the tool had reduced blood glucose, both compared with randomized and matched controls (involving 158 and 204 users, respectively), as well as improved systolic blood pressure, body weight and insulin resistance. The improvement was sustained during the entire follow-up (average 730 days). A pathophysiological subgroup of obese insulin-resistant individuals had a pronounced glycemic response, enabling identification of those who would benefit in particular from lifestyle treatment. Natural language processing showed that the metabolic improvement was coupled with the self-reflective element of the tool. The treatment is cost-saving because of improved risk factor control for cardiovascular complications. The findings open an avenue for self-managed lifestyle treatment with long-term metabolic efficacy that is cost-saving and can reach large numbers of people.

Stress-related consequences of the coronavirus disease 2019 pandemic on symptoms of Crohn's disease.

OBJECTIVES: A link between stress and Crohn's disease activity suggests an association, but results have been conflicting. The purpose of this study was to assess whether the stress related to the coronavirus disease 2019 (COVID-19) pandemic affected disease activity in patients with Crohn's disease. BASIC METHODS: An anonymous survey was distributed to patients through gastroenterology clinics and networks. Patients were asked to report their Crohn's disease symptoms in the months prior to the COVID-19 pandemic and again during the early stages of the COVID-19 pandemic using the Manitoba inflammatory bowel disease index in addition to questions about stress, perception of reasons for symptom change and personal impact. MAIN RESULTS: Out of 243 individuals with a confirmed diagnosis of Crohn's disease, there was a 24% relative increase in active symptoms between the pre-COVID-19 period to the during-COVID-19 period (P < 0.0001) reflecting an absolute change from 45 to 56%, respectively. The most frequent reported reason for a change in symptoms was 'Increased stress/and or feeling overwhelmed' (118/236), and personal impact of the pandemic was, 'I'm worrying a lot about the future' (113/236), both reported by approximately half of respondents. PRINCIPAL CONCLUSIONS: This study serves as a 'proof of concept' demonstrating the impact of a significant and uniquely uniform stressor as a natural experiment on Crohn's disease activity. The severity of symptoms of Crohn's disease increased during the COVID-19 pandemic. The primary reported reason for symptom change was an increase in stress, not a change in diet, exercise or other lifestyle behaviours, corroborating the hypothesis that stress affects Crohn's disease activity.

Small interfering RNA screens reveal enhanced cisplatin cytotoxicity in tumor cells having both BRCA network and TP53 disruptions.

RNA interference technology allows the systematic genetic analysis of the molecular alterations in cancer cells and how these alterations affect response to therapies. Here we used small interfering RNA (siRNA) screens to identify genes that enhance the cytotoxicity (enhancers) of established anticancer chemotherapeutics. Hits identified in drug enhancer screens of cisplatin, gemcitabine, and paclitaxel were largely unique to the drug being tested and could be linked to the drug's mechanism of action. Hits identified by screening of a genome-scale siRNA library for cisplatin enhancers in TP53-deficient HeLa cells were significantly enriched for genes with annotated functions in DNA damage repair as well as poorly characterized genes likely having novel functions in this process. We followed up on a subset of the hits from the cisplatin enhancer screen and validated a number of enhancers whose products interact with BRCA1 and/or BRCA2. TP53(+/-) matched-pair cell lines were used to determine if knockdown of BRCA1, BRCA2, or validated hits that associate with BRCA1 and BRCA2 selectively enhances cisplatin cytotoxicity in TP53-deficient cells. Silencing of BRCA1, BRCA2, or BRCA1/2-associated genes enhanced cisplatin cytotoxicity approximately 4- to 7-fold more in TP53-deficient cells than in matched TP53 wild-type cells. Thus, tumor cells having disruptions in BRCA1/2 network genes and TP53 together are more sensitive to cisplatin than cells with either disruption alone.

Detecting the impact of subject characteristics on machine learning-based diagnostic applications.

Collection of high-dimensional, longitudinal digital health data has the potential to support a wide-variety of research and clinical applications including diagnostics and longitudinal health tracking. Algorithms that process these data and inform digital diagnostics are typically developed using training and test sets generated from multiple repeated measures collected across a set of individuals. However, the inclusion of repeated measurements is not always appropriately taken into account in the analytical evaluations of predictive performance. The assignment of repeated measurements from each individual to both the training and the test sets ("record-wise" data split) is a common practice and can lead to massive underestimation of the prediction error due to the presence of "identity confounding." In essence, these models learn to identify subjects, in addition to diagnostic signal. Here, we present a method that can be used to effectively calculate the amount of identity confounding learned by classifiers developed using a record-wise data split. By applying this method to several real datasets, we demonstrate that identity confounding is a serious issue in digital health studies and that record-wise data splits for machine learning- based applications need to be avoided.

How molecular profiling could revolutionize drug discovery.

Information from genomic, proteomic and metabolomic measurements has already benefited target discovery and validation, assessment of efficacy and toxicity of compounds, identification of disease subgroups and the prediction of responses of individual patients. Greater benefits can be expected from the application of these technologies on a significantly larger scale; by simultaneously collecting diverse measurements from the same subjects or cell cultures; by exploiting the steadily improving quantitative accuracy of the technologies; and by interpreting the emerging data in the context of underlying biological models of increasing sophistication. The benefits of applying molecular profiling to drug discovery and development will include much lower failure rates at all stages of the drug development pipeline, faster progression from discovery through to clinical trials and more successful therapies for patient subgroups. Upheavals in existing organizational structures in the current 'conveyor belt' models of drug discovery might be required to take full advantage of these methods.

Combining accurate tumor genome simulation with crowdsourcing to benchmark somatic structural variant detection.

BACKGROUND: The phenotypes of cancer cells are driven in part by somatic structural variants. Structural variants can initiate tumors, enhance their aggressiveness, and provide unique therapeutic opportunities. Whole-genome sequencing of tumors can allow exhaustive identification of the specific structural variants present in an individual cancer, facilitating both clinical diagnostics and the discovery of novel mutagenic mechanisms. A plethora of somatic structural variant detection algorithms have been created to enable these discoveries; however, there are no systematic benchmarks of them. Rigorous performance evaluation of somatic structural variant detection methods has been challenged by the lack of gold standards, extensive resource requirements, and difficulties arising from the need to share personal genomic information. RESULTS: To facilitate structural variant detection algorithm evaluations, we create a robust simulation framework for somatic structural variants by extending the BAMSurgeon algorithm. We then organize and enable a crowdsourced benchmarking within the ICGC-TCGA DREAM Somatic Mutation Calling Challenge (SMC-DNA). We report here the results of structural variant benchmarking on three different tumors, comprising 204 submissions from 15 teams. In addition to ranking methods, we identify characteristic error profiles of individual algorithms and general trends across them. Surprisingly, we find that ensembles of analysis pipelines do not always outperform the best individual method, indicating a need for new ways to aggregate somatic structural variant detection approaches. CONCLUSIONS: The synthetic tumors and somatic structural variant detection leaderboards remain available as a community benchmarking resource, and BAMSurgeon is available at https://github.com/adamewing/bamsurgeon .

A gene-expression signature as a predictor of survival in breast cancer.

BACKGROUND: A more accurate means of prognostication in breast cancer will improve the selection of patients for adjuvant systemic therapy. METHODS: Using microarray analysis to evaluate our previously established 70-gene prognosis profile, we classified a series of 295 consecutive patients with primary breast carcinomas as having a gene-expression signature associated with either a poor prognosis or a good prognosis. All patients had stage I or II breast cancer and were younger than 53 years old; 151 had lymph-node-negative disease, and 144 had lymph-node-positive disease. We evaluated the predictive power of the prognosis profile using univariable and multivariable statistical analyses. RESULTS: Among the 295 patients, 180 had a poor-prognosis signature and 115 had a good-prognosis signature, and the mean (+/-SE) overall 10-year survival rates were 54.6+/-4.4 percent and 94.5+/-2.6 percent, respectively. At 10 years, the probability of remaining free of distant metastases was 50.6+/-4.5 percent in the group with a poor-prognosis signature and 85.2+/-4.3 percent in the group with a good-prognosis signature. The estimated hazard ratio for distant metastases in the group with a poor-prognosis signature, as compared with the group with the good-prognosis signature, was 5.1 (95 percent confidence interval, 2.9 to 9.0; P<0.001). This ratio remained significant when the groups were analyzed according to lymph-node status. Multivariable Cox regression analysis showed that the prognosis profile was a strong independent factor in predicting disease outcome. CONCLUSIONS: The gene-expression profile we studied is a more powerful predictor of the outcome of disease in young patients with breast cancer than standard systems based on clinical and histologic criteria.

Toxicogenomics and drug discovery: will new technologies help us produce better drugs?

Acting on reports in the late 1980s that most drug candidates fail in development, pharmaceutical discovery programmes responded by devising ways to increase the number of chemicals in the pipeline. With discovery now driven primarily by chemistry and high-throughput screening, the biological effects and, in particular, the toxicity of new compounds are largely not appreciated until a compound enters development. Arguably, this paradigm has produced more failures rather than delivering more successes--with more chemicals to examine, much less is known about any single agent before costly development studies are initiated. The emerging field of toxicogenomics is enabling us to ask detailed questions about drug effects very early on, thereby fundamentally changing our approach to drug discovery.

The Use of Smartphones for Health Research.

Because of their growing popularity and functionality, smartphones are increasingly valuable potential tools for health and medical research. Using ResearchKit, Apple's open-source platform to build applications ("apps") for smartphone research, collaborators have developed apps for researching asthma, breast cancer, cardiovascular disease, type 2 diabetes, and Parkinson disease. These research apps enhance widespread participation by removing geographical barriers to participation, provide novel ways to motivate healthy behaviors, facilitate high-frequency assessments, and enable more objective data collection. Although the studies have great potential, they also have notable limitations. These include selection bias, identity uncertainty, design limitations, retention, and privacy. As smartphone technology becomes increasingly available, researchers must recognize these factors to ensure that medical research is conducted appropriately. Despite these limitations, the future of smartphones in health research is bright. Their convenience grants unprecedented geographic freedom to researchers and participants alike and transforms the way clinical research can be conducted.

Sulforaphane reduces hepatic glucose production and improves glucose control in patients with type 2 diabetes.

A potentially useful approach for drug discovery is to connect gene expression profiles of disease-affected tissues ("disease signatures") to drug signatures, but it remains to be shown whether it can be used to identify clinically relevant treatment options. We analyzed coexpression networks and genetic data to identify a disease signature for type 2 diabetes in liver tissue. By interrogating a library of 3800 drug signatures, we identified sulforaphane as a compound that may reverse the disease signature. Sulforaphane suppressed glucose production from hepatic cells by nuclear translocation of nuclear factor erythroid 2-related factor 2 (NRF2) and decreased expression of key enzymes in gluconeogenesis. Moreover, sulforaphane reversed the disease signature in the livers from diabetic animals and attenuated exaggerated glucose production and glucose intolerance by a magnitude similar to that of metformin. Finally, sulforaphane, provided as concentrated broccoli sprout extract, reduced fasting blood glucose and glycated hemoglobin (HbA1c) in obese patients with dysregulated type 2 diabetes.