Using a network-based approach and targeted maximum likelihood estimation to evaluate the effect of adding pre-exposure prophylaxis to an ongoing test-and-treat trial.

BACKGROUND: Several cluster-randomized trials are underway to investigate the implementation and effectiveness of a universal test-and-treat strategy on the HIV epidemic in sub-Saharan Africa. We consider nesting studies of pre-exposure prophylaxis within these trials. Pre-exposure prophylaxis is a general strategy where high-risk HIV- persons take antiretrovirals daily to reduce their risk of infection from exposure to HIV. We address how to target pre-exposure prophylaxis to high-risk groups and how to maximize power to detect the individual and combined effects of universal test-and-treat and pre-exposure prophylaxis strategies. METHODS: We simulated 1000 trials, each consisting of 32 villages with 200 individuals per village. At baseline, we randomized the universal test-and-treat strategy. Then, after 3 years of follow-up, we considered four strategies for targeting pre-exposure prophylaxis: (1) all HIV- individuals who self-identify as high risk, (2) all HIV- individuals who are identified by their HIV+ partner (serodiscordant couples), (3) highly connected HIV- individuals, and (4) the HIV- contacts of a newly diagnosed HIV+ individual (a ring-based strategy). We explored two possible trial designs, and all villages were followed for a total of 7 years. For each village in a trial, we used a stochastic block model to generate bipartite (male-female) networks and simulated an agent-based epidemic process on these networks. We estimated the individual and combined intervention effects with a novel targeted maximum likelihood estimator, which used cross-validation to data-adaptively select from a pre-specified library the candidate estimator that maximized the efficiency of the analysis. RESULTS: The universal test-and-treat strategy reduced the 3-year cumulative HIV incidence by 4.0% on average. The impact of each pre-exposure prophylaxis strategy on the 4-year cumulative HIV incidence varied by the coverage of the universal test-and-treat strategy with lower coverage resulting in a larger impact of pre-exposure prophylaxis. Offering pre-exposure prophylaxis to serodiscordant couples resulted in the largest reductions in HIV incidence (2% reduction), and the ring-based strategy had little impact (0% reduction). The joint effect was larger than either individual effect with reductions in the 7-year incidence ranging from 4.5% to 8.8%. Targeted maximum likelihood estimation, data-adaptively adjusting for baseline covariates, substantially improved power over the unadjusted analysis, while maintaining nominal confidence interval coverage. CONCLUSION: Our simulation study suggests that nesting a pre-exposure prophylaxis study within an ongoing trial can lead to combined intervention effects greater than those of universal test-and-treat alone and can provide information about the efficacy of pre-exposure prophylaxis in the presence of high coverage of treatment for HIV+ persons.

Realizing the potential of mobile mental health: new methods for new data in psychiatry.

Smartphones are now ubiquitous and can be harnessed to offer psychiatry a wealth of real-time data regarding patient behavior, self-reported symptoms, and even physiology. The data collected from smartphones meet the three criteria of big data: velocity, volume, and variety. Although these data have tremendous potential, transforming them into clinically valid and useful information requires using new tools and methods as a part of assessment in psychiatry. In this paper, we introduce and explore numerous analytical methods and tools from the computational and statistical sciences that appear readily applicable to psychiatric data collected using smartphones. By matching smartphone data with appropriate statistical methods, psychiatry can better realize the potential of mobile mental health and empower both patients and providers with novel clinical tools.

Leveraging Contact Network Information in Clustered Randomized Studies of Contagion Processes.

In a randomized study, leveraging covariates related to the outcome (e.g. disease status) may produce less variable estimates of the effect of exposure. For contagion processes operating on a contact network, transmission can only occur through ties that connect affected and unaffected individuals; the outcome of such a process is known to depend intimately on the structure of the network. In this paper, we investigate the use of contact network features as efficiency covariates in exposure effect estimation. Using augmented generalized estimating equations (GEE), we estimate how gains in efficiency depend on the network structure and spread of the contagious agent or behavior. We apply this approach to simulated randomized trials using a stochastic compartmental contagion model on a collection of model-based contact networks and compare the bias, power, and variance of the estimated exposure effects using an assortment of network covariate adjustment strategies. We also demonstrate the use of network-augmented GEEs on a clustered randomized trial evaluating the effects of wastewater monitoring on COVID-19 cases in residential buildings at the the University of California San Diego.

Estimating longitudinal depressive symptoms from smartphone data in a transdiagnostic cohort.

BACKGROUND: Passive measures collected using smartphones have been suggested to represent efficient proxies for depression severity, but the performance of such measures across diagnoses has not been studied. METHODS: We enrolled a cohort of 45 individuals (11 with major depressive disorder, 11 with bipolar disorder, 11 with schizophrenia or schizoaffective disorder, and 12 individuals with no axis I psychiatric disorder). During the 8-week study period, participants were evaluated with a rater-administered Montgomery-Åsberg Depression Rating Scale (MADRS) biweekly, completed self-report PHQ-8 measures weekly on their smartphone, and consented to collection of smartphone-based GPS and accelerometer data in order to learn about their behaviors. We utilized linear mixed models to predict depression severity on the basis of phone-based PHQ-8 and passive measures. RESULTS: Among the 45 individuals, 38 (84%) completed the 8-week study. The average root-mean-squared error (RMSE) in predicting the MADRS score (scale 0-60) was 4.72 using passive data alone, 4.27 using self-report measures alone, and 4.30 using both. CONCLUSIONS: While passive measures did not improve MADRS score prediction in our cross-disorder study, they may capture behavioral phenotypes that cannot be measured objectively, granularly, or over long-term via self-report.

An Online Calculator for the Prediction of Survival in Glioblastoma Patients Using Classical Statistics and Machine Learning.

BACKGROUND: Although survival statistics in patients with glioblastoma multiforme (GBM) are well-defined at the group level, predicting individual patient survival remains challenging because of significant variation within strata. OBJECTIVE: To compare statistical and machine learning algorithms in their ability to predict survival in GBM patients and deploy the best performing model as an online survival calculator. METHODS: Patients undergoing an operation for a histopathologically confirmed GBM were extracted from the Surveillance Epidemiology and End Results (SEER) database (2005-2015) and split into a training and hold-out test set in an 80/20 ratio. Fifteen statistical and machine learning algorithms were trained based on 13 demographic, socioeconomic, clinical, and radiographic features to predict overall survival, 1-yr survival status, and compute personalized survival curves. RESULTS: In total, 20 821 patients met our inclusion criteria. The accelerated failure time model demonstrated superior performance in terms of discrimination (concordance index = 0.70), calibration, interpretability, predictive applicability, and computational efficiency compared to Cox proportional hazards regression and other machine learning algorithms. This model was deployed through a free, publicly available software interface (https://cnoc-bwh.shinyapps.io/gbmsurvivalpredictor/). CONCLUSION: The development and deployment of survival prediction tools require a multimodal assessment rather than a single metric comparison. This study provides a framework for the development of prediction tools in cancer patients, as well as an online survival calculator for patients with GBM. Future efforts should improve the interpretability, predictive applicability, and computational efficiency of existing machine learning algorithms, increase the granularity of population-based registries, and externally validate the proposed prediction tool.

Design and results of a smartphone-based digital phenotyping study to quantify ALS progression.

OBJECTIVE: The amyotrophic lateral sclerosis (ALS) trial outcome measures are clinic based. Active and passive smartphone data can provide important longitudinal information about ALS progression outside the clinic. METHODS: We used Beiwe, a research platform for smartphone-based digital phenotyping, to collect active (self-report ALSFRS-R surveys and speech recordings) and passive (phone sensors and logs) data from patients with ALS for approximately 24 weeks. In clinics, at baseline and every 3 months, we collected vital capacity, ALSFRS-R, and ALS-CBS at enrollment, week 12, and week 24. We also collected ALSFRS-R by telephone at week 6. RESULTS: Baseline in-clinic ALSFRS-R and smartphone self-report correlation was 0.93 (P < 0.001). ALSFRS-R slopes were equivalent and within-subject standard deviation was smaller for smartphone-based self-report (0.26 vs. 0.56). Use of Beiwe afforded weekly collection of speech samples amenable to a variety of analyses, and we found mean pause time to increase by 0.02 sec per month across the sample. INTERPRETATION: Smartphone-based digital phenotyping in people with ALS is feasible and informative. Self-administered smartphone ALSFRS-R scores correlate highly with clinic-based ALSFRS-R scores, have low variability, and could be used in clinical trials. More research is required to fully analyze speech recordings and passive data, and to identify optimal digital markers for use in future ALS clinical trials.

Relapse prediction in schizophrenia through digital phenotyping: a pilot study.

Among individuals diagnosed, hospitalized, and treated for schizophrenia, up to 40% of those discharged may relapse within 1 year even with appropriate treatment. Passively collected smartphone behavioral data present a scalable and at present underutilized opportunity to monitor patients in order to identify possible warning signs of relapse. Seventeen patients with schizophrenia in active treatment at a state mental health clinic in Boston used the Beiwe app on their personal smartphone for up to 3 months. By testing for changes in mobility patterns and social behavior over time as measured through smartphone use, we were able to identify statistically significant anomalies in patient behavior in the days prior to relapse. We found that the rate of behavioral anomalies detected in the 2 weeks prior to relapse was 71% higher than the rate of anomalies during other time periods. Our findings show how passive smartphone data, data collected in the background during regular phone use without active input from the subjects, can provide an unprecedented and detailed view into patient behavior outside the clinic. Real-time detection of behavioral anomalies could signal the need for an intervention before an escalation of symptoms and relapse occur, therefore reducing patient suffering and reducing the cost of care.

Beyond smartphones and sensors: choosing appropriate statistical methods for the analysis of longitudinal data.

Objectives: As smartphones and sensors become more prominently used in mobile health, the methods used to analyze the resulting data must also be carefully considered. The advantages of smartphone-based studies, including large quantities of temporally dense longitudinally captured data, must be matched with the appropriate statistical methods in order draw valid conclusions. In this paper, we review and provide recommendations in 3 critical domains of analysis for these types of temporally dense longitudinal data and highlight how misleading results can arise from improper use of these methods. Target Audience: Clinicians, biostatisticians, and data analysts who have digital phenotyping data or are interested in performing a digital phenotyping study or any other type of longitudinal study with frequent measurements taken over an extended period of time. Scope: We cover the following topics: 1) statistical models using longitudinal repeated measures, 2) multiple comparisons of correlated tests, and 3) dimension reduction for correlated behavioral covariates. While these 3 classes of methods are frequently used in digital phenotyping data analysis, we demonstrate via actual clinical studies data that they may sometimes not perform as expected when applied to novel digital data.

Characterizing the clinical relevance of digital phenotyping data quality with applications to a cohort with schizophrenia.

Digital phenotyping, or the moment-by-moment quantification of the individual-level human phenotype in situ using data from personal digital devices and smartphones, in particular, holds great potential for behavioral monitoring of patients. However, realizing the potential of digital phenotyping requires understanding of the smartphone as a scientific data collection tool. In this pilot study, we detail a procedure for estimating data quality for phone sensor samples and model the relationship between data quality and future symptom-related survey responses in a cohort with schizophrenia. We find that measures of empirical coverage of collected accelerometer and GPS data, as well as survey timing and survey completion metrics, are significantly associated with future survey scores for a variety of symptom domains. We also find evidence that specific measures of data quality are indicative of domain-specific future survey outcomes. These results suggest that for smartphone-based digital phenotyping, metadata is not independent of patient-reported survey scores, and is therefore potentially useful in predicting future clinical outcomes. This work raises important questions and considerations for future studies; we explore and discuss some of these implications.

The HOPE Pilot Study: Harnessing Patient-Reported Outcomes and Biometric Data to Enhance Cancer Care.

PURPOSE: Integrating patient-reported outcomes (PROs) into clinical practice is an increasingly promising strategy for improving patients' symptoms, communication, and clinical outcomes. The objective of the current study was to assess the feasibility, acceptability, and perceived effectiveness of a mobile health intervention that was designed to collect PROs and activity data as a measure of health status. PATIENTS AND METHODS: This work was a pilot intervention with 10 patients with gynecologic cancers who received palliative chemotherapy. The HOPE (Helping Our Patients Excel) study used wearable accelerometers to assess physical activity and the Beiwe research platform to collect PROs, stratify patient responses by risk, provide tailored symptom management, and notify patients and clinicians of high-risk symptoms. Feasibility and acceptability were assessed through enrollment and adherence rates, and perceived effectiveness was evaluated by patients and oncologists at study completion. RESULTS: The approach-to-consent rate was 100%, and participants were 90% and 70% adherent to the wearable accelerometers and smartphone surveys, respectively. Participants' mean daily step count was 3,973 (standard deviation [SD], 2,305 steps) and increased from week 1 (mean, 3,520 steps; SD, 1,937 steps) to week 3 (mean, 4,136 steps; SD, 1,578 steps). Active monitoring of participants' heart rates, daily steps, and PROs throughout the study identified anomalies in participants' behavior patterns that suggested poor health for two patients (20%). Patients and clinicians indicated that the intervention improved physical activity, communication, and symptom management. CONCLUSION: A mobile health intervention that collects PROs and activity data as a measure of health status is feasible, acceptable, and was perceived to be effective in improving symptom management in patients with advanced gynecologic cancers. A larger, multisite, randomized clinical trial to assess the efficacy of the HOPE intervention on patients' symptoms, health-related quality of life, clinical outcomes, and health care use is warranted.

Information-Based Medicine in Glioma Patients: A Clinical Perspective.

Glioma constitutes the most common type of primary brain tumor with a dismal survival, often measured in terms of months or years. The thin line between treatment effectiveness and patient harm underpins the importance of tailoring clinical management to the individual patient. Randomized trials have laid the foundation for many neuro-oncological guidelines. Despite this, their findings focus on group-level estimates. Given our current tools, we are limited in our ability to guide patients on what therapy is best for them as individuals, or even how long they should expect to survive. Machine learning, however, promises to provide the analytical support for personalizing treatment decisions, and deep learning allows clinicians to unlock insight from the vast amount of unstructured data that is collected on glioma patients. Although these novel techniques have achieved astonishing results across a variety of clinical applications, significant hurdles remain associated with the implementation of them in clinical practice. Future challenges include the assembly of well-curated cross-institutional datasets, improvement of the interpretability of machine learning models, and balancing novel evidence-based decision-making with the associated liability of automated inference. Although artificial intelligence already exceeds clinical expertise in a variety of applications, clinicians remain responsible for interpreting the implications of, and acting upon, each prediction.

Machine Learning and Neurosurgical Outcome Prediction: A Systematic Review.

OBJECTIVE: Accurate measurement of surgical outcomes is highly desirable to optimize surgical decision-making. An important element of surgical decision making is identification of the patient cohort that will benefit from surgery before the intervention. Machine learning (ML) enables computers to learn from previous data to make accurate predictions on new data. In this systematic review, we evaluate the potential of ML for neurosurgical outcome prediction. METHODS: A systematic search in the PubMed and Embase databases was performed to identify all potential relevant studies up to January 1, 2017. RESULTS: Thirty studies were identified that evaluated ML algorithms used as prediction models for survival, recurrence, symptom improvement, and adverse events in patients undergoing surgery for epilepsy, brain tumor, spinal lesions, neurovascular disease, movement disorders, traumatic brain injury, and hydrocephalus. Depending on the specific prediction task evaluated and the type of input features included, ML models predicted outcomes after neurosurgery with a median accuracy and area under the receiver operating curve of 94.5% and 0.83, respectively. Compared with logistic regression, ML models performed significantly better and showed a median absolute improvement in accuracy and area under the receiver operating curve of 15% and 0.06, respectively. Some studies also demonstrated a better performance in ML models compared with established prognostic indices and clinical experts. CONCLUSIONS: In the research setting, ML has been studied extensively, demonstrating an excellent performance in outcome prediction for a wide range of neurosurgical conditions. However, future studies should investigate how ML can be implemented as a practical tool supporting neurosurgical care.

Characterizing Smartphone Engagement for Schizophrenia: Results of a Naturalist Mobile Health Study.

INTRODUCTION: Despite growing interest in smartphone apps for schizophrenia, little is known about how these apps are utilized in the real world. Understanding how app users are engaging with these tools outside of the confines of traditional clinical studies offers an important information on who is most likely to use apps and what type of data they are willing to share. METHODS: The Schizophrenia and Related Disorders Alliance of America, in partnership with Self Care Catalyst, has created a smartphone app for schizophrenia that is free and publically available on both Apple iTunes and Google Android Play stores. We analyzed user engagement data from this app across its medication tracking, mood tracking, and symptom tracking features from August 16th 2015 to January 1st 2017 using the R programming language. We included all registered app users in our analysis with reported ages less than 100. RESULTS: We analyzed a total of 43,451 mood, medication and symptom entries from 622 registered users, and excluded a single patient with a reported age of 114. Seventy one percent of the 622 users tried the mood-tracking feature at least once, 49% the symptom tracking feature, and 36% the medication-tracking feature. The mean number of uses of the mood feature was two, the symptom feature 10, and the medication feature 14. However, a small subset of users were very engaged with the app and the top 10 users for each feature accounted for 35% or greater of all entries for that feature. We find that user engagement follows a power law distribution for each feature, and this fit was largely invariant when stratifying for age or gender. DISCUSSION: Engagement with this app for schizophrenia was overall low, but similar to prior naturalistic studies for mental health app use in other diseases. The low rate of engagement in naturalistic settings, compared to higher rates of use in clinical studies, suggests the importance of clinical involvement as one factor in driving engagement for mental health apps. Power law relationships suggest strongly skewed user engagement, with a small subset of users accounting for the majority of substantial engagements. There is a need for further research on app engagement in schizophrenia.

A comparison of passive and active estimates of sleep in a cohort with schizophrenia.

Sleep abnormalities are considered an important feature of schizophrenia, yet convenient and reliable sleep monitoring remains a challenge. Smartphones offer a novel solution to capture both self-reported and objective measures of sleep in schizophrenia. In this three-month observational study, 17 subjects with a diagnosis of schizophrenia currently in treatment downloaded Beiwe, a platform for digital phenotyping, on their personal Apple or Android smartphones. Subjects were given tri-weekly ecological momentary assessments (EMAs) on their own smartphones, and passive data including accelerometer, GPS, screen use, and anonymized call and text message logs was continuously collected. We compare the in-clinic assessment of sleep quality, assessed with the Pittsburgh Sleep Questionnaire Inventory (PSQI), to EMAs, as well as sleep estimates based on passively collected accelerometer data. EMAs and passive data classified 85% (11/13) of subjects as exhibiting high or low sleep quality compared to the in-clinic assessments among subjects who completed at least one in-person PSQI. Phone-based accelerometer data used to infer sleep duration was moderately correlated with subject self-assessment of sleep duration (r = 0.69, 95% CI 0.23-0.90). Active and passive phone data predicts concurrent PSQI scores for all subjects with mean average error of 0.75 and future PSQI scores with a mean average error of 1.9, with scores ranging from 0-14. These results suggest sleep monitoring via personal smartphones is feasible for subjects with schizophrenia in a scalable and affordable manner. PATIENT MONITORING: SMARTPHONES CAN TRACK SCHIZOPHRENIA-RELATED SLEEP ABNORMALITIES: Smartphones may one-day offer accessible, clinically-useful insights into schizophrenia patients' sleep quality. Despite the clinical relevance of sleep to disease severity, monitoring technologies still evade convenience and reliability. In search of a preferential method, a group of Harvard University researchers led by Patrick Staples investigated the validity of data collected via patients' own mobile phones. The team, with a cohort of 17 schizophrenia patients, compared the quality of data produced by smartphone sensors and smartphone-delivered questionnaires to that of an in-clinic evaluation. The results significantly showed that smartphone monitoring could generate information that approached the accuracy of in-clinic assessments. The team noted some areas for improvement; however, this study provides convincing justifications for further research into this non-invasive, low-cost, scalable method to monitor the sleep quality of schizophrenic patients.

Barriers, Benefits, and Beliefs of Brain Training Smartphone Apps: An Internet Survey of Younger US Consumers.

BACKGROUND: While clinical evidence for the efficacy of brain training remains in question, numerous smartphone applications (apps) already offer brain training directly to consumers. Little is known about why consumers choose to download these apps, how they use them, and what benefits they perceive. Given the high rates of smartphone ownership in those with internet access and the younger demographics, we chose to approach this question first with a general population survey that would capture primarily this demographic. METHOD: We conducted an online internet-based survey of the US population via mTurk regarding their use, experience, and perceptions of brain training apps. There were no exclusion criteria to partake although internet access was required. Respondents were paid 20 cents for completing each survey. The survey was offered for a 2-week period in September 2015. RESULTS: 3125 individuals completed the survey and over half of these were under age 30. Responses did not significantly vary by gender. The brain training app most frequently used was Lumosity. Belief that a brain-training app could help with thinking was strongly correlated with belief it could also help with attention, memory, and even mood. Beliefs of those who had never used brain-training apps were similar to those who had used them. Respondents felt that data security and lack of endorsement from a clinician were the two least important barriers to use. DISCUSSION: RESULTS suggest a high level of interest in brain training apps among the US public, especially those in younger demographics. The stability of positive perception of these apps among app-naïve and app-exposed participants suggests an important role of user expectations in influencing use and experience of these apps. The low concern about data security and lack of clinician endorsement suggest apps are not being utilized in clinical settings. However, the public's interest in the effectiveness of apps suggests a common theme with the scientific community's concerns about direct to consumer brain training programs.

Corrigendum: Barriers, Benefits, and Beliefs of Brain Training Smartphone Apps: An Internet Survey of Younger US Consumers.

[This corrects the article on p. 180 in vol. 10, PMID: 27148026.].

Incorporating Contact Network Structure in Cluster Randomized Trials.

Whenever possible, the efficacy of a new treatment is investigated by randomly assigning some individuals to a treatment and others to control, and comparing the outcomes between the two groups. Often, when the treatment aims to slow an infectious disease, clusters of individuals are assigned to each treatment arm. The structure of interactions within and between clusters can reduce the power of the trial, i.e. the probability of correctly detecting a real treatment effect. We investigate the relationships among power, within-cluster structure, cross-contamination via between-cluster mixing, and infectivity by simulating an infectious process on a collection of clusters. We demonstrate that compared to simulation-based methods, current formula-based power calculations may be conservative for low levels of between-cluster mixing, but failing to account for moderate or high amounts can result in severely underpowered studies. Power also depends on within-cluster network structure for certain kinds of infectious spreading. Infections that spread opportunistically through highly connected individuals have unpredictable infectious breakouts, making it harder to distinguish between random variation and real treatment effects. Our approach can be used before conducting a trial to assess power using network information, and we demonstrate how empirical data can inform the extent of between-cluster mixing.

Utilizing a Personal Smartphone Custom App to Assess the Patient Health Questionnaire-9 (PHQ-9) Depressive Symptoms in Patients With Major Depressive Disorder.

BACKGROUND: Accurate reporting of patient symptoms is critical for diagnosis and therapeutic monitoring in psychiatry. Smartphones offer an accessible, low-cost means to collect patient symptoms in real time and aid in care. OBJECTIVE: To investigate adherence among psychiatric outpatients diagnosed with major depressive disorder in utilizing their personal smartphones to run a custom app to monitor Patient Health Questionnaire-9 (PHQ-9) depression symptoms, as well as to examine the correlation of these scores to traditionally administered (paper-and-pencil) PHQ-9 scores. METHODS: A total of 13 patients with major depressive disorder, referred by their clinicians, received standard outpatient treatment and, in addition, utilized their personal smartphones to run the study app to monitor their symptoms. Subjects downloaded and used the Mindful Moods app on their personal smartphone to complete up to three survey sessions per day, during which a randomized subset of PHQ-9 symptoms of major depressive disorder were assessed on a Likert scale. The study lasted 29 or 30 days without additional follow-up. Outcome measures included adherence, measured by the percentage of completed survey sessions, and estimates of daily PHQ-9 scores collected from the smartphone app, as well as from the traditionally administered PHQ-9. RESULTS: Overall adherence was 77.78% (903/1161) and varied with time of day. PHQ-9 estimates collected from the app strongly correlated (r=.84) with traditionally administered PHQ-9 scores, but app-collected scores were 3.02 (SD 2.25) points higher on average. More subjects reported suicidal ideation using the app than they did on the traditionally administered PHQ-9. CONCLUSIONS: Patients with major depressive disorder are able to utilize an app on their personal smartphones to self-assess their symptoms of major depressive disorder with high levels of adherence. These app-collected results correlate with the traditionally administered PHQ-9. Scores recorded from the app may potentially be more sensitive and better able to capture suicidality than the traditional PHQ-9.