A pilot acceptability evaluation of MomMind: A digital health intervention for Peripartum Depression prevention and management focused on health disparities.

Health disparities cause significant strain on the wellbeing of individuals and society. In this study, we focus on the health disparities present in the condition of Peripartum Depression (PPD), a significant public health issue. While PPD can be managed through therapy and medication, many women do not receive adequate PPD treatment due to issues of social stigma and limited access to healthcare resources. Digital health technologies can offer practical tools for PPD management. However, current solutions do not integrate behavior theory and are rarely responsive to the transient information needs stemming from women's unique sociodemographic, clinical and psychosocial profiles. We describe a pilot acceptability evaluation of MomMind, a health-disparities focused digital health intervention for the prevention and management of PPD. A crucial MomMind advantage is its basis on behavior change theory and patient engagement as enabled by the Digilego digital health framework. Following an internal usability evaluation, MomMind was evaluated by patients through cross-sectional acceptability surveys, pre-and-post PPD health literacy surveys, and interviews. Survey respondents included n = 30 peripartum women, of whom n = 16 (53.3%) were Hispanic and n = 17 (56.7%) of low-income. Survey results show that 96.6% of participants (n = 29) approved and welcomed MomMind, and 90% (n = 27) found MomMind to be an appealing intervention. Additionally, significant improvements (p< = 0.05) were observed in participants' PPD health literacy, specifically their ability to recognize PPD symptoms and knowledge of how to seek PPD information. Interview main themes include MomMind's straightforward design and influence of others (family members, providers) on use of technology. Results suggest that enhancement of a digital health framework with health literacy theory can support production of digital health solutions acceptable to vulnerable populations. This study incorporates existing theories from different disciplines into a unified approach for mitigating health disparities, and produced a novel solution for promotion of health in a vulnerable population.

Digital health technologies for high-risk pregnancy management: three case studies using Digilego framework.

Objective: High-risk pregnancy (HRP) conditions such as gestational diabetes mellitus (GDM), hypertension (HTN), and peripartum depression (PPD) affect maternal and neonatal health. Patient engagement is critical for effective HRP management (HRPM). While digital technologies and analytics hold promise, emerging research indicates limited and suboptimal support offered by the highly prevalent pregnancy digital solutions within the commercial marketplace. In this article, we describe our efforts to develop a portfolio of digital products leveraging advances in social computing, data science, and digital health. Methods: We describe three studies that leverage core methods from Digilego digital health development framework to (1) conduct large-scale social media analysis (n = 55 301 posts) to understand population-level patterns in women's needs, (2) architect a digital repository to enable women curate HRP related information, and (3) develop a digital platform to support PPD prevention. We applied a combination of qualitative coding, machine learning, theory-mapping, and programmatic implementation of theory-linked digital features. Further, we conducted preliminary testing of the resulting products for acceptance with sample of pregnant women for GDM/HTN information management (n = 10) and PPD prevention (n = 30). Results: Scalable social computing models using deep learning classifiers with reasonable accuracy have allowed us to capture and examine psychosociobehavioral drivers associated with HRPM. Our work resulted in two digital health solutions, MyPregnancyChart and MomMind are developed. Initial evaluation of both tools indicates positive acceptance from potential end users. Further evaluation with MomMind revealed statistically significant improvements (P < .05) in PPD recognition and knowledge on how to seek PPD information. Discussion: Digilego framework provides an integrative methodological lens to gain micro-macro perspective on women's needs, theory integration, engagement optimization, as well as subsequent feature and content engineering, which can be organized into core and specialized digital pathways for women engagement in disease management. Conclusion: Future works should focus on implementation and testing of digital solutions that facilitate women to capture, aggregate, preserve, and utilize, otherwise siloed, prenatal information artifacts for enhanced self-management of their high-risk conditions, ultimately leading to improved health outcomes.

Digital health technologies for peripartum depression management among low-socioeconomic populations: perspectives from patients, providers, and social media channels.

BACKGROUND: Peripartum Depression (PPD) affects approximately 10-15% of perinatal women in the U.S., with those of low socioeconomic status (low-SES) more likely to develop symptoms. Multilevel treatment barriers including social stigma and not having appropriate access to mental health resources have played a major role in PPD-related disparities. Emerging advances in digital technologies and analytics provide opportunities to identify and address access barriers, knowledge gaps, and engagement issues. However, most market solutions for PPD prevention and management are produced generically without considering the specialized needs of low-SES populations. In this study, we examine and portray the information and technology needs of low-SES women by considering their unique perspectives and providers' current experiences. We supplement our understanding of women's needs by harvesting online social discourse in PPD-related forums, which we identify as valuable information resources among these populations. METHODS: We conducted (a) 2 focus groups (n = 9), (b) semi-structured interviews with care providers (n = 9) and low SES women (n = 10), and (c) secondary analysis of online messages (n = 1,424). Qualitative data were inductively analyzed using a grounded theory approach. RESULTS: A total of 134 open concepts resulted from patient interviews, 185 from provider interviews, and 106 from focus groups. These revealed six core themes for PPD management, including "Use of Technology/Features", "Access to Care", and "Pregnancy Education". Our social media analysis revealed six PPD topics of importance in online messages, including "Physical and Mental Health" (n = 725 messages), and "Social Support" (n = 674). CONCLUSION: Our data triangulation allowed us to analyze PPD information and technology needs at different levels of granularity. Differences between patients and providers included a focus from providers on needing better support from administrative staff, as well as better PPD clinical decision support. Our results can inform future research and development efforts to address PPD health disparities.

Identifying Contributing Factors Associated With Dental Adverse Events Through a Pragmatic Electronic Health Record-Based Root Cause Analysis.

OBJECTIVE: This study assessed contributing factors associated with dental adverse events (AEs). METHODS: Seven electronic health record-based triggers were deployed identifying potential AEs at 2 dental institutions. From 4106 flagged charts, 2 reviewers examined 439 charts selected randomly to identify and classify AEs using our dental AE type and severity classification systems. Based on information captured in the electronic health record, we analyzed harmful AEs to assess potential contributing factors; harmful AEs were defined as those that resulted in temporary moderate to severe harm, required hospitalization, or resulted in permanent moderate to severe harm. We classified potential contributing factors according to (1) who was involved (person), (2) what were they doing (tasks), (3) what tools/technologies were they using (tools/technologies), (4) where did the event take place (environment), (5) what organizational conditions contributed to the event? (organization), (6) patient (including parents), and (7) professional-professional collaboration. A blinded panel of dental experts conducted a second review to confirm the presence of an AE. RESULTS: Fifty-nine cases had 1 or more harmful AEs. Pain occurred most frequently (27.1%), followed by nerve injury (16.9%), hard tissue injury (15.2%), and soft tissue injury (15.2%). Forty percent of the cases were classified as "temporary not moderate to severe harm." Person (training, supervision, and fatigue) was the most common contributing factor (31.5%), followed by patient (noncompliance, unsafe practices at home, low health literacy, 17.1%), and professional-professional collaboration (15.3%). CONCLUSIONS: Pain was the most common harmful AE identified. Person, patient, and professional-professional collaboration were the most frequently assessed factors associated with harmful AEs.

Discerning conversational context in online health communities for personalized digital behavior change solutions using Pragmatics to Reveal Intent in Social Media (PRISM) framework.

BACKGROUND: Online health communities (OHCs) have emerged as prominent platforms for behavior modification, and the digitization of online peer interactions has afforded researchers with unique opportunities to model multilevel mechanisms that drive behavior change. Existing studies, however, have been limited by a lack of methods that allow the capture of conversational context and socio-behavioral dynamics at scale, as manifested in these digital platforms. OBJECTIVE: We develop, evaluate, and apply a novel methodological framework, Pragmatics to Reveal Intent in Social Media (PRISM), to facilitate granular characterization of peer interactions by combining multidimensional facets of human communication. METHODS: We developed and applied PRISM to analyze peer interactions (N = 2.23 million) in QuitNet, an OHC for tobacco cessation. First, we generated a labeled set of peer interactions (n = 2,005) through manual annotation along three dimensions: communication themes (CTs), behavior change techniques (BCTs), and speech acts (SAs). Second, we used deep learning models to apply our qualitative codes at scale. Third, we applied our validated model to perform a retrospective analysis. Finally, using social network analysis (SNA), we portrayed large-scale patterns and relationships among the aforementioned communication dimensions embedded in peer interactions in QuitNet. RESULTS: Qualitative analysis showed that the themes of social support and behavioral progress were common. The most used BCTs were feedback and monitoring and comparison of behavior, and users most commonly expressed their intentions using SAs-expressive and emotion. With additional in-domain pre-training, bidirectional encoder representations from Transformers (BERT) outperformed other deep learning models on the classification tasks. Content-specific SNA revealed that users' engagement or abstinence status is associated with the prevalence of various categories of BCTs and SAs, which also was evident from the visualization of network structures. CONCLUSIONS: Our study describes the interplay of multilevel characteristics of online communication and their association with individual health behaviors.

Building Consensus for a Shared Definition of Adverse Events: A Case Study in the Profession of Dentistry.

BACKGROUND: To achieve high-quality health care, adverse events (AEs) must be proactively recognized and mitigated. However, there is often ambiguity in applying guidelines and definitions. We describe the iterative calibration process needed to achieve a shared definition of AEs in dentistry. Our alignment process includes both independent and consensus building approaches. OBJECTIVE: We explore the process of defining dental AEs and the steps necessary to achieve alignment across different care providers. METHODS: Teams from 4 dental institutions across the United States iteratively reviewed patient records after identification of charts using an automated trigger tool. Calibration across teams was supported through negotiated definition of AEs and standardization of evidence provided in review. Interrater reliability was assessed using descriptive and κ statistics. RESULTS: After 5 iterative cycles of calibration, the teams (n = 8 raters) identified 118 cases. The average percent agreement for AE determination was 82.2%. Furthermore, the average, pairwise prevalence and bias-adjusted κ (PABAK) was 57.5% (κ = 0.575) for determining AE presence. The average percent agreement for categorization of the AE type was 78.5%, whereas the PABAK was 48.8%. Lastly, the average percent agreement for categorization of AE severity was 82.2% and the corresponding PABAK was 71.7%. CONCLUSIONS: Successful calibration across reviewers is possible after consensus building procedures. Higher levels of agreement were found when categorizing severity (of identified events) rather than the events themselves. Our results demonstrate the need for collaborative procedures as well as training for the identification and severity rating of AEs.

Mobile Health Applications for Postpartum Depression Management: A Theory-Informed Analysis of Change-Use-Engagement (CUE) Criteria in the Digital Environment.

Postpartum Depression (PPD) is the most common childbirth complication, with approximately 15% of postpartum women experiencing depression symptoms. Mobile applications have potential to expand delivery of mental health interventions. However, our understanding of how these tools engage women with PPD and facilitate positive behavioral changes is limited. In our paper, we analyze 15 commercial PPD applications to understand their role as facilitators of change, engagement, and sustained use. Applications reviewed contained an average of four theory-based behavioral change techniques, and highest patient engagement level reached was to empower patients through patient-generated data. Heuristic violations were identified in areas including user control and freedom, aesthetic and minimalist design, and help and documentation. An inverse correlation was found between the number of theory-based behavior change features and patient engagement. Findings suggest underserved populations may suffer further limitations accessing relevant health resources in the current application market.

Development of a Quality Improvement Dental Chart Review Training Program.

INTRODUCTION: Chart review is central to understanding adverse events (AEs) in medicine. In this article, we describe the process and results of educating chart reviewers assigned to evaluate dental AEs. METHODS: We developed a Web-based training program, "Dental Patient Safety Training," which uses both independent and consensus-based curricula, for identifying AEs recorded in electronic health records in the dental setting. Training included (1) didactic education, (2) skills training using videos and guided walkthroughs, (3) quizzes with feedback, and (4) hands-on learning exercises. In addition, novice reviewers were coached weekly during consensus review discussions. TeamExpert was composed of 2 experienced reviewers, and TeamNovice included 2 chart reviewers in training. McNemar test, interrater reliability, sensitivity, specificity, positive predictive value, and negative predictive value were calculated to compare accuracy rates on the identification of charts containing AEs at the start of training and 7 months after consensus building discussions between the 2 teams. RESULTS: TeamNovice completed independent and consensus development training. Initial chart reviews were conducted on a shared set of charts (n = 51) followed by additional training including consensus building discussions. There was a marked improvement in overall percent agreement, prevalence and bias-adjusted κ correlation, and diagnostic measures (sensitivity, specificity, positive predictive value, and negative predictive value) of reviewed charts between both teams from the phase I training program to phase II consensus building. CONCLUSIONS: This study detailed the process of training new chart reviewers and evaluating their performance. Our results suggest that standardized training and continuous coaching improves calibration between experts and trained chart reviewers.

Pragmatics to Reveal Intent in Social Media Peer Interactions: Mixed Methods Study.

BACKGROUND: Online health communities (OHCs) have emerged as the leading venues for behavior change and health-related information seeking. The soul and success of these digital platforms lie in their ability to foster social togetherness and a sense of community by providing personalized support. However, we have a minimal understanding of how conversational posts in these settings lead to collaborative societies and ultimately result in positive health changes through social influence. OBJECTIVE: Our objective is to develop a content-specific and intent-sensitive methodological framework for analyzing peer interactions in OHCs. METHODS: We developed and applied a mixed-methods approach to understand the manifestation of expressions in peer interactions in OHCs. We applied our approach to describe online social dialogue in the context of two online communities, QuitNet (QN) and the American Diabetes Association (ADA) support community. A total of 3011 randomly selected peer interactions (n=2005 from QN, n=1006 from ADA) were analyzed. Specifically, we conducted thematic analysis to characterize communication content and linguistic expressions (speech acts) embedded within the two data sets. We also developed an empirical user persona based on their engagement levels and behavior profiles. Further, we examined the association between speech acts and communication themes across observed tiers of user engagement and self-reported behavior profiles using the chi-square test or the Fisher test. RESULTS: Although social support, the most prevalent communication theme in both communities, was expressed in several subtle manners, the prevalence of emotions was higher in the tobacco cessation community and assertions were higher in the diabetes self-management (DSM) community. Specific communication theme-speech act relationships were revealed, such as the social support theme was significantly associated (P<.05) with 9 speech acts from a total of 10 speech acts (ie, assertion, commissive, declarative, desire, directive, expressive, question, stance, and statement) within the QN community. Only four speech acts (ie, commissive, emotion, expressive, and stance) were significantly associated (P<.05) with the social support theme in the ADA community. The speech acts were also significantly associated with the users' abstinence status within the QN community and with the users' lifestyle status within the ADA community (P<.05). CONCLUSIONS: Such an overlay of communication intent implicit in online peer interactions alongside content-specific theory-linked characterizations of social media discourse can inform the development of effective digital health technologies in the field of health promotion and behavior change. Our analysis revealed a rich gradient of expressions across a standardized thematic vocabulary, with a distinct variation in emotional and informational needs, depending on the behavioral and disease management profiles within and across the communities. This signifies the need and opportunities for coupling pragmatic messaging in digital therapeutics and care management pathways for personalized support.

Digital Technology Needs in Maternal Mental Health: A Qualitative Inquiry.

Digital technologies offer many opportunities to improve mental healthcare management for women seeking pre- and-postnatal care. They provide a discrete, practical medium that is well-suited for the sensitive nature of mental health. Women who are more prone to experiencing peripartum depression (PPD), such as those of low-socioeconomic background or in high-risk pregnancies, can benefit the most from such technologies. However, current digital interventions directed towards this population provide suboptimal support, and their responsiveness to end user needs is quite limited. Our objective is to understand the digital terrain of information needs for low-socioeconomic status women with high-risk pregnancies, specifically within the management of their mental health. This qualitative study consists of semi-structured focus groups and interviews with a sample of nineteen patients. A total of eleven core themes emerged from participant comments. Resulting themes highlighted the need for digital technologies that promote personalized care, a sense of community, and improved provider communication.

Impact of simulated patients on students' self-assessment of competency in practice of geriatric dentistry.

PURPOSE: The evaluation of the effectiveness of simulation training in dentistry has previously been focused primarily on psychomotor hand skills. This study explored the impact of simulated patient (SP) encounters in simulation on students' self-assessment of competency in their clinical and communication abilities with geriatric patients. METHODS: Students from 2 cohorts were recruited for this study. Cohort 1 (n = 30) participated in the standard curriculum with no simulation training and served as the control group. Cohort 2 (n = 34) participated in a SP experience, simulating the initial stages of a care visit for 2 nursing home patients. Students' perceptions of competency to perform these clinical and communication tasks were assessed. A group debriefing session was held 5 weeks post-simulation where Cohort 2 completed a student feedback form. RESULTS: A statistically significant change (P < 0.00001) was noted for both cohorts in their self-reported competence to perform clinical tasks following exposure to an independent clinical experience. In addition to this gain, individuals in Cohort 2 demonstrated improvements following simulation and expressed different responses of impact to questions related to treatment, pharmacology, and managing a complex medical history. CONCLUSIONS: This study suggests that simulation of patient interactions using SPs can strengthen students' self-assessment of competency in their abilities, leading to more genuine interactions with actual patients. These findings will help inform the design of future SP encounters as a component of an evolving humanistic curriculum.

Incorporating Behavioral Trigger Messages Into a Mobile Health App for Chronic Disease Management: Randomized Clinical Feasibility Trial in Diabetes.

BACKGROUND: Although there is a rise in the use of mobile health (mHealth) tools to support chronic disease management, evidence derived from theory-driven design is lacking. OBJECTIVE: The objective of this study was to determine the impact of an mHealth app that incorporated theory-driven trigger messages. These messages took different forms following the Fogg behavior model (FBM) and targeted self-efficacy, knowledge, and self-care. We assess the feasibility of our app in modifying these behaviors in a pilot study involving individuals with diabetes. METHODS: The pilot randomized unblinded study comprised two cohorts recruited as employees from within a health care system. In total, 20 patients with type 2 diabetes were recruited for the study and a within-subjects design was utilized. Each participant interacted with an app called capABILITY. capABILITY and its affiliated trigger (text) messages integrate components from social cognitive theory (SCT), FBM, and persuasive technology into the interactive health communications framework. In this within-subjects design, participants interacted with the capABILITY app and received (or did not receive) text messages in alternative blocks. The capABILITY app alone was the control condition along with trigger messages including spark and facilitator messages. A repeated-measures analysis of variance (ANOVA) was used to compare adherence with behavioral measures and engagement with the mobile app across conditions. A paired sample t test was utilized on each health outcome to determine changes related to capABILITY intervention, as well as participants' classified usage of capABILITY. RESULTS: Pre- and postintervention results indicated statistical significance on 3 of the 7 health survey measures (general diet: P=.03; exercise: P=.005; and blood glucose: P=.02). When only analyzing the high and midusers (n=14) of capABILITY, we found a statistically significant difference in both self-efficacy (P=.008) and exercise (P=.01). Although the ANOVA did not reveal any statistically significant differences across groups, there is a trend among spark conditions to respond more quickly (ie, shorter log-in lag) following the receipt of the message. CONCLUSIONS: Our theory-driven mHealth app appears to be a feasible means of improving self-efficacy and health-related behaviors. Although our sample size is too small to draw conclusions about the differential impact of specific forms of trigger messages, our findings suggest that spark triggers may have the ability to cue engagement in mobile tools. This was demonstrated with the increased use of capABILITY at the beginning and conclusion of the study depending on spark timing. Our results suggest that theory-driven personalization of mobile tools is a viable form of intervention. TRIAL REGISTRATION: ClinicalTrials.gov NCT04132089; http://clinicaltrials.gov/ct2/show/NCT004122089.

NewCope: A Theory-Linked Mobile Application for Stress Education and Management.

The negative effects of long-term stress on health outcomes are well-documented. Emerging technologies that harness mobile technologies have been linked to positive effects on stress management. However, the ways in which existing inter- and intrapersonal theories of behavior change are integrated into development processes of these mHealth technologies for stress coping are limited. In this paper, we present a novel theory-driven approach to develop and implement a sustainable mobile application for stress education and management. Specifically, we integrate the taxonomy of Behavior Change Techniques and user engagement framework to model and adapt theory-driven techniques in the context of mobile technologies. A total of 12 behavior change techniques were incorporated into our mobile application. Initial user evaluation and usability testing was conducted. Results indicate heuristic modifications could improve overall delivery of content, and potential user satisfaction is likely. We conclude that this novel approach may have implications well beyond stress management.

Characterization of Behavioral Transitions Through Social Media Analysis: A Mixed-Methods Approach.

Unhealthy behaviors are a socioeconomic burden and lead to the development of chronic diseases. Relapse is a common issue that most individuals deal with as they adopt and sustain a positive healthy lifestyle. Proper identification of behavioral transitions can help design agile, adaptive, and just-in-time interventions. In this paper, we present a methodology that integrates qualitative coding, machine learning, and formal data analysis using stage transition probabilities and linguistics-based text analysis to track shifts in stages of behavior change as embedded in journal entries recorded by users in an online community for tobacco cessation. Results indicate that our semi-automated stage identification method has an accuracy of 90%. Further analysis revealed stage-specific language features and transition probabilities. Implications for targeted social interventions are discussed.

Exploring the Social Structure of a Health-Related Online Community for Tobacco Cessation: A Two-Mode Network Approach.

Unhealthy behaviors, such as tobacco use, increase individual health risk while also creating a global economic burden on the healthcare system. Social ties have been seen as an important, yet complex factor, to sustain abstinence from these modifiable risk behaviors. However, the underlying social mechanisms are still opaque and poorly understood. Digital health communities provide opportunities to understand social dependencies of behavior change because peer interactions in these platforms are digitized. In this paper, we present a novel approach that integrates theories of behavior change and Exponential Random Graph Models (ERGMs) to understand structural dependencies between users of an online community and the behavior change techniques that are manifested in their communication using an affiliation network. Results indicate population specific traits in terms of individuals' engagement in peer communication embed behavior change techniques in online social settings. Implications for personalized health promotion technologies are discussed.

Cost-aware active learning for named entity recognition in clinical text.

OBJECTIVE: Active Learning (AL) attempts to reduce annotation cost (ie, time) by selecting the most informative examples for annotation. Most approaches tacitly (and unrealistically) assume that the cost for annotating each sample is identical. This study introduces a cost-aware AL method, which simultaneously models both the annotation cost and the informativeness of the samples and evaluates both via simulation and user studies. MATERIALS AND METHODS: We designed a novel, cost-aware AL algorithm (Cost-CAUSE) for annotating clinical named entities; we first utilized lexical and syntactic features to estimate annotation cost, then we incorporated this cost measure into an existing AL algorithm. Using the 2010 i2b2/VA data set, we then conducted a simulation study comparing Cost-CAUSE with noncost-aware AL methods, and a user study comparing Cost-CAUSE with passive learning. RESULTS: Our cost model fit empirical annotation data well, and Cost-CAUSE increased the simulation area under the learning curve (ALC) scores by up to 5.6% and 4.9%, compared with random sampling and alternate AL methods. Moreover, in a user annotation task, Cost-CAUSE outperformed passive learning on the ALC score and reduced annotation time by 20.5%-30.2%. DISCUSSION: Although AL has proven effective in simulations, our user study shows that a real-world environment is far more complex. Other factors have a noticeable effect on the AL method, such as the annotation accuracy of users, the tiredness of users, and even the physical and mental condition of users. CONCLUSION: Cost-CAUSE saves significant annotation cost compared to random sampling.

Cost-sensitive Active Learning for Phenotyping of Electronic Health Records.

Developing high-throughput and high-performance phenotyping algorithms is critical to the secondary use of electronic health records for clinical research. Supervised machine learning-based methods have shown good performance, but often require large annotated datasets that are costly to build. Simulation studies have shown that active learning (AL) could reduce the number of annotated samples while improving the model performance when assuming that the time of labeling each sample is the same (i.e., cost-insensitive). In this study, we proposed a cost- sensitive AL (CostAL) algorithm for clinical phenotyping, using the identification of breast cancer patients as a use case. CostAL implements a linear regression model to estimate the actual time required for annotating each individual sample. We recruited two annotators to manual review medical records of 766 potential breast cancer patients and recorded the actual time of annotating each sample. We then compared CostAL, AL, and passive learning (PL, aka random sampling) using this annotated dataset and generated learning curves for each method. Our experimental results showed that CostAL achieved the highest area under the curve (AUC) score among the three algorithms (PL, AL, and CostAL are 0.784, 0.8501, and 0.8673 for user 1 and 0.8006, 0.8806 and 0.9006 for user 2). To achieve an accuracy of 0.94, AL and CostAL could save 36% and 60% annotation time for user 1 and 53% and 70% annotation time for user 2, when they were compared with PL, indicating the value of cost-sensitive AL approaches.

Transitioning to the National Institutes of Health single institutional review board model: Piloting the use of the Streamlined, Multi-site, Accelerated Resources for Trials IRB Reliance.

BACKGROUND/AIMS: Obtaining ethical approval from multiple institutional review boards is a long-standing challenge to multi-site clinical trials and often leads to significant delays in study activation and enrollment. As of 25 January 2018, the National Institutes of Health began requiring use of a single institutional review board for US multi-site trials. To learn more and further inform the research and regulatory communities around aspects of transitioning to single institutional review board review, this study evaluated the efficiency, resource use, and user perceptions of a nascent institutional review board reliance model (Streamlined, Multi-site, Accelerated Resources for Trials IRB Reliance). METHODS: This research was embedded within the Influenza Vaccine to Effectively Stop Cardio Thoracic Events and Decompensated Heart Failure trial-a multi-site trial of two influenza vaccine formulations. In the first year of the trial, a sample of sites agreed to use the developing Streamlined, Multi-site, Accelerated Resources for Trials IRB Reliance model and participated in its evaluation. In keeping with a least burdensome approach, short surveys were developed and obtained from each reporting entity (relying sites, non-relying site, lead site, and reviewing institutional review board). Data regarding time to institutional review board approval and site activation, costs, and user perceptions of reliant review were self-reported and collected via the survey form. Quantitative and qualitative analyses were performed, with costs analyzed as actual versus estimated due to the lack of established baseline cost data. RESULTS: A total of 13 sites ceded review and received institutional review board approval. Mean time to approval was substantially faster in sites that ceded review using the Streamlined, Multi-site, Accelerated Resources for Trials IRB Reliance model versus the site that did not cede review (81 vs 121 days). The mean time to approval was also faster than published averages for academic medical centers (81 vs 103 days). Time to first enrollment was faster for ceding sites versus the non-ceding site, and also faster than published averages (126 vs 149 and 169 days, respectively). Costs were higher than estimates for local institutional review board review and approval. Nearly half (47%) the stakeholders reported being very satisfied or satisfied with the reliance experience, although many noted the challenge related to institutional culture change. CONCLUSION: Implementation of a single institutional review board represents a shift in practice and culture for many institutions. Evaluation of the reliance arrangements for this study highlights both the potential of, and challenges for, institutions as they transition to single institutional review board review. Although efficiencies were observed for study start-up, we anticipate a learning curve as institutions and research teams implement necessary process and resource changes to adapt to single institutional review board oversight. Findings may inform research teams but are, however, limited by the relatively small number of sites and lack of a control group.

Clinical text annotation - what factors are associated with the cost of time?

Building high-quality annotated clinical corpora is necessary for developing statistical Natural Language Processing (NLP) models to unlock information embedded in clinical text, but it is also time consuming and expensive. Consequently, it important to identify factors that may affect annotation time, such as syntactic complexity of the text- to-be-annotated and the vagaries of individual user behavior. However, limited work has been done to understand annotation of clinical text. In this study, we aimed to investigate how factors inherent to the text affect annotation time for a named entity recognition (NER) task. We recruited 9 users to annotate a clinical corpus and recorded annotation time for each sample. Then we defined a set of factors that we hypothesized might affect annotation time, and fitted them into a linear regression model to predict annotation time. The linear regression model achieved an R2 of 0.611, and revealed eight time-associated factors, including characteristics of sentences, individual users, and annotation order with implications for the practice of annotation, and the development of cost models for active learning research.