Advantages and disadvantages of using theory-based versus data-driven models with social and behavioral determinants of health data.

OBJECTIVE: Theory-based research of social and behavioral determinants of health (SBDH) found SBDH-related patterns in interventions and outcomes for pregnant/birthing people. The objectives of this study were to replicate the theory-based SBDH study with a new sample, and to compare these findings to a data-driven SBDH study. MATERIALS AND METHODS: Using deidentified public health nurse-generated Omaha System data, 2 SBDH indices were computed separately to create groups based on SBDH (0-5+ signs/symptoms). The data-driven SBDH index used multiple linear regression with backward elimination to identify SBDH factors. Changes in Knowledge, Behavior, and Status (KBS) outcomes, numbers of interventions, and adjusted R-squared statistics were computed for both models. RESULTS: There were 4109 clients ages 13-40 years. Outcome patterns aligned with the original research: KBS increased from admission to discharge with Knowledge improving the most; discharge KBS decreased as SBDH increased; and interventions increased as SBDH increased. Slopes of the data-driven model were steeper, showing clearer KBS trends for data-driven SBDH groups. The theory-based model adjusted R-squared was 0.54 (SE = 0.38) versus 0.61 (SE = 0.35) for the data-driven model with an entirely different set of SBDH factors. CONCLUSIONS: The theory-based approach provided a framework to identity patterns and relationships and may be applied consistently across studies and populations. In contrast, the data-driven approach can provide insights based on novel patterns for a given dataset and reveal insights and relationships not predicted by existing theories. Data-driven methods may be an advantage if there is sufficiently comprehensive SBDH data upon which to create the data-driven models.

Understanding Women's Cardiovascular Health Using MyStrengths+MyHealth: A Patient-Generated Data Visualization Study of Strengths, Challenges, and Needs Differences.

PURPOSE: The purpose of this data visualization study was to identify patterns in patient-generated health data (PGHD) of women with and without Circulation signs or symptoms. Specific aims were to (a) visualize and interpret relationships among strengths, challenges, and needs of women with and without Circulation signs or symptoms; (b) generate hypotheses based on these patterns; and (c) test hypotheses generated in Aim 2. DESIGN: The design of this visualization study was retrospective, observational, case controlled, and exploratory. METHODS: We used existing de-identified PGHD from a mobile health application, MyStrengths+MyHealth (N = 383). From the data, women identified with Circulation signs or symptoms (n = 80) were matched to an equal number of women without Circulation signs or symptoms. Data were analyzed using data visualization techniques and descriptive and inferential statistics. FINDINGS: Based on the patterns, we generated nine hypotheses, of which four were supported. Visualization and interpretation of relationships revealed that women without Circulation signs or symptoms compared to women with Circulation signs or symptoms had more strengths, challenges, and needs-specifically, strengths in connecting; challenges in emotions, vision, and health care; and needs related to info and guidance. CONCLUSIONS: This study suggests that visualization of whole-person health including strengths, challenges, and needs enabled detection and testing of new health patterns. Some findings were unexpected, and perspectives of the patient would not have been detected without PGHD, which should be valued and sought. Such data may support improved clinical interactions as well as policies for standardization of PGHD as sharable and comparable data across clinical and community settings. CLINICAL RELEVANCE: Standardization of patient-generated whole-person health data enabled clinically relevant research that included the patients' perspective.

Towards more Accessible Precision Medicine: Building a more Transferable Machine Learning Model to Support Prognostic Decisions for Micro- and Macrovascular Complications of Type 2 Diabetes Mellitus.

Although machine learning models are increasingly being developed for clinical decision support for patients with type 2 diabetes, the adoption of these models into clinical practice remains limited. Currently, machine learning (ML) models are being constructed on local healthcare systems and are validated internally with no expectation that they would validate externally and thus, are rarely transferrable to a different healthcare system. In this work, we aim to demonstrate that (1) even a complex ML model built on a national cohort can be transferred to two local healthcare systems, (2) while a model constructed on a local healthcare system's cohort is difficult to transfer; (3) we examine the impact of training cohort size on the transferability; and (4) we discuss criteria for external validity. We built a model using our previously published Multi-Task Learning-based methodology on a national cohort extracted from OptumLabs® Data Warehouse and transferred the model to two local healthcare systems (i.e., University of Minnesota Medical Center and Mayo Clinic) for external evaluation. The model remained valid when applied to the local patient populations and performed as well as locally constructed models (concordance: .73-.92), demonstrating transferability. The performance of the locally constructed models reduced substantially when applied to each other's healthcare system (concordance: .62-.90). We believe that our modeling approach, in which a model is learned from a national cohort and is externally validated, produces a transferable model, allowing patients at smaller healthcare systems to benefit from precision medicine.

Assessment of Personal Health Care Management and Chronic Disease Prevalence: Comparative Analysis of Demographic, Socioeconomic, and Health-Related Variables.

BACKGROUND: The use of personal health care management (PHM) is increasing rapidly within the United States because of implementation of health technology across the health care continuum and increased regulatory requirements for health care providers and organizations promoting the use of PHM, particularly the use of text messaging (short message service), Web-based scheduling, and Web-based requests for prescription renewals. Limited research has been conducted comparing PHM use across groups based on chronic conditions. OBJECTIVE: This study aimed to describe the overall utilization of PHM and compare individual characteristics associated with PHM in groups with no reported chronic conditions, with 1 chronic condition, and with 2 or more such conditions. METHODS: Datasets drawn from the National Health Interview Survey were analyzed using multiple logistic regression to determine the level of PHM use in relation to demographic, socioeconomic, or health-related factors. Data from 47,814 individuals were analyzed using logistic regression. RESULTS: Approximately 12.19% (5737/47,814) of respondents reported using PHM, but higher rates of use were reported by individuals with higher levels of education and income. The overall rate of PHM remained stable between 2009 and 2014, despite increased focus on the promotion of patient engagement initiatives. Demographic factors predictive of PHM use included people who were younger, non-Hispanic, and who lived in the western region of the United States. There were also differences in PHM use based on socioeconomic factors. Respondents with college-level education were over 2.5 times more likely to use PHM than respondents without college-level education. Health-related factors were also predictive of PHM use. Individuals with health insurance and a usual place for health care were more likely to use PHM than individuals with no health insurance and no usual place for health care. Individuals reporting a single chronic condition or multiple chronic conditions reported slightly higher levels of PHM use than individuals reporting no chronic conditions. Individuals with no chronic conditions who did not experience barriers to accessing health care were more likely to use PHM than individuals with 1 or more chronic conditions. CONCLUSIONS: The findings of this study illustrated the disparities in PHM use based on the number of chronic conditions and that multiple factors influence the use of PHM, including economics and education. These findings provide evidence of the challenge associated with engaging patients using electronic health information as the health care industry continues to evolve.

Multi-Task Learning to Identify Outcome-Specific Risk Factors that Distinguish Individual Micro and Macrovascular Complications of Type 2 Diabetes.

Because deterioration in overall metabolic health underlies multiple complications of Type 2 Diabetes Mellitus, a substantial overlap among risk factors for the complications exists, and this makes the outcomes difficult to distinguish. We hypothesized each risk factor had two roles: describing the extent of deteriorating overall metabolic health and signaling a particular complication the patient is progressing towards. We aimed to examine feasibility of our proposed methodology that separates these two roles, thereby, improving interpretation of predictions and helping prioritize which complication to target first. To separate these two roles, we built models for six complications utilizing Multi-Task Learning-a machine learning technique for modeling multiple related outcomes by exploiting their commonality-in 80% of EHR data (N=9,793) from a university hospital and validated them in remaining 20% of the data. Additionally, we externally validated the models in claims and EHR data from the OptumLabs™ Data Warehouse (N=72,720). Our methodology successfully separated the two roles, revealing distinguishing outcome-specific risk factors without compromising predictive performance. We believe that our methodology has a great potential to generate more understandable thus actionable clinical information to make a more accurate and timely prognosis for the patients.

Discovering Public Health Nurse-Specific Family Home Visiting Intervention Patterns Using Visualization Techniques.

Visualization is a Big Data method for detecting and validating previously unknown and hidden patterns within large data sets. This study used visualization techniques to discover and test novel patterns in public health nurse (PHN)-client-risk-intervention-outcome relationships. To understand the mechanism underlying risk reduction among high risk mothers, data representing complex social interventions were visualized in a series of three steps, and analyzed with other important contextual factors using standard descriptive and inferential statistics. Overall, client risk decreased after clients received personally tailored PHN services. Clinically important and unique PHN-client-risk-intervention-outcome patterns were discovered through pattern detection using streamgraphs, heat maps, and parallel coordinates techniques. Statistical evaluation validated that PHN intervention tailoring leads to improved client outcomes. The study demonstrates the importance of exploring data to discover ways to improve care quality and client outcomes. Further research is needed to examine additional factors that may influence PHN-client-risk-intervention-outcome patterns, and to test these methods with other data sets.

Using Publicly Available Data to Characterize Consumers Use of Email to Communicate with Healthcare Providers.

The use of patient focused technology has been proclaimed as a means to improve patient satisfaction and improve care outcomes. The Center for Medicaid/Medicare Services, through its EHR Incentive Program, has required eligible hospitals and professionals to send and receive secure messages from patients in order to receive financial incentives and avoid reimbursement penalties. Secure messaging between providers and patients has the potential to improve communication and care outcomes. The purpose of this study was to use National Health Interview Series (NHIS) data to identify the patient characteristics associated with communicating with healthcare providers via email. Individual patient characteristics were analyzed to determine the likelihood of emailing healthcare providers. The use of email for this purpose is associated with educational attainment, having a usual place of receiving healthcare, income, and geography. Publicly available data such as the NHIS may be used to better understand trends in adoption and use of consumer health information technologies.

Determinants of Consumer eHealth Information Seeking Behavior.

Patients are increasingly using the Internet and other technologies to engage in their own healthcare, but little research has focused on the determinants of consumer eHealth behaviors related to Internet use. This study uses data from 115,089 respondents to four years of the National Health Interview Series to identify the associations between one consumer eHealth behavior (information seeking) and demographics, health measures, and Personal Health Information Management (PHIM) (messaging, scheduling, refills, and chat). Individuals who use PHIM are 7.5 times more likely to search the internet for health related information. Just as health has social determinants, the results of this study indicate there are potential social determinants of consumer eHealth behaviors including personal demographics, health status, and healthcare access.

Divisive Hierarchical Clustering towards Identifying Clinically Significant Pre-Diabetes Subpopulations.

Type 2 Diabetes Mellitus is a progressive disease with increased risk of developing serious complications. Identifying subpopulations and their relevant risk factors can contribute to the prevention and effective management of diabetes. We use a novel divisive hierarchical clustering technique to identify clinically interesting subpopulations in a large cohort of Olmsted County, MN residents. Our results show that our clustering algorithm successfully identified clinically interesting clusters consisting of patients with higher or lower risk of diabetes than the general population. The proposed algorithm offers fine control over the granularity of the clustering, has the ability to seamlessly discover and incorporate interactions among the risk factors, and can handle non-proportional hazards, as well. It has the potential to significantly impact clinical practice by recognizing patients with specific risk factors who may benefit from an alternative management approach potentially leading to the prevention of diabetes and its complications.

Effects of time constraints on clinician-computer interaction: a study on information synthesis from EHR clinical notes.

BACKGROUND: Time is a measurable and critical resource that affects the quality of services provided in clinical practice. There is limited insight into the effects of time restrictions on clinicians' cognitive processes with the electronic health record (EHR) in providing ambulatory care. OBJECTIVE: To understand the impact of time constraints on clinicians' synthesis of text-based EHR clinical notes. METHODS: We used an established clinician cognitive framework based on a think-aloud protocol. We studied interns' thought processes as they accomplished a set of four preformed ambulatory care clinical scenarios with and without time restrictions in a controlled setting. RESULTS: Interns most often synthesized details relevant to patients' problems and treatment, regardless of whether or not the time available for task performance was restricted. In contrast to previous findings, subsequent information commonly synthesized by clinicians related most commonly to the chronology of clinical events for the unrestricted time observations and to investigative procedures for the time-restricted sessions. There was no significant difference in the mean number of omission errors and incorrect deductions when interns synthesized the EHR clinical notes with and without time restrictions (3.5±0.5 vs. 2.3±0.5, p=0.14). CONCLUSION: Our results suggest that the incidence of errors during clinicians' synthesis of EHR clinical notes is not increased with modest time restrictions, possibly due to effective adjustments of information processing strategies learned from the usual time-constrained nature of patient visits. Further research is required to investigate the effects of similar or more extreme time variations on cognitive processes employed with different levels of expertise, specialty, and with different care settings.

Systematic refinement of a health information technology time and motion workflow instrument for inpatient nursing care using a standardized interface terminology.

Time and motion (T&M) studies provide an objective method to measure the expenditure of time by clinicians. While some instruments for T&M studies have been designed to evaluate health information technology (HIT), these instruments have not been designed for nursing workflow. We took an existing open source HIT T&M study application designed to evaluate physicians in the ambulatory setting and rationally adapted it through empiric observations to record nursing activities in the inpatient setting and linked this instrument to an existing interface terminology, the Omaha System. Nursing activities involved several dimensions and could include multiple activities occurring simultaneously, requiring significant instrument redesign. 94% of the activities from the study instrument mapped adequately to the Omaha System. T&M study instruments require customization in design optimize them for different environments, such as inpatient nursing, to enable optimal data collection. Interface terminologies show promise as a framework for recording and analyzing T&M study data.

The role of nonverbal and verbal communication in a multimedia informed consent process.

OBJECTIVE: Nonverbal and verbal communication elements enhance and reinforce the consent form in the informed consent process and need to be transferred appropriately to multimedia formats using interaction design when re-designing the process. METHODS: Observational, question asking behavior, and content analyses were used to analyze nonverbal and verbal elements of an informed consent process. RESULTS: A variety of gestures, interruptions, and communication styles were observed. CONCLUSION: In converting a verbal conversation about a textual document to multimedia formats, all aspects of the original process including verbal and nonverbal variation should be one part of an interaction community-centered design approach.

Evaluating the decision accuracy and speed of clinical data visualizations.

Clinicians face an increasing volume of biomedical data. Assessing the efficacy of systems that enable accurate and timely clinical decision making merits corresponding attention. This paper discusses the multiple-reader multiple-case (MRMC) experimental design and linear mixed models as means of assessing and comparing decision accuracy and latency (time) for decision tasks in which clinician readers must interpret visual displays of data. These tools can assess and compare decision accuracy and latency (time). These experimental and statistical techniques, used extensively in radiology imaging studies, offer a number of practical and analytic advantages over more traditional quantitative methods such as percent-correct measurements and ANOVAs, and are recommended for their statistical efficiency and generalizability. An example analysis using readily available, free, and commercial statistical software is provided as an appendix. While these techniques are not appropriate for all evaluation questions, they can provide a valuable addition to the evaluative toolkit of medical informatics research.

A trend map for e-health information.

Health information of varying levels of quality is available and accessed on the web by patients and medical personnel, who base their decisions on the knowledge they find on the sites they visit. Providing a network map showing different types of sources for medical information is a powerful way to visualize the complex, overlapping web of information available to providers and consumers. We present a prototype map representing consumer e-health information web sites.

Improved techniques for quantitatively comparing data visualizations.

Visualizations are increasingly important in helping users manage large data streams. As a result, researchers often need to compare the performance of several visualizations. We present two statistical techniques, multiple-reader multiple-case receiver operating characteristic curve analysis, and generalized linear mixed models, to compare the accuracy and speed of decisions using data visualizations. These techniques have several advantages over simpler strategies for assessing decision quality, and should be made part of the quantitative evaluation of visualizations.

Design and evaluation of a web-based interactive visualization system for lung transplant home monitoring data.

We describe the implementation and evaluation of an interactive system for displaying patient pulmonary information from a lung transplant home monitoring program. Software was designed to take case information and render it as Scalable Vector Graphics (SVG) in a web browser. Twelve clinician readers reviewed twenty randomly-ordered, retrospective cases in each of three display formats (graphical, graphical interactive, and tabular) and determined whether they showed signs of infection or graft rejection. Decision times and reader preferences were also recorded. No statistically significant differences were found with respect to decision accuracy. However, the graphical displays yielded up to 25% faster decision times than numerical tables. Readers overwhelmingly preferred the graphical formats, particularly the interactive displays. We believe that graphical, interactive displays of patient data would be well-accepted and efficacious tools in clinical practice, whether for transplant care, or any care involving the assessment of large bodies of time-oriented, multivariable data.