"How Did We Get Here?": Topic Drift in Online Health Discussions.

BACKGROUND: Patients increasingly use online health communities to exchange health information and peer support. During the progression of health discussions, a change of topic-topic drift-can occur. Topic drift is a frequent phenomenon linked to incoherence and frustration in online communities and other forms of computer-mediated communication. For sensitive topics, such as health, such drift could have life-altering repercussions, yet topic drift has not been studied in these contexts. OBJECTIVE: Our goals were to understand topic drift in online health communities and then to develop and evaluate an automated approach to detect both topic drift and efforts of community members to counteract such drift. METHODS: We manually analyzed 721 posts from 184 threads from 7 online health communities within WebMD to understand topic drift, members' reaction towards topic drift, and their efforts to counteract topic drift. Then, we developed an automated approach to detect topic drift and counteraction efforts. We detected topic drift by calculating cosine similarity between 229,156 posts from 37,805 threads and measuring change of cosine similarity scores from the threads' first posts to their sequential posts. Using a similar approach, we detected counteractions to topic drift in threads by focusing on the irregular increase of similarity scores compared to the previous post in threads. Finally, we evaluated the performance of our automated approaches to detect topic drift and counteracting efforts by using a manually developed gold standard. RESULTS: Our qualitative analyses revealed that in threads of online health communities, topics change gradually, but usually stay within the global frame of topics for the specific community. Members showed frustration when topic drift occurred in the middle of threads but reacted positively to off-topic stories shared as separate threads. Although all types of members helped to counteract topic drift, original posters provided the most effort to keep threads on topic. Cosine similarity scores show promise for automatically detecting topical changes in online health discussions. In our manual evaluation, we achieved an F1 score of .71 and .73 for detecting topic drift and counteracting efforts to stay on topic, respectively. CONCLUSIONS: Our analyses expand our understanding of topic drift in a health context and highlight practical implications, such as promoting off-topic discussions as a function of building rapport in online health communities. Furthermore, the quantitative findings suggest that an automated tool could help detect topic drift, support counteraction efforts to bring the conversation back on topic, and improve communication in these important communities. Findings from this study have the potential to reduce topic drift and improve online health community members' experience of computer-mediated communication. Improved communication could enhance the personal health management of members who seek essential information and support during times of difficulty.

Automatically Detecting Failures in Natural Language Processing Tools for Online Community Text.

BACKGROUND: The prevalence and value of patient-generated health text are increasing, but processing such text remains problematic. Although existing biomedical natural language processing (NLP) tools are appealing, most were developed to process clinician- or researcher-generated text, such as clinical notes or journal articles. In addition to being constructed for different types of text, other challenges of using existing NLP include constantly changing technologies, source vocabularies, and characteristics of text. These continuously evolving challenges warrant the need for applying low-cost systematic assessment. However, the primarily accepted evaluation method in NLP, manual annotation, requires tremendous effort and time. OBJECTIVE: The primary objective of this study is to explore an alternative approach-using low-cost, automated methods to detect failures (eg, incorrect boundaries, missed terms, mismapped concepts) when processing patient-generated text with existing biomedical NLP tools. We first characterize common failures that NLP tools can make in processing online community text. We then demonstrate the feasibility of our automated approach in detecting these common failures using one of the most popular biomedical NLP tools, MetaMap. METHODS: Using 9657 posts from an online cancer community, we explored our automated failure detection approach in two steps: (1) to characterize the failure types, we first manually reviewed MetaMap's commonly occurring failures, grouped the inaccurate mappings into failure types, and then identified causes of the failures through iterative rounds of manual review using open coding, and (2) to automatically detect these failure types, we then explored combinations of existing NLP techniques and dictionary-based matching for each failure cause. Finally, we manually evaluated the automatically detected failures. RESULTS: From our manual review, we characterized three types of failure: (1) boundary failures, (2) missed term failures, and (3) word ambiguity failures. Within these three failure types, we discovered 12 causes of inaccurate mappings of concepts. We used automated methods to detect almost half of 383,572 MetaMap's mappings as problematic. Word sense ambiguity failure was the most widely occurring, comprising 82.22% of failures. Boundary failure was the second most frequent, amounting to 15.90% of failures, while missed term failures were the least common, making up 1.88% of failures. The automated failure detection achieved precision, recall, accuracy, and F1 score of 83.00%, 92.57%, 88.17%, and 87.52%, respectively. CONCLUSIONS: We illustrate the challenges of processing patient-generated online health community text and characterize failures of NLP tools on this patient-generated health text, demonstrating the feasibility of our low-cost approach to automatically detect those failures. Our approach shows the potential for scalable and effective solutions to automatically assess the constantly evolving NLP tools and source vocabularies to process patient-generated text.

Exploring the relationship between changes in weight and utterances in an online weight loss forum: a content and correlational analysis study.

BACKGROUND: There is increasing interest in the use of online forums as a component of eHealth weight loss interventions. Although the research is mixed on the utility of online forums in general, results suggest that there is promise to this, particularly if the systems can be designed well to support healthful interactions that foster weight loss and continued engagement. OBJECTIVE: The purpose of this study was to examine the relationship between the styles of utterances individuals make on an online weight loss forum and week-to-week fluctuations in weight. This analysis was conducted to generate hypotheses on possible strategies that could be used to improve the overall design of online support groups to facilitate more healthful interactions. METHODS: A convenience sample of individuals using an online weight loss forum (N=4132) included data both on online forum use and weight check-in data. All interactions were coded utilizing the Linguistic Inquiry and Word Count (LIWC) system. Mixed model analyses were conducted to examine the relationship between these LIWC variables and weight over time. RESULTS: Results suggested that increased use of past-tense verbs (P=.05) and motion (P=.02) were associated with lower weekly weights whereas increased use of conjunctions (eg, and, but, whereas; P=.001) and exclusion words (eg, but, without, exclude; P=.07) were both associated with higher weight during the weeks when these utterances were used more. CONCLUSIONS: These results provide some insights on the styles of interactions that appear to be associated with weight fluctuations. Future work should explore the stability of these findings and also explore possibilities for fostering these types of interactions more explicitly within online weight loss forums.

Leveraging cues from person-generated health data for peer matching in online communities.

OBJECTIVE: Online health communities offer a diverse peer support base, yet users can struggle to identify suitable peer mentors as these communities grow. To facilitate mentoring connections, we designed a peer-matching system that automatically profiles and recommends peer mentors to mentees based on person-generated health data (PGHD). This study examined the profile characteristics that mentees value when choosing a peer mentor. MATERIALS AND METHODS: Through a mixed-methods user study, in which cancer patients and caregivers evaluated peer mentor recommendations, we examined the relative importance of four possible profile elements: health interests, language style, demographics, and sample posts. Playing the role of mentees, the study participants ranked mentors, then rated both the likelihood that they would hypothetically contact each mentor and the helpfulness of each profile element in helping the make that decision. We analyzed the participants' ratings with linear regression and qualitatively analyzed participants' feedback for emerging themes about choosing mentors and improving profile design. RESULTS: Of the four profile elements, only sample posts were a significant predictor for the likelihood of a mentee contacting a mentor. Communication cues embedded in posts were critical for helping the participants choose a compatible mentor. Qualitative themes offer insight into the interpersonal characteristics that mentees sought in peer mentors, including being knowledgeable, sociable, and articulate. Additionally, the participants emphasized the need for streamlined profiles that minimize the time required to choose a mentor. CONCLUSION: Peer-matching systems in online health communities offer a promising approach for leveraging PGHD to connect patients. Our findings point to interpersonal communication cues embedded in PGHD that could prove critical for building mentoring relationships among the growing membership of online health communities.

Asynchronous communication among clinical researchers: a study for systems design.

Group work is an essential part of modern health care. Although there have been advances in the technology of communication, these have not necessarily led to efficient and effective communication among collaborating health-care professionals. Instead, barriers such as varied organizational cultures, different training backgrounds, and varied time schedules can overwhelm technological solutions and impede efficient communication. We focus on one particular sort of collaboration, that of group work around a clinical trial protocol, where the collaboration is asynchronous and the participants are geographically distributed. In this work setting, we have applied a computer-supported cooperative work (CSCW) approach in two distinct ways. First, we used observational methods to uncover and understand a complex set of communication behaviors and needs. Second, we used participatory design and iterative prototyping to design a system that aims to improve communication and work flow among these collaborators. We found that these methods work well in tandem-our observational study helped better inform our design, and our prototyping cycles provided additional insight into the work. More specifically, we were able to identify a set of communication problems in the work that led us to specify a set of design desiderata for systems that support asynchronous collaboration around an evolving medical document.

Homophily of Vocabulary Usage: Beneficial Effects of Vocabulary Similarity on Online Health Communities Participation.

Online health communities provide popular platforms for individuals to exchange psychosocial support and form ties. Although regular active participation (i.e., posting to interact with other members) in online health communities can provide important benefits, sustained active participation remains challenging for these communities. Leveraging previous literature on homophily (i.e., "love of those who are like themselves"), we examined the relationship between vocabulary similarity (i.e., homophily of word usage) of thread posts and members' future interaction in online health communities. We quantitatively measured vocabulary similarity by calculating, in a vector space model, cosine similarity between the original post and the first reply in 20,499 threads. Our findings across five online health communities suggest that vocabulary similarity is a significant predictor of members' future interaction in online health communities. These findings carry practical implications for facilitating and sustaining online community participation through beneficial effects of homophily in the vocabulary of essential peer support.

A collaborative clinical trial protocol writing system.

Increasing complexity in medicine has caused clinical trial experts with disparate backgrounds from multiple organizations to collaborate when developing clinical trial protocols. Although many protocol-authoring tools provide computerbased decision support to assist in protocol writing, few of them provide sufficient collaboration support for a group of protocol writers. The iterative group writing activities among interdisciplinary clinical trial experts call for advanced tool support. Here we present a web-based protocol writing system with integrated support for collaborative reviewing and collaborative editing. The system uses a shared database to store threaded review comments and version information for electronic protocols. It also captures rich group event information to provide cross-activity awareness and to facilitate self-coordination within the collaborative writing team. We believe that our system can help streamline collaborative clinical trial protocol writing processes.

An ethnographic study of collaborative clinical trial protocol writing.

Clinical trial protocol documents play an important role in clinical research. However, clinical protocol writing remains a complex and relatively un-studied process. Protocols are often written by teams of people, yet little prior research has captured the problems or analyzed the collaboration support needs of protocol writers. Here we present the results of an initial ethnographic study into the clinical trial protocol writing processes at a representative cooperative clinical trial group funded by National Cancer Institute (NCI). We analyzed the collaborative nature of the writing process, identified common problems, derived information and communication support needs of collaborative clinical protocol writers, and provided recommendations to streamline the process. We believe that this paper contributes useful implications for the design of future collaborative clinical protocol writing tools.

Evaluating health interest profiles extracted from patient-generated data.

Patient-generated health data (PGHD) offers a promising resource for shaping patient care, self-management, population health, and health policy. Although emerging technologies bolster opportunities to extract PGHD and profile the needs and experiences of patients, few efforts examine the validity and use of such profiles from the patient's perspective. To address this gap, we explore health interest profiles built automatically from online community posts. Through a user evaluation with community members, we found that extracted profiles not only align with members' stated health interests, but also expand upon those manually entered interests with little user effort. Community members express positive attitudes toward the use and expansion of profiles to connect with peers for support. Despite this promising approach, findings also point to improvements required of biomedical text processing tools to effectively process PGHD. Findings demonstrate opportunities to leverage the wealth of unstructured PGHD available in emerging technologies that patients regularly use.

Patient moderator interaction in online health communities.

An increasing number of people visit online health communities to share experiences and seek health information. Although studies have enumerated reasons for patients' visits to online communities for health information from peers, we know little about how patients gain health information from the moderators in these communities. We qualitatively analyze 480 patient and moderator posts from six communities to understand how moderators fulfill patients' information needs. Our findings show that patients use the community as an integral part of their health management practices. Based on our results, we suggest enhancements to moderated online health communities for their unique role to support patient care.

Barriers to physical activity: a study of self-revelation in an online community.

The importance of regular physical activity to overall health has been well established, yet adults in the United States are leading increasingly sedentary lives. Research suggests that lowering perceived barriers to exercise is an effective strategy for encouraging physical activity. This article describes the top barriers that emerged from a qualitative analysis of message board traffic from a three-month healthy lifestyle intervention that promoted physical activity and healthy eating. The findings further elaborate known barriers to physical activity-two of which are not reported as key barriers in prior research-and illustrate the value of a grounded approach to studying health and fitness behaviors. Based on our analysis, we identify design considerations for technologies that encourage and support physical activity. Understanding the needs of a population is a critical step in the design process, and this paper offers unique insights for those working in this growing domain.

Bringing the Field into Focus: User-centered Design of a Patient Expertise Locator.

Managing personal aspects of health is challenging for many patients, particularly those facing a serious condition such as cancer. Finding experienced patients, who can share their knowledge from managing a similar health situation, is of tremendous value. Users of health-related social software form a large base of such knowledge, yet these tools often lack features needed to locate peers with expertise. Informed directly by our field work with breast cancer patients, we designed a patient expertise locator for users of online health communities. Using feedback from two focus groups with breast cancer survivors, we took our design through two iterations. Focus groups concluded that expertise locating features proved useful for extending social software. They guided design enhancements by suggesting granular user control through (1) multiple mechanisms to identify expertise, (2) detailed user profiles to select expertise, and (3) varied collaboration levels. Our user-centered approach links field work to design through close collaboration with patients. By illustrating trade-offs made when sharing sensitive health information, our findings inform the incorporation of expertise locating features into social software for patients.

Incorporating collaboratory concepts into informatics in support of translational interdisciplinary biomedical research.

Due to its complex nature, modern biomedical research has become increasingly interdisciplinary and collaborative in nature. Although a necessity, interdisciplinary biomedical collaboration is difficult. There is, however, a growing body of literature on the study and fostering of collaboration in fields such as computer supported cooperative work (CSCW) and information science (IS). These studies of collaboration provide insight into how to potentially alleviate the difficulties of interdisciplinary collaborative research. We, therefore, undertook a cross cutting study of science and engineering collaboratories to identify emergent themes. We review many relevant collaboratory concepts: (a) general collaboratory concepts across many domains: communication, common workspace and coordination, and data sharing and management, (b) specific collaboratory concepts of particular biomedical relevance: data integration and analysis, security structure, metadata and data provenance, and interoperability and data standards, (c) environmental factors that support collaboratories: administrative and management structure, technical support, and available funding as critical environmental factors, and (d) future considerations for biomedical collaboration: appropriate training and long-term planning. In our opinion, the collaboratory concepts we discuss can guide planning and design of future collaborative infrastructure by biomedical informatics researchers to alleviate some of the difficulties of interdisciplinary biomedical collaboration.

Locating Patient Expertise in Everyday Life.

Coping with a new health issue often requires individuals to acquire knowledge and skills to manage personal health. Many patients turn to one another for experiential expertise outside the formal bounds of the health-care system. Internet-based social software can facilitate expertise sharing among patients, but provides only limited ways for users to locate sources of patient expertise. Although much prior research has investigated expertise location and systems to augment expertise sharing in workplace organizations, the transferability of this knowledge to other contexts, such as personal health, is unclear. Guided by expertise locating frameworks drawn from prior work, we conducted a field study to investigate expertise locating in the informal and everyday context of women diagnosed with breast cancer. Similarities between patients' expertise locating practices and practices of professionals in workplace organizations suggest similar support strategies could apply in both contexts. However, unlike professionals, unsolicited advice often triggered patients to locate expertise. They identified expertise through various forms of gatekeeping. The high-stakes nature of problems patients faced also led them to use triangulation strategies in anticipation of breakdowns in expertise location. Based on these key differences, we explored five design additions to social software that could support patients in their critical need to locate patient expertise.

Using mobile & personal sensing technologies to support health behavior change in everyday life: lessons learned.

Lifestyle modification is a key facet of the prevention and management of chronic diseases. Mobile devices that people already carry provide a promising platform for facilitating these lifestyle changes. This paper describes key lessons learned from the development and evaluation of two mobile systems for encouraging physical activity. We argue that by supporting persistent cognitive activation of health goals, encouraging an extensive range of relevant healthy behaviors, focusing on long-term patterns of activity, and facilitating social support as an optional but not primary motivator, systems can be developed that effectively motivate behavior change and provide support when and where people make decisions that affect their health.

Participatory design of a collaborative clinical trial protocol writing system.

OBJECTIVE: To explore concrete approaches to socio-technical design of collaborative healthcare information systems and to design a groupware technology for collaborative clinical trial protocol writing. METHOD: We conducted "quick and dirty ethnography" through semi-structured interviews, observational studies, and work artifacts analysis to understand the group work for protocol development. We used participatory design through evolutionary prototyping to explore the feature space of a collaborative writing system. Our design strategies include role-based user advocacy, formative evaluation, and change management. RESULTS: Quick and dirty ethnography helped us efficiently understand relevant work practice, and participatory design helped us engage users into design and bring out their tacit work knowledge. Our approach that intertwined both techniques helped achieve a "work-informed and user-oriented" design. This research leads to a collaborative writing system that supports in situ communication, group awareness, and effective work progress tracking. The usability evaluation results have been satisfactory. The system design is being transferred to an organizational tool for daily use.

Rapid improvement of calcinosis in juvenile dermatomyositis with alendronate therapy.

A 6-year-old boy with improving juvenile dermatomyositis (JDM) developed severe and debilitating calcinosis, unresponsive to diltiazem and probenecid. Alendronate produced dramatic improvement within 1 month and by 12 months calcinosis had virtually resolved. The response was followed by bone mineral content measurements.

Technology, work, and information flows: lessons from the implementation of a wireless alert pager system.

The combination of collaborative work practices and information technology affect the flow of information in clinical settings. The introduction of a new technology into these settings can change not only established work practices but also the information flows. In this paper, we examine the introduction of a wireless alerts pager in a surgical intensive care unit (SICU). Through a qualitative study, we analyze the effects that this new information tool had on both the work practices in the SICU and the information flow in the unit. We describe four challenges that SICU staff members faced with respect to the alerts pagers. We found that the pager provided new routes of information to SICU staff but in doing so disrupted existing work practices and information flows.

Scenario-based participatory design of a collaborative clinical trial protocol authoring system.

We present our experience of using prototype scenarios to actively involve users in the design of a collaborative clinical trial protocol authoring system. This method enables us to do usability testing and elicit prompt user feedback at the early phase of de-sign. We conclude that it is an effective approach to the design of complex medical information systems.

Incorporating ideas from computer-supported cooperative work.

Many information systems have failed when deployed into complex health-care settings. We believe that one cause of these failures is the difficulty in systematically accounting for the collaborative and exception-filled nature of medical work. In this methodological review paper, we highlight research from the field of computer-supported cooperative work (CSCW) that could help biomedical informaticists recognize and design around the kinds of challenges that lead to unanticipated breakdowns and eventual abandonment of their systems. The field of CSCW studies how people collaborate with each other and the role that technology plays in this collaboration for a wide variety of organizational settings. Thus, biomedical informaticists could benefit from the lessons learned by CSCW researchers. In this paper, we provide a focused review of CSCW methods and ideas-we review aspects of the field that could be applied to improve the design and deployment of medical information systems. To make our discussion concrete, we use electronic medical record systems as an example medical information system, and present three specific principles from CSCW: accounting for incentive structures, understanding workflow, and incorporating awareness.