Resource and Response Type Classification for Consumer Health Question Answering.

Health question answering systems often depend on the initial step of question type classification. Practitioners face several modeling choices for this component alone. We evaluate the effectiveness of different modeling choices in both the embeddings and architectural hyper-parameters of the classifier. In the process, we achieve improved performance over previous methods, achieving a new best 5-fold accuracy of 85.3% on the GARD dataset. The contribution of this work is to evaluate the performance of sentence classification methods on the task of consumer health question type classification and to contribute a dataset of 2,882 medical questions annotated for question type.

Evaluating the Quality and Readability of Internet Information on Meningiomas.

BACKGROUND: The Internet is a highly powerful resource for patients and provides an extensive amount of information on medical conditions. It is therefore important that the information accessible is accurate, up to date, and at an appropriate comprehensive level for the general public. This article aims to evaluate the quality of patient information on meningiomas. METHODS: The term meningioma was searched using the following search engines: Google, Bing, Yahoo, Ask, and AOL. The top 100 meningioma Web sites were analyzed for readability using the Flesch Reading Ease score and the Flesch-Kincaid grade level. The quality of each Web page was assessed with the DISCERN instrument and the Centers for Disease Control and Prevention (CDC) Clear Communication Index (CCI). RESULTS: The quality of information on the Internet on meningiomas is highly variable. The overall mean Flesch Reading Ease score was 43.1 (standard deviation = 13.3) and the mean Flesch-Kincaid grade of all the Web sites was 11.2 (standard deviation = 2.3). This finding suggests that the information is on average difficult to read. Only one Web site was at the recommended seventh-grade level and the remainder were above this grade. Only one third of the Web pages had Health On the Net Code of Conduct or The Information Standard certification and were found to be significantly of higher quality: DISCERN (P = 0.022) and CDC CCI (P = 0.027). More than 50% of the Web sites had significantly poor or average DISCERN scores and only 2 Web sites fulfilled the CDC CCI criteria. CONCLUSIONS: It is recommended that clinicians personally research material for their patients to be able to guide them to reliable and accurate Web sites. It is also encouraged to become Health On the Net Code of Conduct/The Information Standard certified because this may indicate information of high quality. In addition, it is also recommended that authors of existing information assess the quality of their online health information against the CDC CCI criteria.

[Comprehensibility of online-based patient education material in ophthalmology]. / Die Lesbarkeit von onlinebasierten Patienteninformationen in der Augenheilkunde.

BACKGROUND: Investigations have shown that the internet as a source of information in medical issues is increasing in importance. For most patients information delivered or supported by hospitals and universities is considered to be the most reliable, however, the comprehensibility of available information is often considered to be wanting. Comprehensibility scores are formulae allowing a quantitative value for the readability of a document to be calculated. OBJECTIVE: The purpose of this study was to assess data by analyzing the comprehensibility of medical information published on the websites of departments for ophthalmology of German university hospitals. We investigated and analyzed medical information dealing with three eye diseases with potentially severe irreversible damage. METHODS: The websites of 32 departments for ophthalmology of German university hospitals were investigated. Information regarding cataracts, glaucoma and retinal detachment (amotio retinae) were identified and analyzed. All information was systematically analyzed regarding comprehensibility by using the analysis program Text-Lab ( http://www.text-lab.de ) by calculation of five readability scores: the Hohenheim comprehensibility index (HVI), the Amstad index, the simple measure of gobbledygook (G-SMOG) index, the Vienna non-fictional text formula (W-STX) and the readability index (LIX). RESULTS: In 59 cases (61.46 %) useful text information from the homepage of the institutions could be detected and analyzed. On average the comprehensibility of the information was identified as being poor (HVI 7.91 ± 3.94, Amstad index 35.45 ± 11.85, Vienna formula 11.19 ± 1.93, G­SMOG 9.77 ± 1.42 and the LIX 54.53 ± 6.67). CONCLUSION: In most of the cases patient information material was written far above the literacy level of the average population. It must be assumed that the presented information is difficult to read for the majority of the patients. A critical evaluation of accessible information material seems to be desirable and available texts should be amended.

The Foundation in Evidence of Medical and Dental Telephone Consultations.

BACKGROUND: Patients can only make well-informed decisions if the information they are given by health professionals is based on scientific evidence. In this study, we assessed the foundation in evidence of free, publicly available telephone consultations in Germany. METHODS: From March 2013 to January 2014, four hidden clients seeking information asked standardized questions about three medical topics (screening for colorectal cancer, for glaucoma, and for trisomy 21) and three dental ones (the sealing of dental fissures, professional dental cleaning, and mercury detoxification). Depending on the topic, the questions addressed such issues as the risk of disease and the purpose, content, validity, benefits, and risks of potential diagnostic and therapeutic measures. All identifiable telephone consultation services that provided counselling on the above topics were included in the study (23 government-sponsored institutions, 31 institutions independently run by physicians, 521 institutions under religious auspices, 25 dental counselling services). RESULTS: Of the 599 telephone consultation services that were identified, 567 were contacted; 404 did not offer any relevant counselling. A total of 293 conversations were held with the remaining 163 consultation services. Six of these conversations fully met predefined criteria for evidence-based counselling. The percentage of appropriate answers to the key questions on each topic was 5% for colorectal cancer screening (7/140), 23.8% for glaucoma screening (25/105), 33.9% for trisomy 21 screening (121/357), 27.5% for the sealing of dental fissures (28/102), 16.2% for professional dental cleaning (19/117), and 12.9% for mercury detoxification (12/93). The percentage of appropriate answers also varied depending on the type of institution: 26.8% for government-sponsored institutions (67/250), 4.5% for institutions independently run by physicians (4/88), and 31.1% for institutions under religious auspices (82/264). CONCLUSION: The medical and dental counselling now offered over the telephone by the types of institutions included in this study does not satisfy the criteria for evidence-based health information.

Internet Information for Patients on Cancer Diets - an Analysis of German Websites.

BACKGROUND: In recent years, the Internet has become an important source of information for cancer patients. Various cancer diets that are publicized on the Web promise significant benefits. The aim of our study was to evaluate the quality of online patient information about cancer diets. MATERIALS AND METHODS: A patient's search for 'cancer diets' on German websites was simulated using the search engine Google. The websites were evaluated utilizing a standardized instrument with formal and content aspects. RESULTS: An analysis of 60 websites revealed that websites from nonprofit associations as well as self-help groups offer the best content and formal ranking. Websites whose owners aim to make a profit, practices that offer cancer diet therapies, and newspapers received the poorest quality score. The majority of content provided on the Web gets published by profit-oriented content groups. CONCLUSION: The divergence between profit-driven websites offering low-quality content and the few trustworthy websites on cancer diets is enormous. The information given online about cancer diets may turn out to be a hazardous pitfall. In order to present evidence-based information about cancer diets, online information should be replenished to create a more accurate picture and give higher visibility to the right information.

[Storytelling in Health Journalism: Online Survey of Health Journalists on Definition and Use]. / Storytelling im Gesundheitsjournalismus: Onlinebefragung von Fachjournalisten über Definition und Anwendung.

Background: Although health information is of great interest and plays an important role in almost all media, there are very few studies on the actual work of health journalists. Methods: A quantitative online survey with qualitative elements on the definition and use of storytelling in health journalism was carried out among members of various professional journalists' associations (n=86). Results: The results suggest that health journalists understand storytelling especially as a term used when an article has a dramatic construction, and the story is about real people. As reasons for using storytelling, health journalists primarily name the understandable and clear presentation of medical issues. They see better chances for identification and establishing a relationship to the readers' lives. Of particular importance seems to be that narrative elements do not distort the facts and protect the privacy rights of persons mentioned in case reports.

[Text Comprehensibility of Hospital Report Cards]. / Verständlichkeit der Texte von Qualitätsvergleichen zu Krankenhausleistungen.

Objectives: Recently, the number of hospital report cards that compare quality of hospitals and present information from German quality reports has greatly increased. Objectives of this study were to a) identify suitable methods for measuring the readability and comprehensibility of hospital report cards, b) to obtain reliable information on the comprehensibility of texts for laymen, c) to give recommendations for improvements and d) to recommend public health actions. Methods: The readability and comprehensibility of the texts were tested with a) a computer-aided evaluation of formal text characteristics (readability indices Flesch (German formula) and 1. Wiener Sachtextformel formula), b) an expert-based heuristic analysis of readability and comprehensibility of texts (counting technical terms and analysis of text simplicity as well as brevity and conciseness using the Hamburg intelligibility model) and c) a survey of subjects about the comprehensibility of individual technical terms, the assessment of the comprehensibility of the presentations and the subjects' decisions in favour of one of the 5 presented clinics due to the better quality of data. In addition, the correlation between the results of the text analysis with the results from the survey of subjects was tested. Results: The assessment of texts with the computer-aided evaluations showed poor comprehensibility values. The assessment of text simplicity using the Hamburg intelligibility model showed poor comprehensibility values (-0.3). On average, 6.8% of the words used were technical terms. A review of 10 technical terms revealed that in all cases only a minority of respondents (from 4.4% to 39.1%) exactly knew what was meant by each of them. Most subjects (62.4%) also believed that unclear terms worsened their understanding of the information offered. The correlation analysis showed that presentations with a lower frequency of technical terms and better values for the text simplicity were better understood. Conclusion: The determination of the frequency of technical terms and the assessment of text simplicity using the Hamburg intelligibility model were suitable methods to determine the readability and comprehensibility of presentations of quality indicators. The analysis showed predominantly poor comprehensibility values and indicated the need to improve the texts of report cards.

E-Patient Reputation in Health Forums.

Online health forums are increasingly used by patients to get information and help related to their health. However, information reliability in these forums is unfortunately not always guaranteed. Obviously, consequences of self-diagnosis may be severe on the patient's health if measures are taken without consulting a doctor. Many works on trust issues related to social media have been proposed, but most of them mainly focus only on the structure part of the social network (number of posts, number of likes, etc.). In the case of online health forums, a lot of trust and distrust is expressed inside the posted messages and cannot be inferred by only considering the structure. In this study, we rather suggest inferring the user's trustworthiness from the replies he receives in the forum. The proposed method is divided into three main steps: First, the recipient(s) of each post must be identified. Next, the trust or distrust expressed in these posts is evaluated. Finally, the user's reputation is computed by aggregating all the posts he received. Conducted experiments using a manually annotated corpus are encouraging.

Characterizing Health Information for Different Target Audiences.

Different groups of audiences in health care: health professionals and health consumers, each have different information needs. Health monographs targeting different audiences are created by leveraging readers' background knowledge. The NCI's Physician Data Query (PDQ®) Cancer Information Summaries provide parallel cancer information and education resources with different target audiences. In this paper, we used targeted audience-specific cancer information PDQs to measure characteristic differences on the element level between audiences. In addition, we compared vocabulary coverage. Results show a significant difference between the professional and patient version of cancer monographs in both content organization and vocabulary. This study provides a new view to assess targeted audience-specific health information, and helps editors to improve the quality and readability of health information.

[Informative Texts for Rehabilitation Patients are Difficult to Understand]. / Informationstexte für Rehabilitanden sind schwer verständlich.

OBJECTIVE: To check the intelligibility of informational texts written for rehabilitation patients. METHOD: Most investigated texts came from rehabilitation providers, patient organizations and health portals (8 texts), and from 50 rehabilitation facilities for musculoskeletal conditions (145 texts). We added a self-written booklet in plain language. All texts dealt with the goals and substance of rehabilitation. Readability formulas were computed for each text, and the Hamburg approach to intelligibility was applied. RESULTS: Only highly educated people will find the documents easily to understand, except for our self-written booklet, which is also comprehensible to lower secondary school leavers. CONCLUSION: The informational texts are of little help for people who are undecided whether or not to participate in rehabilitative measures and for rehab patients in preparatory stages. Issuing organizations are overburdened with the task of wording in a comprehensible manner, and should consider seeking assistance by professional writers or providing training for their staff.

Expertizer: a tool to assess the expert level of online health websites.

Health-related Web sites have become a primary resource to search for information on diseases, diagnoses or treatment options. Various Web sites offer a great variety of such information. However, lay people might have difficulties to assess whether a certain article or Web site fits their individual level of understandability. Hence, they might get overwhelmed with the delivered complexity of medical information. In this paper, we present a Web browser plugin, Expertizer that supports users in order to easily assess the expert level of textual medical Web content. The plugin communicates with a Web service, which leverages pre-computed classification models based on a Support Vector Machine.

General vs health specialized search engine: a blind comparative evaluation of top search results.

This paper presents the results of a blind comparison of top ten search results retrieved by Google.ch (French) and Khresmoi for everyone, a health specialized search engine. Participants--students of the Faculty of Medicine of the University of Geneva had to complete three tasks and select their preferred results. The majority of the participants have largely preferred Google results while Khresmoi results showed potential to compete in specific topics. The coverage of the results seems to be one of the reasons. The second being that participants do not know how to select quality and transparent health web pages. More awareness, tools and education about the matter is required for the students of Medicine to be able to efficiently distinguish trustworthy online health information.

Monitoring food safety violation reports from internet forums.

Food-borne illness is a growing public health concern in the world. Government bodies, which regulate and monitor the state of food safety, solicit citizen feedback about food hygiene practices followed by food establishments. They use traditional channels like call center, e-mail for such feedback collection. With the growing popularity of Web 2.0 and social media, citizens often post such feedback on internet forums, message boards etc. The system proposed in this paper applies text mining techniques to identify and mine such food safety complaints posted by citizens on web data sources thereby enabling the government agencies to gather more information about the state of food safety. In this paper, we discuss the architecture of our system and the text mining methods used. We also present results which demonstrate the effectiveness of this system in a real-world deployment.

Automatically Assessing the Expert Degree of Online Health Content using SVMs.

More and more people search for health information regarding diseases, diagnoses and treatments over the Web. However, lay people often have difficulties in assessing the understandability of related articles. Therefore, they could benefit from a system, which computes the medical expert degree of a corresponding piece of text in advance. In this paper we present an approach to automatically compute this expert degree using a machine learning approach. For evaluation purposes we constructed a large text corpus and tested our trained text classifier, which is based on Support Vector Machines.

App-synopsis: self-assessment on trust or distrust of health-apps.

Mobile devices and the apps running on them have become an integral part of daily life and this also includes the areas of medicine and health. Target audiences include those interested in their health, patients, and professional users. Nevertheless, the sheer number of available apps makes it hard for users to decide whether an app they are interested in is trustable or not. Nevertheless, such a decision should be based on a careful analysis of the available information. If left to their own devices, especially casual users may tend to forget certain aspects when performing such an evaluation. In order to provide them with a guideline on how to perform their evaluations, we propose a checklist covering the most important aspects for deciding whether an app can be entrusted with information that - especially when it comes to personal, health related information - is usually of a highly sensitive nature.

[Are German information materials on colorectal cancer screening understandable or do they fail? Rating of health information by users with different educational backgrounds]. / Sind Informationsmaterialien zur Darmkrebsfrüherkennung in Deutschland verständlich oder verfehlen sie ihre Wirkung? : Bewertung von Gesundheitsinformationen durch Nutzerinnen und Nutzer mit unterschiedlichem Bildungshintergrund.

Evidence-based patient information is an essential part of decision making in health issues. A qualitative study was conducted to examine whether consumers consider the health information on colorectal cancer screening in Germany helpful in supporting their decision making. In this study, eight texts from different German authors about colorectal cancer screening were evaluated with a criteria-based selection. The texts were tested for understandability, structure, design, and effect on readers. Thirteen semi-structured focus groups were formed including 59 higher-educated and 15 lower-educated testers, 46 % of them being male with a mean age of 62.9 years. The transcripts were analyzed by content-analytic assessment. The testers provided detailed comments on the content and design of the texts. They revealed inaccuracies and suggested improvements of the texts. The testers differed from each other in terms of the intensity of their participation in the discussions and comprehension of the material. The reception of the flyers and brochures varied according to the educational level of the testers. Lower-educated testers often skipped passages of texts about risks as well as passages containing numbers and graphics. The texts had different effects on the testers, regardless of the level of education. If adverse effects were presented, some testers became scared while others were reassured because they felt informed. Most of the testers appreciated a call for participation as a central message of the text. Sometimes, if there was no clear appeal to attend the screening, the testers even asked for it. The recently introduced political strategy of promoting an informed choice is applied to an unprepared population who was used to getting a clear yes-or-no recommendation. Thus, at first, the population has to learn how to make an informed choice. This topic must be addressed and taken into account in the process of developing health information. Since this is not always the case, we conclude that current materials are limited in their contribution toward the decision of attending CRC screening.

Engaging patients through your website.

Legislation requires the healthcare industry to directly engage patients through technology. This paper proposes a model that can be used to review hospital websites for features that engage patients in their healthcare. The model describes four levels of patient engagement in website design. The sample consisted of 130 hospital websites from hospitals listed on 2010 and 2011 Most Wired Hospitals. Hospital websites were analyzed for features that encouraged patient interaction with their healthcare according to the levels in the model. Of the four levels identified in the model, websites ranged from "informing" to "collaborative" in website design. There was great variation of features offered on hospital websites with few being engaging and interactive.

Systematic review of the types of methods and approaches used to assess the effectiveness of healthcare information websites.

The aim of this systematic review was to identify types of approaches and methods used to evaluate the effectiveness of healthcare information websites. Simple usage data may not be sufficient to assess whether desired healthcare outcomes were achieved or to determine the relative effectiveness of different web resources on the same health topic. To establish the state of the knowledge base on assessment methods used to determine the effectiveness of healthcare websites, a structured search of the literature was conducted in Ovid Medline, resulting in the retrieval of 1611 articles, of which 240 met the inclusion criteria for the present review. The present review found that diverse evaluation methods were used to measure the effectiveness of healthcare websites. These evaluation methods were used during development, before release and after release. Economic assessment was rare and most evaluations looked at content issues, such as readability scores. Several studies did try to assess the usefulness of websites, but few studies looked at behaviour change or knowledge transfer following engagement with the designated health website. To assess the effectiveness of the knowledge transfer of healthcare information through the online environment, multiple methods may need to be used to evaluate healthcare websites and may need to be undertaken at all stages of the website development process.

Identifying unproven cancer treatments on the health web: addressing accuracy, generalizability and scalability.

Building machine learning models that identify unproven cancer treatments on the Health Web is a promising approach for dealing with the dissemination of false and dangerous information to vulnerable health consumers. Aside from the obvious requirement of accuracy, two issues are of practical importance in deploying these models in real world applications. (a) Generalizability: The models must generalize to all treatments (not just the ones used in the training of the models). (b) Scalability: The models can be applied efficiently to billions of documents on the Health Web. First, we provide methods and related empirical data demonstrating strong accuracy and generalizability. Second, by combining the MapReduce distributed architecture and high dimensionality compression via Markov Boundary feature selection, we show how to scale the application of the models to WWW-scale corpora. The present work provides evidence that (a) a very small subset of unproven cancer treatments is sufficient to build a model to identify unproven treatments on the web; (b) unproven treatments use distinct language to market their claims and this language is learnable; (c) through distributed parallelization and state of the art feature selection, it is possible to prepare the corpora and build and apply models with large scalability.