Readability of English, German, and Russian Disease-Related Wikipedia Pages: Automated Computational Analysis.

BACKGROUND: Wikipedia is a popular encyclopedia for health- and disease-related information in which patients seek advice and guidance on the web. Yet, Wikipedia articles can be unsuitable as patient education materials, as investigated in previous studies that analyzed specific diseases or medical topics with a comparatively small sample size. Currently, no data are available on the average readability levels of all disease-related Wikipedia pages for the different localizations of this particular encyclopedia. OBJECTIVE: This study aimed to analyze disease-related Wikipedia pages written in English, German, and Russian using well-established readability metrics for each language. METHODS: Wikipedia database snapshots and Wikidata metadata were chosen as resources for data collection. Disease-related articles were retrieved separately for English, German, and Russian starting with the main concept of Human Diseases and Disorders (German: Krankheit; Russian: Заболевания человека). In the case of existence, the corresponding International Classification of Diseases, Tenth Revision (ICD-10), codes were retrieved for each article. Next, the raw texts were extracted and readability metrics were computed. RESULTS: The number of articles included in this study for English, German, and Russian Wikipedia was n=6127, n=6024, and n=3314, respectively. Most disease-related articles had a Flesch Reading Ease (FRE) score <50.00, signaling difficult or very difficult educational material (English: 5937/6125, 96.93%; German: 6004/6022, 99.7%; Russian: 2647/3313, 79.9%). In total, 70% (7/10) of the analyzed articles could be assigned an ICD-10 code with certainty (English: 4235/6127, 69.12%; German: 4625/6024, 76.78%; Russian: 2316/3314, 69.89%). For articles with ICD-10 codes, the mean FRE scores were 28.69 (SD 11.00), 20.33 (SD 9.98), and 38.54 (SD 13.51) for English, German, and Russian, respectively. A total of 9 English ICD-10 chapters (11 German and 10 Russian) showed significant differences: chapter F (FRE 23.88, SD 9.95; P<.001), chapter E (FRE 25.14, SD 9.88; P<.001), chapter H (FRE 30.04, SD 10.57; P=.049), chapter I (FRE 30.05, SD 9.07; P=.04), chapter M (FRE 31.17, 11.94; P<.001), chapter T (FRE 32.06, SD 10.51; P=.001), chapter A (FRE 32.63, SD 9.25; P<.001), chapter B (FRE 33.24, SD 9.07; P<.001), and chapter S (FRE 39.02, SD 8.22; P<.001). CONCLUSIONS: Disease-related English, German, and Russian Wikipedia articles cannot be recommended as patient education materials because a major fraction is difficult or very difficult to read. The authors of Wikipedia pages should carefully revise existing text materials for readers with a specific interest in a disease or its associated symptoms. Special attention should be given to articles on mental, behavioral, and neurodevelopmental disorders (ICD-10 chapter F) because these articles were most difficult to read in comparison with other ICD-10 chapters. Wikipedia readers should be supported by editors providing a short and easy-to-read summary for each article.

Temporal and Location Variations, and Link Categories for the Dissemination of COVID-19-Related Information on Twitter During the SARS-CoV-2 Outbreak in Europe: Infoveillance Study.

BACKGROUND: The spread of the 2019 novel coronavirus disease, COVID-19, across Asia and Europe sparked a significant increase in public interest and media coverage, including on social media platforms such as Twitter. In this context, the origin of information plays a central role in the dissemination of evidence-based information about the SARS-CoV-2 virus and COVID-19. On February 2, 2020, the World Health Organization (WHO) constituted a "massive infodemic" and argued that this situation "makes it hard for people to find trustworthy sources and reliable guidance when they need it." OBJECTIVE: This infoveillance study, conducted during the early phase of the COVID-19 pandemic, focuses on the social media platform Twitter. It allows monitoring of the dynamic pandemic situation on a global scale for different aspects and topics, languages, as well as regions and even whole countries. Of particular interest are temporal and geographical variations of COVID-19-related tweets, the situation in Europe, and the categories and origin of shared external resources. METHODS: Twitter's Streaming application programming interface was used to filter tweets based on 16 prevalent hashtags related to the COVID-19 outbreak. Each tweet's text and corresponding metadata as well as the user's profile information were extracted and stored into a database. Metadata included links to external resources. A link categorization scheme-introduced in a study by Chew and Eysenbach in 2009-was applied onto the top 250 shared resources to analyze the relative proportion for each category. Moreover, temporal variations of global tweet volumes were analyzed and a specific analysis was conducted for the European region. RESULTS: Between February 9 and April 11, 2020, a total of 21,755,802 distinct tweets were collected, posted by 4,809,842 distinct Twitter accounts. The volume of #covid19-related tweets increased after the WHO announced the name of the new disease on February 11, 2020, and stabilized at the end of March at a high level. For the regional analysis, a higher tweet volume was observed in the vicinity of major European capitals or in densely populated areas. The most frequently shared resources originated from various social media platforms (ranks 1-7). The most prevalent category in the top 50 was "Mainstream or Local News." For the category "Government or Public Health," only two information sources were found in the top 50: US Centers for Disease Control and Prevention at rank 25 and the WHO at rank 27. The first occurrence of a prevalent scientific source was Nature (rank 116). CONCLUSIONS: The naming of the disease by the WHO was a major signal to address the public audience with public health response via social media platforms such as Twitter. Future studies should focus on the origin and trustworthiness of shared resources, as monitoring the spread of fake news during a pandemic situation is of particular importance. In addition, it would be beneficial to analyze and uncover bot networks spreading COVID-19-related misinformation.

Crawling the German Health Web: Exploratory Study and Graph Analysis.

BACKGROUND: The internet has become an increasingly important resource for health information. However, with a growing amount of web pages, it is nearly impossible for humans to manually keep track of evolving and continuously changing content in the health domain. To better understand the nature of all web-based health information as given in a specific language, it is important to identify (1) information hubs for the health domain, (2) content providers of high prestige, and (3) important topics and trends in the health-related web. In this context, an automatic web crawling approach can provide the necessary data for a computational and statistical analysis to answer (1) to (3). OBJECTIVE: This study demonstrates the suitability of a focused crawler for the acquisition of the German Health Web (GHW) which includes all health-related web content of the three mostly German speaking countries Germany, Austria and Switzerland. Based on the gathered data, we provide a preliminary analysis of the GHW's graph structure covering its size, most important content providers and a ratio of public to private stakeholders. In addition, we provide our experiences in building and operating such a highly scalable crawler. METHODS: A support vector machine classifier was trained on a large data set acquired from various German content providers to distinguish between health-related and non-health-related web pages. The classifier was evaluated using accuracy, recall and precision on an 80/20 training/test split (TD1) and against a crowd-validated data set (TD2). To implement the crawler, we extended the open-source framework StormCrawler. The actual crawl was conducted for 227 days. The crawler was evaluated by using harvest rate and its recall was estimated using a seed-target approach. RESULTS: In total, n=22,405 seed URLs with country-code top level domains .de: 85.36% (19,126/22,405), .at: 6.83% (1530/22,405), .ch: 7.81% (1749/22,405), were collected from Curlie and a previous crawl. The text classifier achieved an accuracy on TD1 of 0.937 (TD2=0.966), a precision on TD1 of 0.934 (TD2=0.954) and a recall on TD1 of 0.944 (TD2=0.989). The crawl yields 13.5 million presumably relevant and 119.5 million nonrelevant web pages. The average harvest rate was 19.76%; recall was 0.821 (4105/5000 targets found). The resulting host-aggregated graph contains 215,372 nodes and 403,175 edges (network diameter=25; average path length=6.466; average degree=1.872; average in-degree=1.892; average out-degree=1.845; modularity=0.723). Among the 25 top-ranked pages for each country (according to PageRank), 40% (30/75) were web sites published by public institutions. 25% (19/75) were published by nonprofit organizations and 35% (26/75) by private organizations or individuals. CONCLUSIONS: The results indicate, that the presented crawler is a suitable method for acquiring a large fraction of the GHW. As desired, the computed statistical data allows for determining major information hubs and important content providers on the GHW. In the future, the acquired data may be used to assess important topics and trends but also to build health-specific search engines.

Computer-Based Readability Testing of Information Booklets for German Cancer Patients.

Understandable health information is essential for treatment adherence and improved health outcomes. For readability testing, several instruments analyze the complexity of sentence structures, e.g., Flesch-Reading Ease (FRE) or Vienna-Formula (WSTF). Moreover, the vocabulary is of high relevance for readers. The aim of this study is to investigate the agreement of sentence structure and vocabulary-based (SVM) instruments. A total of 52 freely available German patient information booklets on cancer were collected from the Internet. The mean understandability level L was computed for 51 booklets. The resulting values of FRE, WSTF, and SVM were assessed pairwise for agreement with Bland-Altman plots and two-sided, paired t tests. For the pairwise comparison, the mean L values are LFRE = 6.81, LWSTF = 7.39, LSVM = 5.09. The sentence structure-based metrics gave significantly different scores (P < 0.001) for all assessed booklets, confirmed by the Bland-Altman analysis. The study findings suggest that vocabulary-based instruments cannot be interchanged with FRE/WSTF. However, both analytical aspects should be considered and checked by authors to linguistically refine texts with respect to the individual target group. Authors of health information can be supported by automated readability analysis. Health professionals can benefit by direct booklet comparisons allowing for time-effective selection of suitable booklets for patients.

Understandability of Patient Information Booklets for Patients with Cancer.

The improvement of health literacy in general and the information of individual patient is a major concern of the German national cancer plan and similar initiatives in other western countries. The aim of our study was to assess the readability and understandability of information booklets for cancer patients available at German Web sites. A support vector machine (SVM) was used to discriminate between laymen- and expert-centric patient information booklets about nine most common tumor types. All booklets had to be available for free at the Internet. A total of 52 different patient booklets were downloaded and assessed. Overall, the assessment of all booklets showed that an understandability level L of 4.6 and therefore increased medical background knowledge is required to understand a random text selected from the sample. The assessed information booklets on cancer show very limited suitability for laymen. We were able to demonstrate that a medical background is necessary to understand the examined booklets. The current study highlights the need to create information material adjusted to the needs of laymen. Assessing understandability before publication, especially for laymen with low health literacy, could ensure the suitability and thus quality of the information material.

Readability and topics of the German Health Web: Exploratory study and text analysis.

BACKGROUND: The internet has become an increasingly important resource for health information, especially for lay people. However, the information found does not necessarily comply with the user's health literacy level. Therefore, it is vital to (1) identify prominent information providers, (2) quantify the readability of written health information, and (3) to analyze how different types of information sources are suited for people with differing health literacy levels. OBJECTIVE: In previous work, we showed the use of a focused crawler to "capture" and describe a large sample of the "German Health Web", which we call the "Sampled German Health Web" (sGHW). It includes health-related web content of the three mostly German speaking countries Germany, Austria, and Switzerland, i.e. country-code top-level domains (ccTLDs) ".de", ".at" and ".ch". Based on the crawled data, we now provide a fully automated readability and vocabulary analysis of a subsample of the sGHW, an analysis of the sGHW's graph structure covering its size, its content providers and a ratio of public to private stakeholders. In addition, we apply Latent Dirichlet Allocation (LDA) to identify topics and themes within the sGHW. METHODS: Important web sites were identified by applying PageRank on the sGHW's graph representation. LDA was used to discover topics within the top-ranked web sites. Next, a computer-based readability and vocabulary analysis was performed on each health-related web page. Flesch Reading Ease (FRE) and the 4th Vienna formula (WSTF) were used to assess the readability. Vocabulary was assessed by a specifically trained Support Vector Machine classifier. RESULTS: In total, n = 14,193,743 health-related web pages were collected during the study period of 370 days. The resulting host-aggregated web graph comprises 231,733 nodes connected via 429,530 edges (network diameter = 25; average path length = 6.804; average degree = 1.854; modularity = 0.723). Among 3000 top-ranked pages (1000 per ccTLD according to PageRank), 18.50%(555/3000) belong to web sites from governmental or public institutions, 18.03% (541/3000) from nonprofit organizations, 54.03% (1621/3000) from private organizations, 4.07% (122/3000) from news agencies, 3.87% (116/3000) from pharmaceutical companies, 0.90% (27/3000) from private bloggers, and 0.60% (18/3000) are from others. LDA identified 50 topics, which we grouped into 11 themes: "Research & Science", "Illness & Injury", "The State", "Healthcare structures", "Diet & Food", "Medical Specialities", "Economy", "Food production", "Health communication", "Family" and "Other". The most prevalent themes were "Research & Science" and "Illness & Injury" accounting for 21.04% and 17.92% of all topics across all ccTLDs and provider types, respectively. Our readability analysis reveals that the majority of the collected web sites is structurally difficult or very difficult to read: 84.63% (2539/3000) scored a WSTF ≥ 12, 89.70% (2691/3000) scored a FRE ≤ 49. Moreover, our vocabulary analysis shows that 44.00% (1320/3000) web sites use vocabulary that is well suited for a lay audience. CONCLUSIONS: We were able to identify major information hubs as well as topics and themes within the sGHW. Results indicate that the readability within the sGHW is low. As a consequence, patients may face barriers, even though the vocabulary used seems appropriate from a medical perspective. In future work, the authors intend to extend their analyses to identify trustworthy health information web sites.

The Security State of the German Health Web: An Exploratory Study.

The internet has become an important resource for health information and for interactions with healthcare providers. However, information of all types can go through many servers and networks before reaching its intended destination and any of these has the potential to intercept or even manipulate the exchanged information if data's transfer is not adequately protected. As trust is a fundamental concept in healthcare relationships, it is crucial to offer a secure medical website to maintain the same level of trust as provided in a face-to-face meeting. This study provides a first analysis of the SSL/TLS security of and the security headers used within the health-related web limited to web pages in German, the German health web (GHW). METHODS: testssl.sh and TLS-Scanner were used to analyze the URLs of the 1,000 top-ranked health-related web sites (according to PageRank) for each of the country- code top level domains: ".de", ".at" and ".ch". RESULTS: Our study revealed that most websites in the GHW are potentially vulnerable to common SSL/TLS security vulnerabilities, offer deprecated SSL/TLS protocol versions and mostly do not implement HTTP security headers at all. CONCLUSIONS: These findings question the concept of trust within the GHW. Website owners should reconsider the use of outdated SSL/TLS protocol versions for compatibility reasons. Additionally, HTTP security headers should be implemented more consequently to provide additional security aspects. In future work, the authors intend to repeat this study and to incorporate a website's category, i.e. governmental or public health, to get a more detailed view of the GHW's security.

PrositNG - A Machine Learning Supported Disease Model Generation Software.

Decision models (DM), especially Markov Models, play an essential role in the economic evaluation of new medical interventions. The process of DM generation requires expert knowledge of the medical domain and is a time-consuming task. Therefore, the authors propose a new model generation software PrositNG that is connectable to database systems of real-world routine care data. The structure of the model is derived from the entries in a database system by the help of Machine Learning algorithms. The software was implemented with the programming language Java. Two data sources were successfully utilized to demonstrate the value of PrositNG. However, a good understanding of the local documentation routine and software is paramount to use real-world data for model generation.

Analyzing the Readability of Health Information Booklets on Cardiovascular Diseases.

A low level of patient health literacy represents a major reason for worse prognosis or reduced therapy adherence. Health information booklets are a major tool for improving patient's health literacy. This paper presents a computer-based readability analysis of patient information booklets from the cardiovascular domain. The study relies on 34 English booklets mostly on heart disease, prevention and procedures. It compares five different, well-established readability instruments. On average, readers of the assessed booklets have to visit school at least until the 9th U.S. school grade when applying the Flesch-Kincaid formula. According to the Gunning-Fog metric, readers would have to attend school until the 11th grade. The presented study demonstrates the feasibility of a fully automated text processing tool-chain for patient information booklets. The results reveal that readability metrics should be carefully interpreted and only be interchanged with caution.

Technology Adoption, Motivational Aspects, and Privacy Concerns of Wearables in the German Running Community: Field Study.

BACKGROUND: Despite the availability of a great variety of consumer-oriented wearable devices, perceived usefulness, user satisfaction, and privacy concerns have not been fully investigated in the field of wearable applications. It is not clear why healthy, active citizens equip themselves with wearable technology for running activities, and what privacy and data sharing features might influence their individual decisions. OBJECTIVE: The primary aim of the study was to shed light on motivational and privacy aspects of wearable technology used by healthy, active citizens. A secondary aim was to reevaluate smart technology adoption within the running community in Germany in 2017 and to compare it with the results of other studies and our own study from 2016. METHODS: A questionnaire was designed to assess what wearable technology is used by runners of different ages and sex. Data on motivational factors were also collected. The survey was conducted at a regional road race event in May 2017, paperless via a self-implemented app. The demographic parameters of the sample cohort were compared with the event's official starter list. In addition, the validation included comparison with demographic parameters of the largest German running events in Berlin, Hamburg, and Frankfurt/Main. Binary logistic regression analysis was used to investigate whether age, sex, or course distance were associated with device use. The same method was applied to analyze whether a runner's age was predictive of privacy concerns, openness to voluntary data sharing, and level of trust in one's own body for runners not using wearables (ie, technological assistance considered unnecessary in this group). RESULTS: A total of 845 questionnaires were collected. Use of technology for activity monitoring during events or training was prevalent (73.0%, 617/845) in this group. Male long-distance runners and runners in younger age groups (30-39 years: odds ratio [OR] 2.357, 95% CI 1.378-4.115; 40-49 years: OR 1.485, 95% CI 0.920-2.403) were more likely to use tracking devices, with ages 16 to 29 years as the reference group (OR 1). Where wearable technology was used, 42.0% (259/617) stated that they were not concerned if data might be shared by a device vendor without their consent. By contrast, 35.0% (216/617) of the participants would not accept this. In the case of voluntary sharing, runners preferred to exchange tracked data with friends (51.7%, 319/617), family members (43.4%, 268/617), or a physician (32.3%, 199/617). A large proportion (68.0%, 155/228) of runners not using technology stated that they preferred to trust what their own body was telling them rather than trust a device or an app (50-59 years: P<.001; 60-69 years: P=.008). CONCLUSIONS: A total of 136 distinct devices by 23 vendors or manufacturers and 17 running apps were identified. Out of 4, 3 runners (76.8%, 474/617) always trusted in the data tracked by their personal device. Data privacy concerns do, however, exist in the German running community, especially for older age groups (30-39 years: OR 1.041, 95% CI 0.371-0.905; 40-49 years: OR 1.421, 95% CI 0.813-2.506; 50-59 years: OR 2.076, 95% CI 1.813-3.686; 60-69 years: OR 2.394, 95% CI 0.957-6.183).

Accuracy and Adoption of Wearable Technology Used by Active Citizens: A Marathon Event Field Study.

BACKGROUND: Today, runners use wearable technology such as global positioning system (GPS)-enabled sport watches to track and optimize their training activities, for example, when participating in a road race event. For this purpose, an increasing amount of low-priced, consumer-oriented wearable devices are available. However, the variety of such devices is overwhelming. It is unclear which devices are used by active, healthy citizens and whether they can provide accurate tracking results in a diverse study population. No published literature has yet assessed the dissemination of wearable technology in such a cohort and related influencing factors. OBJECTIVE: The aim of this study was 2-fold: (1) to determine the adoption of wearable technology by runners, especially "smart" devices and (2) to investigate on the accuracy of tracked distances as recorded by such devices. METHODS: A pre-race survey was applied to assess which wearable technology was predominantly used by runners of different age, sex, and fitness level. A post-race survey was conducted to determine the accuracy of the devices that tracked the running course. Logistic regression analysis was used to investigate whether age, sex, fitness level, or track distance were influencing factors. Recorded distances of different device categories were tested with a 2-sample t test against each other. RESULTS: A total of 898 pre-race and 262 post-race surveys were completed. Most of the participants (approximately 75%) used wearable technology for training optimization and distance recording. Females (P=.02) and runners in higher age groups (50-59 years: P=.03; 60-69 years: P<.001; 70-79 year: P=.004) were less likely to use wearables. The mean of the track distances recorded by mobile phones with combined app (mean absolute error, MAE=0.35 km) and GPS-enabled sport watches (MAE=0.12 km) was significantly different (P=.002) for the half-marathon event. CONCLUSIONS: A great variety of vendors (n=36) and devices (n=156) were identified. Under real-world conditions, GPS-enabled devices, especially sport watches and mobile phones, were found to be accurate in terms of recorded course distances.

Expertizer: a tool to assess the expert level of online health websites.

Health-related Web sites have become a primary resource to search for information on diseases, diagnoses or treatment options. Various Web sites offer a great variety of such information. However, lay people might have difficulties to assess whether a certain article or Web site fits their individual level of understandability. Hence, they might get overwhelmed with the delivered complexity of medical information. In this paper, we present a Web browser plugin, Expertizer that supports users in order to easily assess the expert level of textual medical Web content. The plugin communicates with a Web service, which leverages pre-computed classification models based on a Support Vector Machine.

Proactive Support of Internet Browsing when Searching for Relevant Health Information.

Many people use the Internet as one of the primary sources of health information. This is due to the high volume and easy access of freely available information regarding diseases, diagnoses and treatments. However, users may find it difficult to retrieve information which is easily understandable and does not require a deep medical background. In this paper, we present a new kind of Web browser add-on, in order to proactively support users when searching for relevant health information. Our add-on not only visualizes the understandability of displayed medical text but also provides further recommendations of Web pages which hold similar content but are potentially easier to comprehend.

[Patient guidelines in oncology: objectives, procedures and first experiences with this format]. / Patientenleitlinien in der Onkologie: Zielsetzung, Vorgehen und erste Erfahrungen mit dem Format.

For several years patient versions of guidelines have become mandatory in the German Guidelines Program in Oncology (GGPO). Based on the methodology that has been developed for the German National Disease Management Guidelines Program, patient versions of guidelines translate the recommendations of clinical practice guideline into plain language and provide information about the harms and benefits of the interventions being addressed in the guideline. They are developed by a group of guideline authors (experts as well as patients), they are consensus-based and aim to create transparency in recommendations for physicians and their rationales. An automated analysis of readability shows that patient versions of guidelines are specific to the target group of educated lay people. Moreover, the responses to a reader feedback questionnaire indicate that comprehensibility, level of detail and depth of information are considered highly relevant and positive by users. Thus, patient versions of guidelines meet the needs of a specific target group. Nevertheless, the development of other formats for readers with low levels of health literacy or cognitive competencies is desirable. Currently it remains unclear if these simplified formats are able to reflect the complexity of high quality clinical practice guidelines.

Automatically Assessing the Expert Degree of Online Health Content using SVMs.

More and more people search for health information regarding diseases, diagnoses and treatments over the Web. However, lay people often have difficulties in assessing the understandability of related articles. Therefore, they could benefit from a system, which computes the medical expert degree of a corresponding piece of text in advance. In this paper we present an approach to automatically compute this expert degree using a machine learning approach. For evaluation purposes we constructed a large text corpus and tested our trained text classifier, which is based on Support Vector Machines.