Mining user reviews of COVID contact-tracing apps: An exploratory analysis of nine European apps.

CONTEXT: More than 78 countries have developed COVID contact-tracing apps to limit the spread of coronavirus. However, many experts and scientists cast doubt on the effectiveness of those apps. For each app, a large number of reviews have been entered by end-users in app stores. OBJECTIVE: Our goal is to gain insights into the user reviews of those apps, and to find out the main problems that users have reported. Our focus is to assess the "software in society" aspects of the apps, based on user reviews. METHOD: We selected nine European national apps for our analysis and used a commercial app-review analytics tool to extract and mine the user reviews. For all the apps combined, our dataset includes 39,425 user reviews. RESULTS: Results show that users are generally dissatisfied with the nine apps under study, except the Scottish ("Protect Scotland") app. Some of the major issues that users have complained about are high battery drainage and doubts on whether apps are really working. CONCLUSION: Our results show that more work is needed by the stakeholders behind the apps (e.g., app developers, decision-makers, public health experts) to improve the public adoption, software quality and public perception of these apps.

A large scale analysis of mHealth app user reviews.

The global mHealth app market is rapidly expanding, especially since the COVID-19 pandemic. However, many of these mHealth apps have serious issues, as reported in their user reviews. Better understanding their key user concerns would help app developers improve their apps' quality and uptake. While app reviews have been used to study user feedback in many prior studies, many are limited in scope, size and/or analysis. In this paper, we introduce a very large-scale study and analysis of mHealth app reviews. We extracted and translated over 5 million user reviews for 278 mHealth apps. These reviews were then classified into 14 different aspects/categories of issues reported. Several mHealth app subcategories were examined to reveal differences in significant areas of user concerns, and to investigate the impact of different aspects of mhealth apps on their ratings. Based on our findings, women's health apps had the highest satisfaction ratings. Fitness activity tracking apps received the lowest and most unfavourable ratings from users. Over half of users who reported troubles leading them to uninstall mHealth apps gave a 1-star rating. Half of users gave the account and logging aspect only one star due to faults and issues encountered while registering or logging in. Over a third of users who expressed privacy concerns gave the app a 1-star rating. However, only 6% of users gave apps a one-star rating due to UI/UX concerns. 20% of users reported issues with handling of user requests and internationalisation concerns. We validated our findings by manually analysing a sample of 1,000 user reviews from each investigated aspect/category. We developed a list of recommendations for mHealth apps developers based on our user review analysis.

User Perspectives of Diet-Tracking Apps: Reviews Content Analysis and Topic Modeling.

BACKGROUND: The availability and use of mobile apps in health and nutrition management are increasing. Ease of access and user friendliness make diet-tracking apps an important ally in their users' efforts to lose and manage weight. To foster motivation for long-term use and to achieve goals, it is necessary to better understand users' opinions and needs for dietary self-monitoring. OBJECTIVE: The aim of this study was to identify the key topics and issues that users highlight in their reviews of diet-tracking apps on Google Play Store. Identifying the topics that users frequently mention in their reviews of these apps, along with the user ratings for each of these apps, allowed us to identify areas where further improvement of the apps could facilitate app use, and support users' weight loss and intake management efforts. METHODS: We collected 72,084 user reviews from Google Play Store for 15 diet-tracking apps that allow users to track and count calories. After a series of text processing operations, two text-mining techniques (topic modeling and topical n-grams) were applied to the corpus of user reviews of diet-tracking apps. RESULTS: Using the topic modeling technique, 11 separate topics were extracted from the pool of user reviews. Most of the users providing feedback were generally satisfied with the apps they use (average rating of 4.4 out of 5 for the 15 apps). Most topics referred to the positive evaluation of the apps and their functions. Negatively rated topics mostly referred to app charges and technical difficulties encountered. We identified the positive and negative topic trigrams (3-word combinations) among the most frequently mentioned topics. Usability and functionality (tracking options) of apps were rated positively on average. Negative ratings were associated with trigrams related to adding new foods, technical issues, and app charges. CONCLUSIONS: Motivating users to use an app over time could help them better achieve their nutrition goals. Although user reviews generally showed positive opinions and ratings of the apps, developers should pay more attention to users' technical problems and inform users about expected payments, along with their refund and cancellation policies, to increase user loyalty.

Quality Assessment of Health-apps using a Public Agency Quality and Safety Seal. The Appsaludable Case.

Standards and validation practices regarding mobile health apps need to be established to ensure their proper use and integration into medical practice. This paper proposes an innovative and integrative approach to examine and compare the quality of a certified health mobile applications set. A double classification framework (verified and perceived quality) is proposed and validated to evaluate the quality of 22 verified health mobile apps, particularly to understand whether these apps provide the service demanded by its users. Evaluation of verified quality was based on a certification public program. To analysis user satisfaction, we used content analysis to examine and extract words and expressions contained in online reviews (1,574 reviews were analyzed). The R language was used to scrape data and to perform all statistical analyses. The results did not confirm a significant relationship between both classifications (p > .05). However, the requirements with the highest degree of compliance ("design and relevance" and "quality and safety") of the app accreditation program were the best rated by the users of the apps in their comments. Reflections derived from this paper represent an important contribution to the knowledge base in the expanding research area of mobile health.

Medical concept normalization in social media posts with recurrent neural networks.

Text mining of scientific libraries and social media has already proven itself as a reliable tool for drug repurposing and hypothesis generation. The task of mapping a disease mention to a concept in a controlled vocabulary, typically to the standard thesaurus in the Unified Medical Language System (UMLS), is known as medical concept normalization. This task is challenging due to the differences in the use of medical terminology between health care professionals and social media texts coming from the lay public. To bridge this gap, we use sequence learning with recurrent neural networks and semantic representation of one- or multi-word expressions: we develop end-to-end architectures directly tailored to the task, including bidirectional Long Short-Term Memory, Gated Recurrent Units with an attention mechanism, and additional semantic similarity features based on UMLS. Our evaluation against a standard benchmark shows that recurrent neural networks improve results over an effective baseline for classification based on convolutional neural networks. A qualitative examination of mentions discovered in a dataset of user reviews collected from popular online health information platforms as well as a quantitative evaluation both show improvements in the semantic representation of health-related expressions in social media.

Python data odyssey: Mining user feedback from google play store.

Context: The Google Play Store is widely recognized as one of the largest platforms for downloading applications, both free and paid. On a daily basis, millions of users avail themselves of this marketplace, sharing their thoughts through various means such as star ratings, user comments, suggestions, and feedback. These insights, in the form of comments and feedback, constitute a valuable resource for organizations, competitors, and emerging companies seeking to expand their market presence. These comments provide insights into app deficiencies, suggestions for new features, identified issues, and potential enhancements. Unlocking the potential of this repository of suggestions holds significant value. Objective: This study sought to gather and analyze user reviews from the Google Play store for leading game apps. The primary aim was to construct a dataset for subsequent analysis utilizing requirements engineering, machine learning, and competitive assessment. Methodology: The authors employed a Python-based web scraping method to extract a comprehensive set of over 429,000+ reviews from the Google Play pages of selected apps. The scraped data encompassed reviewer names (removed due to privacy), ratings, and the textual content of the reviews. Results: The outcome was a dataset comprising the extracted user reviews, ratings, and associated metadata. A total of 429,000+ reviews were acquired through the scraping process for popular apps like Subway Surfers, Candy Crush Saga, PUBG Mobile, among others. This dataset not only serves as a valuable educational resource for instructors, aiding in the training of students in data analysis, but also offers practitioners the opportunity for in-depth examination and insights (in the past data of top apps).

Mining software insights: uncovering the frequently occurring issues in low-rating software applications.

In today's digital world, app stores have become an essential part of software distribution, providing customers with a wide range of applications and opportunities for software developers to showcase their work. This study elaborates on the importance of end-user feedback for software evolution. However, in the literature, more emphasis has been given to high-rating & popular software apps while ignoring comparatively low-rating apps. Therefore, the proposed approach focuses on end-user reviews collected from 64 low-rated apps representing 14 categories in the Amazon App Store. We critically analyze feedback from low-rating apps and developed a grounded theory to identify various concepts important for software evolution and improving its quality including user interface (UI) and user experience (UX), functionality and features, compatibility and device-specific, performance and stability, customer support and responsiveness and security and privacy issues. Then, using a grounded theory and content analysis approach, a novel research dataset is curated to evaluate the performance of baseline machine learning (ML), and state-of-the-art deep learning (DL) algorithms in automatically classifying end-user feedback into frequently occurring issues. Various natural language processing and feature engineering techniques are utilized for improving and optimizing the performance of ML and DL classifiers. Also, an experimental study comparing various ML and DL algorithms, including multinomial naive Bayes (MNB), logistic regression (LR), random forest (RF), multi-layer perception (MLP), k-nearest neighbors (KNN), AdaBoost, Voting, convolutional neural network (CNN), long short-term memory (LSTM), bidirectional long short term memory (BiLSTM), gated recurrent unit (GRU), bidirectional gated recurrent unit (BiGRU), and recurrent neural network (RNN) classifiers, achieved satisfactory results in classifying end-user feedback to commonly occurring issues. Whereas, MLP, RF, BiGRU, GRU, CNN, LSTM, and Classifiers achieved average accuracies of 94%, 94%, 92%, 91%, 90%, 89%, and 89%, respectively. We employed the SHAP approach to identify the critical features associated with each issue type to enhance the explainability of the classifiers. This research sheds light on areas needing improvement in low-rated apps and opens up new avenues for developers to improve software quality based on user feedback.

An experimental study on the performance of collaborative filtering based on user reviews for large-scale datasets.

Collaborative filtering (CF) approaches generate user recommendations based on user similarities. These similarities are calculated based on the overall (explicit) user ratings. However, in some domains, such ratings may be sparse or unavailable. User reviews can play a significant role in such cases, as implicit ratings can be derived from the reviews using sentiment analysis, a natural language processing technique. However, most current studies calculate the implicit ratings by simply aggregating the scores of all sentiment words appearing in reviews and, thus, ignoring the elements of sentiment degrees and aspects of user reviews. This study addresses this issue by calculating the implicit rating differently, leveraging the rich information in user reviews by using both sentiment words and aspect-sentiment word pairs to enhance the CF performance. It proposes four methods to calculate the implicit ratings on large-scale datasets: the first considers the degree of sentiment words, while the second exploits the aspects by extracting aspect-sentiment word pairs to calculate the implicit ratings. The remaining two methods combine explicit ratings with the implicit ratings generated by the first two methods. The generated ratings are then incorporated into different CF rating prediction algorithms to evaluate their effectiveness in enhancing the CF performance. Evaluative experiments of the proposed methods are conducted on two large-scale datasets: Amazon and Yelp. Results of the experiments show that the proposed ratings improved the accuracy of CF rating prediction algorithms and outperformed the explicit ratings in terms of three predictive accuracy metrics.

Review disagreements, cultural capital, and cultural discount on imported hollywood movies in China.

Within the context of mounting competition faced by Hollywood movies in the global cinema market, particularly in emerging markets, this paper examines the influence of review disagreements and cultural capital on the cultural discount of imported Hollywood blockbusters in China's movie market. Using a dataset of 652 movies from the U.S. spanning 1994-2019, we find that review disagreements between U.S. and Chinese consumers exacerbate the cultural discount on imported Hollywood movies in China. However, cultural capital, measured by the accumulated revenue of specific imported movie genres in China, effectively mitigates this discount. We also observe a reduction in the discount for movie genres with fewer language barriers. Accounting for endogeneity due to reverse causality and selection bias, we identify a significant structural break in 2012. In the post-2012 era, collaboration involving China's censorship, quota rules, and film-production laws has improved the market mechanism and cultural capital accumulation, enhancing the performance of imported Hollywood movies in China. Minority movies, characterized by being less mainstream and commercial, are more sensitive to review disagreements, while cultural capital plays a greater role in mitigating the discount for mainstream Hollywood movies. These findings have significant implications for professionals involved in the distribution and screening of Hollywood movies in China.

A Study of the Ping An Health App Based on User Reviews with Sentiment Analysis.

By mining the dimensional sentiment and dimension weight of the Ping An Health App reviews, this paper explores the changing trend of the influence of dimensions on user satisfaction and provides suggestions for the Ping An Health App operators to improve user satisfaction. Firstly, the topic model is used to identify the topic of user comments, and then the fine-grained sentiment analysis method is used to calculate the sentiment and weight of each dimension. Finally, the changing trend of the weight of each dimension and the changing trend of user satisfaction of each dimension are drawn. Based on the reviews of the Ping An Health App in the Apple App Store, users' concerns about Ping An Health App can be summarized into seven main dimensions: Usage, Bug report, Reliability, Feature information, Services, Other apps, and User Background. The "Feature information" dimension and "Reliability" dimension have a great impact on user satisfaction with the Ping An Health App, while the "Bug report" dimension has the lowest user satisfaction.

Evaluating User Feedback for an Artificial Intelligence-Enabled, Cognitive Behavioral Therapy-Based Mental Health App (Wysa): Qualitative Thematic Analysis.

BACKGROUND: Digital mental health apps are rapidly becoming a common source of accessible support across the world, but their effectiveness is often influenced by limited helpfulness and engagement. OBJECTIVE: This study's primary objective was to analyze feedback content to understand users' experiences with engaging with a digital mental health app. As a secondary objective, an exploratory analysis captured the types of mental health app users. METHODS: This study utilized a user-led approach to understanding factors for engagement and helpfulness in digital mental health by analyzing feedback (n=7929) reported on Google Play Store about Wysa, a mental health app (1-year period). The analysis of keywords in the user feedback categorized and evaluated the reported user experience into the core domains of acceptability, usability, usefulness, and integration. The study also captured key deficits and strengths of the app and explored salient characteristics of the types of users who benefit from accessible digital mental health support. RESULTS: The analysis of user feedback found the app to be overwhelmingly positively reviewed (6700/7929, 84.50% 5-star rating). The themes of engaging exercises, interactive interface, and artificial intelligence (AI) conversational ability indicated the acceptability of the app, while the nonjudgmentality and ease of conversation highlighted its usability. The app's usefulness was portrayed by themes such as improvement in mental health, convenient access, and cognitive restructuring exercises. Themes of privacy and confidentiality underscored users' preference for the integrated aspects of the app. Further analysis revealed 4 predominant types of individuals who shared app feedback on the store. CONCLUSIONS: Users reported therapeutic elements of a comfortable, safe, and supportive environment through using the digital mental health app. Digital mental health apps may expand mental health access to those unable to access traditional forms of mental health support and treatments.

Do Critical Reviews Affect Box Office Revenues Through Community Engagement and User Reviews?

With advances in technology and the popularity of the Internet, consumers increasingly rely on various sources of electronic word-of-mouth (eWOM), such as online user reviews and critical reviews, in their decision-making processes. Despite general consensus on the importance of eWOM and the ability of critical reviews to influence product sales, very little is known about the mediation between critical reviews and user reviews. Therefore, we used path analysis to examine how critical reviews and user reviews simultaneously affect box office revenues using eWOM data collected from Metacritic.com and IMDb.com, and box office revenue information collected from BoxOfficeMojo.com. The results showed that critical reviews valence not only directly affects box office revenues but also increases active postings in the community and user reviews volume, thus indirectly leading to greater box office revenues. The study provides strategic guidance and practical implications for eWOM communication management.

A feature-oriented analysis of developers' descriptions and user reviews of top mHealth applications for diabetes and hypertension.

BACKGROUND AND OBJECTIVE: Diabetes and hypertension are two prevalent and related chronic conditions. To inform the design and development of mobile health applications (mHealth apps) for people living with multiple chronic conditions, this paper examines features mentioned in developers' descriptions and user reviews of mHealth apps, along with users' attitudes toward associated features. MATERIALS AND METHODS: Eleven top apps for diabetes and hypertension were identified from Google Play as of January 2020. Based on a stratified sampling strategy, 1,100 user reviews were selected to form the final dataset. Developers' descriptions were also collected for analysis. Using the grounded theory approach, we developed a feature-oriented coding scheme, which was used to identify three levels of features mentioned in app descriptions and user reviews: feature group (the highest level), feature type (the second level), and individual feature (the lowest level). Users' attitudes toward app features mentioned in user reviews were also analyzed. RESULTS: Most top-rated apps for diabetes and hypertension under study were multifeatured, incorporating self-management, information sharing, and decision support features. At the feature-group level, most informative user reviews commented on features related to self-management, followed by decision support and information sharing. The four most frequently mentioned feature types were data entry, data export/import, data visualization, and assessment. Users expressed overwhelming positive attitudes toward app features across all feature categories. Based on users' assessments of existing features and requests for additional features, design implications for app development are provided. CONCLUSIONS: Despite the diversity of app features provided by mHealth apps and users' primarily positive attitudes toward existing app features, more comprehensive and personalized features are expected by app users to satisfy their health needs. Beyond identifying app features in user reviews, future research may seek more in-depth feedback from real-life patients for app development and design using methods like interviews and focus groups, to further enhance the overall quality of relevant mHealth apps to better support users.

Research on the Characteristics and Usefulness of User Reviews of Online Mental Health Consultation Services: A Content Analysis.

Online consultation based on Internet technology is gradually becoming the main way to seek health information and professional assistance. Online user reviews, such as content reviews and star ratings, are an important basis for reflecting users' views on the effectiveness of health services. Here, we used user reviews related to online psychological consultation services for content feature mining and usefulness analyses. We used a professional online psychological counseling service platform in China to collect user reviews that were liked by users as a data sample for a content analysis. An LDA topic model, dictionary-based sentiment analysis, and the NRC Word-Emotion Association Lexicon were used to extract the topic, sentiment, and context features of the content of 4254 useful reviews, and the influence of these features on the usefulness of the reviews was verified by a multiple linear regression analysis. Our results show that the content of online reviews by psychological counseling users presented a positive emotional attitude as a whole and expressed more views on the process, effects, and future expectations of counseling than on other topics. There was a significant correlation between the topic, sentiment, and context features of a user review and its usefulness: reviews giving high scores and containing topics such as "ease emotions" and "consulting expectations" received more user likes. However, the usefulness of a review was significantly reduced if it was in existence for too long. This research provides valuable suggestions for understanding the needs and emotional attitudes of users with mental health problems in terms of online psychological consultation; identifying the factors that affect the number of likes a review receives can help platform users write better consultation evaluations and thereby provide greater usefulness. In addition, the use of online reviews generated by users for content analysis effectively supplements the current research on online psychological counseling in terms of data and methods.

Banned from the sharing economy: an agent-based model of a peer-to-peer marketplace for consumer goods and services.

The emergence of profit-based online platforms related to the Sharing Economy, such as BlaBlaCar and Airbnb, provides new means for end users to create an income from their possessions. With this opportunity, participants have to make strategic economic decisions despite limited formal expertise and information. Decentralization (using digital technologies) and reputation (using user reviews) are the core mechanisms chosen by these platforms to mitigate these limitations and to work efficiently as online matchmakers. We test the performance of these two mechanisms by studying the allocative efficiency (in terms of value and volume of transactions) of simulated marketplaces under different types of motivation from the participants and control from the platforms. As a result, we find an inverted-U relationship between the decision-making leeway available to the participants and the platform's allocative efficiency. From the participants' perspectives, too much freedom or too many barriers lead to market failures affecting specific participants: low-end consumers are banned from the marketplace while high-end providers experience lower levels of activity. As governance advice for these platforms, we show the limitations of promoting these platforms on the sole motive of monetary rewards.