Digital health applications from a government-regulated directory of reimbursable health apps in Germany-a systematic review for evidence and bias.

Background: The Digital Healthcare Act, passed in November 2019, authorizes healthcare providers in Germany to prescribe digital health applications (DiGA) to patients covered by statutory health insurance. If DiGA meet specific efficacy requirements, they may be listed in a special directory maintained by the German Federal Institute for Drugs and Medical Devices. Due to the lack of well-founded app evaluation tools, the objectives were to assess (I) the evidence quality situation for DiGA in the literature and (II) how DiGA manufacturers deal with this issue, as reflected by the apps available in the aforementioned directory. Methods: A systematic review of the literature on DiGA using PubMed, Scopus, and Web of Science was started on February 4, 2023. Papers addressing the evidence for applications listed in the directory were included, while duplicates and mere study protocols not reporting on data were removed. The remaining publications were used to assess the quality of the evidence or potential gaps in this regard. Results were aggregated in tabular form. Results: The review identified fourteen relevant publications. Six studies suggested inadequate scientific evidence, five mentioned shortcomings of tools for validating DiGA-related evidence, and four publications described a high potential for bias, potentially influencing the validity of the results. Concerns about limited external generalizability were also raised. Conclusions: The literature review found evidence-related gaps that must be addressed with adequate measures. Our findings can serve as a basis for a plea for a more detailed examination of the quality of evidence in the DiGA context.

Evaluation of a semiautomated App Store analysis for the identification of health apps for cardiac arrhythmias.

BACKGROUND: Smartphone apps are increasingly utilised by patients and physicians for medical purposes. Thus, numerous applications are provided on the App Store platforms. OBJECTIVES: The aim of the study was to establish a novel, expanded approach of a semiautomated retrospective App Store analysis (SARASA) to identify and characterise health apps in the context of cardiac arrhythmias. MATERIALS AND METHODS: An automated total read-out of the "Medical" category of Apple's German App Store was performed in December 2022 by analysing the developer-provided descriptions and other metadata using a semiautomated multilevel approach. Search terms were defined, based on which the textual information of the total extraction results was automatically filtered. RESULTS: A total of 435 of 31,564 apps were identified in the context of cardiac arrhythmias. Of those, 81.4% were found to deal with education, decision support, or disease management, and 26.2% (additionally) provided the opportunity to derive information on heart rhythm. The apps were intended for healthcare professionals in 55.9%, students in 17.5%, and/or patients in 15.9%. In 31.5%, the target population was not specified in the description texts. In all, 108 apps (24.8%) provided a telemedicine treatment approach; 83.7% of the description texts did not reveal any information on medical product status; 8.3% of the apps indicated that they have and 8.0% that they do not have medical product status. CONCLUSION: Through the supplemented SARASA method, health apps in the context of cardiac arrhythmias could be identified and assigned to the target categories. Clinicians and patients have a wide choice of apps, although the app description texts do not provide sufficient information about the intended use and quality.

COVID-19 Mobile Apps Trends Derived from Long-Term Google Play Analysis.

In this paper, we describe the 5-year trends of COVID-related mobile apps in the Google Play platform obtained by retrospectively analyzing app descriptions. Out of 21764 and 48750 unique apps available free of charge in the "medical" and "health and fitness", there were 161 and 143 COVID-related apps, respectively. The prominentrise in apps' prevalence occurred in January 2021.

Conciliation and Approval of Quality Principles for the Self-Declaration of the Quality of Health Apps.

This poster describes the conciliation and approval process of the unified set of criteria for self-declaration of health app quality. The timeline underlines the necessity of transparency and open communication in regulations.

Do Health Apps Stigmatize People with Obesity?

Apps in the "Medicine" category of Apple's App Store were examined concerning the potential stigmatization of people with obesity through word and image language. Only 5/71 potentially stigmatizing apps related to obesity were identified. Stigmatization in this context can occur, for example, through the excessive promotion of very slim people in connection with weight loss-related apps.

An Automated System to Distribute Students to Clerkships.

Participating in clerkships with general practitioners (GPs) is integral to studying medicine. The students gain deep and valuable insights into the everyday working practice of GPs. The central challenge is organizing these clerkships to distribute the students to the participating doctors' offices. This process becomes even more complex and time-consuming if students can state their preferences. To support faculty staff and involve students in the process, we developed an application to support the distribution process via an automated system and applied it to allocate over 700 students over the course of 2.5 years.

Medical Apps for Android and iOS: Differences and Similarities.

Google Play and Apple's App Store dominate the mobile health app market. We analyzed the metadata and descriptive texts of apps in the medical category using semi-automated retrospective app store analysis (SARASA) and compared the store offerings in terms of their number, descriptive texts, user ratings, medical device status, diseases, and conditions (both keyword-based). Relatively speaking, the store listings for the selected items were comparable.

Evidence of Digital Health Applications from a State-Regulated Repository for Reimbursable Health Applications in Germany.

17 RCTs for 15 digital health applications (DiGA) permanently listed in the state-regulated register were analyzed descriptively for methodological study aspects relevant to evidence analysis. The analysis revealed that several underlying studies had limitations, at least worthy of discussion, in terms of their power concerning sample size, intervention and control group specifications, drop-out rates, and blinding.

Orthopaedic and trauma surgeons' prioritisation of app quality principles based on their demographic background.

BACKGROUND: Although apps are becoming increasingly relevant in healthcare, there is limited knowledge about how healthcare professionals perceive "quality" in this context and how quality principles that can aid them in assessing health-related apps may be prioritised. The objective was to investigate physicians' views of predefined (general) quality principles for health apps and to determine whether a ranking algorithm applied to the acquired data can provide stable results against various demographic influences and may thus be appropriate for prioritisation. METHODS: Participants of an online survey of members of two German professional orthopaedics associations conducted between 02/12/2019 and 02/01/2020 were asked about their perception of a set of quality principles for health apps (i.e., "practicality," "risk adequacy," "ethical soundness," "legal conformity," "content validity," "technical adequacy," "usability," "resource efficiency," and "transparency"). Structured as a Kano survey, for each principle, there were questions about its perceived relevance and opinions regarding the presence or absence of corresponding characteristics. The available data were evaluated descriptively, and a newly developed method for prioritisation of the principles was applied overall and to different demographic strata (for validation). RESULTS: Three hundred eighty-two datasets from 9503 participants were evaluated. Legal conformity, content validity, and risk adequacy filled ranks one to three, followed by practicability, ethical soundness, and usability (ranks 4 to 6). Technical adequacy, transparency, and resource efficiency ranked last (ranks 7 to 9). The ranking based on the proposed method was relatively stable, irrespective of demographic factors. The principles were seen as essential, with one exception ("resource efficiency"). Only those with little to no interest in digitisation (22/382, 5.8%) rated the nine principles indifferently. CONCLUSIONS: The specified quality principles and their prioritisation can lay a foundation for future assessments of apps in the medical field. Professional societies build upon this to highlight opportunities for digital transformations in medicine and encourage their members to participate.

Exploring the weight bias of professionals working in the field of obesity with a mobile IAT: a pilot study.

Background: Obesity is common in many industrialized nations and often accompanied by related health issues. Furthermore, individuals living with overweight or obesity are often confronted with stigmatization in their daily lives. These problems may be aggravated if the objectivity of health care professionals is compromised due to (unconscious) prejudices. If pharmaceutical companies, regulatory agencies, and health insurers are also susceptible to these biases, decisions related to the development, approval, and reimbursement of obesity-related therapies may be negatively impacted. Materials and Methods: The 'Implicit Association Test' (IAT) is a psychometric test allowing to measure these attitudes and could therefore assist to reveal unconscious preferences. A self-developed mobile version, in the form of a ResearchKit-based IAT app was employed in the presented study. The objective was to determine (potential) weight bias and its characteristics for professionals attending a national obesity-related conference in Germany (G1), compared to a control group (without stated interest in the topic, G2) - both using the mobile app - and a historical control (G3) based on data provided by Project Implicit acquired by a web app. Results: Explicit evaluations of G1 were neutral at a higher percentage compared with G2 and G3, while implicit preference toward lean individuals did not differ significantly between G2 and G3, and G1. Conclusion: The greater discrepancy between the (more neutral) explicit attitude and the unconscious preference pointing in the anti-obesity direction could indicate an underestimated bias for the professional participants in G1. Implicit preference is often ingrained from childhood on, and difficult to overcome. Thus, even for professionals, it may unconsciously influence decisions made in the care they provide. Professionals in any given health care sector directed at obesity care should thus be made aware of this inconsistency to enable them to consciously counteract this potential effect.

Implementation of Mobile Psychological Testing on Smart Devices: Evaluation of a ResearchKit-Based Design Approach for the Implicit Association Test.

Objective: To determine whether a framework-based approach for mobile apps is appropriate for the implementation of psychological testing, and equivalent to established methods. Methods: Apple's ResearchKit was used for implementing native implicit association test methods (IAT), and an exemplary app was developed to examine users' implicit attitudes toward overweight or thin individuals. For comparison, a web-based IAT app, based on code provided by Project Implicit, was used. Adult volunteers were asked to test both versions on an iPad with touch as well as keyboard input (altogether four tests per participant, random order). Latency values were recorded and used to calculate parameters relevant to the implicit setting. Measurements were analyzed with respect to app type and input method, as well as test order (ANOVA and χ2 tests). Results: Fifty-one datasets were acquired (female, n = 21; male, n = 30, average age 35 ± 4.66 years). Test order and combination of app type and input method influenced the latency values significantly (both P<0.001). This was not mirrored for the D scores or average number of errors vs. app type combined with input method (D scores: P = 0.66; number of errors: P = 0.733) or test order (D scores: P = 0.096; number of errors: P = 0.85). Post-hoc power analysis of the linear ANOVA showed 0.8 by f 2=0.25, with α = 0.05 and 4 predictors. Conclusions: The results suggest that a native mobile implementation of the IAT may be comparable to established implementations. The validity of the acquired measurements seems to depend on the properties of the chosen test rather than the specifics of the chosen platform or input method.

Prioritization of Quality Principles for Health Apps Using the Kano Model: Survey Study.

BACKGROUND: Health apps are often used without adequately taking aspects related to their quality under consideration. This may partially be due to inadequate awareness about necessary criteria and how to prioritize them when evaluating an app. OBJECTIVE: The aim of this study was to introduce a method for prioritizing quality attributes in the mobile health context. To this end, physicians were asked about their assessment of nine app quality principles relevant in health contexts and their responses were used as a basis for designing a method for app prioritization. Ultimately, the goal was to aid in making better use of limited resources (eg, time) by assisting with the decision as to the specific quality principles that deserve priority in everyday medical practice and those that can be given lower priority, even in cases where the overall principles are rated similarly. METHODS: A total of 9503 members of two German professional societies in the field of orthopedics were invited by email to participate in an anonymous online survey over a 1-month period. Participants were asked to rate a set of nine app quality principles using a Kano survey with functional and dysfunctional (ie, positively and negatively worded) questions. The evaluation was based on the work of Kano (baseline), supplemented by a self-designed approach. RESULTS: Among the 9503 invited members, 382 completed relevant parts of the survey (return rate of 4.02%). These participants were equally and randomly assigned to two groups (test group and validation group, n=191 each). Demographic characteristics did not significantly differ between groups (all P>.05). Participants were predominantly male (328/382, 85.9%) and older than 40 years (290/382, 75.9%). Given similar ratings, common evaluation strategies for Kano surveys did not allow for conclusive prioritization of the principles, and the same was true when using the more elaborate approach of satisfaction and dissatisfaction indices following the work of Timko. Therefore, an extended, so-called "in-line-of-sight" method was developed and applied for this evaluation. Modified from the Timko method, this approach is based on a "point of view" (POV) metric, which generates a ranking coefficient. Although the principles were previously almost exclusively rated as must-be (with the exception of resource efficiency), which was not conducive to their prioritization, the new method applied from the must-be POV resulted in identical rankings for the test and validation groups: (1) legal conformity, (2) content validity, (3) risk adequacy, (4) practicality, (5) ethical soundness, (6) usability, (7) transparency, (8) technical adequacy, and (9) resource efficiency. CONCLUSIONS: Established survey methodologies based on the work of Kano predominantly seek to categorize the attributes to be evaluated. The methodology presented here is an interesting option for prioritization, and enables focusing on the most important criteria, thus saving valuable time when reviewing apps for use in the medical field, even with otherwise largely similar categorization results. The extent to which this approach is applicable beyond the scenario presented herein requires further investigation.

[Great potential, little substance : Overview of quality and quantity of urological apps in the German app store]. / Viel Potenzial, wenig Substanz : Eine aktuelle Übersicht über Quantität und Qualität urologischer Apps im deutschen App-Store.

BACKGROUND: The range of medical apps is broad and diverse. The previous evaluations are inconsistent and limited to the respective areas of application. OBJECTIVES: The main objective of this work is to comprehensively present, organize, and evaluate the current range of urological apps with the help of a semi-automatic retrospective app store analysis (SARASA). METHODS: Application of an adaptable method based on filter processes according to predefined criteria by means of SARASA to characterize urological apps from various subject areas in the Apple App Store with subsequent manual filtering and evaluation. RESULTS: From the original list of 34,830 apps in the "Medicine" category of the Apple App Store on 27 September 2021, 3556 apps remained after apps without a German-language store description were removed. 43 subject-specific apps remained for further analysis and description. The number of reviews, rating, topicality, urological issues, technical support and richness of content were taken into account. The two most relevant apps for each topic are presented in detail. CONCLUSION: SARASA offers an easy-to-use method for applying filter processes to identify apps in app stores that meet predefined, formal criteria. The highest number of apps can be classified in categories of patient information and further education and training. An app officially listed by the German Federal Institute for Drugs and Medical Products (BfArM) in the sense of a digital health application (DiGA), the costs of which are reimbursed by the health insurance companies, is available for urology in only one single case. The authors see great potential for future developments in this regard.

The Digital Healthcare Act - a Turning Point in the German Digitisation Strategy? / Das Digitale-Versorgung-Gesetz ­ eine Zäsur der deutschen Digitalisierungsstrategie?

The recently ratified Digital Healthcare Act - DVG - has paved the way, among other issues, for the prescription of health apps. The German DVG creates an entitlement to health apps for the very first time. However, this entitlement is initially limited to low-risk medical devices that have been included in the publicly accessible Register for Digital Health Applications by the German Federal Institute for Drugs and Medical Devices. Listing in the register is granted, if the manufacturer has submitted a health app and verified that it meets basic requirements for medical devices and data security as well as positive health care effects. It is questionable, whether the DVG will lead to sustainable improvements in future patient care. In order to pursue this question and align the DGOUC's digitization strategy closely with its basis, an online survey was conducted among the respective members on the DVG's content and its associated opportunities and risks. A total of 461 German-speaking, predominantly male, experienced and elderly physicians in leading positions took part in the survey. In this study, it was shown that the majority of the participating German orthopaedic and trauma surgeons is not familiar with the DVG's contents. Despite a fundamentally positive attitude towards digitization, scepticism about the use of "apps on prescription" and potential risks still prevails at present. The closing of ranks between medicine, IT and the legislature via interdisciplinary expert groups and the involvement of medical societies might be obligatory.

Assessment of a Mobile App by Adolescents and Young Adults With Cystic Fibrosis: Pilot Evaluation.

BACKGROUND: Cystic fibrosis (CF) continues to be the most common life-limiting chronic pulmonary disease in adolescents and young adults. Treatment of CF demands a high treatment time investment to slow the progression of lung function decline, the most important contributor to morbidity and mortality. Adherence is challenging in CF due to the high treatment burden and the lack of immediate health consequences in case of nonadherence. Lung function decline is particularly pronounced in the transition phase between 12 and 24 years of age. The improvement of self-management and self-responsibility and independence from parents and desire for normalcy are conflicting aspects for many adolescents with CF, which influence adherence to the time-consuming pulmonary therapy. Mobile health (mHealth) care apps could help to support self-management and independence and thereby reconcile seemingly conflicting goals to improve adherence, quality of life, and ultimately CF life expectancy. OBJECTIVE: This study aimed to (1) assess user behavior and satisfaction among adolescents and young adults with CF over an observation period of three months using an mHealth app; (2) identify areas of improvement for this mHealth app; and (3) compare overall and disease-specific satisfaction, lung function, and anthropometry before and after using the mHealth app. METHODS: A total of 27 adolescents and young adults with CF (age range 12-24 years, mean age 16 years, SD 3 years; 14 females, 11 males) used a free mHealth app for three months of whom 25 provided questionnaire data for analysis at the end of the study. Data collection was carried out using questionnaires on usage characteristics and life satisfaction, and standardized assessment of lung function and anthropometry. RESULTS: The use of the reminder function for medication declined from 70% (15/21) of the participants at week 4 to 65% (13/20) at week 8 of the observation period. At the end of the study, only 17% (4/23) of the participants wanted to continue using the app. Nevertheless, 56% (14/25) of participants saw the mobile app as a support for everyday life. Potential improvements targeting hedonistic qualities were identified to improve mHealth app adherence. Comparisons of satisfaction with different life aspects hinted at improvements or stabilization for the subitem respiration and the subitem lack of handicap by CF, suggesting that app use might stabilize certain CF-specific aspects of the weighted satisfaction with life. Lung function and anthropometry were not affected consistently. CONCLUSIONS: Most of the patients did not want to continue using the app after the study period. Only a few CF-specific aspects of weighted life satisfaction were possibly stabilized by the mHealth app; clinical parameters were not affected. Adaptation of the functions to adolescent-specific needs could improve the long-term use and thus positively affect the disease course.

Quality Awareness and Its Influence on the Evaluation of App Meta-Information by Physicians: Validation Study.

BACKGROUND: Meta-information provided about health apps on app stores is often the only readily available source of quality-related information before installation. OBJECTIVE: The purpose of this study was to assess whether physicians deem a predefined set of quality principles as relevant for health apps; whether they are able to identify corresponding information in a given sample of app descriptions; and whether, and how, this facilitates their informed usage decisions. METHODS: All members of the German Society for Internal Medicine were invited by email to participate in an anonymous online survey over a 6-week period. Participants were randomly assigned one app description focusing on cardiology or pulmonology. In the survey, participants were asked three times about whether the assigned description sufficed for a usage decision: they were asked (1) after giving an appraisal of the relevance of nine predefined app quality principles, (2) after determining whether the descriptions covered the quality principles, and (3) after they assessed the availability of detailed quality information by means of 25 additional key questions. Tests for significance of changes in their decisions between assessments 1 and 2, and between assessments 2 and 3, were conducted with the McNemar-Bowker test of symmetry. The effect size represents the discordant proportion ratio sum as a quotient of the test statistics of the Bowker test and the number of observation units. The significance level was set to alpha=.05 with a power of 1-beta=.95. RESULTS: A total of 441 of 724 participants (60.9%) who started the survey fully completed the questionnaires and were included in the evaluation. The participants predominantly rated the specified nine quality principles as important for their decision (approximately 80%-99% of ratings). However, apart from the practicality criterion, information provided in the app descriptions was lacking for both groups (approximately 51%-92%). Reassessment of the apps led to more critical assessments among both groups. After having familiarized themselves with the nine quality principles, approximately one-third of the participants (group A: 63/220, 28.6%; group B: 62/221, 28.1%) came to more critical usage decisions in a statistically significant manner (McNemar-Bowker test, groups A and B: P<.001). After a subsequent reassessment with 25 key questions, critical appraisals further increased, although not in a statistically significant manner (McNemar-Bowker, group A: P=.13; group B: P=.05). CONCLUSIONS: Sensitizing physicians to the topic of quality principles via questions about attitudes toward established quality principles, and letting them apply these principles to app descriptions, lead to more critical appraisals of the sufficiency of the information they provided. Even working with only nine generic criteria was sufficient to bring about the majority of decision changes. This may lay the foundation for aiding physicians in their app-related decision processes, without unduly taking up their valuable time.

Quality Principles of App Description Texts and Their Significance in Deciding to Use Health Apps as Assessed by Medical Students: Survey Study.

BACKGROUND: Currently, there are no binding requirements for manufacturers prescribing which information must be included in the app descriptions of health apps. OBJECTIVE: The aim of this study was to investigate how medical students perceive a selection of quality principles, intended for usage decisions in the app context, and establish whether the information presented in a sample of app descriptions is perceived as sufficient for facilitating an informed usage decision. METHODS: A total of 123 students (mean age 24.2 years, SD 3.4) participating in a 6-week teaching module covering cardiology and pulmonology at the University of Göttingen (original enrollment 152 students, response rate 80.9%) were included. Students were asked to read 3 store description texts of cardiological or pneumological apps and initially assess whether the descriptions sufficed for a usage decision. Subsequently, they were queried on their perception of the relevance of 9 predefined quality principles, formulated for usage decisions. An appraisal of whether the app description texts contained sufficient information to satisfy these quality principles followed. By means of 20 guiding questions, participants were then asked to identify relevant information (or a lack thereof) within the descriptions. A reassessment of whether the description texts sufficed for making a usage decision ensued. A total of 343 complete datasets were obtained. RESULTS: A majority of the quality principles were described as "very important" and "important" for making a usage decision. When accessed via the predefined principles, students felt unable to identify sufficient information within the app descriptions in 68.81% (2124/3087) of cases. Notably, information regarding undesired effects (91.8%, 315/343), ethical soundness (90.1%, 309/343), measures taken to avert risks (89.2%, 306/343), conflicts of interest (88.3%, 303/343), and the location of data storage (87.8%, 301/343) was lacking. Following participants' engagement with the quality principles, statistically significant changes in their assessment of whether the app descriptions sufficed for a usage decision can be seen-McNemar-Bowker test (3)=45.803919, P<.001, Cohen g=.295. In 34.1% (117/343) cases, the assessment was revised. About 3 quarters of changed assessments were seen more critically (76.9%, 90/117). Although, initially, 70% (240/343) had been considered "sufficient," this rate was reduced to 54.2% (186/343) in the second assessment. CONCLUSIONS: In a considerable number of app descriptions, participants were unable to locate the information necessary for making an informed usage decision. Participants' sensitization to the quality principles led to changes in their assessment of app descriptions as a tool for usage decisions. Better transparency in app descriptions released by manufacturers and the exposure of users to quality principles could collectively form the basis for well-founded usage decisions.

Description of Cardiological Apps From the German App Store: Semiautomated Retrospective App Store Analysis.

BACKGROUND: In the app stores of mobile platforms, consumers are confronted with an enormous number of mobile apps. Over the past few years, considerable research has been undertaken into to identifying, characterizing, and evaluating apps, be it in health-related or other contexts. However, many of these projects are restricted to specific areas of application and offer little flexibility in adapting the applied criteria. OBJECTIVE: This paper presents an adaptable method for selecting and characterizing mobile apps listed in a mobile App Store (the Apple App Store). The method is based on filtering processes using predefined criteria, through a semiautomated retrospective App Store analysis (SARASA). METHODS: To illustrate the SARASA process, keyword-based filtering and metadata-based description, review, and ranking steps were applied to a dataset, more specifically, an April 2018 readout of the Medical category of the German App Store, with the aim of obtaining a list of cardiology-related apps. RESULTS: From the original list of 39,427 apps within the "Medical" category of the App Store on April 14, 2018, 34,382 apps with store descriptions in languages other than German were removed. For the remaining 5045 apps, keywords related to cardiology were applied to filter the output, obtaining a final total of 335 subject-specific apps for further analysis and description. CONCLUSIONS: SARASA provides an easy to use method for applying filtering processes to identify apps matching predefined, formal criteria from app stores. The criteria can be well adapted to the needs of users. Automatic and manual analyses are easily combined when using SARASA. In the future, additional features, such as algorithmic topic analyses, may supplement the process. Although the area of application is currently limited to Apple's App Store, expansion to other stores is planned. The method stands or falls with the transparency of the app store providers and the manufacturers to make relevant meta-information available. It is up to them to liberalize information and restrict censorship to provide clients, customers, and users truly fair circumstances finding their way around the app market.

Relevance of Trust Marks and CE Labels in German-Language Store Descriptions of Health Apps: Analysis.

BACKGROUND: In addition to mandatory CE marking ("CE" representing Conformité Européenne, with the CE marking being a symbol of free marketability in the European Economic Area) for medical devices, there are various seals, initiatives, action groups, etc, in the health app context. However, whether manufacturers use them to distinguish their apps and attach relevance to them is unclear. OBJECTIVE: The objective was to take a snapshot of quality seals, regulatory marks, and other orientation aids available on the German app market and to determine whether manufacturers deem such labels relevant enough to apply them to their apps, namely as reflected by mentions in app description texts in a typical app store (ie, Apple's App Store). METHODS: A full survey of the metadata of 103,046 apps from Apple's German App Store in the Medicine and Health & Fitness categories was carried out. For apps with German-language store descriptions (N=8767), these were automatically searched for the occurrence of relevant keywords and validated manually (N=41). In addition, the websites of various app seal providers were checked for assigned seals. RESULTS: Few manufacturers referenced seals in the descriptions (5/41), although this would have been expected more often based on the seals we were able to identify from the seal providers' Web pages, and there were 34 of 41 that mentioned CE status in the descriptions. Two apps referenced an app directory curated by experts; however, this is not an alternative to CE marks and seals of approval. CONCLUSIONS: Currently, quality seals seem to be irrelevant for manufacturers. In line with regulatory requirements, mentions of medical device status are more frequent; however, neither characteristic is effective for identifying high-quality apps. To improve this situation, a possibly legally obligatory, standardized reporting system should be implemented.