Attributes, Methods, and Frameworks Used to Evaluate Wearables and Their Companion mHealth Apps: Scoping Review.

BACKGROUND: Wearable devices, mobile technologies, and their combination have been accepted into clinical use to better assess the physical fitness and quality of life of patients and as preventive measures. Usability is pivotal for overcoming constraints and gaining users' acceptance of technology such as wearables and their companion mobile health (mHealth) apps. However, owing to limitations in design and evaluation, interactive wearables and mHealth apps have often been restricted from their full potential. OBJECTIVE: This study aims to identify studies that have incorporated wearable devices and determine their frequency of use in conjunction with mHealth apps or their combination. Specifically, this study aims to understand the attributes and evaluation techniques used to evaluate usability in the health care domain for these technologies and their combinations. METHODS: We conducted an extensive search across 4 electronic databases, spanning the last 30 years up to December 2021. Studies including the keywords "wearable devices," "mobile apps," "mHealth apps," "physiological data," "usability," "user experience," and "user evaluation" were considered for inclusion. A team of 5 reviewers screened the collected publications and charted the features based on the research questions. Subsequently, we categorized these characteristics following existing usability and wearable taxonomies. We applied a methodological framework for scoping reviews and the PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews) checklist. RESULTS: A total of 382 reports were identified from the search strategy, and 68 articles were included. Most of the studies (57/68, 84%) involved the simultaneous use of wearables and connected mobile apps. Wrist-worn commercial consumer devices such as wristbands were the most prevalent, accounting for 66% (45/68) of the wearables identified in our review. Approximately half of the data from the medical domain (32/68, 47%) focused on studies involving participants with chronic illnesses or disorders. Overall, 29 usability attributes were identified, and 5 attributes were frequently used for evaluation: satisfaction (34/68, 50%), ease of use (27/68, 40%), user experience (16/68, 24%), perceived usefulness (18/68, 26%), and effectiveness (15/68, 22%). Only 10% (7/68) of the studies used a user- or human-centered design paradigm for usability evaluation. CONCLUSIONS: Our scoping review identified the types and categories of wearable devices and mHealth apps, their frequency of use in studies, and their implementation in the medical context. In addition, we examined the usability evaluation of these technologies: methods, attributes, and frameworks. Within the array of available wearables and mHealth apps, health care providers encounter the challenge of selecting devices and companion apps that are effective, user-friendly, and compatible with user interactions. The current gap in usability and user experience in health care research limits our understanding of the strengths and limitations of wearable technologies and their companion apps. Additional research is necessary to overcome these limitations.

Requirements analysis for an AI-based clinical decision support system for general practitioners: a user-centered design process.

BACKGROUND: As the first point of contact for patients with health issues, general practitioners (GPs) are frequently confronted with patients presenting with non-specific symptoms of unclear origin. This can result in delayed, prolonged or false diagnoses. To accelerate and improve the diagnosis of diseases, clinical decision support systems would appear to be an appropriate tool. The objective of the project 'Smart physician portal for patients with unclear disease' (SATURN) is to employ a user-centered design process based on the requirements analysis presented in this paper to develop an artificial Intelligence (AI)-based diagnosis support system that specifically addresses the needs of German GPs. METHODS: Requirements analysis for a GP-specific diagnosis support system was conducted in an iterative process with five GPs. First, interviews were conducted to analyze current workflows and the use of digital applications in cases of diagnostic uncertainty (as-is situation). Second, we focused on collecting and prioritizing tasks to be performed by an ideal smart physician portal (to-be situation) in a workshop. We then developed a task model with corresponding user requirements. RESULTS: Numerous GP-specific user requirements were identified concerning the tasks and subtasks: performing data entry (open system, enter patient data), reviewing results (receiving and evaluating results), discussing results (with patients and colleagues), scheduling further diagnostic procedures, referring to specialists (select, contact, make appointments), and case closure. Suggested features particularly concerned the process of screening and assessing results: e.g., the system should focus more on atypical patterns of common diseases than on rare diseases only, display probabilities of differential diagnoses, ensure sources and results are transparent, and mark diagnoses that have already been ruled out. Moreover, establishing a means of using the platform to communicate with colleagues and transferring patient data directly from electronic patient records to the system was strongly recommended. CONCLUSIONS: Essential user requirements to be considered in the development and design of a diagnosis system for primary care could be derived from the analysis. They form the basis for mockup-development and system engineering.

Context-Sensitive Common Data Models for Genetic Rare Diseases - A Concept.

Current challenges of rare diseases need to involve patients, physicians, and the research community to generate new insights on comprehensive patient cohorts. Interestingly, the integration of patient context has been insufficiently considered, but might tremendously improve the accuracy of predictive models for individual patients. Here, we conceptualized an extension of the European Platform for Rare Disease Registration data model with contextual factors. This extended model can serve as an enhanced baseline and is well-suited for analyses using artificial intelligence models for improved predictions. The study is an initial result that will develop context-sensitive common data models for genetic rare diseases.

Empowering Researchers to Query Medical Data and Biospecimens by Ensuring Appropriate Usability of a Feasibility Tool: Evaluation Study.

BACKGROUND: The Aligning Biobanking and Data Integration Centers Efficiently project aims to harmonize technologies and governance structures of German university hospitals and their biobanks to facilitate searching for patient data and biospecimens. The central element will be a feasibility tool for researchers to query the availability of samples and data to determine the feasibility of their study project. OBJECTIVE: The objectives of the study were as follows: an evaluation of the overall user interface usability of the feasibility tool, the identification of critical usability issues, comprehensibility of the underlying ontology operability, and analysis of user feedback on additional functionalities. From these, recommendations for quality-of-use optimization, focusing on more intuitive usability, were derived. METHODS: To achieve the study goal, an exploratory usability test consisting of 2 main parts was conducted. In the first part, the thinking aloud method (test participants express their thoughts aloud throughout their use of the tool) was complemented by a quantitative questionnaire. In the second part, the interview method was combined with supplementary mock-ups to collect users' opinions on possible additional features. RESULTS: The study cohort rated global usability of the feasibility tool based on the System Usability Scale with a good score of 81.25. The tasks assigned posed certain challenges. No participant was able to solve all tasks correctly. A detailed analysis showed that this was mostly because of minor issues. This impression was confirmed by the recorded statements, which described the tool as intuitive and user friendly. The feedback also provided useful insights regarding which critical usability problems occur and need to be addressed promptly. CONCLUSIONS: The findings indicate that the prototype of the Aligning Biobanking and Data Integration Centers Efficiently feasibility tool is headed in the right direction. Nevertheless, we see potential for optimization primarily in the display of the search functions, the unambiguous distinguishability of criteria, and the visibility of their associated classification system. Overall, it can be stated that the combination of different tools used to evaluate the feasibility tool provided a comprehensive picture of its usability.

A pragmatic methodical framework for the user-centred development of an electronic process support for the sleep laboratory patients' management.

Objective: Limited capacities and ineffective care pathways result in long waiting times for patients and sporadic treatment controls in sleep medicine. As one objective of the 'Telesleep Medicine' project, a portal should be developed, which supports sleep specialists in an efficient and resource-saving patient management. On account of the limited project timeframe, the 'classical' user-centred design and evaluation methods could not be comprehensively implemented. Therefore, a pragmatic methodical framework was developed. Methods: For the iterative development of the portal, a combination of low-cost and quick-to-implement methods was used. In chronological order, these were: context interviews, personas, the development of an as-is model, a web search of design standards and good design aspects of similar systems, the development of a to-be model, the creation of an overarching mind map, and the iterative creation of mockups with simplified usability walkthroughs. Results: The feasibility of the pragmatic methodological framework for the development of a prototype for the portal was demonstrated. The used method combination resulted in a prototype based on the needs and requirements of the sleep specialists, taking into account their specific workflow and the technical implementation conditions. Conclusions: The presented pragmatic methodological framework can be a valuable resource for developers of comparable projects. The combination of methods worked well together regarding the limited timeframe and resources for concept development. For the future, we plan to implement and test the portal in the clinical field and thus enrich our framework with additional methods.

Visualization of Medical Wearable-Data Using SMART-on-FHIR - A Concept for an Interdisciplinary Complex Practical Course.

For the success of digital applications, especially AI applications, it is essential that both developers and medical professionals are enabled to understand each other's perspective. For this reason, a new concept for an interdisciplinary complex practical course was developed for the master's program in computer science at a German university, based on online learning nuggets and a hackathon on site. The core of the concept is a real-world medical application task: extracting ECG patient data from a smartwatch to support primary care physicians in making decisions regarding an action. The concept was developed based on the so-called constructive alignment concept. An initial application of the concept showed that it was rated as very positive in terms of learning experience and working atmosphere.

User-Centred Development of a Diagnosis Support System for Rare Diseases.

The diagnosis of rare diseases is often challenging for physicians, but can be supported by Clinical Decision Support Systems. The MIRACUM consortia, which includes ten university hospitals in Germany, develops a Clinical Decision Support System to support the diagnosis of patients with rare diseases. The users are involved in different phases using a user-centred design process. This publication has the objective to summarize the results of all studies performed in context of the requirements elicitation and to derive concrete requirements for the development of the system. Several studies were performed for requirements elicitation: a cross-sectional survey, expert interviews and a focus group. Participants were experts in rare diseases of the MIRACUM locations. 32 requirements were derived and implemented in a prototype. The prototype allows similarity analyses as a decision support functionality by comparing patients without a diagnosis to patients with a rare disease. In the final evaluation, the prototype was rated with a good usability. Since the system is limited in its functionality, further work and improvements are necessary to make it ready for clinical usage.

The Architecture of a Feasibility Query Portal for Distributed COVID-19 Fast Healthcare Interoperability Resources (FHIR) Patient Data Repositories: Design and Implementation Study.

BACKGROUND: An essential step in any medical research project after identifying the research question is to determine if there are sufficient patients available for a study and where to find them. Pursuing digital feasibility queries on available patient data registries has proven to be an excellent way of reusing existing real-world data sources. To support multicentric research, these feasibility queries should be designed and implemented to run across multiple sites and securely access local data. Working across hospitals usually involves working with different data formats and vocabularies. Recently, the Fast Healthcare Interoperability Resources (FHIR) standard was developed by Health Level Seven to address this concern and describe patient data in a standardized format. The Medical Informatics Initiative in Germany has committed to this standard and created data integration centers, which convert existing data into the FHIR format at each hospital. This partially solves the interoperability problem; however, a distributed feasibility query platform for the FHIR standard is still missing. OBJECTIVE: This study described the design and implementation of the components involved in creating a cross-hospital feasibility query platform for researchers based on FHIR resources. This effort was part of a large COVID-19 data exchange platform and was designed to be scalable for a broad range of patient data. METHODS: We analyzed and designed the abstract components necessary for a distributed feasibility query. This included a user interface for creating the query, backend with an ontology and terminology service, middleware for query distribution, and FHIR feasibility query execution service. RESULTS: We implemented the components described in the Methods section. The resulting solution was distributed to 33 German university hospitals. The functionality of the comprehensive network infrastructure was demonstrated using a test data set based on the German Corona Consensus Data Set. A performance test using specifically created synthetic data revealed the applicability of our solution to data sets containing millions of FHIR resources. The solution can be easily deployed across hospitals and supports feasibility queries, combining multiple inclusion and exclusion criteria using standard Health Level Seven query languages such as Clinical Quality Language and FHIR Search. Developing a platform based on multiple microservices allowed us to create an extendable platform and support multiple Health Level Seven query languages and middleware components to allow integration with future directions of the Medical Informatics Initiative. CONCLUSIONS: We designed and implemented a feasibility platform for distributed feasibility queries, which works directly on FHIR-formatted data and distributed it across 33 university hospitals in Germany. We showed that developing a feasibility platform directly on the FHIR standard is feasible.

Improving COVID-19 Research of University Hospitals in Germany: Formative Usability Evaluation of the CODEX Feasibility Portal.

BACKGROUND: Within the German "Network University Medicine," a portal is to be developed to enable researchers to query on novel coronavirus disease 2019 (COVID-19) data from university hospitals for assessing the feasibility of a clinical study. OBJECTIVES: The usability of a prototype for federated feasibility queries was evaluated to identify design strengths and weaknesses and derive improvement recommendations for further development. METHODS: In the course of a remote usability test with the thinking-aloud method and posttask interviews, 15 clinical researchers evaluated the usability of a prototype of the Feasibility Portal. The identified usability problems were rated according to severity, and improvement recommendations were derived. RESULTS: The design of the prototype was rated as simple, intuitive, and as usable with little effort. The usability test reported a total of 26 problems, 8 of these were rated as "critical." Usability problems and revision recommendations focus primarily on improving the visual distinguishability of selected inclusion and exclusion criteria, enabling a flexible approach to criteria linking, and enhancing the free-text search. CONCLUSION: Improvement proposals were developed for these user problems which will guide further development and the adaptation of the portal to user needs. This is an important prerequisite for correct and efficient use in everyday clinical work in the future. Results can provide developers of similar systems with a good starting point for interface conceptualizations. The methodological approach/the developed test guideline can serve as a template for similar evaluations.

Correction: Evaluation of Three Feasibility Tools for Identifying Patient Data and Biospecimen Availability: Comparative Usability Study.

[This corrects the article DOI: 10.2196/25531.].

[Interdisciplinary care path and potential IT support for people with rare diseases in Germany]. / Interdisziplinärer Versorgungspfad und mögliche IT-Unterstützung für Menschen mit Seltenen Erkrankungen in Deutschland.

INTRODUCTION: Due to the high variability and, at the same time, rare occurrence of rare diseases, the diagnosis of these patients (approx. 4 million people in Germany) can turn into an odyssey. The large time interval between the appearance of symptoms and the final diagnosis of the rare disease leads to a delay in the appropriate treatment. The often long period of uncertainty about the cause of symptoms as well as non-specific or even wrong therapies can have negative effects on both the course of disease and the patients' quality of life. For a better understanding of the current care situation and IT landscape, the interdisciplinary care pathway for people with rare diseases will be modelled and the possible uses of IT applications identified that have the potential to improve diagnosis, treatment and therapy of rare diseases. METHODS: In order to achieve these goals, an initial care pathway was modelled on the basis of process descriptions which are commonly used in the literature, discussed in detail, and agreed upon in a first workshop with six experts from outpatient and inpatient care as well as employees of Centers for Rare Diseases. In a second workshop, ten experts analyzed the resulting care pathway with regard to the possible use of IT applications, and the identification was agreed upon. The experts included those involved in the process, in particular physicians, patients / patient representatives, health care researchers, and experts in hospital IT, IT security, and data protection. RESULTS: The two workshops resulted in process models including the specification of possible uses for IT applications. The most important steps in the care pathway for people with rare diseases in Germany include: neonatal screening, seeking medical advice, outpatient care by general practitioners, outpatient care by specialists, care by specialist outpatient clinic, care by clinic, care by a Center for Rare Diseases: case review and case conference and treatment and therapy. The discussion of the possible uses of IT applications resulted in a focus on registers (e. g. with regard to experts, treatment and therapy options) as well as on digital tools, such as "digital findings and findings platform" and "digital referral with referral tracking". DISCUSSION: Our results show that the care pathway is very heterogeneous and complex. Thus, the sub-processes show different variants with many branches and repetitions. They also illustrate that the care for people with rare diseases requires a high level of interdisciplinary collaboration; diagnosis as well as treatment and therapy often take place across sectors and in cooperation between different medical health care institutions and professions. When analyzing the current IT landscape, it becomes clear that IT applications can be used at many process steps in the care for people with rare diseases and have a high potential. Therefore, they must be used to inform decisions about the adequate diagnosis and treatment as well as communication about the clinical pictures and the patient's case between practitioners and medical care sectors. CONCLUSION: The interdisciplinary collaboration highlights the need for cooperation between the various parties involved in the process, which requires the identification and implementation of interfaces between the stakeholders and their systems. However, it is not enough to include the view of the processes; the data perspective is also required. Creating interoperability also enables the use of IT applications. The basis for this is the results obtained.

Development of a Dashboard for Rare Diseases - A Technical Case Report.

About 30 million people suffer from a rare disease in Europe. Those affected face a variety of problems. These include the lack of information and difficult access to scientific knowledge for physicians. For a higher visibility of rare diseases and high-quality research, effective documentation and use of data are essential. The aim of this work is to optimize the processing, use and accessibility of data on rare diseases and thus increase the added value from existing information. While dashboards are already being used to visualize clinical data, it is unclear what requirements are prevalent for rare diseases and how these can be implemented with available development tools so that a highly accepted dashboard can be designed. For this purpose, based on an analysis of the current situation and a requirements analysis, a prototype dashboard for the visualization of up-to-date key figures on rare diseases was developed at the University Hospital Carl Gustav Carus in Dresden. The development was based on the user-centered design process in order to achieve a high-level user-friendliness. The requirements analysis identified parameters that stakeholders wanted to see, focusing primarily on statistical analyses. The dashboard handles the automated calculation of statistics as well as their preparation and provision. The evaluations showed the prototypical dashboard would be considered valuable and used by potential users. This work demonstrates that stakeholders are interested in access to prepared information and exemplifies a way to implement it. The dashboard can increase the usage of existing information in terms of a higher accessibility and thus improve the knowledge about rare diseases.

Evaluation of Three Feasibility Tools for Identifying Patient Data and Biospecimen Availability: Comparative Usability Study.

BACKGROUND: To meet the growing importance of real-word data analysis, clinical data and biosamples must be timely made available. Feasibility platforms are often the first contact point for determining the availability of such data for specific research questions. Therefore, a user-friendly interface should be provided to enable access to this information easily. The German Medical Informatics Initiative also aims to establish such a platform for its infrastructure. Although some of these platforms are actively used, their tools still have limitations. Consequently, the Medical Informatics Initiative consortium MIRACUM (Medical Informatics in Research and Care in University Medicine) committed itself to analyzing the pros and cons of existing solutions and to designing an optimized graphical feasibility user interface. OBJECTIVE: The aim of this study is to identify the system that is most user-friendly and thus forms the best basis for developing a harmonized tool. To achieve this goal, we carried out a comparative usability evaluation of existing tools used by researchers acting as end users. METHODS: The evaluation included three preselected search tools and was conducted as a qualitative exploratory study with a randomized design over a period of 6 weeks. The tools in question were the MIRACUM i2b2 (Informatics for Integrating Biology and the Bedside) feasibility platform, OHDSI's (Observational Health Data Sciences and Informatics) ATLAS, and the Sample Locator of the German Biobank Alliance. The evaluation was conducted in the form of a web-based usability test (usability walkthrough combined with a web-based questionnaire) with participants aged between 26 and 63 years who work as medical doctors. RESULTS: In total, 17 study participants evaluated the three tools. The overall evaluation of usability, which was based on the System Usability Scale, showed that the Sample Locator, with a mean System Usability Scale score of 77.03 (SD 20.62), was significantly superior to the other two tools (Wilcoxon test; Sample Locator vs i2b2: P=.047; Sample Locator vs ATLAS: P=.001). i2b2, with a score of 59.83 (SD 25.36), performed significantly better than ATLAS, which had a score of 27.81 (SD 21.79; Wilcoxon test; i2b2 vs ATLAS: P=.005). The analysis of the material generated by the usability walkthrough method confirmed these findings. ATLAS caused the most usability problems (n=66), followed by i2b2 (n=48) and the Sample Locator (n=22). Moreover, the Sample Locator achieved the highest ratings with respect to additional questions regarding satisfaction with the tools. CONCLUSIONS: This study provides data to develop a suitable basis for the selection of a harmonized tool for feasibility studies via concrete evaluation and a comparison of the usability of three different types of query builders. The feedback obtained from the participants during the usability test made it possible to identify user problems and positive design aspects of the individual tools and compare them qualitatively.

Evaluation of a clinical decision support system for rare diseases: a qualitative study.

BACKGROUND: Rare Diseases (RDs) are difficult to diagnose. Clinical Decision Support Systems (CDSS) could support the diagnosis for RDs. The Medical Informatics in Research and Medicine (MIRACUM) consortium developed a CDSS for RDs based on distributed clinical data from eight German university hospitals. To support the diagnosis for difficult patient cases, the CDSS uses data from the different hospitals to perform a patient similarity analysis to obtain an indication of a diagnosis. To optimize our CDSS, we conducted a qualitative study to investigate usability and functionality of our designed CDSS. METHODS: We performed a Thinking Aloud Test (TA-Test) with RDs experts working in Rare Diseases Centers (RDCs) at MIRACUM locations which are specialized in diagnosis and treatment of RDs. An instruction sheet with tasks was prepared that the participants should perform with the CDSS during the study. The TA-Test was recorded on audio and video, whereas the resulting transcripts were analysed with a qualitative content analysis, as a ruled-guided fixed procedure to analyse text-based data. Furthermore, a questionnaire was handed out at the end of the study including the System Usability Scale (SUS). RESULTS: A total of eight experts from eight MIRACUM locations with an established RDC were included in the study. Results indicate that more detailed information about patients, such as descriptive attributes or findings, can help the system perform better. The system was rated positively in terms of functionality, such as functions that enable the user to obtain an overview of similar patients or medical history of a patient. However, there is a lack of transparency in the results of the CDSS patient similarity analysis. The study participants often stated that the system should present the user with an overview of exact symptoms, diagnosis, and other characteristics that define two patients as similar. In the usability section, the CDSS received a score of 73.21 points, which is ranked as good usability. CONCLUSIONS: This qualitative study investigated the usability and functionality of a CDSS of RDs. Despite positive feedback about functionality of system, the CDSS still requires some revisions and improvement in transparency of the patient similarity analysis.

Development and Evaluation of a Web-Based Paediatric Drug Information System for Germany.

Background: Off-label use is frequent in paediatrics but that does not necessarily mean that the risk-benefit ratio is negative. Nevertheless, evidence-based data is essential for safe drug therapy. In Germany, there is no publicly available compendium providing transparent, evidence-based information for paediatric pharmacotherapy to date. This work describes the development of a web-based paediatric drug information system (PDIS) for Germany and its evaluation by health care professionals (HCP). Methods: Since 2012, a PDIS is being developed by the authors and is supported by the Federal Ministry of Health since 2016. Dosing recommendations were established based on systematic literature reviews and subsequent evaluation by clinical experts. The prototype was evaluated by HCP. Based on the results, the further development was concluded. Results: 92% of HCP believed that the PDIS could improve the quality of prescribing, as currently available information is deficient. Besides the license and formulations, dosing recommendations were the most relevant modules. A dosage calculator was the most wanted improvement. To facilitate sustainability of future development, a collaboration with the Dutch Kinderformularium was established. As of 2021, the database will be available to German HCP. Conclusion: The fundamentals for a German PDIS were established, and vital steps were taken towards successful continuation.

[Evaluation of a future scenario concerning the use of big data applications to improve the care of people with rare diseases]. / Evaluation eines Zukunftsszenarios zur Nutzung von Big-Data-Anwendungen für die Verbesserung der Versorgung von Menschen mit seltenen Erkrankungen.

INTRODUCTION: In Germany there are about 4 million people living with a rare disease. Studies have shown that big data applications can improve diagnosis of and research on rare diseases more effectively. However, no concrete comprehensive concept for the use of big data in the care of people with rare diseases has so far been established in Germany. As part of the project "BIDA-SE", which is funded by the German Ministry of Health, a first scenario has been designed to show how big data applications can be usefully incorporated into the care of people with rare diseases. METHODS: The aim of the present study was to evaluate this scenario with regard to acceptance, (clinical) benefits, economic aspects, and limitations and barriers to its implementation. To evaluate the scenario, an online survey was conducted in Germany in October/November 2019 amongst a total of N = 9 physicians, N = 69 patients with rare diseases/patient representatives, N = 14 IT experts and N = 21 health care researchers. The online questionnaire consisted of both standardized, validated questions taken from already tested survey instruments and additional questions which were constructed on the basis of a preceding literature analysis. The evaluation of the survey was primarily descriptive, with a calculation of frequencies, mean values and standard deviations. RESULTS: The results of the evaluation show that the scenario has been accepted by a majority of all groups surveyed (physicians, patients/patient representatives, IT experts and health care researchers). From the point of view of physicians, patients/patient representatives and health care researchers, the scenario has the potential to accelerate the diagnosis and initiation of therapy and to improve cross-sectoral treatment. From the physician's and health care researcher's perspective, investments in the application presented in the scenario would be profitable. Financing the scenario would, however, require adjusting the reimbursement situation. The limitations and barriers identified by all groups for a medium-term implementation of the scenario can be grouped into seven thematic areas where action is needed: (1) financing and investment, (2) data protection and data security, (3) standards/data sources/data quality, (4) acceptance of technology, (5) integration into the daily work routine, (6) knowledge about availability as well as (7) habits and preferences/physician's role. DISCUSSION: With the present study, a first interdisciplinary, practical scenario using big data applications was evaluated with regard to acceptance, benefits and limitations/barriers. The scenario is widely accepted among the majority of all surveyed target groups and is considered (clinically) useful, although legal, organisational and technical barriers still need to be overcome for its medium-term implementation. The evaluation results contribute to the derivation of recommendations for action to ensure the medium-term implementation of the scenario and to channel access to the Centres for Rare Diseases in the future. CONCLUSION: Many activities have been initiated at a national level to improve the health care situation of people with rare diseases. The scenario developed in the "BIDA-SE" project complements these research activities and illustrates how big data applications can be usefully implemented into practice to improve the diagnosis and therapy of people with rare diseases in a sustainable way.

Implementation of patient-reported outcome assessment in routine cancer care: A systematic review of multicentric programs in Europe.

INTRODUCTION: To give an overview of patient-reported outcome measures (PROMs) programs in routine cancer care that allow for both major purposes of PROM assessment: 1) monitoring of an individual patient's outcome to assist treatment decision making, and 2) use in quality improvement initiatives including the benchmarking of providers. We synthesize information on program elements like the mode of assessment and questionnaire used, as well as information relevant for adaptation following a PDCA scheme. METHOD: We carried out a systematic literature research in the databases PubMed and EMBASE using MeSH terms and keywords related to PROM assessment in routine cancer care to identify eligible studies published between January 2003 and November 2018 (PROSPERO reg. no. CRD42019141402). We included studies in which PROM assessment programs had been reported as being implemented in clinical practice as well as collected multicentrically with at least one site in Europe and in which PROMs had been collected before and at least once after intervention. Study authors were queried to verify or correct the program elements extracted and merged during the review. Study quality assessment was not done, since it is not expedient for the objective of this review. RESULTS: Overall, 5,545 unique references were identified, 5,416 of which were excluded after the screening of titles and abstracts. Of the 29 references assessed, five programs were identified and included in the synthesis. The programs included those from Germany, Austria, Denmark, the Netherlands and the UK, and patients with different cancer types and tumor stages, which used paper-based or purely electronic PROM assessment. DISCUSSION AND CONCLUSION: Few reports have so far been published on PROM programs that allow for both the monitoring of an individual patient's outcome and use in quality improvement initiatives. The studies revealed relevant information on existing PROM programs and gave valuable insight into issues that need to be considered when setting up such an infrastructure. Some critical issues, however, were hardly addressed, among them costs, staff resources and methods of reporting and responding.

User-Centered Development of an Online Platform for Drug Dosing Recommendations in Pediatrics.

BACKGROUND: Drug therapy in pediatric patients is a complex process. Children are subject to continuous growth and variation in drug-metabolizing enzyme activity, requiring continuous adaption of dosages. In Germany, currently no publicly available database exists that provides evidence-based information on drug dosages in pediatrics. For local drug dosing support, a prototype database has been developed within the Children's Hospital, Erlangen. A user-centered development process was initiated to establish an online platform for evidence-based dosing recommendations, as well as pharmacological and pharmaceutical drug information in pediatrics. OBJECTIVES: The objectives of the study were to survey the demand for such a platform and to assess the usability of the different versions of the developed system. METHODS: The developed prototype was evaluated in a pluralistic walkthrough with prospective end users. After a redesign, the second prototype of the online platform underwent an online usability testing based on a tailored questionnaire and the System Usability Scale (SUS) (n = 12). RESULTS: Eleven of 12 participants expressed a demand for an online platform for pediatric dosing recommendations. The majority of the participants requested the integration of extended features, such as drug-drug interaction alerts, or information on adverse effects, pharmacokinetics, and pharmacodynamics. Particularly noteworthy is the demand for an online calculator; 5 of a total of 15 participants explicitly requested a calculator for dosages (based on age, weight, body surface) and glomerular filtration rate. The usability of the second prototype was rated "good to excellent" with a median SUS of 81.25. CONCLUSION: Local domain experts demand an online platform for pediatric dosing recommendations. The application of the user-centered design approach enabled the development of a prototype suitable for practical use. Multiple additional required functionalities have been identified, whereby the importance of an online calculator for patient-individual dosing recommendations was particularly emphasized.

A Digital Cognitive Aid for Anesthesia to Support Intraoperative Crisis Management: Results of the User-Centered Design Process.

BACKGROUND: Stressful situations during intraoperative emergencies have negative impact on human cognitive functions. Consequently, task performance may decrease and patient safety may be compromised. Cognitive aids can counteract these effects and support anesthesiologists in their crisis management. The Professional Association of German Anesthesiologists set up a project to develop a comprehensive set of digital cognitive aids for intraoperative emergencies. A parallel development for several software platforms and stationary and mobile devices will accommodate the inhomogeneity of the information technology infrastructure within German anesthesia departments. OBJECTIVE: This paper aimed to provide a detailed overview of how the task of developing a digital cognitive aid for intraoperative crisis management in anesthesia was addressed that meets user requirements and is highly user-friendly. METHODS: A user-centered design (UCD) process was conducted to identify, specify, and supplement the requirements for a digital cognitive aid. The study covered 4 aspects: analysis of the context of use, specification of user requirements, development of design solutions, and evaluation of design solutions. Three prototypes were developed and evaluated by end users of the application. Following each evaluation, the new requirements were prioritized and used for redesign. For the first and third prototype, the System Usability Scale (SUS) score was determined. The second prototype was evaluated with an extensive Web-based questionnaire. The evaluation of the third prototype included a think-aloud protocol. RESULTS: The chosen methods enabled a comprehensive collection of requirements and helped to improve the design of the application. The first prototype achieved an average SUS score of 74 (SD 12), indicating good usability. The second prototype included the following main revisions: 2-column layout, initial selection of patient type (infant, adult, or parturient), 4 offered search options, and the option to check off completed action steps. Its evaluation identified the following major revision points: add quick selection for resuscitation checklists, design the top bar and tabs slightly larger, and add more pictograms to the text. The third prototype achieved an average SUS score of 77 (SD 15). The evaluation of the think-aloud protocol revealed a good intuitiveness of the application and identified a missing home button as the main issue. CONCLUSIONS: Anesthesiology-as an acute medical field-is particularly characterized by its high demands on decision making and action in dynamic, or time-critical situations. The integration of usability aspects is essential for everyday and emergency suitability. The UCD process allowed us to develop a prototypical digital cognitive aid, exhibiting high usability and user satisfaction in the demanding environment of anesthesiological emergencies. Both aspects are essential to increase the acceptance of the application in later stages. The study approach, combining different methods for determining user requirements, may be useful for other implementation projects in a highly demanding environment.

User-centered design of a mobile medication management.

BACKGROUND: The use of a nationwide medication plan has been promoted as an effective strategy to improve patient safety in Germany. However, the medication plan only exists as a paper-based version, which is related to several problems, that could be circumvented by an electronic alternative. OBJECTIVE: The main objective of this study was to report on the development of a mobile interface concept to support the management of medication information. METHODS: The human-centered design (UCD) process was chosen. First the context of use was analyzed, and personas and an interaction concept were designed. Next, a paper prototype was developed and evaluated by experts. Based on those results, a medium-fidelity prototype was created and assessed by seven end-users who performed a thinking-aloud test in combination with a questionnaire based on the System Usability Scale (SUS). RESULTS: Initially for one persona/user type, an interface design concept was developed, which received an average SUS-Score of 92.1 in the user test. Usability problems have been solved so that the design concept could be fixed for a future implementation. Contribution: The approach of the UCD process and the methods involved can be applied by other researchers as a framework for the development of similar applications.