Correction: Online Guide for Electronic Health Evaluation Approaches: Systematic Scoping Review and Concept Mapping Study.

[This corrects the article DOI: .].

Online Guide for Electronic Health Evaluation Approaches: Systematic Scoping Review and Concept Mapping Study.

BACKGROUND: Despite the increase in use and high expectations of digital health solutions, scientific evidence about the effectiveness of electronic health (eHealth) and other aspects such as usability and accuracy is lagging behind. eHealth solutions are complex interventions, which require a wide array of evaluation approaches that are capable of answering the many different questions that arise during the consecutive study phases of eHealth development and implementation. However, evaluators seem to struggle in choosing suitable evaluation approaches in relation to a specific study phase. OBJECTIVE: The objective of this project was to provide a structured overview of the existing eHealth evaluation approaches, with the aim of assisting eHealth evaluators in selecting a suitable approach for evaluating their eHealth solution at a specific evaluation study phase. METHODS: Three consecutive steps were followed. Step 1 was a systematic scoping review, summarizing existing eHealth evaluation approaches. Step 2 was a concept mapping study asking eHealth researchers about approaches for evaluating eHealth. In step 3, the results of step 1 and 2 were used to develop an "eHealth evaluation cycle" and subsequently compose the online "eHealth methodology guide." RESULTS: The scoping review yielded 57 articles describing 50 unique evaluation approaches. The concept mapping study questioned 43 eHealth researchers, resulting in 48 unique approaches. After removing duplicates, 75 unique evaluation approaches remained. Thereafter, an "eHealth evaluation cycle" was developed, consisting of six evaluation study phases: conceptual and planning, design, development and usability, pilot (feasibility), effectiveness (impact), uptake (implementation), and all phases. Finally, the "eHealth methodology guide" was composed by assigning the 75 evaluation approaches to the specific study phases of the "eHealth evaluation cycle." CONCLUSIONS: Seventy-five unique evaluation approaches were found in the literature and suggested by eHealth researchers, which served as content for the online "eHealth methodology guide." By assisting evaluators in selecting a suitable evaluation approach in relation to a specific study phase of the "eHealth evaluation cycle," the guide aims to enhance the quality, safety, and successful long-term implementation of novel eHealth solutions.

Professionals' Use of a Multidisciplinary Communication Tool for Patients With Dementia in Primary Care.

In this descriptive study, the use of a professional e-communication tool, Congredi, is evaluated. Ninety-six Congredi records of patients with dementia could be divided into the subgroups low-complex care (n = 43) and high-complex care (n = 53). If Congredi is an adequate communication tool for professionals, the changing involvement of caregivers must also be reflected within the two subgroups. We hypothesized that use would be more intensive in the high-complex group in comparison with the low-complex group. Data were gathered during 42 weeks. Results showed that the mean number of care activities in the high-complex group was significantly higher than in the low-complex group (10.43 vs 5.61, P = .001). The number of professionals involved with the high-complex care group (3.58) was higher compared to the low-complex care group (2.51) (P = .000). The most frequent use was by case managers and nurses (43.4%) in the high-complex group and by several case managers (41.9%) in the low-complex group. It was concluded that professionals used Congredi adequately in the multidisciplinary care of patients with dementia because the changing involvement of caregivers and the level of care activities were reflected in the use of Congredi.

Evaluation of a tertiary teledermatology service between peripheral and academic dermatologists in the Netherlands.

BACKGROUND: Tertiary teledermatology (TTD)-secondary-care to tertiary-care teleconsultation-is applied rarely compared with the frequently applied secondary teledermatology (primary to secondary care). The objective of this study was to determine the effect of TTD on referrals from peripheral dermatologists to the tertiary center and to evaluate acceptance of TTD. MATERIALS AND METHODS: From May 2010 to May 2012, 39 dermatologists could send in teleconsultations to one of two tertiary centers. Physical referrals to the tertiary centers prevented by TTD were calculated based on questions before and after TTD. The acceptance of the TTD system was evaluated through questionnaires, a focus group meeting, and personal interviews. RESULTS: Eighty-five teleconsultations were sent by 13 peripheral dermatologists from nine care institutions and answered by 8 tertiary dermatologists. Of the patients, 62% (n=53) would have been referred physically to the tertiary center if teledermatology were not available. In this group, teledermatology prevented 81% (n=43) of physical referrals. Dermatologists indicated that TTD had important advantages such as fast response time, formalized records, and data and privacy security. However, the current work process using telephone and e-mail was preferred because of its ease of use and direct personal network. The following conditions that could lead to successful implementation were indicated: (1) a national TTD system including all dermatologists indexed according their subspecialty, (2) ability to send the TTD consultation to a dermatologist personally, (3) ability to discuss a case with multiple dermatologists, (4) connections to electronic health records, and (5) change in policies of tertiary centers or health insurers, where they would stimulate the use of TTD consultation for all referrals and questions. CONCLUSIONS: Although quantitative results indicate that TTD can be used to improve triage between secondary and tertiary centers and dermatologists perceived advantages of TTD, the motivation to use TTD in this setting was lacking as current work processes were easier to use.

The Store-and-Forward Telemedicine Service User-satisfaction Questionnaire: Development and validation of a questionnaire to monitor and assess health care providers' experiences.

INTRODUCTION: With wider adoption of store-and-forward telemedicine accelerating, particularly post-coronavirus disease 2019, it is essential to understand health care providers' (HCPs) satisfaction with digital solutions offered by telemedicine organizations to (continuously) guarantee service quality. We developed the Store-and-Forward Telemedicine Service User-satisfaction Questionnaire to monitor and assess HCPs' experiences with contracted telemedicine organizations. METHODS: Questionnaire construction (phase 1) consisted of exploratory literature search on validated telemedicine satisfaction questionnaires, a telemedicine domain and human factors expert focus group, stakeholder focus group (customer service employee and telemedicine account managers), and two pre-testing rounds among 18 HCPs. The pilot questionnaire (phase 2) was sent to 2179 HCPs for validity and reliability assessment. RESULTS: Phase 1: Two validated questionnaires (73 items overall) were used as input for Store-and-Forward Telemedicine Service User-satisfaction Questionnaire. Revisions resulted in 61 items. Phase 2: the pilot 61-item Store-and-Forward Telemedicine Service User-satisfaction Questionnaire instrument was completed by 181 of 2179 invited HCPs. Forty-one mandatory items of the pilot Store-and-Forward Telemedicine Service User-satisfaction Questionnaire rated on a 5-point Likert scale were included in psychometric analyses and resulted in six reliable scales: training, communication, organization policy and strategy, interaction platform, usage platform, and working conditions. DISCUSSION: The Store-and-Forward Telemedicine Service User-satisfaction Questionnaire is a reliable and valid questionnaire for measuring HCPs' satisfaction with store-and-forward telemedicine services as part of a continuous quality improvement cycle. Reimbursement questions were excluded due to low response. As adoption of telemedicine may be impeded by financial compensation issues, this requires consideration in future telemedicine questionnaires. Store-and-Forward Telemedicine Service User-satisfaction Questionnaire including video consultation items is needed to monitor also synchronous services as these expanded in the coronavirus disease 2019 pandemic.

Tertiary teledermatology: a systematic review.

Telemedicine is becoming widely used in healthcare. Dermatology, because of its visual character, is especially suitable for telemedicine applications. Most common is teledermatology between general practitioners and dermatologists (secondary teledermatology). Another form of the teledermatology process is communication among dermatologists (tertiary teledermatology). The objective of this systematic review is to give an overview of studies on tertiary teledermatology with emphasis on the categories of use. A systematic literature search on tertiary teledermatology studies used all databases of the Cochrane Library, MEDLINE (1966-November 2007) and EMBASE (1980-November 2007). Categories of use were identified for all included articles and the modalities of tertiary teledermatology were extracted, together with technology, the setting the outcome measures, and their results. The search resulted in 1,377 publications, of which 11 were included. Four categories of use were found: getting an expert opinion from a specialized, often academic dermatologist (6/11); resident training (2/11); continuing medical education (4/11); and second opinion from a nonspecialized dermatologist (2/11). Three modalities were found: a teledermatology consultation application (7/11), a Web site (2/11), and an e-mail list (1/11). The majority (7/11) used store-and-forward, and 3/11 used store-and-forward and real-time. Outcome measures mentioned were learning effect (6), costs (5), diagnostic accuracy (1), validity (2) and reliability (2), patient and physician satisfaction (1), and efficiency improvement (3). Tertiary teledermatology's main category of use is getting an expert opinion from a specialized, often academic dermatologist. Tertiary teledermatology research is still in early development. Future research should focus on identifying the scale of tertiary teledermatology and on what modality of teledermatology is most suited for what purpose in communication among dermatologists.

The Value of Teledermoscopy to the Expertise of General Practitioners Diagnosing Skin Disorders Based on ICD-10 Coding.

Early recognition of skin cancer is vital to enhance patient outcomes. Teledermoscopy (TDsc), a telemedicine service, supports general practitioners (GPs) in gaining fast access to dermatologists' feedback to detect skin cancer. This study aimed to assess if GPs gain expertise in diagnosing skin disorders after continued use of TDsc, based on diagnosis classification by the International Statistical Classification of Diseases and Related Health Problems (ICD-10). A retrospective study was conducted on TDsc consultations sent by GPs to teledermatologists in the Netherlands (July 2015 - June 2018). GP sensitivity and confirmed cases in diagnosing skin disorders slightly increased over time. However, the total positive predictive value showed a decrease. In three years, 43 melanomas were diagnosed by the TD for which the GP did not provide a (correct) pre-diagnose. Though GPs appear to improve their expertise in skin disorder detection after continued TDsc use, TDsc remains imperative to early melanoma detection.

Is Teledermoscopy Improving General Practitioner Skin Cancer Care?

Skin cancer incidences have tripled in the Netherlands for the last twenty years and are expected to increase even more in the coming years. Teledermoscopy (TDsc) is implemented in Dutch practice to support and enhance early skin cancer detection by general practitioners (GPs) through remote consultation with dermatologists. This study assesses the effect of TDsc consultation on the quality and efficiency of skin cancer care in the primary setting by analyzing 10,184 TDsc consultations.

Improving cooperation between general practitioners and dermatologists via telemedicine: study protocol of the cluster-randomized controlled TeleDerm study.

BACKGROUND: Internationally, teledermatology has proven to be a viable alternative to conventional physical referrals. Travel cost and referral times are reduced while patient safety is preserved. Especially patients from rural areas benefit from this healthcare innovation. Despite these established facts and positive experiences from EU neighboring countries like the Netherlands or the United Kingdom, Germany has not yet implemented store-and-forward teledermatology in routine care. METHODS: The TeleDerm study will implement and evaluate store-and-forward teledermatology in 50 general practitioner (GP) practices as an alternative to conventional referrals. TeleDerm aims to confirm that the possibility of store-and-forward teledermatology in GP practices is going to lead to a 15% (n = 260) reduction in referrals in the intervention arm. The study uses a cluster-randomized controlled trial design. Randomization is planned for the cluster "county". The main observational unit is the GP practice. Poisson distribution of referrals is assumed. The evaluation of secondary outcomes like acceptance, enablers and barriers uses a mixed-methods design with questionnaires and interviews. DISCUSSION: Due to the heterogeneity of GP practice organization, patient management software, information technology service providers, GP personal technical affinity and training, we expect several challenges in implementing teledermatology in German GP routine care. Therefore, we plan to recruit 30% more GPs than required by the power calculation. The implementation design and accompanying evaluation is expected to deliver vital insights into the specifics of implementing telemedicine in German routine care. TRIAL REGISTRATION: German Clinical Trials Register, DRKS00012944 . Registered prospectively on 31 August 2017.

Automatic differentiation of color fundus images containing drusen or exudates using a contextual spatial pyramid approach.

We developed an automatic system to identify and differentiate color fundus images containing no lesions, drusen or exudates. Drusen and exudates are lesions with a bright appearance, associated with age-related macular degeneration and diabetic retinopathy, respectively. The system consists of three lesion detectors operating at pixel-level, combining their outputs using spatial pooling and classification with a random forest classifier. System performance was compared with ratings of two independent human observers using human-expert annotations as reference. Kappa agreements of 0.89, 0.97 and 0.92 and accuracies of 0.93, 0.98 and 0.95 were obtained for the system and observers, respectively.

The course of chronic plaque-type psoriasis in placebo groups of randomized controlled studies.

OBJECTIVE: To determine the outcome in placebo-treated patients with plaque-type psoriasis. DATA SOURCES: Online search of MEDLINE and EMBASE until January 2001 and the Cochrane Library (2001, issue 1), supplemented by references, reviews, guidelines, and textbooks. STUDY SELECTION: Randomized controlled induction of remission trials of patients with chronic plaque-type psoriasis with systemic treatments with a placebo group not treated with antipsoriatic medication. Identified studies were examined by 2 independent reviewers. Through MEDLINE, 290 studies could be identified. Twenty-seven placebo-controlled studies were included (488 patients). DATA EXTRACTION: Two independent reviewers extracted data on first author, year of publication, design, comparison, placebo treatment, number of patients, treatment duration, type of psoriasis and baseline severity in the placebo group, mean relative change in outcome measures, and/or percentage of patients with worsening of psoriasis; no change; minimal, moderate, or good improvement; or complete clearance. DATA SYNTHESIS: Owing to substantial heterogeneity and differences in the way outcomes were reported, no summary estimates could be obtained. The outcome of placebo treatment was poor in most studies. Some reported a mean relative change of 11% to 47%. The highest percentages of patients ended up in the worsening, no change, or minimal improvement categories. Also, complete clearance was possible. No explanation for the differences in outcome between placebo groups could be found. Description of placebo groups was often insufficient. CONCLUSIONS: The effect of treatment in placebo groups varied across studies in an unpredictable way. To evaluate the variability, improvement of the standardization of study designs, entry criteria, and outcome measures is necessary in psoriasis trials.

Telepulmonology and telespirometry.

Taking and interpreting spirometry tests has proven difficult in primary care practice. This may lead to mis- or underdiagnosis of pulmonary diseases, among others chronic obstructive pulmonary disease. Telespirometry and telepulmonology consultation (TPC) may play a role in monitoring and improving the quality of the spirometry tests, supporting GPs in interpreting spirometry test results and reducing the number of physical referrals to the pulmonologist. In telespirometry up to 10% of spirometry results uploaded by GPs were randomly sent to a pulmonologist. Both the GP or practice nurse and the pulmonologist interpreted the spirometry results and gave their diagnostic findings. Additionally the pulmonologist assessed the quality of the test. In TPC a GP could digitally consult a pulmonologist for advice or referral of patients. On sending and closing the TPC consult the GP was presented a number of questions. Based on these questions the percentage of prevented physical referrals and the educational effect experienced by the GPs were determined. Almost a third of the 227 telespirometry tests was of Moderate or Bad quality. The Kappa of the interobserver agreement on diagnostic findings between GP and pulmonologist was 0.38. Between April 2009 and January 2014, GPS sent 4.488 TPCs to pulmonologists. Sixty-nine percent of the TPCs were sent to gain advice, the others were sent in order to prevent a physical referral. Overall telepulmonology reduced the number of physical referrals by 22%. In 90% of the TPCs the GPs indicated they had learned from the consult.

Telepulmonology: effect on quality and efficiency of care.

BACKGROUND: Interpreting spirometry results has proven challenging in primary care practice, among others potentially leading to under- and misdiagnosis of COPD. In telepulmonology a general practitioner (GP) digitally consults a pulmonologist to support the interpretation of spirometry results. This study assessed the effect of telepulmonology on quality and efficiency of care. METHODS: Quality of care was measured by five indicators, among others the percentage of TelePulmonology Consultations (TPCs) sent by GPs for advice, percentage of those TPCs resulting in a physical referral, and educational effect of telepulmonology as experienced by GPs. Efficiency was defined as the percentage of prevented unnecessary physical referrals of patients to the pulmonologist. RESULTS: Between April 2009 and November 2012 1.958 TPCs were sent by 158 GPs to 32 pulmonologists. Sixty-nine percent of the TPCs were sent for advice. Based on the advice of the pulmonologist 18% of these TPCs led to a physical referral of patients who would not have been referred without telepulmonology. Thirty-one percent of the TPCs were intended to prevent a physical referral, 68% of these actually prevented a physical referral to a pulmonologist. CONCLUSION: The results show telepulmonology can contribute to quality of care by supporting GPs and can additionally prevent unnecessary physical referrals.

Telepulmonology in the Netherlands: effect on quality and efficiency of care.

In telepulmonology a general practitioner (GP) digitally consults a local pulmonologist. This study assessed the effect of telepulmonology on quality and efficiency of care. Efficiency of care was measured as the percentage of prevented physical referrals. Quality of care was measured using 5 indicators. Thirty-one percent of the TelePulmonology Consultations (TPCs) were sent to prevent a physical referral, the other TPCs were sent to ask for advice of the pulmonologist. Sixty-eight percent of the TPCs sent to prevent a physical referral indeed prevented a physical referral. Eighteen percent of the TPCs sent for advice resulted in a physical referral on advice of the pulmonologist. These patients would not have been referred without telepulmonology.

Accuracy and reliability of teledermatoscopy with images taken by general practitioners during everyday practice.

We assessed the accuracy and reliability of teledermatoscopy with images taken by a general practitioner (GP) compared to face-to-face dermatological examination. GPs selected patients for teledermatoscopy and took both macro and dermatoscopic photographs. Patients were then referred to the local dermatologist for face-to-face examination. Accuracy and inter-observer reliability were calculated for the diagnosis and management plan. Image quality was rated by two observers on a three-point scale. A total of 108 teledermatoscopy consultations sent by 13 GPs were assessed by four dermatologists. Agreement was 0.61 (kappa) on diagnostic group and 0.23 on management plan. The inter-observer reliability was 0.65 on diagnostic group and 0.36 on management plan. The image quality was reported as bad in 36% of cases, reasonable in 28% and good in 36%. Agreement for cases with good quality images was 0.66 on diagnostic group and 0.42 on management plan. Teledermatoscopy in general practice had overall a lower accuracy and reliability than face-to-face consultation. In cases where a good quality image was reported, the accuracy increased, which emphasises that teledermatoscopy is highly dependent on a good quality images.

A pilot study on tertiary teledermatology: feasibility and acceptance of telecommunication among dermatologists.

Tertiary teledermatology (TTD), where a general dermatologist consults a specialized dermatologist on difficult cases, is a relatively new telemedicine service. We evaluated TTD in a Dutch university hospital, where 13 general dermatologists used TTD to consult 11 specialized dermatologists and two residents at the university medical centre. We measured the avoided referrals to the university centre, the usability of the system and the user acceptance of it. During a three-month study, general dermatologists consulted via TTD 28 times. In 17 of the consultations (61%), the general dermatologists would have referred their patients to the university centre if teledermatology had not been available. Referral was not necessary after teledermatology for 12 of these 17 consultations (71%). The mean usability score (0-100) of all the users was 80. All dermatologists were satisfied with TTD (mean satisfaction of 7.6 on a 10-point scale) and acceptance was high. The baseline measurements showed that half of tertiary referrals were suitable for TTD. These results suggest that TTD reduces unnecessary physical referrals and that users are satisfied with it. A large-scale evaluation is now required.