Development of a localization-based algorithm for the prediction of leg ulcer etiology.

BACKGROUND: Diagnostic work-up of leg ulcers is time- and cost-intensive. This study aimed at evaluating ulcer location as a diagnostic criterium and providing a diagnostic algorithm to facilitate differential diagnosis. PATIENTS AND METHODS: The study consisted of 277 patients with lower leg ulcers. The following five groups were defined: Venous leg ulcer, arterial ulcers, mixed ulcer, arteriolosclerosis, and vasculitis. Using computational surface rendering, predilection sites of different ulcer types were evaluated. The results were integrated in a multinomial logistic regression model to calculate the likelihood of a specific diagnosis depending on location, age, bilateral involvement, and ulcer count. Additionally, neural network image analysis was performed. RESULTS: The majority of venous ulcers extended to the medial malleolar region. Arterial ulcers were most frequently located on the dorsal aspect of the forefoot. Arteriolosclerotic ulcers were distinctly localized at the middle third of the lower leg. Vasculitic ulcers appeared to be randomly distributed and were markedly smaller, multilocular and bilateral. The multinomial logistic regression model showed an overall satisfactory performance with an estimated accuracy of 0.68 on unseen data. CONCLUSIONS: The presented algorithm based on ulcer location may serve as a basic tool to narrow down potential diagnoses and guide further diagnostic work-up.

Interval cancer after colonoscopy in the Austrian National Screening Programme: influence of physician and patient factors.

OBJECTIVE: Postscreening colorectal cancer (PSCRC) after screening colonoscopy is associated with endoscopists' performance and characteristics of resected lesions. Prior studies have shown that adenoma detection rate (ADR) is a decisive factor for PSCRC, but correlations with other parameters need further analysis and ADR may change over time. DESIGN: Cohort study including individuals undergoing screening colonoscopy between 1/2008 and 12/2019 performed by physicians participating in a quality assurance programme in Austria. Data were linked with hospitalisation data for the diagnosis of PSCRC (defined as CRC diagnosis >6 months after colonoscopy). ADR was defined dynamically in relation to the time point of subsequent colonoscopies; high-risk groups of patients were those with an adenoma ≥10 mm, or with high-grade dysplasia, or villous or tubulovillous histology, or a serrated lesion ≥10 mm or with dysplasia, or colonoscopies with ≥3 lesions. Main outcome was PSCRC for each risk group (negative colonoscopy, hyperplastic polyps, low-risk and high-risk group of patients) after colonoscopy by endoscopists with an ADR <20% compared with endoscopists with an ADR ≥20%. RESULTS: 352 685 individuals were included in the study (51.0% women, median age 60 years) of which 10.5% were classified as high-risk group. During a median follow-up of 55.4 months, 241 (0.06%) PSCRC were identified; of 387 participating physicians, 19.6% had at least one PSCRC (8.4% two or more). While higher endoscopist ADR decreased PSCRC incidence (HR per 1% increase 0.97, 95% CI 0.95 to 0.98), affiliation to the high-risk group of patients was also associated with higher PSCRC incidence (HR 3.27, 95% CI 2.36 to 4.00). Similar correlations were seen with regards to high-risk, and advanced adenomas. The risk for PSCRC was significantly higher after colonoscopy by an endoscopist with an ADR <20% as compared with an endoscopist with an ADR ≥20% in patients after negative colonoscopy (HR 2.01, 95% CI 1.35 to 3.0, p<0.001) and for the high-risk group of patients (HR 2.51, 95% CI 1.49 to 4.22, p<0.001). CONCLUSION: A dynamic calculation of the ADR takes into account changes over time but confirms the correlation of ADR and interval cancer. Both lesion characteristics and endoscopists ADR may play a similar role for the risk of PSCRC. This should be considered in deciding about appropriate surveillance intervals in the future.

Utilization of potentially inappropriate medication and risk of adverse drug events among older adults with chronic renal insufficiency: a population-wide cohort study.

BACKGROUND: The use of potentially inappropriate medication (PIM) in population of older adults may result in adverse drug events (ADE) already after short term exposure, especially when it is prescribed to patients with chronic kidney disease (CKD). In order to limit ADE in the treatment of older adults PIM lists have been constructed as a source of information for healthcare professionals. The aim of this study was to estimate the utilization of PIM and incidence of ADE in older adults (≥70 years) with CKD. METHODS: We conducted a retrospective population-wide cohort study including patients from Lower Austria who were 70 years or older and diagnosed with CKD in the period from 2008 to 2011. Utilization of PIM was estimated from prescriptions filled by target population. We estimated risks of hospitalization due to ADE within 30 days after incident PIM prescription and compared them to a PIM-free control group by using marginal structural models (MSM). RESULTS: We identified 11,547 patients (women: 50.6%, median age in 2008: 78 years) who fulfilled the inclusion criteria. In total 24.7 and 8.1% of all prescriptions from that period contained a medication with a substance listed in the EU (7)-PIM and AT-PIM list, respectively. Proton pump inhibitors and Ginkgo biloba were the most often prescribed PIMs in this population. 94.6 and 79.3% patients filled at least one EU(7)-PIM and AT-PIM prescription, respectively. Despite the relatively high utilization of PIM there was only a low incidence of clinically relevant ADE. No event type exceeded the threshold level of 1% in the analysis of risks of ADE after filling a prescription for PIM. Nevertheless, MSM analysis showed an increased risk for 11 drugs and reduced risk for 4 drugs. CONCLUSIONS: PIM prescription was common among older adults with CKD, however, only a small number of these drugs eventually led to hospitalization due to ADE within 30 days after incident PIM was filled. In the absence of a clinically important PIM-related increase in risk, an assessment of potential ADE severity to a PIM list by using a warning score system seems prudent.

The role of prescription drugs in female overactive bladder syndrome-A population-wide cohort study.

PURPOSE: Overactive bladder (OAB) syndrome has severe effects on quality of life. Certain drugs are known risk factors for OAB but have not been investigated in a population-wide cohort. The objective of this study was to investigate the role of prescription drugs in the etiology of the OAB. METHODS: Retrospective cohort study using a population-wide database of 4 185 098 OAB-naïve women followed Strengthening the Reporting of Observational Studies in Epidemiology guidelines. We investigated the subscription use of anticholinergic medication and 188 chemical substances, which are suspected triggers for OAB (trigger medications [TMs]). We hypothesized a relationship between the prescription for one or more TM and the prescription for anticholinergic medication against OAB (marker medication [MM]). RESULTS: The use of MM in Austria increased from 2009 to 2012 on average by 0.025 percentage points per year (95% confidence interval [CI]: 0.015-0.036). In December 2012, 1 in 123 women filled a prescription for any MM, equaling an average utilization of 0.84%. The relative risk of filling a prescription for a MM 6 months after filling a prescription for a TM was 2.70 (95% CI: 2.64-2.77). All investigated medication classes showed a higher risk for the prescription for MM. Medication from classes "genitourinary system and sex hormones" and "systemic anti-infectives" caused the highest increase in risk (109% and 89%, respectively). Prescriptions for class "cardiovascular system" caused the lowest increase in the risk (15%). CONCLUSION: Certain prescription medications are a significant risk factor for the need to take anticholinergic medication as a consequence.

Analysis of Collective Human Intelligence for Diagnosis of Pigmented Skin Lesions Harnessed by Gamification Via a Web-Based Training Platform: Simulation Reader Study.

BACKGROUND: The diagnosis of pigmented skin lesion is error prone and requires domain-specific expertise, which is not readily available in many parts of the world. Collective intelligence could potentially decrease the error rates of nonexperts. OBJECTIVE: The aim of this study was to evaluate the feasibility and impact of collective intelligence for the detection of skin cancer. METHODS: We created a gamified study platform on a stack of established Web technologies and presented 4216 dermatoscopic images of the most common benign and malignant pigmented skin lesions to 1245 human raters with different levels of experience. Raters were recruited via scientific meetings, mailing lists, and social media posts. Education was self-declared, and domain-specific experience was tested by screening tests. In the target test, the readers had to assign 30 dermatoscopic images to 1 of the 7 disease categories. The readers could repeat the test with different lesions at their own discretion. Collective human intelligence was achieved by sampling answers from multiple readers. The disease category with most votes was regarded as the collective vote per image. RESULTS: We collected 111,019 single ratings, with a mean of 25.2 (SD 18.5) ratings per image. As single raters, nonexperts achieved a lower mean accuracy (58.6%) than experts (68.4%; mean difference=-9.4%; 95% CI -10.74% to -8.1%; P<.001). Collectives of nonexperts achieved higher accuracies than single raters, and the improvement increased with the size of the collective. A collective of 4 nonexperts surpassed single nonexperts in accuracy by 6.3% (95% CI 6.1% to 6.6%; P<.001). The accuracy of a collective of 8 nonexperts was 9.7% higher (95% CI 9.5% to 10.29%; P<.001) than that of single nonexperts, an improvement similar to single experts (P=.73). The sensitivity for malignant images increased for nonexperts (66.3% to 77.6%) and experts (64.6% to 79.4%) for answers given faster than the intrarater mean. CONCLUSIONS: A high number of raters can be attracted by elements of gamification and Web-based marketing via mailing lists and social media. Nonexperts increase their accuracy to expert level when acting as a collective, and faster answers correspond to higher accuracy. This information could be useful in a teledermatology setting.

Comparison of the accuracy of human readers versus machine-learning algorithms for pigmented skin lesion classification: an open, web-based, international, diagnostic study.

BACKGROUND: Whether machine-learning algorithms can diagnose all pigmented skin lesions as accurately as human experts is unclear. The aim of this study was to compare the diagnostic accuracy of state-of-the-art machine-learning algorithms with human readers for all clinically relevant types of benign and malignant pigmented skin lesions. METHODS: For this open, web-based, international, diagnostic study, human readers were asked to diagnose dermatoscopic images selected randomly in 30-image batches from a test set of 1511 images. The diagnoses from human readers were compared with those of 139 algorithms created by 77 machine-learning labs, who participated in the International Skin Imaging Collaboration 2018 challenge and received a training set of 10 015 images in advance. The ground truth of each lesion fell into one of seven predefined disease categories: intraepithelial carcinoma including actinic keratoses and Bowen's disease; basal cell carcinoma; benign keratinocytic lesions including solar lentigo, seborrheic keratosis and lichen planus-like keratosis; dermatofibroma; melanoma; melanocytic nevus; and vascular lesions. The two main outcomes were the differences in the number of correct specific diagnoses per batch between all human readers and the top three algorithms, and between human experts and the top three algorithms. FINDINGS: Between Aug 4, 2018, and Sept 30, 2018, 511 human readers from 63 countries had at least one attempt in the reader study. 283 (55·4%) of 511 human readers were board-certified dermatologists, 118 (23·1%) were dermatology residents, and 83 (16·2%) were general practitioners. When comparing all human readers with all machine-learning algorithms, the algorithms achieved a mean of 2·01 (95% CI 1·97 to 2·04; p<0·0001) more correct diagnoses (17·91 [SD 3·42] vs 19·92 [4·27]). 27 human experts with more than 10 years of experience achieved a mean of 18·78 (SD 3·15) correct answers, compared with 25·43 (1·95) correct answers for the top three machine algorithms (mean difference 6·65, 95% CI 6·06-7·25; p<0·0001). The difference between human experts and the top three algorithms was significantly lower for images in the test set that were collected from sources not included in the training set (human underperformance of 11·4%, 95% CI 9·9-12·9 vs 3·6%, 0·8-6·3; p<0·0001). INTERPRETATION: State-of-the-art machine-learning classifiers outperformed human experts in the diagnosis of pigmented skin lesions and should have a more important role in clinical practice. However, a possible limitation of these algorithms is their decreased performance for out-of-distribution images, which should be addressed in future research. FUNDING: None.

Feasibility of Mapping Austrian Health Claims Data to the OMOP Common Data Model.

The Main Association of Austrian Social Security Institutions collects pseudonymized claims data from Austrian social security institutions and information about hospital stays in a database for research purposes. For new studies the same data are repeatedly reprocessed and it is difficult to compare different study results even though the data is already preprocessed and prepared in a proprietary data model. Based on a study on adverse drug events in relation to inappropriate medication in geriatric patients the suitability of the Observational Medical Outcomes Partnership (OMOP) common data model (CDM) is analyzed and data is transformed into the OMOP CDM. 1,023 (99.7%) of drug codes and 3,812 (99.2%) of diagnoses codes coincide with the OMOP vocabularies. The biggest obstacles are missing mappings for the Local Vocabularies like the Austrian pharmaceutical registration numbers and the Socio-Economic Index to the OMOP vocabularies. OMOP CDM is a promising approach for the standardization of Austrian claims data. In the long run, the benefits of standardization and reproducibility of research should outweigh this initial drawback.

Accuracy of dermatoscopy for the diagnosis of nonpigmented cancers of the skin.

BACKGROUND: Nonpigmented skin cancer is common, and diagnosis with the unaided eye is error prone. OBJECTIVE: To investigate whether dermatoscopy improves the diagnostic accuracy for nonpigmented (amelanotic) cutaneous neoplasms. METHODS: We collected a sample of 2072 benign and malignant neoplastic lesions and inflammatory conditions and presented close-up images taken with and without dermatoscopy to 95 examiners with different levels of experience. RESULTS: The area under the curve was significantly higher with than without dermatoscopy (0.68 vs 0.64, P < .001). Among 51 possible diagnoses, the correct diagnosis was selected in 33.1% of cases with and 26.4% of cases without dermatoscopy (P < .001). For experts, the frequencies of correct specific diagnoses of a malignant lesion improved from 40.2% without to 51.3% with dermatoscopy. For all malignant neoplasms combined, the frequencies of appropriate management strategies increased from 78.1% without to 82.5% with dermatoscopy. LIMITATIONS: The study deviated from a real-life clinical setting and was potentially affected by verification and selection bias. CONCLUSIONS: Dermatoscopy improves the diagnosis and management of nonpigmented skin cancer and should be used as an adjunct to examination with the unaided eye.

Langzeitauswertung des Nutzens der digitalen Dermatoskopie an einem Referenzzentrum.

HINTERGRUND UND ZIELE: Wir untersuchten den Nutzen der sequentiellen digitalen Dermatoskopie für die Kontrolle von Patienten mit multiplen Nävi in einem spezialisierten Zentrum. PATIENTEN UND METHODIK: Das ist eine retrospektive Kohortenstudie mit 2 824 Patienten und 23 241 kontrollierten Läsionen, wobei für Schlüsselparameter, wie der Anzahl der kontrollierten und exzidierten Melanome und Nävi, eine Trendberechnung durchgeführt wurde. ERGEBNISSE: Im Zuge der Kontrolluntersuchungen haben wir bei 709 Patienten 1 266 Läsionen exzidiert, darunter waren 146 (11,5 %) Melanome. Der Anteil der im Zuge der Kontrolle detektierten in-situ-Melanome war signifikant höher als bei der Erstuntersuchung. (46,6 % gegenüber 23,4 %; p ≤ 0,001). Bei Patienten mit einem Melanom bei der Erstuntersuchung war das Risiko, während der Nachsorge ein Melanom zu detektieren, höher als bei Patienten ohne Melanom (relatives Risiko: 3,59; 95 % CI: 2,15 bis 6,00). Die Zahl der dokumentierten Läsionen korrelierte positiv mit dem Verhältnis von gutartigen zu bösartigen excidierten Läsionen und die Exzisionen bei Erstuntersuchungen gingen während des Untersuchungszeitraums signifikant zurück. SCHLUSSFOLGERUNGEN: Die Überwachung mit digitaler Dermatoskopie verbessert die Detektion dünner Melanome bei Patienten mit multiplen Nävi. Patienten mit einem Melanom im Rahmen der Erstuntersuchung haben ein erhöhtes Risiko, während der Nachsorge ein Melanom zu entwickeln und sollten daher eine Zielgruppe für die sequentielle Dermatoskopie sein.

Long-term evaluation of the efficacy of digital dermatoscopy monitoring at a tertiary referral center.

BACKGROUND AND OBJECTIVES: We examined the value of monitoring patients with multiple nevi using sequential digital dermatoscopy imaging at a tertiary referral center. PATIENTS AND METHODS: This is a retrospective cohort study including 2,824 patients and 23,241 monitored lesions. We calculated trends in key parameters such as the number of melanomas and nevi monitored and excised. RESULTS: During follow-up, we excised 1,266 lesions in 709 patients, including 146 (11.5 %) melanomas. The percentage of in situ melanomas detected at follow-up was significantly higher than at baseline (46.6 % versus 23.4 %, p ≤ 0.001). The risk of detecting a melanoma during follow-up was higher for patients with a melanoma at baseline, compared to those without (relative risk: 3.59, 95 % CI: 2.15 to 6.00). The number of documented lesions showed a positive correlation with the benign/malignant ratio, and excisions at baseline decreased significantly over the course of the study period. CONCLUSION: Digital dermatoscopy monitoring improves the detection of thin melanomas in patients with multiple nevi. Patients with a melanoma at baseline are at an increased risk of developing a melanoma during follow-up and should therefore be a target group for sequential dermatoscopy imaging.

Towards plug-and-play integration of archetypes into legacy electronic health record systems: the ArchiMed experience.

BACKGROUND: The dual model approach represents a promising solution for achieving semantically interoperable standardized electronic health record (EHR) exchange. Its acceptance, however, will depend on the effort required for integrating archetypes into legacy EHR systems. METHODS: We propose a corresponding approach that: (a) automatically generates entry forms in legacy EHR systems from archetypes; and (b) allows the immediate export of EHR documents that are recorded via the generated forms and stored in the EHR systems' internal format as standardized and archetype-compliant EHR extracts. As a prerequisite for applying our approach, we define a set of basic requirements for the EHR systems. RESULTS: We tested our approach with an EHR system called ArchiMed and were able to successfully integrate 15 archetypes from a test set of 27. For 12 archetypes, the form generation failed owing to a particular type of complex structure (multiple repeating subnodes), which was prescribed by the archetypes but not supported by ArchiMed's data model. CONCLUSIONS: Our experiences show that archetypes should be customized based on the planned application scenario before their integration. This would allow problematic structures to be dissolved and irrelevant optional archetype nodes to be removed. For customization of archetypes, openEHR templates or specialized archetypes may be employed. Gaps in the data types or terminological features supported by an EHR system will often not preclude integration of the relevant archetypes. More work needs to be done on the usability of the generated forms.

Evaluating a CDS Hook for FHIR Questionnaires in a SMART on FHIR App and an Existing Dermatological CDS System.

BACKGROUND: To deploy clinical decision support (CDS) systems in routine patient care they have to be certified as a medical device. The European Medical Device Regulation explicitly asks for the use of standards and interoperability in the approval process. OBJECTIVES: We extended an existing dermatological CDS system with emerging standards for CDS interoperability, to facilitate a future integration into existing healthcare infrastructure, and approval as a medical device. METHODS: The data collection part of a CDS system was extended with the endpoints required by the CDS Hooks specification. FHIR QuestionnaireResponse resources trigger a newly defined hook. RESULTS: One hundred and seventeen clinical observations and patient variables needed for the ranking of a disease were mapped to SNOMED CT or LOINC and modeled as FHIR Questionnaire which is rendered using LHC LForms in a SMART on FHIR app with the SMART Dev Sandbox. CONCLUSION: SMART on FHIR in combination with CDS Hooks facilitates the integration of existing CDS systems into EHR systems, potentially improving education and patient care.

ELGA Terminology Server for Clinical Decision Support: A Case-Study Using an Existing Knowledge Base, CDS Hooks and FHIR.

BACKGROUND: There are many medical knowledge bases with potential for supporting medical professionals in their decision-making during routine care, yet usage of these sources remains low. Standardized linking of Clinical Decision Support (CDS) applications and existing medical knowledge bases is not a common practice. OBJECTIVES: Using existing eHealth standards to increase the utilization of knowledge bases and implement a prototype. METHODS: Linking an existing online knowledge base via a FHIR CodeSystem supplement to the Austrian national EHR (ELGA) terminology server and accessing these data using CDS Hooks and FHIR. RESULTS: We tested the approach by incorporating photosensitivity data of medications into a local copy of the Austrian terminology server. These data are directly used by a CDS Hooks compliant CDS service. CONCLUSION: The Austrian Terminology Server could be an important interface to access existing knowledge bases from within EHR systems. FHIR and CDS Hooks could lead the way for a simple and open integration of CDS services into EHR systems.

A reinforcement learning model for AI-based decision support in skin cancer.

We investigated whether human preferences hold the potential to improve diagnostic artificial intelligence (AI)-based decision support using skin cancer diagnosis as a use case. We utilized nonuniform rewards and penalties based on expert-generated tables, balancing the benefits and harms of various diagnostic errors, which were applied using reinforcement learning. Compared with supervised learning, the reinforcement learning model improved the sensitivity for melanoma from 61.4% to 79.5% (95% confidence interval (CI): 73.5-85.6%) and for basal cell carcinoma from 79.4% to 87.1% (95% CI: 80.3-93.9%). AI overconfidence was also reduced while simultaneously maintaining accuracy. Reinforcement learning increased the rate of correct diagnoses made by dermatologists by 12.0% (95% CI: 8.8-15.1%) and improved the rate of optimal management decisions from 57.4% to 65.3% (95% CI: 61.7-68.9%). We further demonstrated that the reward-adjusted reinforcement learning model and a threshold-based model outperformed naïve supervised learning in various clinical scenarios. Our findings suggest the potential for incorporating human preferences into image-based diagnostic algorithms.

Archetype based search in an IHE XDS environment.

To prevent information overload of physicians when accessing EHRs we introduce a method to extend the IHE XDS profile metadata-based search towards a content-based search. Detailed queries are created based on predefined information needs mapped to ISO/EN 13606 Archetypes. They are aggregated to a metadata-based query to retrieve all relevant documents, which are then analyzed for the desired contents. The results are presented in a tabular form. The content-based search in IHE-XDS could be implemented efficiently and was found helpful by the evaluating physicians.

Comparison of Convolutional Neural Network Architectures for Robustness Against Common Artefacts in Dermatoscopic Images.

Introduction: Classification of dermatoscopic images via neural networks shows comparable performance to clinicians in experimental conditions but can be affected by artefacts like skin markings or rulers. It is unknown whether specialized neural networks are more robust to artefacts. Objectives: Analyze robustness of 3 neural network architectures, namely ResNet-34, Faster R-CNN and Mask R-CNN. Methods: We identified common artefacts in the HAM10000, PH2 and the 7-point criteria evaluation datasets, and established a template-based method to superimpose artefacts on dermatoscopic images. The HAM10000-dataset with and without superimposed artefacts was used to train the networks, followed by analyzing their robustness against artefacts in test images. Performance was assessed via area under the precision recall curve and classification results. Results: ResNet-34 and Faster R-CNN models trained on regular images perform worse than Mask R-CNN on images with superimposed artefacts. Artefacts added to all tested images led to a decrease in area under the precision-recall curve values of 0.030 for ResNet-34 and 0.045 for Faster R-CNN in comparison to only 0.011 for Mask R-CNN. However, changes in model performance only became significant with 40% or more of the images having superimposed artefacts. A loss in performance occurred when the training was biased by selectively superimposing artefacts on images belonging to a certain class. Conclusions: As Mask R-CNN showed the least decrease in performance when confronted with artefacts, instance segmentation architectures may be helpful to counter the effects of artefacts, warranting further research on related architectures. Our artefact insertion mechanism could be useful for future research.

Validation of artificial intelligence prediction models for skin cancer diagnosis using dermoscopy images: the 2019 International Skin Imaging Collaboration Grand Challenge.

BACKGROUND: Previous studies of artificial intelligence (AI) applied to dermatology have shown AI to have higher diagnostic classification accuracy than expert dermatologists; however, these studies did not adequately assess clinically realistic scenarios, such as how AI systems behave when presented with images of disease categories that are not included in the training dataset or images drawn from statistical distributions with significant shifts from training distributions. We aimed to simulate these real-world scenarios and evaluate the effects of image source institution, diagnoses outside of the training set, and other image artifacts on classification accuracy, with the goal of informing clinicians and regulatory agencies about safety and real-world accuracy. METHODS: We designed a large dermoscopic image classification challenge to quantify the performance of machine learning algorithms for the task of skin cancer classification from dermoscopic images, and how this performance is affected by shifts in statistical distributions of data, disease categories not represented in training datasets, and imaging or lesion artifacts. Factors that might be beneficial to performance, such as clinical metadata and external training data collected by challenge participants, were also evaluated. 25 331 training images collected from two datasets (in Vienna [HAM10000] and Barcelona [BCN20000]) between Jan 1, 2000, and Dec 31, 2018, across eight skin diseases, were provided to challenge participants to design appropriate algorithms. The trained algorithms were then tested for balanced accuracy against the HAM10000 and BCN20000 test datasets and data from countries not included in the training dataset (Turkey, New Zealand, Sweden, and Argentina). Test datasets contained images of all diagnostic categories available in training plus other diagnoses not included in training data (not trained category). We compared the performance of the algorithms against that of 18 dermatologists in a simulated setting that reflected intended clinical use. FINDINGS: 64 teams submitted 129 state-of-the-art algorithm predictions on a test set of 8238 images. The best performing algorithm achieved 58·8% balanced accuracy on the BCN20000 data, which was designed to better reflect realistic clinical scenarios, compared with 82·0% balanced accuracy on HAM10000, which was used in a previously published benchmark. Shifted statistical distributions and disease categories not included in training data contributed to decreases in accuracy. Image artifacts, including hair, pen markings, ulceration, and imaging source institution, decreased accuracy in a complex manner that varied based on the underlying diagnosis. When comparing algorithms to expert dermatologists (2460 ratings on 1269 images), algorithms performed better than experts in most categories, except for actinic keratoses (similar accuracy on average) and images from categories not included in training data (26% correct for experts vs 6% correct for algorithms, p<0·0001). For the top 25 submitted algorithms, 47·1% of the images from categories not included in training data were misclassified as malignant diagnoses, which would lead to a substantial number of unnecessary biopsies if current state-of-the-art AI technologies were clinically deployed. INTERPRETATION: We have identified specific deficiencies and safety issues in AI diagnostic systems for skin cancer that should be addressed in future diagnostic evaluation protocols to improve safety and reliability in clinical practice. FUNDING: Melanoma Research Alliance and La Marató de TV3.

Creating ISO/EN 13606 archetypes based on clinical information needs.

Archetypes model individual EHR contents and build the basis of the dual-model approach used in the ISO/EN 13606 EHR architecture. We present an approach to create archetypes using an iterative development process. It includes automated generation of electronic case report forms from archetypes. We evaluated our approach by developing 128 archetypes which represent 446 clinical information items from the diabetes domain.

Clinical situations and information needs of physicians during treatment of diabetes mellitus patients: a triangulation study.

Physicians should have access to the information they need to provide the most effective health care. Medical knowledge and patient-oriented information is dynamic and expanding rapidly so there is a rising risk of information overload. We investigated the information needs of physicians during treatment of Diabetes mellitus patients, using a combination of interviews, observations, literature research and analysis of recorded medical information in hospitals as part of a methodical triangulation. 446 information items were identified, structured in a set of 9 main categories each, as well as 6 time windows, 10 clinical situations and 68 brief queries. The physician's information needs as identified in this study will now be used to develop sophisticated query tools to efficiently support finding of information in an electronic health record.