Diabetic foot ulcers segmentation challenge report: Benchmark and analysis.

Monitoring the healing progress of diabetic foot ulcers is a challenging process. Accurate segmentation of foot ulcers can help podiatrists to quantitatively measure the size of wound regions to assist prediction of healing status. The main challenge in this field is the lack of publicly available manual delineation, which can be time consuming and laborious. Recently, methods based on deep learning have shown excellent results in automatic segmentation of medical images, however, they require large-scale datasets for training, and there is limited consensus on which methods perform the best. The 2022 Diabetic Foot Ulcers segmentation challenge was held in conjunction with the 2022 International Conference on Medical Image Computing and Computer Assisted Intervention, which sought to address these issues and stimulate progress in this research domain. A training set of 2000 images exhibiting diabetic foot ulcers was released with corresponding segmentation ground truth masks. Of the 72 (approved) requests from 47 countries, 26 teams used this data to develop fully automated systems to predict the true segmentation masks on a test set of 2000 images, with the corresponding ground truth segmentation masks kept private. Predictions from participating teams were scored and ranked according to their average Dice similarity coefficient of the ground truth masks and prediction masks. The winning team achieved a Dice of 0.7287 for diabetic foot ulcer segmentation. This challenge has now entered a live leaderboard stage where it serves as a challenging benchmark for diabetic foot ulcer segmentation.

Monitoring Flow-Forming Processes Using Design of Experiments and a Machine Learning Approach Based on Randomized-Supervised Time Series Forest and Recursive Feature Elimination.

The machines of WF Maschinenbau process metal blanks into various workpieces using so-called flow-forming processes. The quality of these workpieces depends largely on the quality of the blanks and the condition of the machine. This creates an urgent need for automated monitoring of the forming processes and the condition of the machine. Since the complexity of the flow-forming processes makes physical modeling impossible, the present work deals with data-driven modeling using machine learning algorithms. The main contributions of this work lie in showcasing the feasibility of utilizing machine learning and sensor data to monitor flow-forming processes, along with developing a practical approach for this purpose. The approach includes an experimental design capable of providing the necessary data, as well as a procedure for preprocessing the data and extracting features that capture the information needed by the machine learning models to detect defects in the blank and the machine. To make efficient use of the small number of experiments available, the experimental design is generated using Design of Experiments methods. They consist of two parts. In the first part, a pre-selection of influencing variables relevant to the forming process is performed. In the second part of the design, the selected variables are investigated in more detail. The preprocessing procedure consists of feature engineering, feature extraction and feature selection. In the feature engineering step, the data set is augmented with time series variables that are meaningful in the domain. For feature extraction, an algorithm was developed based on the mechanisms of the r-STSF, a state-of-the-art algorithm for time series classification, extending them for multivariate time series and metric target variables. This feature extraction algorithm itself can be seen as an additional contribution of this work, because it is not tied to the application domain of monitoring flow-forming processes, but can be used as a feature extraction algorithm for multivariate time series classification in general. For feature selection, a Recursive Feature Elimination is employed. With the resulting features, random forests are trained to detect several quality features of the blank and defects of the machine. The trained models achieve good prediction accuracy for most of the target variables. This shows that the application of machine learning is a promising approach for the monitoring of flow-forming processes, which requires further investigation for confirmation.

Systematic comparison of 3D Deep learning and classical machine learning explanations for Alzheimer's Disease detection.

Black-box deep learning (DL) models trained for the early detection of Alzheimer's Disease (AD) often lack systematic model interpretation. This work computes the activated brain regions during DL and compares those with classical Machine Learning (ML) explanations. The architectures used for DL were 3D DenseNets, EfficientNets, and Squeeze-and-Excitation (SE) networks. The classical models include Random Forests (RFs), Support Vector Machines (SVMs), eXtreme Gradient Boosting (XGBoost), Light Gradient Boosting (LightGBM), Decision Trees (DTs), and Logistic Regression (LR). For explanations, SHapley Additive exPlanations (SHAP) values, Local Interpretable Model-agnostic Explanations (LIME), Gradient-weighted Class Activation Mapping (GradCAM), GradCAM++ and permutation-based feature importance were implemented. During interpretation, correlated features were consolidated into aspects. All models were trained on the Alzheimer's Disease Neuroimaging Initiative (ADNI) dataset. The validation includes internal and external validation on the Australian Imaging and Lifestyle flagship study of Ageing (AIBL) and the Open Access Series of Imaging Studies (OASIS). DL and ML models reached similar classification performances. Regarding the brain regions, both types focus on different regions. The ML models focus on the inferior and middle temporal gyri, and the hippocampus, and amygdala regions previously associated with AD. The DL models focus on a wider range of regions including the optical chiasm, the entorhinal cortices, the left and right vessels, and the 4th ventricle which were partially associated with AD. One explanation for the differences is the input features (textures vs. volumes). Both types show reasonable similarity to a ground truth Voxel-Based Morphometry (VBM) analysis. Slightly higher similarities were measured for ML models.

Medication event extraction in clinical notes: Contribution of the WisPerMed team to the n2c2 2022 challenge.

In this work, we describe the findings of the 'WisPerMed' team from their participation in Track 1 (Contextualized Medication Event Extraction) of the n2c2 2022 challenge. We tackle two tasks: (i) medication extraction, which involves extracting all mentions of medications from the clinical notes, and (ii) event classification, which involves classifying the medication mentions based on whether a change in the medication has been discussed. To address the long lengths of clinical texts, which often exceed the maximum token length that models based on the transformer-architecture can handle, various approaches, such as the use of ClinicalBERT with a sliding window approach and Longformer-based models, are employed. In addition, domain adaptation through masked language modeling and preprocessing steps such as sentence splitting are utilized to improve model performance. Since both tasks were treated as named entity recognition (NER) problems, a sanity check was performed in the second release to eliminate possible weaknesses in the medication detection itself. This check used the medication spans to remove false positive predictions and replace missed tokens with the highest softmax probability of the disposition types. The effectiveness of these approaches is evaluated through multiple submissions to the tasks, as well as with post-challenge results, with a focus on the DeBERTa v3 model and its disentangled attention mechanism. Results show that the DeBERTa v3 model performs well in both the NER task and the event classification task.

Explainable AI in medical imaging: An overview for clinical practitioners - Saliency-based XAI approaches.

Since recent achievements of Artificial Intelligence (AI) have proven significant success and promising results throughout many fields of application during the last decade, AI has also become an essential part of medical research. The improving data availability, coupled with advances in high-performance computing and innovative algorithms, has increased AI's potential in various aspects. Because AI rapidly reshapes research and promotes the development of personalized clinical care, alongside its implementation arises an urgent need for a deep understanding of its inner workings, especially in high-stake domains. However, such systems can be highly complex and opaque, limiting the possibility of an immediate understanding of the system's decisions. Regarding the medical field, a high impact is attributed to these decisions as physicians and patients can only fully trust AI systems when reasonably communicating the origin of their results, simultaneously enabling the identification of errors and biases. Explainable AI (XAI), becoming an increasingly important field of research in recent years, promotes the formulation of explainability methods and provides a rationale allowing users to comprehend the results generated by AI systems. In this paper, we investigate the application of XAI in medical imaging, addressing a broad audience, especially healthcare professionals. The content focuses on definitions and taxonomies, standard methods and approaches, advantages, limitations, and examples representing the current state of research regarding XAI in medical imaging. This paper focuses on saliency-based XAI methods, where the explanation can be provided directly on the input data (image) and which naturally are of special importance in medical imaging.

NanoDefiner Framework and e-Tool Revisited According to the European Commission's Nanomaterial Definition 2022/C 229/01.

The new recommended definition of a nanomaterial, 2022/C 229/01, adopted by the European Commission in 2022, will have a considerable impact on European Union legislation addressing chemicals, and therefore tools to implement this new definition are urgently needed. The updated NanoDefiner framework and its e-tool implementation presented here are such instruments, which help stakeholders to find out in a straightforward way whether a material is a nanomaterial or not. They are two major outcomes of the NanoDefine project, which is explicitly referred to in the new definition. This work revisits the framework and e-tool, and elaborates necessary adjustments to make these outcomes applicable for the updated recommendation. A broad set of case studies on representative materials confirms the validity of these adjustments. To further foster the sustainability and applicability of the framework and e-tool, measures for the FAIRification of expert knowledge within the e-tool's knowledge base are elaborated as well. The updated framework and e-tool are now ready to be used in line with the updated recommendation. The presented approach may serve as an example for reviewing existing guidance and tools developed for the previous definition 2011/696/EU, particularly those adopting NanoDefine project outcomes.

Explainable AI in medical imaging: An overview for clinical practitioners - Beyond saliency-based XAI approaches.

Driven by recent advances in Artificial Intelligence (AI) and Computer Vision (CV), the implementation of AI systems in the medical domain increased correspondingly. This is especially true for the domain of medical imaging, in which the incorporation of AI aids several imaging-based tasks such as classification, segmentation, and registration. Moreover, AI reshapes medical research and contributes to the development of personalized clinical care. Consequently, alongside its extended implementation arises the need for an extensive understanding of AI systems and their inner workings, potentials, and limitations which the field of eXplainable AI (XAI) aims at. Because medical imaging is mainly associated with visual tasks, most explainability approaches incorporate saliency-based XAI methods. In contrast to that, in this article we would like to investigate the full potential of XAI methods in the field of medical imaging by specifically focusing on XAI techniques not relying on saliency, and providing diversified examples. We dedicate our investigation to a broad audience, but particularly healthcare professionals. Moreover, this work aims at establishing a common ground for cross-disciplinary understanding and exchange across disciplines between Deep Learning (DL) builders and healthcare professionals, which is why we aimed for a non-technical overview. Presented XAI methods are divided by a method's output representation into the following categories: Case-based explanations, textual explanations, and auxiliary explanations.

Static and Dynamic Accuracy and Occlusion Robustness of SteamVR Tracking 2.0 in Multi-Base Station Setups.

The tracking of objects and person position, orientation, and movement is relevant for various medical use cases, e.g., practical training of medical staff or patient rehabilitation. However, these demand high tracking accuracy and occlusion robustness. Expensive professional tracking systems fulfill these demands, however, cost-efficient and potentially adequate alternatives can be found in the gaming industry, e.g., SteamVR Tracking. This work presents an evaluation of SteamVR Tracking in its latest version 2.0 in two experimental setups, involving two and four base stations. Tracking accuracy, both static and dynamic, and occlusion robustness are investigated using a VIVE Tracker (3.0). A dynamic analysis further compares three different velocities. An error evaluation is performed using a Universal Robots UR10 robotic arm as ground-truth system under nonlaboratory conditions. Results are presented using the Root Mean Square Error. For static experiments, tracking errors in the submillimeter and subdegree range are achieved by both setups. Dynamic experiments achieved errors in the submillimeter range as well, yet tracking accuracy suffers from increasing velocity. Four base stations enable generally higher accuracy and robustness, especially in the dynamic experiments. Both setups enable adequate accuracy for diverse medical use cases. However, use cases demanding very high accuracy should primarily rely on SteamVR Tracking 2.0 with four base stations.

Intracranial Aneurysm Classifier Using Phenotypic Factors: An International Pooled Analysis.

Intracranial aneurysms (IAs) are usually asymptomatic with a low risk of rupture, but consequences of aneurysmal subarachnoid hemorrhage (aSAH) are severe. Identifying IAs at risk of rupture has important clinical and socio-economic consequences. The goal of this study was to assess the effect of patient and IA characteristics on the likelihood of IA being diagnosed incidentally versus ruptured. Patients were recruited at 21 international centers. Seven phenotypic patient characteristics and three IA characteristics were recorded. The analyzed cohort included 7992 patients. Multivariate analysis demonstrated that: (1) IA location is the strongest factor associated with IA rupture status at diagnosis; (2) Risk factor awareness (hypertension, smoking) increases the likelihood of being diagnosed with unruptured IA; (3) Patients with ruptured IAs in high-risk locations tend to be older, and their IAs are smaller; (4) Smokers with ruptured IAs tend to be younger, and their IAs are larger; (5) Female patients with ruptured IAs tend to be older, and their IAs are smaller; (6) IA size and age at rupture correlate. The assessment of associations regarding patient and IA characteristics with IA rupture allows us to refine IA disease models and provide data to develop risk instruments for clinicians to support personalized decision-making.

Bayesian network analysis reveals the interplay of intracranial aneurysm rupture risk factors.

Clinical decision making regarding the treatment of unruptured intracranial aneurysms (IA) benefits from a better understanding of the interplay of IA rupture risk factors. Probabilistic graphical models can capture and graphically display potentially causal relationships in a mechanistic model. In this study, Bayesian networks (BN) were used to estimate IA rupture risk factors influences. From 1248 IA patient records, a retrospective, single-cohort, patient-level data set with 9 phenotypic rupture risk factors (n=790 complete entries) was extracted. Prior knowledge together with score-based structure learning algorithms estimated rupture risk factor interactions. Two approaches, discrete and mixed-data additive BN, were implemented and compared. The corresponding graphs were learned using non-parametric bootstrapping and Markov chain Monte Carlo, respectively. The BN models were compared to standard descriptive and regression analysis methods. Correlation and regression analyses showed significant associations between IA rupture status and patient's sex, familial history of IA, age at IA diagnosis, IA location, IA size and IA multiplicity. BN models confirmed the findings from standard analysis methods. More precisely, they directly associated IA rupture with familial history of IA, IA size and IA location in a discrete framework. Additive model formulation, enabling mixed-data, found that IA rupture was directly influenced by patient age at diagnosis besides additional mutual influences of the risk factors. This study establishes a data-driven methodology for mechanistic disease modelling of IA rupture and shows the potential to direct clinical decision-making in IA treatment, allowing personalised prediction.

Data analysis with Shapley values for automatic subject selection in Alzheimer's disease data sets using interpretable machine learning.

BACKGROUND: For the recruitment and monitoring of subjects for therapy studies, it is important to predict whether mild cognitive impaired (MCI) subjects will prospectively develop Alzheimer's disease (AD). Machine learning (ML) is suitable to improve early AD prediction. The etiology of AD is heterogeneous, which leads to high variability in disease patterns. Further variability originates from multicentric study designs, varying acquisition protocols, and errors in the preprocessing of magnetic resonance imaging (MRI) scans. The high variability makes the differentiation between signal and noise difficult and may lead to overfitting. This article examines whether an automatic and fair data valuation method based on Shapley values can identify the most informative subjects to improve ML classification. METHODS: An ML workflow was developed and trained for a subset of the Alzheimer's Disease Neuroimaging Initiative (ADNI) cohort. The validation was executed for an independent ADNI test set and for the Australian Imaging, Biomarker and Lifestyle Flagship Study of Ageing (AIBL) cohort. The workflow included volumetric MRI feature extraction, feature selection, sample selection using Data Shapley, random forest (RF), and eXtreme Gradient Boosting (XGBoost) for model training as well as Kernel SHapley Additive exPlanations (SHAP) values for model interpretation. RESULTS: The RF models, which excluded 134 of the 467 training subjects based on their RF Data Shapley values, outperformed the base models that reached a mean accuracy of 62.64% by 5.76% (3.61 percentage points) for the independent ADNI test set. The XGBoost base models reached a mean accuracy of 60.00% for the AIBL data set. The exclusion of those 133 subjects with the smallest RF Data Shapley values could improve the classification accuracy by 2.98% (1.79 percentage points). The cutoff values were calculated using an independent validation set. CONCLUSION: The Data Shapley method was able to improve the mean accuracies for the test sets. The most informative subjects were associated with the number of ApolipoproteinE ε4 (ApoE ε4) alleles, cognitive test results, and volumetric MRI measurements.

Deep learning in diabetic foot ulcers detection: A comprehensive evaluation.

There has been a substantial amount of research involving computer methods and technology for the detection and recognition of diabetic foot ulcers (DFUs), but there is a lack of systematic comparisons of state-of-the-art deep learning object detection frameworks applied to this problem. DFUC2020 provided participants with a comprehensive dataset consisting of 2,000 images for training and 2,000 images for testing. This paper summarizes the results of DFUC2020 by comparing the deep learning-based algorithms proposed by the winning teams: Faster R-CNN, three variants of Faster R-CNN and an ensemble method; YOLOv3; YOLOv5; EfficientDet; and a new Cascade Attention Network. For each deep learning method, we provide a detailed description of model architecture, parameter settings for training and additional stages including pre-processing, data augmentation and post-processing. We provide a comprehensive evaluation for each method. All the methods required a data augmentation stage to increase the number of images available for training and a post-processing stage to remove false positives. The best performance was obtained from Deformable Convolution, a variant of Faster R-CNN, with a mean average precision (mAP) of 0.6940 and an F1-Score of 0.7434. Finally, we demonstrate that the ensemble method based on different deep learning methods can enhance the F1-Score but not the mAP.

Genome-wide association study of intracranial aneurysms identifies 17 risk loci and genetic overlap with clinical risk factors.

Rupture of an intracranial aneurysm leads to subarachnoid hemorrhage, a severe type of stroke. To discover new risk loci and the genetic architecture of intracranial aneurysms, we performed a cross-ancestry, genome-wide association study in 10,754 cases and 306,882 controls of European and East Asian ancestry. We discovered 17 risk loci, 11 of which are new. We reveal a polygenic architecture and explain over half of the disease heritability. We show a high genetic correlation between ruptured and unruptured intracranial aneurysms. We also find a suggestive role for endothelial cells by using gene mapping and heritability enrichment. Drug-target enrichment shows pleiotropy between intracranial aneurysms and antiepileptic and sex hormone drugs, providing insights into intracranial aneurysm pathophysiology. Finally, genetic risks for smoking and high blood pressure, the two main clinical risk factors, play important roles in intracranial aneurysm risk, and drive most of the genetic correlation between intracranial aneurysms and other cerebrovascular traits.

Sociodemographic data and APOE-ε4 augmentation for MRI-based detection of amnestic mild cognitive impairment using deep learning systems.

Detection and diagnosis of early and subclinical stages of Alzheimer's Disease (AD) play an essential role in the implementation of intervention and prevention strategies. Neuroimaging techniques predominantly provide insight into anatomic structure changes associated with AD. Deep learning methods have been extensively applied towards creating and evaluating models capable of differentiating between cognitively unimpaired, patients with Mild Cognitive Impairment (MCI) and AD dementia. Several published approaches apply information fusion techniques, providing ways of combining several input sources in the medical domain, which contributes to knowledge of broader and enriched quality. The aim of this paper is to fuse sociodemographic data such as age, marital status, education and gender, and genetic data (presence of an apolipoprotein E (APOE)-ε4 allele) with Magnetic Resonance Imaging (MRI) scans. This enables enriched multi-modal features, that adequately represent the MRI scan visually and is adopted for creating and modeling classification systems capable of detecting amnestic MCI (aMCI). To fully utilize the potential of deep convolutional neural networks, two extra color layers denoting contrast intensified and blurred image adaptations are virtually augmented to each MRI scan, completing the Red-Green-Blue (RGB) color channels. Deep convolutional activation features (DeCAF) are extracted from the average pooling layer of the deep learning system Inception_v3. These features from the fused MRI scans are used as visual representation for the Long Short-Term Memory (LSTM) based Recurrent Neural Network (RNN) classification model. The proposed approach is evaluated on a sub-study containing 120 participants (aMCI = 61 and cognitively unimpaired = 59) of the Heinz Nixdorf Recall (HNR) Study with a baseline model accuracy of 76%. Further evaluation was conducted on the ADNI Phase 1 dataset with 624 participants (aMCI = 397 and cognitively unimpaired = 227) with a baseline model accuracy of 66.27%. Experimental results show that the proposed approach achieves 90% accuracy and 0.90 F1-Score at classification of aMCI vs. cognitively unimpaired participants on the HNR Study dataset, and 77% accuracy and 0.83 F1-Score on the ADNI dataset.

Nano or Not Nano? A Structured Approach for Identifying Nanomaterials According to the European Commission's Definition.

Identifying nanomaterials (NMs) according to European Union legislation is challenging, as there is an enormous variety of materials, with different physico-chemical properties. The NanoDefiner Framework and its Decision Support Flow Scheme (DSFS) allow choosing the optimal method to measure the particle size distribution by matching the material properties and the performance of the particular measurement techniques. The DSFS leads to a reliable and economic decision whether a material is an NM or not based on scientific criteria and respecting regulatory requirements. The DSFS starts beyond regulatory requirements by identifying non-NMs by a proxy approach based on their volume-specific surface area. In a second step, it identifies NMs. The DSFS is tested on real-world materials and is implemented in an e-tool. The DSFS is compared with a decision flowchart of the European Commission's (EC) Joint Research Centre (JRC), which rigorously follows the explicit criteria of the EC NM definition with the focus on identifying NMs, and non-NMs are identified by exclusion. The two approaches build on the same scientific basis and measurement methods, but start from opposite ends: the JRC Flowchart starts by identifying NMs, whereas the NanoDefiner Framework first identifies non-NMs.

Mimicking the radiologists' workflow: Estimating pediatric hand bone age with stacked deep neural networks.

Pediatric endocrinologists regularly order radiographs of the left hand to estimate the degree of bone maturation in order to assess their patients for advanced or delayed growth, physical development, and to monitor consecutive therapeutic measures. The reading of such images is a labor-intensive task that requires a lot of experience and is normally performed by highly trained experts like pediatric radiologists. In this paper we build an automated system for pediatric bone age estimation that mimics and accelerates the workflow of the radiologist without breaking it. The complete system is based on two neural network based models: on the one hand a detector network, which identifies the ossification areas, on the other hand gender and region specific regression networks, which estimate the bone age from the detected areas. With a small annotated dataset an ossification area detection network can be trained, which is stable enough to work as part of a multi-stage approach. Furthermore, our system achieves competitive results on the RSNA Pediatric Bone Age Challenge test set with an average error of 4.56 months. In contrast to other approaches, especially purely encoder-based architectures, our two-stage approach provides self-explanatory results. By detecting and evaluating the individual ossification areas, thus simulating the workflow of the Tanner-Whitehouse procedure, the results are interpretable for a radiologist.

NanoDefiner e-Tool: An Implemented Decision Support Framework for Nanomaterial Identification.

The European Commission's recommendation on the definition of nanomaterial (2011/696/EU) established an applicable standard for material categorization. However, manufacturers face regulatory challenges during registration of their products. Reliable categorization is difficult and requires considerable expertise in existing measurement techniques (MTs). Additionally, organizational complexity is increased as different authorities' registration processes require distinct reporting. The NanoDefine project tackled these obstacles by providing the NanoDefiner e-tool: A decision support expert system for nanomaterial identification in a regulatory context. It provides MT recommendations for categorization of specific materials using a tiered approach (screening/confirmatory), and was constructed with experts from academia and industry to be extensible, interoperable, and adaptable for forthcoming revisions of the nanomaterial definition. An implemented MT-driven material categorization scheme allows detailed description. Its guided workflow is suitable for a variety of user groups. Direct feedback and explanation enable transparent decisions. Expert knowledge is held in a knowledge base for representation of MT performance criteria and physicochemical particle type properties. Continuous revision ensured data quality and validity. Recommendations were validated by independent case studies on industry-relevant particulate materials. Besides supporting material identification and registration, the free and open-source e-tool may serve as template for other expert systems within the nanoscience domain.

Association of single and multiple aneurysms with tobacco abuse: an @neurIST risk analysis.

OBJECTIVE: Although several studies have suggested that the incidence of intracranial aneurysms (IAs) is higher in smokers, the higher prevalence of subarachnoid hemorrhage (SAH) in smokers remains uncertain. It is unclear whether smoking additionally contributes to the formation of multiple aneurysms and the risk of rupture. The aim of this study was to determine whether smoking is associated with IA formation, multiplicity, or rupture. METHODS: Patients from the prospective multicenter @neurIST database (n = 1410; 985 females [69.9%]) were reviewed for the presence of SAH, multiple aneurysms, and smoking status. The prevalence of smokers in the population of patients diagnosed with at least one IA was compared with that of smokers in the general population. RESULTS: The proportion of smokers was higher in patients with IAs (56.2%) than in the reference population (51.4%; p < 0.001). A significant association of smoking with the presence of an IA was found throughout group comparisons (p = 0.01). The presence of multiple IAs was also significantly associated with smoking (p = 0.003). A trend was found between duration of smoking and the presence of multiple IAs (p = 0.057). However, the proportion of smokers among patients suffering SAH was similar to that of smokers among patients diagnosed with unruptured IAs (p = 0.48). CONCLUSIONS: Smoking is strongly associated with IA formation. Once an IA is present, however, smoking does not appear to increase the risk of rupture compared with IAs in the nonsmoking population. The trend toward an association between duration of smoking and the presence of multiple IAs stresses the need for counseling patients with IAs regarding lifestyle modification.

Augmented Reality in Medicine: Systematic and Bibliographic Review.

BACKGROUND: Augmented reality (AR) is a technology that integrates digital information into the user's real-world environment. It offers a new approach for treatments and education in medicine. AR aids in surgery planning and patient treatment and helps explain complex medical situations to patients and their relatives. OBJECTIVE: This systematic and bibliographic review offers an overview of the development of apps in AR with a medical use case from March 2012 to June 2017. This work can aid as a guide to the literature and categorizes the publications in the field of AR research. METHODS: From March 2012 to June 2017, a total of 1309 publications from PubMed and Scopus databases were manually analyzed and categorized based on a predefined taxonomy. Of the total, 340 duplicates were removed and 631 publications were excluded due to incorrect classification or unavailable technical data. The remaining 338 publications were original research studies on AR. An assessment of the maturity of the projects was conducted on these publications by using the technology readiness level. To provide a comprehensive process of inclusion and exclusion, the authors adopted the Preferred Reporting Items for Systematic Reviews and Meta-Analyses statement. RESULTS: The results showed an increasing trend in the number of publications on AR in medicine. There were no relevant clinical trials on the effect of AR in medicine. Domains that used display technologies seemed to be researched more than other medical fields. The technology readiness level showed that AR technology is following a rough bell curve from levels 4 to 7. Current AR technology is more often applied to treatment scenarios than training scenarios. CONCLUSIONS: This work discusses the applicability and future development of augmented- and mixed-reality technologies such as wearable computers and AR devices. It offers an overview of current technology and a base for researchers interested in developing AR apps in medicine. The field of AR is well researched, and there is a positive trend in its application, but its use is still in the early stages in the field of medicine and it is not widely adopted in clinical practice. Clinical studies proving the effectiveness of applied AR technologies are still lacking.