Accuracy of a Generative Artificial Intelligence Model in a Complex Diagnostic Challenge.

This study assesses the diagnostic accuracy of the Generative Pre-trained Transformer 4 (GPT-4) artificial intelligence (AI) model in a series of challenging cases.

Boundary-Sensitive Loss Function With Location Constraint for Hard Region Segmentation.

In computer-aided diagnosis and treatment planning, accurate segmentation of medical images plays an essential role, especially for some hard regions including boundaries, small objects and background interference. However, existing segmentation loss functions including distribution-, region- and boundary-based losses cannot achieve satisfactory performances on these hard regions. In this paper, a boundary-sensitive loss function with location constraint is proposed for hard region segmentation in medical images, which provides three advantages: i) our Boundary-Sensitive loss (BS-loss) can automatically pay more attention to the hard-to-segment boundaries (e.g., thin structures and blurred boundaries), thus obtaining finer object boundaries; ii) BS-loss also can adjust its attention to small objects during training to segment them more accurately; and iii) our location constraint can alleviate the negative impact of the background interference, through the distribution matching of pixels between prediction and Ground Truth (GT) along each axis. By resorting to the proposed BS-loss and location constraint, the hard regions in both foreground and background are considered. Experimental results on three public datasets demonstrate the superiority of our method. Specifically, compared to the second-best method tested in this study, our method improves performance on hard regions in terms of Dice similarity coefficient (DSC) and 95% Hausdorff distance (95%HD) of up to 4.17% and 73% respectively. In addition, it also achieves the best overall segmentation performance. Hence, we can conclude that our method can accurately segment these hard regions and improve the overall segmentation performance in medical images.

Protocol for the diagnosis of keratoconus using convolutional neural networks.

Keratoconus is the corneal disease with the highest reported incidence of 1:2000. The treatment's level of success highly depends on how early it was started. Subsequently, a fast and highly capable diagnostic tool is crucial. While there are many computer-based systems that are capable of the analysis of medical image data, they only provide parameters. These have advanced quite far, though full diagnosis does not exist. Machine learning has provided the capabilities for the parameters, and numerous similar scientific fields have developed full image diagnosis based on neural networks. The Homburg Keratoconus Center has been gathering almost 2000 patient datasets, over 1000 of them over the course of their disease. Backed by this databank, this work aims to develop a convolutional neural network to tackle diagnosis of keratoconus as the major corneal disease.

Emergent cardiac outcomes in patients with normal electrocardiograms in the emergency department.

BACKGROUND: Emergency physicians (EP) are frequently interrupted to screen electrocardiograms (ECG) from Emergency Department (ED) patients undergoing triage. Our objective was to identify discrepancies between the computer ECG interpretation and the cardiologist ECG interpretation and if any patients with normal ECGs underwent emergent cardiac intervention. We hypothesized that computer-interpreted normal ECGs do not require immediate review by an EP. METHODS: This was a retrospective study of adult (≥ 18 years old) ED patients with computer-interpreted normal ECGs. Laboratory, diagnostic testing and clinical outcomes were abstracted following accepted methodologic guidelines. The primary outcome was emergent cardiac catheterization (within four hours of ED arrival). All ECGs underwent final cardiologist interpretation. When cardiology interpretation differed from the computer (discrepant ECG interpretation), the difference was classified as potentially clinically significant or not clinically significant. Data was described with simple descriptive statistics. MAIN FINDINGS: 989 ECGs interpreted as normal by the computer were analyzed with a mean age of 50.4 ± 16.8 years (range 18-96 years) and 527 (53%) female. Discrepant ECG interpretations were identified in 184 cases including 124 (12.5%, 95% CI 10.4, 14.7%) not clinically significant and 60 (6.1%, 95% CI 4.6, 7.7%) potentially clinically significant. The 60 potentially clinically significant changes included: ST/T wave changes 45 (75%), T wave inversions 6 (10%), prolonged QT 3 (5%), and possible ischemia 10 (17%). Of these 60, 21 (35%) patients were admitted. Six patients had potassium levels >6.0 mEq/L, with one having a potentially clinically significant ECG change. No patient (0%, 95% CI 0, 0.3%) underwent immediate (within four hours) cardiac catherization whereas two underwent delayed cardiac interventions. CONCLUSIONS: Cardiologists frequently disagree with a computer-interpreted normal ECG. Patients with computer-interpreted normal ECGs, however, rarely had significant ischemic events. A rare number of patients will have important cardiac outcomes regardless of the computer-generated normal ECG interpretation. Immediate EP review of the ECG, however, would not have changed these patients' ED courses.

A Computational Tumor-Infiltrating Lymphocyte Assessment Method Comparable with Visual Reporting Guidelines for Triple-Negative Breast Cancer.

BACKGROUND: Tumor-infiltrating lymphocytes (TILs) are clinically significant in triple-negative breast cancer (TNBC). Although a standardized methodology for visual TILs assessment (VTA) exists, it has several inherent limitations. We established a deep learning-based computational TIL assessment (CTA) method broadly following VTA guideline and compared it with VTA for TNBC to determine the prognostic value of the CTA and a reasonable CTA workflow for clinical practice. METHODS: We trained three deep neural networks for nuclei segmentation, nuclei classification and necrosis classification to establish a CTA workflow. The automatic TIL (aTIL) score generated was compared with manual TIL (mTIL) scores provided by three pathologists in an Asian (n = 184) and a Caucasian (n = 117) TNBC cohort to evaluate scoring concordance and prognostic value. FINDINGS: The intraclass correlations (ICCs) between aTILs and mTILs varied from 0.40 to 0.70 in two cohorts. Multivariate Cox proportional hazards analysis revealed that the aTIL score was associated with disease free survival (DFS) in both cohorts, as either a continuous [hazard ratio (HR)=0.96, 95% CI 0.94-0.99] or dichotomous variable (HR=0.29, 95% CI 0.12-0.72). A higher C-index was observed in a composite mTIL/aTIL three-tier stratification model than in the dichotomous model, using either mTILs or aTILs alone. INTERPRETATION: The current study provides a useful tool for stromal TIL assessment and prognosis evaluation for patients with TNBC. A workflow integrating both VTA and CTA may aid pathologists in performing risk management and decision-making tasks. FUNDING: National Natural Science Foundation of China, Guangdong Medical Research Foundation, Guangdong Natural Science Foundation.

A prognostic and predictive computational pathology image signature for added benefit of adjuvant chemotherapy in early stage non-small-cell lung cancer.

BACKGROUND: We developed and validated a prognostic and predictive computational pathology risk score (CoRiS) using H&E stained tissue images from patients with early-stage non-small cell lung cancer (ES-NSCLC). METHODS: 1330 patients with ES-NSCLC were acquired from 3 independent sources and divided into four cohorts D1-4. D1 comprised 100 surgery treated patients and was used to identify prognostic features via an elastic-net Cox model to predict overall and disease-free survival. CoRiS was constructed using the Cox model coefficients for the top features. The prognostic performance of CoRiS was evaluated on D2 (N=331), D3 (N=657) and D4 (N=242). Patients from D2 and D3 which comprised surgery + chemotherapy were used to validate CoRiS as predictive of added benefit to adjuvant chemotherapy (ACT) by comparing survival between different CoRiS defined risk groups. FINDINGS: CoRiS was found to be prognostic on univariable analysis, D2 (hazard ratio (HR) = 1.41, adjusted (adj.) P = .01) and D3 (HR = 1.35, adj. P < .001). Multivariable analysis showed CoRiS was independently prognostic, D2 (HR = 1.41, adj. P < .001) and D3 (HR = 1.35, adj. P < .001), after adjusting for clinico-pathologic factors. CoRiS was also able to identify high-risk patients who derived survival benefit from ACT D2 (HR = 0.42, adj. P = .006) and D3 (HR = 0.46, adj. P = .08). INTERPRETATION: CoRiS is a tissue non-destructive, quantitative and low-cost tool that could potentially help guide management of ES-NSCLC patients. FUNDING: Data collection, anlaysis, and computation resources of the research reported in this publication was supported by the National Cancer Institute of the National Institutes of Health under award numbers: 1U24CA199374-01, R01CA202752-01A1, R01CA208236-01A1, R01 CA216579-01A1, R01 CA220581-01A1, 1U01 CA239055-01. National Center for Research Resources under award number 1 C06 RR12463-01. VA Merit Review Award IBX004121A from the United States Department of Veterans Affairs Biomedical Laboratory Research and Development Service, the DOD Prostate Cancer Idea Development Award (W81XWH-15-1-0558), the DOD Lung Cancer Investigator-Initiated Translational Research Award (W81XWH-18-1-0440), the DOD Peer Reviewed Cancer Research Program (W81XWH-16-1-0329), the Ohio Third Frontier Technology Validation Fund, the Wallace H. Coulter Foundation Program in the Department of Biomedical Engineering and the Clinical and Translational Science Award Program (CTSA) at Case Western Reserve University.

Reliability of a computer-based neurobehavioral assessment test battery for Bangladeshi adolescent children.

In developing countries, there is a need for low-cost neurobehavioral (NB) test batteries for vulnerable populations, particularly for children exposed to environmental neurotoxicants. The objective of the current study was to assess the feasibility and test-retest reliability of the Behavioral Assessment and Research System (BARS) in children from a rural community in Bangladesh. Fifty healthy adolescents living in the Health Effects of Arsenic Longitudinal Study (HEALS) area in Araihazar, Bangladesh completed all six tests from the BARS in two test sessions scheduled two weeks apart. The BARS tests evaluated NB functions such as motor coordination, attention, memory, and information processing speed. The reliability assessment, evaluated by test-retest correlations demonstrated moderate to strong correlations (i.e., correlation coefficients ranged from 0.43 to 0.85), which were statistically significant (p < 0.05). Paired t-tests for comparing the test and retest outcomes indicated significant improvement in NB performance, highlighting learning and practice effects. NB performance improved with increasing age in most cases. Adolescent boys performed better than the girls in Finger Tapping, Digit Span, and Simple Reaction Time, whereas the girls performed better in Continuous Performance and Symbol Digit tests. The reliability scores (Pearson's correlations 0.43-0.85) were consistent with other children studies in different cultural settings. The effects of age and sex on NB tests were also consistent with findings reported in other countries. Overall, the findings of the study support the feasibility of using this computer-based test system to assess vulnerability of brain health due to environmental exposures among rural Bangladeshi children.

Automatic lumen detection and magnetic alignment control for magnetic-assisted capsule colonoscope system optimization.

We developed a magnetic-assisted capsule colonoscope system with integration of computer vision-based object detection and an alignment control scheme. Two convolutional neural network models A and B for lumen identification were trained on an endoscopic dataset of 9080 images. In the lumen alignment experiment, models C and D used a simulated dataset of 8414 images. The models were evaluated using validation indexes for recall (R), precision (P), mean average precision (mAP), and F1 score. Predictive performance was evaluated with the area under the P-R curve. Adjustments of pitch and yaw angles and alignment control time were analyzed in the alignment experiment. Model D had the best predictive performance. Its R, P, mAP, and F1 score were 0.964, 0.961, 0.961, and 0.963, respectively, when the area of overlap/area of union was at 0.3. In the lumen alignment experiment, the mean degrees of adjustment for yaw and pitch in 160 trials were 21.70° and 13.78°, respectively. Mean alignment control time was 0.902 s. Finally, we compared the cecal intubation time between semi-automated and manual navigation in 20 trials. The average cecal intubation time of manual navigation and semi-automated navigation were 9 min 28.41 s and 7 min 23.61 s, respectively. The automatic lumen detection model, which was trained using a deep learning algorithm, demonstrated high performance in each validation index.

Objective differential diagnosis of Noonan and Williams-Beuren syndromes in diverse populations using quantitative facial phenotyping.

INTRODUCTION: Patients with Noonan and Williams-Beuren syndrome present similar facial phenotypes modulated by their ethnic background. Although distinctive facial features have been reported, studies show a variable incidence of those characteristics in populations with diverse ancestry. Hence, a differential diagnosis based on reported facial features can be challenging. Although accurate diagnoses are possible with genetic testing, they are not available in developing and remote regions. METHODS: We used a facial analysis technology to identify the most discriminative facial metrics between 286 patients with Noonan and 161 with Williams-Beuren syndrome with diverse ethnic background. We quantified the most discriminative metrics, and their ranges both globally and in different ethnic groups. We also created population-based appearance images that are useful not only as clinical references but also for training purposes. Finally, we trained both global and ethnic-specific machine learning models with previous metrics to distinguish between patients with Noonan and Williams-Beuren syndromes. RESULTS: We obtained a classification accuracy of 85.68% in the global population evaluated using cross-validation, which improved to 90.38% when we adapted the facial metrics to the ethnicity of the patients (p = 0.024). CONCLUSION: Our facial analysis provided for the first time quantitative reference facial metrics for the differential diagnosis Noonan and Williams-Beuren syndromes in diverse populations.

Prospective Development and Validation of the Computerized Adaptive Screen for Suicidal Youth.

Importance: The rate of suicide among adolescents is rising in the US, yet many adolescents at risk are unidentified and receive no mental health services. Objective: To develop and independently validate a novel computerized adaptive screen for suicidal youth (CASSY) for use as a universal screen for suicide risk in medical emergency departments (EDs). Design, Setting, and Participants: Study 1 of this prognostic study prospectively enrolled adolescent patients at 13 geographically diverse US EDs in the Pediatric Emergency Care Applied Research Network. They completed a baseline suicide risk survey and participated in 3-month telephone follow-ups. Using 3 fixed Ask Suicide-Screening Questions items as anchors and additional items that varied in number and content across individuals, we derived algorithms for the CASSY. In study 2, data were collected from patients at 14 Pediatric Emergency Care Applied Research Network EDs and 1 Indian Health Service hospital. Algorithms were independently validated in a prospective cohort of adolescent patients who also participated in 3-month telephone follow-ups. Adolescents aged 12 to 17 years were consecutively approached during randomly assigned shifts. Exposures: Presentation at an ED. Main Outcome and Measure: A suicide attempt between ED visit and 3-month follow-up, measured via patient and/or parent report. Results: The study 1 CASSY derivation sample included 2075 adolescents (1307 female adolescents [63.0%]; mean [SD] age, 15.1 [1.61] years) with 3-month follow-ups (72.9% retention [2075 adolescents]). The study 2 validation sample included 2754 adolescents (1711 female adolescents [62.1%]; mean [SD] age, 15.0 [1.65] years), with 3-month follow-ups (69.5% retention [2754 adolescents]). The CASSY algorithms had excellent predictive accuracy for suicide attempt (area under the curve, 0.89 [95% CI, 0.85-0.91]) in study 1. The mean number of adaptively administered items was 11 (range, 5-21). At a specificity of 80%, the CASSY had a sensitivity of 83%. It also demonstrated excellent accuracy in the study 2 validation sample (area under the curve, 0.87 [95% CI, 0.85-0.89]). In this study, the CASSY had a sensitivity of 82.4% for prediction of a suicide attempt at the 80% specificity cutoff established in study 1. Conclusions and Relevance: In this study, the adaptive and personalized CASSY demonstrated excellent suicide attempt risk recognition, which has the potential to facilitate linkage to services.

A content validity evaluation of the PEDI-CAT Speedy Mobility domain.

Objective: To evaluate the content validity of the PEDI-CAT Speedy Mobility domain through analysis of item and content area exposure, score range and scoring precision.Methods: Retrospective analysis of 3,364 items from assessments (n = 301) completed from 2013 to 2017. Content validity was appraised through analysis of item and content area exposure (item, content area, response frequency), score range (floor and ceiling effect) and scoring precision (person fit, score reliability, item information function).Results: Sixty-five of the 75 general mobility items from the PEDI-CAT Mobility domain item bank were exposed. "Stands up from the middle of the floor" (68%) was the most frequently exposed non-mandatory item. Almost half (49%) of all items were from the Basic Mobility and Transfers content area. Scaled scores ranged from 26.77 to 69.40 with a floor (scores ≤27; n = 51, 17%) but no ceiling effect. Person fit statistics were acceptable for 238 (79%), suggesting limited outliers. Score reliability was sufficient with 68% of scores above threshold (>0.9). Item information function plot indicated less discriminating items at the lower end of the score range.Conclusion: Content is adequately and reliably measuring the intended construct, but additional items at the lower end of the scale could improve score precision.

Prediction of disease progression and outcomes in multiple sclerosis with machine learning.

Multiple Sclerosis is a chronic inflammatory disease, affecting the Central Nervous System and leading to irreversible neurological damage, such as long term functional impairment and disability. It has no cure and the symptoms vary widely, depending on the affected regions, amount of damage, and the ability to activate compensatory mechanisms, which constitutes a challenge to evaluate and predict its course. Additionally, relapsing-remitting patients can evolve its course into a secondary progressive, characterized by a slow progression of disability independent of relapses. With clinical information from Multiple Sclerosis patients, we developed a machine learning exploration framework concerning this disease evolution, more specifically to obtain three predictions: one on conversion to secondary progressive course and two on disease severity with rapid accumulation of disability, concerning the 6th and 10th years of progression. For the first case, the best results were obtained within two years: AUC=[Formula: see text], sensitivity=[Formula: see text] and specificity=[Formula: see text]; and for the second, the best results were obtained for the 6th year of progression, also within two years: AUC=[Formula: see text], sensitivity=[Formula: see text], and specificity=[Formula: see text]. The Expanded Disability Status Scale value, the majority of functional systems, affected functions during relapses, and age at onset were described as the most predictive features. These results demonstrate the possibility of predicting Multiple Sclerosis progression by using machine learning, which may help to understand this disease's dynamics and thus, advise physicians on medication intake.

Automatic Assessment of Cognitive Tests for Differentiating Mild Cognitive Impairment: A Proof of Concept Study of the Digit Span Task.

BACKGROUND: Current conventional cognitive assessments are limited in their efficiency and sensitivity, often relying on a single score such as the total correct items. Typically, multiple features of response go uncaptured. OBJECTIVES: We aim to explore a new set of automatically derived features from the Digit Span (DS) task that address some of the drawbacks in the conventional scoring and are also useful for distinguishing subjects with Mild Cognitive Impairment (MCI) from those with intact cognition. METHODS: Audio-recordings of the DS tests administered to 85 subjects (22 MCI and 63 healthy controls, mean age 90.2 years) were transcribed using an Automatic Speech Recognition (ASR) system. Next, five correctness measures were generated from Levenshtein distance analysis of responses: number correct, incorrect, deleted, inserted, and substituted words compared to the test item. These per-item features were aggregated across all test items for both Forward Digit Span (FDS) and Backward Digit Span (BDS) tasks using summary statistical functions, constructing a global feature vector representing the detailed assessment of each subject's response. A support vector machine classifier distinguished MCI from cognitively intact participants. RESULTS: Conventional DS scores did not differentiate MCI participants from controls. The automated multi-feature DS-derived metric achieved 73% on AUC-ROC of the SVM classifier, independent of additional clinical features (77% when combined with demographic features of subjects); well above chance, 50%. CONCLUSION: Our analysis verifies the effectiveness of introduced measures, solely derived from the DS task, in the context of differentiating subjects with MCI from those with intact cognition.

Histopathological imaging features- versus molecular measurements-based cancer prognosis modeling.

For lung and many other cancers, prognosis is essentially important, and extensive modeling has been carried out. Cancer is a genetic disease. In the past 2 decades, diverse molecular data (such as gene expressions and DNA mutations) have been analyzed in prognosis modeling. More recently, histopathological imaging data, which is a "byproduct" of biopsy, has been suggested as informative for prognosis. In this article, with the TCGA LUAD and LUSC data, we examine and directly compare modeling lung cancer overall survival using gene expressions versus histopathological imaging features. High-dimensional penalization methods are adopted for estimation and variable selection. Our findings include that gene expressions have slightly better prognostic performance, and that most of the gene expressions are weakly correlated imaging features. This study may provide additional insight into utilizing the two types of important data in cancer prognosis modeling and into lung cancer overall survival.

Toward standardized premarket evaluation of computer aided diagnosis/detection products: insights from FDA-approved products.

INTRODUCTION: Computer aided detection and diagnosis (CADe and CADx) products are an emerging branch of medical device industry. However, limited technical standard has been developed for product verification and validation. It will be helpful to investigate the current practice of preclinical and clinical evaluation of approved products and provide insights for future standardization. AREAS COVERED: Document review was conducted on 56 products approved by the United States Food and Drug Administration, including Summary of Safety and Effectiveness Data, 510(k) decision and de novo decision summaries. Key parameters describing product characteristics, preclinical studies and clinical studies were collected. Evaluation strategies for CADe/CADx products were analyzed and assessed. EXPERT OPINION: Preclinical studies were widely adopted in the verification of CADe/CADx products. Standalone performance testing was a common procedure, but the selection of testing dataset and performance metrics showed significant variability and flexibility among manufacturers. Clinical studies were reported by all class III products and some class II products, and Multi-Reader Multi-Case design was commonly used. However, statistical analysis and presentation/interpretation of results was oftentimes incomplete. To resolve above issues, systematic development of standards of CADe/CADx is encouraged, which can be implemented at different aspects through the product lifecycle.

The Impact of Artificial Intelligence in the Endoscopic Assessment of Premalignant and Malignant Esophageal Lesions: Present and Future.

In the gastroenterology field, the impact of artificial intelligence was investigated for the purposes of diagnostics, risk stratification of patients, improvement in quality of endoscopic procedures and early detection of neoplastic diseases, implementation of the best treatment strategy, and optimization of patient prognosis. Computer-assisted diagnostic systems to evaluate upper endoscopy images have recently emerged as a supporting tool in endoscopy due to the risks of misdiagnosis related to standard endoscopy and different expertise levels of endoscopists, time-consuming procedures, lack of availability of advanced procedures, increasing workloads, and development of endoscopic mass screening programs. Recent research has tended toward computerized, automatic, and real-time detection of lesions, which are approaches that offer utility in daily practice. Despite promising results, certain studies might overexaggerate the diagnostic accuracy of artificial systems, and several limitations remain to be overcome in the future. Therefore, additional multicenter randomized trials and the development of existent database platforms are needed to certify clinical implementation. This paper presents an overview of the literature and the current knowledge of the usefulness of different types of machine learning systems in the assessment of premalignant and malignant esophageal lesions via conventional and advanced endoscopic procedures. This study makes a presentation of the artificial intelligence terminology and refers also to the most prominent recent research on computer-assisted diagnosis of neoplasia on Barrett's esophagus and early esophageal squamous cell carcinoma, and prediction of invasion depth in esophageal neoplasms. Furthermore, this review highlights the main directions of future doctor-computer collaborations in which machines are expected to improve the quality of medical action and routine clinical workflow, thus reducing the burden on physicians.

Multivariate Data Analysis and Machine Learning for Prediction of MCI-to-AD Conversion.

There has always been a need for discovering efficient and dependable Alzheimer's disease (AD) diagnostic biomarkers. Like the majority of diseases, the earlier the diagnosis, the most effective the treatment. (Semi)-automated structural magnetic resonance imaging (MRI) processing approaches are very popular in AD research. Mild cognitive impairment (MCI) is considered to be a stage between normal cognitive ageing and dementia. MCI can often be the prodromal stage of AD. Around 10-15% of MCI patients convert to AD per year. In this study, we used three supervised machine learning (ML) techniques to differentiate MCI converters (MCIc) from MCI non-converters (MCInc) and predict their conversion rates from baseline MRI data (cortical thickness (CTH) and hippocampal volume (HCV)). A total of 803 participants from the ADNI cohort were included in this study (188 AD, 107 MCIc, 257 MCInc and 156 healthy controls (HC)). We studied the classification abilities of three different WEKA classifiers (support vector machine (SVM), decision trees (J48) and Naive Bayes (NB)). We built six different classification models, three models based on CTH and three based on HCV (CTH-SVM, CTH-J48, CTH-NB, HCV-SVM, HCV-J48 and HCV-NB). For the classification experiments, we obtained up to 71% sensitivity and up to 56% specificity. The prediction of conversion showed accuracy for up to 84%. The value of certain multivariate models derived from the classification experiments has exhibited robust and effective results in MCIc identification. However, there was a limitation in this study since we could not compare the CTH with the HCV models seeing as the data used originated from different subjects. As future direction, we propose the creation of a model that would combine various features with data originating from the same subjects, thus being a far more reliable and accurate prognostic tool.

Problems in variation interpretation guidelines and in their implementation in computational tools.

BACKGROUND: ACMG/AMP and AMP/ASCO/CAP have released guidelines for variation interpretation, and ESHG for diagnostic sequencing. These guidelines contain recommendations including the use of computational prediction methods. The guidelines per se and the way they are implemented cause some problems. METHODS: Logical reasoning based on domain knowledge. RESULTS: According to the guidelines, several methods have to be used and they have to agree. This means that the methods with the poorest performance overrule the better ones. The choice of the prediction method(s) should be made by experts  based on systematic benchmarking studies reporting all the relevant performance measures. Currently variation interpretation methods have been applied mainly to amino acid substitutions and splice site variants; however, predictors for some other types of variations are available and there will be tools for new application areas in the near future. Common problems in prediction method usage are discussed. The number of features used for method training or the number of variation types predicted by a tool are not indicators of method performance. Many published gene, protein or disease-specific benchmark studies suffer from too small dataset rendering the results useless. In the case of binary predictors, equal number of positive and negative cases is beneficial for training, the imbalance has to be corrected for performance assessment. Predictors cannot be better than the data they are based on and used for training and testing. Minor allele frequency (MAF) can help to detect likely benign cases, but the recommended MAF threshold is apparently too high. The fact that many rare variants are disease-causing or -related does not mean that rare variants in general would be harmful. How large a portion of the tested variants a tool can predict (coverage) is not a quality measure. CONCLUSION: Methods used for variation interpretation have to be carefully selected. It should be possible to use only one predictor, with proven good performance or a limited number of complementary predictors with state-of-the-art performance. Bear in mind that diseases and pathogenicity have a continuum and variants are not dichotomic i.e. either pathogenic or benign, either.