Development of a dental digital data set for research in artificial intelligence: The importance of labeling performed by radiologists.

OBJECTIVE: The aim of this study was to present the development of a database (dataset) of panoramic radiographs. STUDY DESIGN: Three radiologists labeled an image set consisting of 936 panoramic radiographs. Labeling includes tooth numbering (including teeth present and missing) and annotation of dental conditions (e.g., caries, dental restoration, residual root, endodontic treatment, implant, fixed prosthesis, incisal wear). The annotation process was performed in a Picture Archive and Communication System software customized for the study purposes using a small bounding box to delimit the entire tooth and items for radiographic diagnosis and a large bounding box to simultaneously delimit the 2 dental arches (maxilla and mandible). A JSON file was generated for each annotation. RESULTS: The database encompassed 23,619 annotations; disagreement between radiologists occurred in 0.7% of the notes. CONCLUSIONS: This work aims to inform researchers about the importance of the labeling process, in addition to providing the scientific community with a bank of labeled images to implement artificial intelligence systems in clinical practice.

Prediction of mortality in Intensive Care Units: a multivariate feature selection.

CONTEXT: The critical nature of patients in Intensive Care Units (ICUs) demands intensive monitoring of their vital signs as well as highly qualified professional assistance. The combination of these needs makes ICUs very expensive, which requires investment to be prioritized. Administrative issues emerge, and health institutions face dilemmas such as: "How many beds should an ICU provide to serve the population, at the lowest costs" and "Which is the most critical body information to monitor in an ICU?". Due to financial and ethical implications, these judgments require technical and precise knowledge. Decisions have usually relied on clinical scores, like the APACHE (Acute Physiology And Chronic Health Evaluation) and SOFA (Sequential Organ Failure Assessment) scores, which are imprecise and outdated. The popularization of machine learning techniques has shed some light on the topic as a way to renew score purposes. In 2012, the PhysioNet/Computing in Cardiology launched the Challenge - ICU Patients. This Challenge aimed to stimulate the development of techniques to predict mortality in ICUs. Based on biometric and physiological features collected from patients, the participants predicted the patient's death risk by using their classifiers. Several participants achieved results that were better than the results produced by the SOFA and the APACHE scores; the prediction levels were ≈54%, which is weak. OBJECTIVES: Here, we investigate the reasons that led to these results as a means to ground our solution. Then, we propose alternative practices in an attempt to improve the results. Our main goal is to improve the prediction of mortality in ICUs by using the same data employed during the 2012 PhysioNet Challenge. Our specific objectives are (i) to simplify the problem by reducing the dimensionality; (ii) to reduce the uncontrolled variance, and (iii) to make classifiers less dependent on the training set. METHODS: Accordingly, we propose a methodology based on extensive steps, including sample filter and data normalization. To select features and to reduce the intra-group variance, we employ multivariate data analysis by using Principal Component Analysis, Factor Analysis, Spectral Clustering, and Tukey's HSD Test, recursively. After that, we use machine learning techniques to create classifiers according to different methods. We evaluate our results with the same metrics proposed by the 2012 PhysioNet Challenge. RESULTS: For classifiers constructed and tested by using independent datasets, our best classifier was a linear SVM, which provided results of ≈0.73. These results were significantly better than the ≈0.54 achieved in previous work at >99% confidence interval. Furthermore, our approach only demanded twelve features, which was consistently smaller than the number of features required by the previous approaches. CONCLUSION: Our results indicated that our approach presented: (a) higher performance to predict death risks (+20%); (b) smaller dependence on the training set; and (c) lower costs for ICU monitoring (few features). Besides the better prediction power, our approach also demanded lower costs for implementation and a more extensive range of potential ICUs. Future studies should employ our proposal to investigate the possibility of including some physiological features that were not available for the 2012 PhysioNet Challenge.

A Health Surveillance Software Framework to deliver information on preventive healthcare strategies.

A software framework can reduce costs related to the development of an application because it allows developers to reuse both design and code. Recently, companies and research groups have announced that they have been employing health software frameworks. This paper presents the design, proof-of-concept implementations and experimentation of the Health Surveillance Software Framework (HSSF). The HSSF is a framework that tackles the demand for the recommendation of surveillance information aiming at supporting preventive healthcare strategies. Examples of such strategies are the automatic recommendation of surveillance levels to patients in need of healthcare and the automatic recommendation of scientific literature that elucidates epigenetic problems related to patients. HSSF was created from two systems we developed in our previous work on health surveillance systems: the Automatic-SL and CISS systems. The Automatic-SL system aims to assist healthcare professionals in making decisions and in identifying children with developmental problems. The CISS service associates genetic and epigenetic risk factors related to chronic diseases with patient's clinical records. Towards evaluating the HSSF framework, two new systems, CISS+ and CISS-SW, were created by means of abstractions and instantiations of the framework (design and code). We show that HSSF supported the development of the two new systems given that they both recommend scientific papers using medical records as queries even though they exploit different computational technologies. In an experiment using simulated patients' medical records, we show that CISS, CISS+, and CISS-SW systems recommended more closely related and somewhat related documents than Google, Google Scholar and PubMed. Considering recall and precision measures, CISS+ surpasses CISS-SW in terms of precision.

A multi-label approach using binary relevance and decision trees applied to functional genomics.

Many classification problems, especially in the field of bioinformatics, are associated with more than one class, known as multi-label classification problems. In this study, we propose a new adaptation for the Binary Relevance algorithm taking into account possible relations among labels, focusing on the interpretability of the model, not only on its performance. Experiments were conducted to compare the performance of our approach against others commonly found in the literature and applied to functional genomic datasets. The experimental results show that our proposal has a performance comparable to that of other methods and that, at the same time, it provides an interpretable model from the multi-label problem.

Surveillance for the prevention of chronic diseases through information association.

BACKGROUND: Research on Genomic medicine has suggested that the exposure of patients to early life risk factors may induce the development of chronic diseases in adulthood, as the presence of premature risk factors can influence gene expression. The large number of scientific papers published in this research area makes it difficult for the healthcare professional to keep up with individual results and to establish association between them. Therefore, in our work we aim at building a computational system that will offer an innovative approach that alerts health professionals about human development problems such as cardiovascular disease, obesity and type 2 diabetes. METHODS: We built a computational system called Chronic Illness Surveillance System (CISS), which retrieves scientific studies that establish associations (conceptual relationships) between chronic diseases (cardiovascular diseases, diabetes and obesity) and the risk factors described on clinical records. To evaluate our approach, we submitted ten queries to CISS as well as to three other search engines (Google™, Google Scholar™ and Pubmed®;) - the queries were composed of terms and expressions from a list of risk factors provided by specialists. RESULTS: CISS retrieved a higher number of closely related (+) and somewhat related (+/-) documents, and a smaller number of unrelated (-) and almost unrelated (-/+) documents, in comparison with the three other systems. The results from the Friedman's test carried out with the post-hoc Holm procedure (95% confidence) for our system (control) versus the results for the three other engines indicate that our system had the best performance in three of the categories (+), (-) and (+/-). This is an important result, since these are the most relevant categories for our users. CONCLUSION: Our system should be able to assist researchers and health professionals in finding out relationships between potential risk factors and chronic diseases in scientific papers.

Evaluation of Stacking on Biomedical Data / Avaliação de Stacking em Dados Biomédicos / Evaluación de Stacking en Datos Biomédicos

Objectives: Stacking is a well-known ensemble technique, but some of its aspects still need to be explored, e.g., there are few recommendations on which and how many algorithms should be used at level-0 or even which algorithm should be used to compose the level-1 meta-classifier. The literature indicates the meta-algorithm at level-1 should be simple, and Naive Bayes has been typically used in these studies. Methods: In this work, we have analyzed stacking on biomedical datasets, using three different paradigms of machine learning algorithms to compose the meta-classifier. Results: The experiments indicate simple meta-algorithms do not provide good results. Conclusion: the meta-classifier must have a degree of complexity to provide a nice performance.

Objetivos: Stacking é uma técnica bem conhecida de combinação de classificadores, mas alguns de seus aspectos ainda precisam ser explorados, por exemplo, existem poucas recomendações sobre quais e quantos algoritmos devem ser utilizados no nível-0 ou ainda qual algoritmo deve ser usado para compor o meta-classificador do nível-1. A literatura indica que o meta-algoritmo no nível-1 deve ser simples e geralmente Naive Bayes tem sido usado nestes estudos. Métodos: Neste trabalho, o algoritmo de stacking foi avaliado em dados biomédicos, usando três algoritmos de aprendizado de máquina de diferentes paradigmas para compor o meta-classificador. Resultados: Os experimentos indicam que meta-algoritmos simples não fornecem bons resultados. Conclusão: O meta-classificador deve ter um grau de complexidade para oferecer um bom desempenho.

Objetivos: Stacking es una técnica de combinación de clasificadores bien conocida, pero algunos aspectos quedan por explorar, por ejemplo, existen pocas recomendaciones sobre cuales o cuantos algoritmos deben utilizarse en el nível-0 o aun cual algoritmo debe usarse para componer el nível-1. La literatura indica que el meta-algoritmo debe ser simple y, generalmente, Naive Bayes ha sido usado en estos estudios. Métodos: En este trabajo, se analiza el algoritmo de stacking con datos biomédicos, utilizando tres algoritmos de aprendizaje automático de distintos paradigmas para componer el meta-clasificador. Resultados: Los experimentos indican que meta-algoritmos simples no ofrecen buenos resultados. Conclusión: El meta-clasificador debe tener un grado de complejidad para obtener un buen rendimiento.

SigrafW: An easy-to-use program for fitting enzyme kinetic data.

SigrafW is Windows-compatible software developed using the Microsoft® Visual Basic Studio program that uses the simplified Hill equation for fitting kinetic data from allosteric and Michaelian enzymes. SigrafW uses a modified Fibonacci search to calculate maximal velocity (V), the Hill coefficient (n), and the enzyme-substrate apparent dissociation constant (K). The estimation of V, K, and the sum of the squares of residuals is performed using a Wilkinson nonlinear regression at any Hill coefficient (n). In contrast to many currently available kinetic analysis programs, SigrafW shows several advantages for the determination of kinetic parameters of both hyperbolic and nonhyperbolic saturation curves. No initial estimates of the kinetic parameters are required, a measure of the goodness-of-the-fit for each calculation performed is provided, the nonlinear regression used for calculations eliminates the statistical bias inherent in linear transformations, and the software can be used for enzyme kinetic simulations either for educational or research purposes. Persons interested in receiving a free copy of the software should contact Dr. F. A. Leone.