Artificial neural networks: opening the black box.

Artificial neural networks now are used in many fields. They have become well established as viable, multipurpose, robust computational methodologies with solid theoretic support and with strong potential to be effective in any discipline, especially medicine. For example, neural networks can extract new medical information from raw data, build computer models that are useful for medical decision-making, and aid in the distribution of medical expertise. Because many important neural network applications currently are emerging, the authors have prepared this article to bring a clearer understanding of these biologically inspired computing paradigms to anyone interested in exploring their use in medicine. They discuss the historical development of neural networks and provide the basic operational mathematics for the popular multilayered perceptron. The authors also describe good training, validation, and testing techniques, and discuss measurements of performance and reliability, including the use of bootstrap methods to obtain confidence intervals. Because it is possible to predict outcomes for individual patients with a neural network, the authors discuss the paradigm shift that is taking place from previous "bin-model" approaches, in which patient outcome and management is assumed from the statistical groups in which the patient fits. The authors explain that with neural networks it is possible to mediate predictions for individual patients with prevalence and misclassification cost considerations using receiver operating characteristic methodology. The authors illustrate their findings with examples that include prostate carcinoma detection, coronary heart disease risk prediction, and medication dosing. The authors identify and discuss obstacles to success, including the need for expanded databases and the need to establish multidisciplinary teams. The authors believe that these obstacles can be overcome and that neural networks have a very important role in future medical decision support and the patient management systems employed in routine medical practice.

Prevalence-value-accuracy plots: a new method for comparing diagnostic tests based on misclassification costs.

The clinical accuracy of diagnostic tests commonly is assessed by ROC analysis. ROC plots, however, do not directly incorporate the effect of prevalence or the value of the possible test outcomes on test performance, which are two important factors in the practical utility of a diagnostic test. We describe a new graphical method, referred to as a prevalence-value-accuracy (PVA) plot analysis, which includes, in addition to accuracy, the effect of prevalence and the cost of misclassifications (false positives and false negatives) in the comparison of diagnostic test performance. PVA plots are contour plots that display the minimum cost attributable to misclassifications (z-axis) at various optimum decision thresholds over a range of possible values for prevalence (x-axis) and the unit cost ratio (UCR; y-axis), which is an index of the cost of a false-positive vs a false-negative test result. Another index based on the cost of misclassifications can be derived from PVA plots for the quantitative comparison of test performance. Depending on the region of the PVA plot that is used to calculate the misclassification cost index, it can potentially lead to a different interpretation than the ROC area index on the relative value of different tests. A PVA-threshold plot, which is a variation of a PVA plot, is also described for readily identifying the optimum decision threshold at any given prevalence and UCR. In summary, the advantages of PVA plot analysis are the following: (a) it directly incorporates the effect of prevalence and misclassification costs in the analysis of test performance; (b) it yields a quantitative index based on the costs of misclassifications for comparing diagnostic tests; (c) it provides a way to restrict the comparison of diagnostic test performance to a clinically relevant range of prevalence and UCR; and (d) it can be used to directly identify an optimum decision threshold based on prevalence and misclassification costs.

Pleural fluid chemical analysis in parapneumonic effusions. A meta-analysis.

Controversy exists regarding the clinical utility of pleural fluid pH, lactate dehydrogenase (LDH), and glucose for identifying complicated parapneumonic effusions that require drainage. In this report, we performed a meta-analysis of pertinent studies, using receiver operating characteristic (ROC) techniques, to assess the diagnostic accuracy of these tests, to determine appropriate decision thresholds, and to evaluate the quality of the primary studies. Seven primary studies reporting values for pleural fluid pH (n = 251), LDH (n = 114), or glucose (n = 135) in pneumonia patients were identified. We found that pleural fluid pH had the highest diagnostic accuracy for all patients with parapneumonic effusions as measured by the area under the ROC curve (AUC = 0.92) compared with pleural fluid glucose (AUC = 0.84) or LDH (AUC = 0.82). After excluding patients with purulent effusions, pH (AUC = 0.89) retained the highest diagnostic accuracy. Pleural fluid pH decision thresholds varied between 7.21 and 7.29 depending on cost-prevalence considerations. The quality of the primary studies was the major limitation in determining the value of pleural fluid chemical analysis. We conclude that meta-analysis of the available data refines the application of pleural fluid chemical analysis but a clearer understanding of the usefulness of these tests awaits more rigorous primary investigations.