Long-term results of microcoil embolization for colonic haemorrhage: how common is rebleeding?

OBJECTIVE: To determine the long-term results of patients undergoing transcatheter coil embolization for the treatment of acute colonic haemorrhage. METHODS: Patients undergoing angiography for suspected colonic bleeding between January 2002 and December 2012 were reviewed (average age, 60 years; 38.4% male). Baseline, procedural and outcome parameters were recorded following the Society of Interventional Radiology guidelines. Primary outcome measures included early (<30 days) and delayed (>30 days) rebleeding events and adverse procedure-related complication. Average follow-up time was 996 days (median, 232 days; range, 30-3663 days). RESULTS: One or multiple sites of bleeding were identified in 40 cases. Coil embolization was performed in 39 patients, 26 (66.7%, 26/39) of whom were treated successfully without technical/clinical failure (n = 12) or loss to follow-up (n = 1). Three patients (11.5%, 3/26) rebled in the early period within 30 days; one patient went on to hemicolectomy. Four patients (15.3%, 4/26) experienced delayed rebleeding after 30 days; two of whom also underwent hemicolectomy. No major complication occurred. One minor complication of short segment arterial dissection was seen in the clinical failure group. One case of asymptomatic ischaemia was identified on a patient undergoing pre-operative colonoscopy for elective bowel resection. No instances of ischaemic stricture were seen. All-cause mortality of successfully treated and all patients at 1 year was 31% (8/26) and 30% (12/40), respectively. CONCLUSION: Transcatheter coil embolization is a durable treatment option with a technical and clinical success rate of 67% in the setting of acute colonic haemorrhage. A modest level of rebleeding was seen among successfully treated patients in both the early and delayed periods; in the majority of patients, embolization proved to be definitive therapy. ADVANCES IN KNOWLEDGE: Transcatheter coil embolization is a durable and potentially definitive therapy in the management of acute colonic haemorrhage.

Automated detection of mass lesions in dedicated breast CT: a preliminary study.

PURPOSE: To develop an automated method to detect breast masses on dedicated breast CT (BCT) volumes and to conduct a preliminary evaluation of its performance. This method can be used in a computer-aided detection (CADe) system for noncontrast enhanced BCT. METHODS: The database included patient images, which were acquired under an IRB-approved protocol. The database in this study consisted of 132 cases. 50 cases contained 58 malignant masses, and 23 cases contained 24 benign masses. 59 cases did not contain any biopsy-proven lesions. Each case consisted of an unenhanced CT volume of a single breast. First, each breast was segmented into adipose and glandular tissues using a fuzzy c-means clustering algorithm. The glandular breast regions were then sampled at a resolution of 2 mm. At each sampling step, a 3.5-cm(3) volume-of-interest was subjected to constrained region segmentation and 17 characteristic features were extracted, yielding 17 corresponding feature volumes. Four features were selected using step-wise feature selection and merged with linear discriminant analysis trained in the task of distinguishing between normal breast glandular regions and masses. Detection performance was measured using free-response receiver operating characteristic analysis (FROC) with leave-one-case-out evaluation. RESULTS: The feature selection stage selected features that characterized the shape and margin strength of the segmented region. CADe sensitivity per case was 84% (std = 4.2%) at 2.6 (std = 0.06) false positives per volume, or 6 × 10(-3) per slice (at an average of 424 slices per volume in this data set). CONCLUSIONS: This preliminary study demonstrates the feasibility of our approach for CADe for BCT.

SU-E-E-03: Ethics and Professionalism Education in Medical Physics: A Needs Assessment Study.

PURPOSE: To perform a needs assessment survey of ethics/professionalism education in medical physics and ethical/professional challenges in clinical,research and educational settings with the intent of supplementing and customizing TG159 recommended ethics curriculum for medical physics trainees. METHODS: A web-based survey was conducted among AAPM members to assess current practices, attitudes and perceptions pertaining to ethics/professionalism education and ethical/professional misconduct or questionable behavior and practices in the field. RESULTS: The survey was distributed by AAPM to 7708 members via email; 1362 (17.7%) responded. Seventy-five percent of the respondents were male. Sixty percent (805/1345) stated they received no education in ethics/professionalism. Eighty-one percent (126/155) of current trainees received instruction in ethics/professionalism, as opposed to 35% (392/1130) of those who are post-training. There was strong support (>90%) for continuing education in ethics/professionalism; seventy-five percent (1019/1354) supported sessions on ethics and professionalism at national meetings. Most preferred method of ethics instruction was periodic discussion sessions involving faculty and trainees, with the least interest expressed for a separate course. Many reported direct personal knowledge of one or more instances of a variety of professional/ethical misconduct or questionable behavior. Thirty eight percent (458/1192) reported poor mentorship, with women reporting this concern more often than men (129/281,46% versus 316/877, 36%, p<.05). Over one-fourth of respondents reported being asked to perform low educational value tasks and expressed concerns about fairness. A significant minority also reported questionable behavior with respect to authorship assignment (346/920, 38%), data fabrication (107/924, 12%), data falsification (94/919, 10%); concerns about research subject privacy and confidentiality were lower (64/887, 7%). CONCLUSIONS: Data gathered through the survey is guiding our efforts to develop a case-based ethics curriculum and instructional materials for medical physics trainees at our institution. This effort may be useful to other medical physics programs which offer ethics training/education. This work has been funded in part by a grant from the National Institutes of Health, T32 EB002103-22S1.

TU-E-217BCD-07: Pilot Study on Consistency in Size Metrics for a Multimodality PEM/MR Breast Imaging Approach.

PURPOSE: In this study, we evaluated the degree of consistency between size metrics obtained from PEM and MRI to determine the intrinsic effectiveness of a multimodality approach using these two systems for breast imaging. METHODS: Under an IRB-approved protocol, 42 cases were considered (16 patients with 28 lesions), each consisting of an MRI data set and corresponding PEM data set, with an inclusion criterion of being obtained within 2 weeks of each other. Lesions were delineated manually on the PEM images and semi-automatically on the MRI images for efficiency. In addition to volume, equivalent sphere diameter (ESD) was evaluated for each lesion. This metric describes the diameter of a sphere with the same volume as that of the lesion, and is useful for minimizing the cubic impact of a single voxel contribution inherent to volume calculations. RESULTS: The relationship between PEM based volumes and MRI based volumes showed a linear trend around VolMRI=VolPEM demonstrating a degree of consistency in the two volumes. The correlation between VolMRI and VolPEM was calculated as 0.547 with a corresponding p-value of 0.00018 demonstrating a significant correlation. The associated concordance was calculated as 0.534. The ESD metric showed a more significant linear trend with correlation 0.733 and corresponding p-value 3.5×10-8, indicating consistency in lesion size. The concordance is 0.717, indicating high reproducibility. CONCLUSION: Multimodality PEM/MRI breast imaging has the potential to combine functional and molecular imaging information for a powerful tool in cancer staging and evaluation of response to therapy. The resulting, expanded data set is of use only if fundamental size metrics are consistent between the two modalities. Our pilot data demonstrates that size metrics as we extracted from the image data are consistent between PEM and MRI breast image sets.Maryellen Giger is a stockholder in R2 Technology/Hologic, has equity in Quantitative Insights, and receives royalties from Hologic, GE Medical Systems, MEDIAN Technologies, Riverain Medical, Mitsubishi and Toshiba. It is the University of Chicago Conflict of Interest Policy that investigators disclose publicly actual or potential significant financial interest that would reasonably appear to be directly and significantly affected by the research activities.

Radiographic texture analysis of densitometer-generated calcaneus images differentiates postmenopausal women with and without fractures.

INTRODUCTION: Bone fragility is determined by bone mass, measured as bone mineral density (BMD), and by trabecular structure, which cannot be easily measured using currently available noninvasive methods. In previous studies, radiographic texture analysis (RTA) performed on the radiographic images of the spine, proximal femur, and os calcis differentiated subjects with and without osteoporotic fractures. The present cross-sectional study was undertaken to determine whether such differentiation could also be made using high-resolution os calcis images obtained on a peripheral densitometer. METHODS: In 170 postmenopausal women (42 with and 128 without prevalent vertebral fractures) who had no secondary causes of osteoporosis and were not receiving treatment for osteoporosis, BMD of the lumbar spine, proximal femur, and os calcis was measured using dual energy x-ray absorptiometry. Vertebral fractures were diagnosed on densitometric spine images. RTA, including Fourier-based and fractal analyses, was performed on densitometric images of os calcis. RESULTS: BMD at all three sites and all texture features was significantly different in subjects with and without fractures, with the most significant differences observed for the femoral neck and total hip measurements and for the RTA feature Minkowski fractal (p<0.001). In univariate logistic regression analysis, Minkowski fractal predicted the presence of vertebral fractures as well as femoral neck BMD (p<0.001). In multivariate logistic regression analysis, both femoral neck BMD and Minkowski fractal yielded significant predictive effects (p=0.001), and when age was added to the model, the effect of RTA remained significant (p=0.002), suggesting that RTA reflects an aspect of bone fragility that is not captured by age or BMD. Finally, when RTA was compared in 42 fracture patients and 42 nonfracture patients matched for age and BMD, the RTA features were significantly different between the groups (p=0.003 to p=0.04), although BMD and age were not. CONCLUSION: This study suggests that RTA of densitometer-generated calcaneus images provides an estimate of bone fragility independent of and complementary to BMD measurement and age.

Computerized mass detection for digital breast tomosynthesis directly from the projection images.

Digital breast tomosynthesis (DBT) has recently emerged as a new and promising three-dimensional modality in breast imaging. In DBT, the breast volume is reconstructed from 11 projection images, taken at source angles equally spaced over an arc of 50 degrees. Reconstruction algorithms for this modality are not fully optimized yet. Because computerized lesion detection in the reconstructed breast volume will be affected by the reconstruction technique, we are developing a novel mass detection algorithm that operates instead on the set of raw projection images. Mass detection is done in three stages. First, lesion candidates are obtained for each projection image separately, using a mass detection algorithm that was initially developed for screen-film mammography. Second, the locations of a lesion candidate are backprojected into the breast volume. In this feature volume, voxel intensities are a combined measure of detection frequency (e.g., the number of projections in which a given lesion candidate was detected), and a measure of the angular range over which a given lesion was detected. Third, features are extracted after reprojecting the three-dimensional (3-D) locations of lesion candidates into projection images. Features are combined using linear discriminant analysis. The database used to test the algorithm consisted of 21 mass cases (13 malignant, 8 benign) and 15 cases without mass lesions. Based on this database, the algorithm yielded a sensitivity of 90% at 1.5 false positives per breast volume. Algorithm performance is positively biased because this dataset was used for development, training, and testing, and because the number of algorithm parameters was approximately the same as the number.of patient cases. Our results indicate that computerized mass detection in the sequence of projection images for DBT may be effective despite the higher noise level in those images.

Computerized detection of mass lesions in digital breast tomosynthesis images using two- and three dimensional radial gradient index segmentation.

Initial results for a computerized mass lesion detection scheme for digital breast tomosynthesis (DBT) images are presented. The algorithm uses a radial gradient index feature for the initial lesion detection and for segmentation of lesion candidates. A set of features is extracted for each segmented partition. Performance of two- and three dimensional features was compared. For gradient features, the additional dimension provided no improvement in classification performance. For shape features, classification using 3D features was improved compared to the 2D equivalent features. The preliminary overall performance was 76% sensitivity at 11 false positives per exam, estimated based on DBT image data of 21 masses. A larger database will allow for further development and improvement in our computer aided detection scheme.

Ideal observer approximation using Bayesian classification neural networks.

It is well understood that the optimal classification decision variable is the likelihood ratio or any monotonic transformation of the likelihood ratio. An automated classifier which maps from an input space to one of the likelihood ratio family of decision variables is an optimal classifier or "ideal observer." Artificial neural networks (ANNs) are frequently used as classifiers for many problems. In the limit of large training sample sizes, an ANN approximates a mapping function which is a monotonic transformation of the likelihood ratio, i.e., it estimates an ideal observer decision variable. A principal disadvantage of conventional ANNs is the potential over-parameterization of the mapping function which results in a poor approximation of an optimal mapping function for smaller training samples. Recently, Bayesian methods have been applied to ANNs in order to regularize training to improve the robustness of the classifier. The goal of training a Bayesian ANN with finite sample sizes is, as with unlimited data, to approximate the ideal observer. We have evaluated the accuracy of Bayesian ANN models of ideal observer decision variables as a function of the number of hidden units used, the signal-to-noise ratio of the data and the number of features or dimensionality of the data. We show that when enough training data are present, excess hidden units do not substantially degrade the accuracy of Bayesian ANNs. However, the minimum number of hidden units required to best model the optimal mapping function varies with the complexity of the data.

Automated detection of lung nodules in CT scans: preliminary results.

We have developed a fully automated computerized method for the detection of lung nodules in helical computed tomography (CT) scans of the thorax. This method is based on two-dimensional and three-dimensional analyses of the image data acquired during diagnostic CT scans. Lung segmentation proceeds on a section-by-section basis to construct a segmented lung volume within which further analysis is performed. Multiple gray-level thresholds are applied to the segmented lung volume to create a series of thresholded lung volumes. An 18-point connectivity scheme is used to identify contiguous three-dimensional structures within each thresholded lung volume, and those structures that satisfy a volume criterion are selected as initial lung nodule candidates. Morphological and gray-level features are computed for each nodule candidate. After a rule-based approach is applied to greatly reduce the number of nodule candidates that corresponds to nonnodules, the features of remaining candidates are merged through linear discriminant analysis. The automated method was applied to a database of 43 diagnostic thoracic CT scans. Receiver operating characteristic (ROC) analysis was used to evaluate the ability of the classifier to differentiate nodule candidates that correspond to actual nodules from false-positive candidates. The area under the ROC curve for this categorization task attained a value of 0.90 during leave-one-out-by-case evaluation. The automated method yielded an overall nodule detection sensitivity of 70% with an average of 1.5 false-positive detections per section when applied to the complete 43-case database. A corresponding nodule detection sensitivity of 89% with an average of 1.3 false-positive detections per section was achieved with a subset of 20 cases that contained only one or two nodules per case.

Automatic segmentation of breast lesions on ultrasound.

In this paper we present a computationally efficient segmentation algorithm for breast masses on sonography that is based on maximizing a utility function over partition margins defined through gray-value thresholding of a preprocessed image. The performance of the segmentation algorithm is evaluated on a database of 400 cases in two ways. Of the 400 cases, 124 were complex cysts, 182 were benign solid lesions, and 94 were malignant lesions. In the first evaluation, the computer-delineated margins were compared to manually delineated margins. At an overlap threshold of 0.40, the segmentation algorithm correctly delineated 94% of the lesions. In the second evaluation, the performance of our computer-aided diagnosis method on the computer-delineated margins was compared to the performance of our method on the manually delineated margins. Round robin evaluation yielded Az values of 0.90 and 0.87 on the manually delineated margins and the computer-delineated margins, respectively, in the task of distinguishing between malignant and nonmalignant lesions.

Computerized analysis of multiple-mammographic views: potential usefulness of special view mammograms in computer-aided diagnosis.

PURPOSE: To investigate the potential usefulness of special view mammograms in the computer-aided diagnosis of mammographic breast lesions. MATERIALS AND METHODS: Previously, we developed a computerized method for the classification of mammographic mass lesions on standard-view mammograms, i.e., mediolateral oblique (MLO) view and/or cranial caudal (CC) views. In this study, we evaluate the performance of our computerized classification method on an independent database consisting of 70 cases (33 malignant and 37 benign cases), each having CC, MLO, and special view mammograms (spot compression or spot compression magnification views). The mass lesion identified in each of the three mammographic views was analyzed using our previously developed and trained computerized classification method. Performance in the task of distinguishing between malignant and benign lesions was evaluated using receiver operating characteristic analysis. On this independent database, we compared the performance of individual computer-extracted mammographic features, as well as the computer-estimated likelihood of malignancy, for the standard and special views. RESULTS: Computerized analysis of special view mammograms alone in the task of distinguishing between malignant and benign lesions yielded an Az of 0.95, which is significantly higher (p < 0.005) than that obtained from the MLO and CC views (Az values of 0.78 and 0.75, respectively). Use of only the special views correctly classified 19 of 33 benign cases (a specificity of 58%) at 100% sensitivity, whereas use of the CC and MLO views alone correctly classified 4 and 8 of 33 benign cases (specificities of 12% and 24%, respectively). In addition, we found that the average computer output of the three views (Az of 0.95) yielded a significantly better performance than did the maximum computer output from the mammographic views. CONCLUSIONS: Computerized analysis of special view mammograms provides an improved prediction of the benign versus malignant status of mammographic mass lesions.

Computerized classification of benign and malignant masses on digitized mammograms: a study of robustness.

RATIONALE AND OBJECTIVES: The purpose of this study was to evaluate the robustness of a computerized method developed for the classification of benign and malignant masses with respect to variations in both case mix and film digitization. MATERIALS AND METHODS: The classification method included automated segmentation of mass regions, automated feature-extraction, and automated lesion characterization. The method was evaluated independently with a 110-case database consisting of 50 malignant and 60 benign cases. Mammograms were digitized twice with two different digitizers (Konica and Lumisys). Performance of the method in differentiating benign from malignant masses was evaluated with receiver operating characteristic (ROC) analysis. Effects of variations in both case mix and film digitization on performance of the method also were assessed. RESULTS: Categorization of lesions as malignant or benign with an artificial neural network (or a hybrid) classifier achieved an area under the ROC curve, Az, value of 0.90 (0.94 for the hybrid) on the previous training database in a round-robin evaluation and Az values of 0.82 (0.81) and 0.81 (0.82) on the independent database for the Konica and Lumisys formats, respectively. These differences, however, were not statistically significant (P > .10). CONCLUSION: The computerized method for the classification of lesions on mammograms was robust with respect to variations in case mix and film digitization.

Computer-aided detection and diagnosis of breast cancer.

The limitations of radiologists when interpreting mammogram examinations provides a reasonable, if not compelling, basis for application of computer techniques that have the potential to improve diagnostic performance. Computer algorithms, at their present state of development, show great promise for clinical use. It can be expected that such use will only improve as computer technology and computer methods continue to become more formidable. The eventual role of computers in mammographic detection and diagnosis has not been fully defined, but their effect on practice may one day be very significant.

Automated registration of frontal and lateral radionuclide lung scans with digital chest radiographs.

RATIONALE AND OBJECTIVES: The purpose of this study was to develop and evaluate a fully automated method that spatially registers anterior, posterior, and lateral ventilation/perfusion (V/Q) images with posteroanterior and lateral digital chest radiographs to retrospectively combine the physiologic information contained in the V/Q scans with the anatomic detail in the chest radiographs. MATERIALS AND METHODS: Gray-level thresholding techniques were used to segment the aerated lung regions in the radiographic images. A variable-thresholding technique combined with an analysis of image noise was used to segment the adequately perfused or ventilated lung regions in the scintigraphic images. The physical dimensions of the segmented lung regions in images from both modalities were used to properly scale the radiographic images relative to the radionuclide images. Computer-determined locations of anatomic landmarks were then used to rotate and translate the images to achieve registration. Pairs of corresponding radionuclide and radiographic images were enhanced with color and then merged to create superimposed images. RESULTS: Five observers used a five-point rating scale to subjectively evaluate four image combinations for each of 50 cases. Of these ratings, 95.5% reflected very good, good, or fair registration. CONCLUSION: The automated method for the registration of radionuclide lung scans with digital chest radiographs to produce images that combine functional and structural information should benefit nuclear medicine physicians and radiologists, who must visually correlate images that differ greatly in physical size, resolution properties, and information content.

Computerized analysis of mammographic parenchymal patterns for breast cancer risk assessment: feature selection.

Our purpose in this study was to identify computer-extracted, mammographic parenchymal patterns that are associated with breast cancer risk. We extracted 14 features from the central breast region on digitized mammograms to characterize the mammographic parenchymal patterns of women at different risk levels. Two different approaches were employed to relate these mammographic features to breast cancer risk. In one approach, the features were used to distinguish mammographic patterns seen in low-risk women from those who inherited a mutated form of the BRCA1/BRCA2 gene, which confers a very high risk of developing breast cancer. In another approach, the features were related to risk as determined from existing clinical models (Gail and Claus models), which use well-known epidemiological factors such as a woman's age, her family history of breast cancer, reproductive history, etc. Stepwise linear discriminant analysis was employed to identify features that were useful in differentiating between "low-risk" women and BRCA1/BRCA2-mutation carriers. Stepwise linear regression analysis was employed to identify useful features in predicting the risk, as estimated from the Gail and Claus models. Similar computer-extracted mammographic features were identified in the two approaches. Results show that women at high risk tend to have dense breasts and their mammographic patterns tend to be coarse and low in contrast.

Computerized analysis of radiographic bone patterns: effect of imaging conditions on performance.

We are developing computerized methods for characterizing the bone texture pattern from digitized skeletal radiographs. For this method to be useful clinically, it must be able to distinguish between weak and strong bone under the range of exposure conditions potentially encountered in the clinical setting. In this study, we examined the effect of exposure conditions on Fourier-based texture features. Thirty-four femoral specimens from total hip arthroplasties were radiographed multiple times under different exposure conditions. The specimens underwent mechanical strength testing from which load to failure values were obtained. The performance of the texture features were investigated in the task of distinguishing between strong and weak bone as characterized by the load to failure values. The texture features showed no dependence upon focal spot size of the x-ray tube or magnification. The texture features did show a dependence with relative exposure, peak kilovoltage, and amount of scattering material.

Normalized BMD as a predictor of bone strength.

RATIONALE AND OBJECTIVES: In the noninvasive evaluation of bone quality, bone mineral density (BMD) has been shown to be the single most important predictor of bone strength and osteoporosis-related fracture. Among the methods of measuring BMD, dual x-ray absorptiometry (DXA) has widespread acceptance due to its low radiation, low cost, and high precision. However, DXA measures area BMD instead of true volumetric density; thus, a larger bone will tend to have a high BMD than will a smaller bone. Therefore, the comparison of BMDs of bones of different sizes can be misleading. In this study, the authors tried to compensate for the size effect by normalizing the area BMD with bone size as measured from a standard pelvic radiograph. MATERIALS AND METHODS: The overall method for calculation of normalized BMD included conventional area-based BMD from DXA and the extraction of geometric measures from pelvic radiographs. The database for analysis included 34 femoral neck specimens. Regression analysis was performed between the normalized volumetric BMD, measured from femoral neck region, and the mechanical properties obtained from trabecular bone cubes machined from the same region. RESULTS: After normalization of the area BMD, the coefficient of determination increased from 0.30 to 0.43 for the Young modulus and from 0.27 to 0.37 for bone compressive strength. CONCLUSION: A noninvasive method of normalizing BMD can improve the prediction of bone mechanical properties and has potential in monitoring changes in growing skeletons and in the clinical evaluation of bone quality.

Computerized radiographic texture measures for characterizing bone strength: a simulated clinical setup using femoral neck specimens.

We are investigating computerized methods to ultimately characterize bone trabecular pattern from clinical skeletal radiographs. In this paper, we present a "phantom" for potential use in the development and evaluation of computerized methods for characterizing radiographic trabecular patterns and ultimately bone strength. Femoral neck specimens were excised during total hip arthroplasties from subjects exhibiting a range of diseases. To mimic the femoral neck in vivo, a "simulated clinical" setup was implemented in which specimens were exposed under conditions that yielded radiographs similar in appearance to standard pelvis radiographs. Fourier-based and fractal-based texture measures were used in the computer analysis; including RMS variation, first moment of the power spectrum, angular-dependent forms of these measures, and fractal dimension. The texture measures obtained from the "simulated clinical" specimen films correlated modestly with those from direct exposure "verification" films of the specimens (r= 0.59-0.69; p<0.0001). From our study, we conclude that the femoral neck specimen "phantoms" may be useful in the development and evaluation of computerized methods for analyzing bone trabecular patterns from skeletal radiographs. The use of a phantom that simulates the clinical radiographic examination allows for repeat exposures without the concern of excessive radiation exposure to a patient.

Feature selection with limited datasets.

Computer-aided diagnosis has the potential of increasing diagnostic accuracy by providing a second reading to radiologists. In many computerized schemes, numerous features can be extracted to describe suspect image regions. A subset of these features is then employed in a data classifier to determine whether the suspect region is abnormal or normal. Different subsets of features will, in general, result in different classification performances. A feature selection method is often used to determine an "optimal" subset of features to use with a particular classifier. A classifier performance measure (such as the area under the receiver operating characteristic curve) must be incorporated into this feature selection process. With limited datasets, however, there is a distribution in the classifier performance measure for a given classifier and subset of features. In this paper, we investigate the variation in the selected subset of "optimal" features as compared with the true optimal subset of features caused by this distribution of classifier performance. We consider examples in which the probability that the optimal subset of features is selected can be analytically computed. We show the dependence of this probability on the dataset sample size, the total number of features from which to select, the number of features selected, and the performance of the true optimal subset. Once a subset of features has been selected, the parameters of the data classifier must be determined. We show that, with limited datasets and/or a large number of features from which to choose, bias is introduced if the classifier parameters are determined using the same data that were employed to select the "optimal" subset of features.