Image processing algorithms for digital mammography: a pictorial essay.

Digital mammography systems allow manipulation of fine differences in image contrast by means of image processing algorithms. Different display algorithms have advantages and disadvantages for the specific tasks required in breast imaging-diagnosis and screening. Manual intensity windowing can produce digital mammograms very similar to standard screen-film mammograms but is limited by its operator dependence. Histogram-based intensity windowing improves the conspicuity of the lesion edge, but there is loss of detail outside the dense parts of the image. Mixture-model intensity windowing enhances the visibility of lesion borders against the fatty background, but the mixed parenchymal densities abutting the lesion may be lost. Contrast-limited adaptive histogram equalization can also provide subtle edge information but might degrade performance in the screening setting by enhancing the visibility of nuisance information. Unsharp masking enhances the sharpness of the borders of mass lesions, but this algorithm may make even an indistinct mass appear more circumscribed. Peripheral equalization displays lesion details well and preserves the peripheral information in the surrounding breast, but there may be flattening of image contrast in the nonperipheral portions of the image. Trex processing allows visualization of both lesion detail and breast edge information but reduces image contrast.

Registration of 3D cerebral vessels with 2D digital angiograms: clinical evaluation.

RATIONALE AND OBJECTIVES: The purpose of this study was to evaluate the accuracy and speed of a new, semiautomatic method of three-dimensional (3D)-two-dimensional (2D) vascular registration. This method should help guide endovascular procedures by allowing interpretation of each digital subtraction angiographic (DSA) image in terms of precreated, 3D vessel trees that contain "parent-child" connectivity information. MATERIALS AND METHODS: Connected, 3D vessel trees were created from segmented magnetic resonance (MR) angiograms. Eleven total DSA images were registered with such trees by using both our method and the current standard (manual registration). The accuracy of each method was compared by using repeated-measures analysis of variance with correction for heterogeneity of variance to evaluate separation of curve pairs on the view plane. Subjective clinical comparisons of the two registration methods were evaluated with the sign test. Registration times were evaluated for both methods and also as a function of the error in the initial estimate of MR angiographic position. RESULTS: The new registration method produced results that were numerically superior to those of manual registration (P < .001) and was subjectively judged to be as good as or better by clinical reviewers. Registration time with the new method was faster (P < .001). If the rotational error in the initial estimate of MR angiographic position is less than 10 degrees around each axis, the registration itself took only 1-2 minutes. CONCLUSION: This method is quicker than and produces results as good as or better than those of manual registration. This method should be able to calculate an initial registration matrix during endovascular embolization and adjust that matrix intermittently with registration updates provided by automatic tracking systems.

Segmentation, registration, and measurement of shape variation via image object shape.

A model of object shape by nets of medial and boundary primitives is justified as richly capturing multiple aspects of shape and yet requiring representation space and image analysis work proportional to the number of primitives. Metrics are described that compute an object representation's prior probability of local geometry by reflecting variabilities in the net's node and link parameter values, and that compute a likelihood function measuring the degree of match of an image to that object representation. A paradigm for image analysis of deforming such a model to optimize a posteriori probability is described, and this paradigm is shown to be usable as a uniform approach for object definition, object-based registration between images of the same or different imaging modalities, and measurement of shape variation of an abnormal anatomical object, compared with a normal anatomical object. Examples of applications of these methods in radiotherapy, surgery, and psychiatry are given.

Medial-node models to identify and measure objects in real-time 3-D echocardiography.

A method is proposed for the automatic, rapid, and stable identification and measurement of objects in three-dimensional (3-D) images. It is based on local shape properties derived statistically from populations of medial primitives sought throughout the image space. These shape properties are measured at medial locations within the object and include scale, orientation, endness, and medial dimensionality. Medial dimensionality is a local shape property differentiating sphere-like, cylinder-like, and slab-like structures, with intermediate dimensionality also possible. Endness is a property found at the cap of a cylinder or the edge of a slab. In terms of an application, the cardiac left ventricle (LV) during systole is modeled as a large dark cylinder with an apical cap, terminated at the other end by a thin bright slab-like mitral valve (MV). Such a model, containing medial shape properties at just a few locations, along with the relative distances and orientations between these locations, is intuitive and robust and permits automated detection of the LV axis in vivo, using real-time 3-D (RT3D) echocardiography. The statistical nature of these shape properties allows their extraction, even in the presence of noise, and permits statistical geometric measurements without exact delineation of boundaries, as demonstrated in determining the volume of balloons in RT3D scans. The inherent high speed of the method is appropriate for real-time clinical use.

Contrast limited adaptive histogram equalization image processing to improve the detection of simulated spiculations in dense mammograms.

The purpose of this project was to determine whether Contrast Limited Adaptive Histogram Equalization (CLAHE) improves detection of simulated spiculations in dense mammograms. Lines simulating the appearance of spiculations, a common marker of malignancy when visualized with masses, were embedded in dense mammograms digitized at 50 micron pixels, 12 bits deep. Film images with no CLAHE applied were compared to film images with nine different combinations of clip levels and region sizes applied. A simulated spiculation was embedded in a background of dense breast tissue, with the orientation of the spiculation varied. The key variables involved in each trial included the orientation of the spiculation, contrast level of the spiculation and the CLAHE settings applied to the image. Combining the 10 CLAHE conditions, 4 contrast levels and 4 orientations gave 160 combinations. The trials were constructed by pairing 160 combinations of key variables with 40 backgrounds. Twenty student observers were asked to detect the orientation of the spiculation in the image. There was a statistically significant improvement in detection performance for spiculations with CLAHE over unenhanced images when the region size was set at 32 with a clip level of 2, and when the region size was set at 32 with a clip level of 4. The selected CLAHE settings should be tested in the clinic with digital mammograms to determine whether detection of spiculations associated with masses detected at mammography can be improved.

Three-dimensional reconstruction of curves from pairs of projection views in the presence of error. I. Algorithms.

We have previously described an approach to 3D intracerebral vascular reconstruction that uses an MRA as a reconstruction base. Additional vessels seen only by angiography are added by segmenting 2D curves from projection angiograms and reconstructing these curves into 3D, building upon the MRA. Intracerebral vascular reconstruction is difficult for at least two reasons. First, 2D curves must be associated on projection images even when the human eye cannot do so. Second, 3D curves must be reconstructed in the presence of errors such as misregistration, image distortion, and misdefinition of 2D curves. This paper is the first of two that address the specific issue of reconstruction of a 3D curve from a given pair of 2D curves in the presence of error. The method explicitly separates what can and cannot be determined from a pair of projection views. It is also capable of recognizing interruptions produced by viewplane errors, of continuing reconstruction beyond such interruptions, and of localizing and estimating the magnitude of the interruptions. These measurements can also be used to estimate the lengths of regional disparities between a pair of 2D curves, leading to a quantitative estimate of the capacity of a pair of 2D curves to combine to create a 3D object (match value). Match values can be used, in turn, as part of the strategy for automatically associating pairs of 2D curves. This paper provides methods for reconstructing a given pair of 2D curves into 3D in the presence of error and for calculating match values. Error analysis is given in the companion report.

Three-dimensional reconstruction of curves from pairs of projection views in the presence of error. II. Analysis of error.

We have previously described an approach to 3D intracerebral vascular reconstruction that uses an MRA as a reconstruction base. Additional vessels seen only by angiography are added by segmenting 2D curves from projection angiograms and reconstructing these curves into 3D, building upon the MRA. This paper is the second of two that discuss the specific problem of reconstructing a 3D curve from a given pair of 2D curves in the presence of error. The method presented is capable of detecting and handling many errors produced by misregistration, image distortion, or misdefinition of 2D curves. The first paper gives an algorithm. The current paper discusses factors affecting the accuracy of a reconstructed curve, with emphasis upon registration error. We analyze the spatial accuracy of a reconstructed point in terms of the relationships between pixel size, relative viewing angle, 3D point location, and registration error. We provide a theoretical framework that, given the known error properties of a registration algorithm, allows optimization of the viewing geometry so as to produce the highest precision of point reconstruction. A major focus is the effect of registration error upon the reconstruction of a curve. We subdivide registration error into two types, one of which produces smoothly continuous point placement errors and the other of which produces pixel pairing errors. We test our ability to reconstruct a 3D curve in the presence of both. Finally, we summarize approaches to other sources of error. We conclude with a list of recommendations to optimize reconstruction accuracy. When projection points are associated by the rules of epipolar geometry, viewplane point displacements should not exceed 1.5-2 mm along the axis perpendicular to epipolar planes.

Methods for displaying intracerebral vascular anatomy.

We are developing three-dimensional imaging methods to portray vascular anatomy better, including noise-free display of vessels extracted from 3-D data sets, tree-based display, and reconstruction of angiographic data (preliminary work has resulted in the successful reconstruction of aneurysms from angiographic data). Fast, interactive display permits real-time manipulation of viewing orientation.

Three-dimensional reconstruction of intracranial vessels from biplane projection views.

The three-dimensional (3D) reconstruction of intracerebral vessels from two-dimensional (2D) projection views is an important clinical problem that, so far, has eluded solution. This report describes a new approach that uses projection images to build arterial trees progressively from an underlying 3D network, using a new method to pair shadow images on widely separated projection views. As a test of our general methodology, we have reconstructed a middle cerebral arterial tree from two projection views of a magnetic resonance dataset and have tested the accuracy of reconstruction against the original 3D dataset. This report describes the general approach to 3D vascular reconstruction and the computer program used to perform the final reconstruction step. The results suggest that accurate, 3D reconstruction of intracranial vessels is indeed possible from as few as two projection views.

Linking object boundaries at scale: a common mechanism for size and shape judgments.

The area over which boundary information contributes to the determination of the center of an extended object was inferred from results of a bisection task. The object to be bisected was a rectangle with two long sinusoidally modulated sides, i.e. a wiggly rectangle. The spatial frequency and amplitude of the edge modulation were varied. Two object widths were tested. The modulation of the perceived center approximately equaled that of the edges at very low edge modulation frequencies and decreased in amplitude with increasing edge modulation frequency. The edge modulation had a greater modulating effect on the perceived center for the narrower object than for the wider object. This scaling with object width didn't follow perfect zoom invariance but was precisely matched by the scaling of the bisection threshold with width, strongly supporting the idea that the same mechanism determines both the location of the perceived center for these stimuli and its variance. We propose that this mechanism is the linking of object boundaries at a scale determined by the object width.

A New Technique for CT/MR Fusion For Skull Base Imaging.

This paper presents our initial experience utilizing a new technique which allows CT and MR image fusion in patients with skull base lesions. Eleven patients with a variety of skull base lesions underwent CT and MR imaging prior to surgery. Both sets of images were coregistered using customized software. The CT and MR data sets were then combined and viewed in a single interactive image formar using a high-speed graphic computing system. Image fusion allowed simultaneous visualization of the bony skull base anatomy (CT) and detailed soft tissue anatomy (MR) using a single image format. Combining both modalities was felt to provide a better assessment of the extent of lesions and improve understanding of their relationship to adjacent bony and neurovascular anatomy. Specifically, image fusion enhanced awareness of location of skill base lesions with respect to the cavernous sinuses. Gasserian ganglia, carotid arteries, and jugular foramina. For tumors arising within the internal auditory canal (IAC), fused images allowed better delineation of the lateral aspect of the lesion with respect to the fundus of the IAC. Thus, fusion of CT and MR studies provides a unique image format which has advantages over single modality display. We believe image fusion is beneficial for surgical planning and for treatment planning of complex skull base malignancies treated with radiotherapy.

Object representation by cores: identifying and representing primitive spatial regions.

We propose a model of the spatial visual processes underlying the identification and representation of the shape of primitive spatial regions. We propose that a region's boundaries are sensed at multiple scales by boundariness detectors that give graded responses, that stimulated boundariness detectors of similar scale, sigma, connect to one another across a distance that is proportional to their scale, and that they connect via cores, where a core encodes the middles and widths of the region and hence is a trace in (chi, gamma, sigma), i.e. 3-D scale space.

A method for determination of optimal image enhancement for the detection of mammographic abnormalities.

We present a paradigm for empirical evaluation of digital image enhancement algorithms for mammography that uses psychophysical methods for implementation and analysis of a clinically relevant detection task. In the experiment, the observer is asked to detect and assign to a quadrant, or indicate the absence of, a simulated mammographic structure characteristic of cancer embedded in a background image of normal breast tissue. Responses are indicated interactively on a computer workstation. The parameter values for the enhancement applied to the composite image may be varied on each trial, and structure detection performance is estimated for each enhancement condition. Preliminary investigations have provided insight into an appropriate viewing duration, and furthermore, suggest that nonradiologists may be used under this methodology for the tasks investigated thus far, for predicting parameter values for clinical investigation. We are presently using this method in evaluating several contrast enhancement algorithms of possible benefit in mammography. These methods enable an objective, clinically relevant evaluation, for the purpose of optimal parameter determination or performance assessment, of digital image-processing methods potentially used in mammography.

Portal film enhancement: technique and clinical utility.

We report on the results a 3-year project which had as its goal the development of methods to enhance radiation portal films to improve their readability. We had previously reported on a portal film enhancement technique, contrast limited adaptive histogram equalization, which could enhance low contrast detail, but degraded sharply contrasted edges. A new method, unsharp masking followed by contrast limited adaptive histogram equalization, now appears to overcome this problem. A clinical trial to test whether enhanced portal films could be read more accurately than standard ones was undertaken. The trial involved 12 readers from two institutions doing 276 readings. In this trial the enhanced films were judged to be of higher quality than the non-enhanced films (p < .001) and were read more accurately (p = .026). The usefulness and difficulties of routinely performing portal film enhancement in a busy radiation therapy department are discussed.

Visual increment and decrement threshold curves as a function of luminance range and noise in simulated computed tomographic scans.

OBJECTIVES: The effect of luminance range compression on the visual threshold of a target in a computed tomographic (CT) scan was investigated in eight experiments. METHODS: Both visual increment thresholds of a hot target and visual decrement thresholds of a cold target were obtained. Realistic noisy CT images simulating a scan reconstruction and partial volume blurring were tested as representative of complex medical images. Negative versions, high luminance versions, and noise-free versions of these CT images also were tested. RESULTS AND CONCLUSIONS: In all the noisy images, as the luminance range of the image was compressed, proportionately smaller physical luminance differences between the target and its local background were needed to reach visual threshold. However, the thresholds were the same in terms of the difference in CT numbers between target and background. Noise in terms of CT numbers sets the threshold for a wide range of display conditions. In the noise-free CTs, as the luminance range was compressed, the luminance differences between the target and background needed to reach threshold also decreased, but only marginally. However, in terms of CT numbers, the thresholds were increased.

Three-dimensional high-resolution volume rendering (HRVR) of computed tomography data: applications to otolaryngology-head and neck surgery.

Conventional computed tomographic display formats are not optimal for demonstrating three-dimensional anatomic relationships. In otolaryngology--head and neck surgery these critical relationships are often highly complex, and their complete understanding is essential to a successful surgical outcome. A new computer-generated image display format, high-resolution volume rendering (HRVR), facilities the understanding of these critical anatomic relationships by transforming conventional imaging data into clinically relevant 3-D images. Unlike many other 3-D reconstruction algorithms, HRVR suffers minimal data loss in the conversion process, which in turn provides for superior image resolution. This better allows the application of 3-D technology to small or complicated anatomic structures such as those frequently encountered in otolaryngology--head and neck surgery. Advances in computer-controlled manipulations that further enhance the evaluation of desired pathologic features have been achieved. This pilot study contains representative clinical cases chosen to illustrate the potential utility of HRVR in otolaryngology--head and neck surgery. The authors believe HRVR images will enhance the surgeon's understanding of the 3-D anatomic relationships that exist between critical pathologic features and surrounding vital structures.

Fundamental properties of medical image perception.

With a mind toward the effective acquisition, processing, presentation, and reading of radiological images, a survey of how the human visual system perceives images is presented here. The level is chosen to be suitable for the radiologist, and the relative emphasis on the various visual cues of luminance, color, form, texture, motion, and depth is chosen based on their importance with radiological images. Examples of the radiological relevance of the various visual properties are given. We cover first what the visual system's behavior is and then survey some of the properties of the physiological mechanisms that provide this behavior.

Visual psychophysics and medical imaging: nonparametric adaptive method for rapid threshold estimation in sensitivity experiments.

The M-AFC (alternatively forced choice) transformed up-down adaptive method for the rapid determination of visual thresholds in medical images is described. The method is very efficient in obtaining thresholds to medical imaging parameters; in addition, it is free from criterion bias, an important concern in radiology, and it is free from parametric assumptions about the stimulus scale, which is often unknown due to the complexity of medical images. Issues of experimental design are presented that arise in the use of this method, and the psychological caveats which should be followed with human observers are noted. Two experiments are presented to demonstrate the method's efficacy in determining thresholds and psychometric functions in medical images.