Observer performance using virtual pathology slides: impact of LCD color reproduction accuracy.

The use of color LCDs in medical imaging is growing as more clinical specialties use digital images as a resource in diagnosis and treatment decisions. Telemedicine applications such as telepathology, teledermatology, and teleophthalmology rely heavily on color images. However, standard methods for calibrating, characterizing, and profiling color displays do not exist, resulting in inconsistent presentation. To address this, we developed a calibration, characterization, and profiling protocol for color-critical medical imaging applications. Physical characterization of displays calibrated with and without the protocol revealed high color reproduction accuracy with the protocol. The present study assessed the impact of this protocol on observer performance. A set of 250 breast biopsy virtual slide regions of interest (half malignant, half benign) were shown to six pathologists, once using the calibration protocol and once using the same display in its "native" off-the-shelf uncalibrated state. Diagnostic accuracy and time to render a decision were measured. In terms of ROC performance, Az (area under the curve) calibrated = 0.8570 and Az uncalibrated = 0.8488. No statistically significant difference (p = 0.4112) was observed. In terms of interpretation speed, mean calibrated = 4.895 s; mean uncalibrated = 6.304 s which is statistically significant (p = 0.0460). Early results suggest a slight advantage diagnostically for a properly calibrated and color-managed display and a significant potential advantage in terms of improved workflow. Future work should be conducted using different types of color images that may be more dependent on accurate color rendering and a wider range of LCDs with varying characteristics.

Using a visual discrimination model for the detection of compression artifacts in virtual pathology images.

A major issue in telepathology is the extremely large and growing size of digitized "virtual" slides, which can require several gigabytes of storage and cause significant delays in data transmission for remote image interpretation and interactive visualization by pathologists. Compression can reduce this massive amount of virtual slide data, but reversible (lossless) methods limit data reduction to less than 50%, while lossy compression can degrade image quality and diagnostic accuracy. "Visually lossless" compression offers the potential for using higher compression levels without noticeable artifacts, but requires a rate-control strategy that adapts to image content and loss visibility. We investigated the utility of a visual discrimination model (VDM) and other distortion metrics for predicting JPEG 2000 bit rates corresponding to visually lossless compression of virtual slides for breast biopsy specimens. Threshold bit rates were determined experimentally with human observers for a variety of tissue regions cropped from virtual slides. For test images compressed to their visually lossless thresholds, just-noticeable difference (JND) metrics computed by the VDM were nearly constant at the 95th percentile level or higher, and were significantly less variable than peak signal-to-noise ratio (PSNR) and structural similarity (SSIM) metrics. Our results suggest that VDM metrics could be used to guide the compression of virtual slides to achieve visually lossless compression while providing 5-12 times the data reduction of reversible methods.

Monochrome versus color softcopy displays for teleradiology: observer performance and visual search efficiency.

This study evaluated the potential clinical utility for teleradiology of a high-performance (3-mega-pixel) color softcopy display compared with two monochrome softcopy displays: one of comparable luminance (250 cd/m2) and one of higher luminance (450 cd/m2). Six radiologists viewed 50 chest images, half with nodules and half without, once on each display. Eye position was recorded on a subset of the images to characterize visual search efficiency. There was no statistically significant difference in diagnostic performance as a function of monitor (F=1.176, p=0.3127), although the higher luminance display yielded slightly better performance. In terms of total viewing time, there were no statistically significant differences between the three monitors (F=1.478, p=0.2298). The dwell times associated with true- and false-positive decisions were shortest for the high luminance monochrome display, longer for the low luminance monochrome, and longest for the low luminance color display. Dwells for the false-negative decisions were longest for the high luminance monochrome display, shorter for the low luminance monochrome, and shortest for the low luminance color display. The true negative dwells were not significantly different. The study suggests that high-performance color displays can be used for teleradiologic interpretation of diagnostic images without negatively impacting diagnostic accuracy or visual search efficiency to a significant degree.

Does the age of liquid crystal displays influence observer performance?

RATIONALE AND OBJECTIVES: As liquid crystal displays (LCDs) age, the whitepoint shifts toward a yellow hue, changing the appearance of the displayed images. This study examined whether this shift in whitepoint influences observer performance and visual search efficiency of radiologists interpreting clinical images. MATERIALS AND METHODS: Six radiologists viewed 50 digital radiography chest images (half with a solitary pulmonary nodule, half without) on three LCDs that had their whitepoint adjusted to simulate monitor age: new, 1 year old, and 2.5 years old. Presence or absence of nodules was reported along with reader confidence. Results were analyzed using receiver operating characteristic techniques. Visual search was measured on a subset of 15 images using eye position recording techniques. RESULTS: There was no statistically significant difference in receiver operating characteristic performance as a function of monitor age (F = 0.4901, P = .6187). In terms of total viewing time, there were not statistically significant differences between the three monitors (F = 0.056, P = .9452). The dwell times associated with each decision type (true and false, positive and negative) did not differ significantly as a function of monitor age for any decision. CONCLUSION: At least up to 2.5 years of age, the shift in whitepoint toward the yellow range does not significantly impact diagnostic accuracy or visual search efficiency of radiologists.

Using a human visual system model to optimize soft-copy mammography display: influence of veiling glare.

RATIONALE AND OBJECTIVES: This project evaluated human observer performance and that of a human visual system model (JNDmetrix) to assess whether the veiling glare of a digital display influences observer performance during soft-copy interpretation of mammographic images for the detection of masses. MATERIALS AND METHODS: A set of 160 mammographic images, half containing a single mass, was processed to simulate four levels of veiling glare: none, comparable to a medical grade monochrome curved-screen cathode ray tube (CRT) display, double that of the CRT and quadruple that of the CRT. The images were shown to six observers in a randomized presentation order on a liquid crystal display (LCD) that had essentially no veiling glare. The images were also analyzed using the JNDmetrix human visual system model. RESULTS: Observer performance as measured using receiver operating characteristic techniques declined with increasing veiling glare (F = 6.884, P = .0035), with quadruple veiling glare yielding significantly lower performance than the lower veiling glare levels. The JNDmetrix model did not show a large reduction in performance as a function of veiling glare, and correlation with the human observer data was modest (0.588). CONCLUSIONS: Soft-copy display veiling glare can influence observer performance, but only at extreme levels. The impact of veiling glare on performance may be more pronounced for less experienced readers.

Determining the MTF of medical imaging displays using edge techniques.

The modulation transfer function (MTF) of a medical imaging display is typically determined by measuring its response to square waves (bar patterns), white noise, and/or line stimuli. However, square waves and white noise methods involve capture and analysis of multiple images and are thus quite tedious. Measurement of the line-spread function (LSF) offers a good alternative. However, as previously reported, low-frequency response obtained from the LSF method is not as good as that obtained from measurement of edge-spread function (ESF). In this paper, we present two methods for evaluating the MTF of a medical imaging display from its ESF. High degree of accuracy in the higher frequency region (near the Nyquist frequency of the system) was achieved by reducing the noise. In the first method, which is a variant of the Gans' original method, the periodic raster noise is reduced by subtracting a shifted ESF from the ESF. The second method employs a low-pass differentiator (LPD). A novel near maximally flat LPD with the desired cut-off frequency was designed for this purpose. Noise reduction in both the methods was also achieved by averaging over large portions of the image data to form the ESF. Experimental results show that the MTF obtained by these methods is comparable to that obtained from the square wave response. Furthermore, the MTFs of rising and falling edges of a cathode ray tube (CRT) were measured. The results show that the rising and falling vertical MTFs are practically the same, whereas the rising horizontal MTF is poorer than the falling horizontal MTF in the midfrequency region.

Differential use of image enhancement techniques by experienced and inexperienced observers.

Full-field digital mammography (FFDM) systems are currently being used to acquire mammograms in digital format, but digital displays are less than ideal compared to traditional film-screen display. Certain physical properties of softcopy displays [e.g., modulation transfer function (MTF)] are less than optimal compared to film. We developed methods to compensate for some of these softcopy display deficiencies, based on careful physical characterization of the displays and image-processing software. A series of 100 FFDM and 60 digitized images was shown to six observers-half experienced (mammographers) and half inexperienced (radiology residents). The observers had to decide if a mass or microcalcification cluster was present and classify it as benign or malignant. A window could be activated that brought the image detail within the window to full resolution and corrected for the nonisotropic MTF of the Cathode Ray Tube (CRT) display. Experienced readers had better diagnostic performance and took less time to view the images. Experienced readers used window/level more than inexperienced readers, but inexperienced readers used magnification and the MTF compensation tool more often. Use of the magnification and the MTF tool increased reader decision confidence. Experienced and inexperienced readers use image-processing tools differently, with certain tools increasing reader confidence. Understanding how observers use image-processing tools may help in the development of better and more automated user interfaces.

On-axis and off-axis viewing of images on CRT displays and LCDs: observer performance and vision model predictions.

RATIONALE AND OBJECTIVES: Although cathode-ray tube (CRT) displays typically are used for softcopy display of radiographs in the digital reading environment, liquid crystal displays (LCDs) currently are being used as an alternative. LCDs have many desirable viewing properties compared with a CRT, but significant image degradation can occur with off-axis viewing. This study compares observer performance and predictions from a human visual system model for on-axis and off-axis viewing for CRT displays versus LCDs. MATERIALS AND METHODS: A set of mammograms with different lesion contrasts (n = 400) were shown to six radiologists on CRT and LCD monitors, once on-axis and once off-axis. Observer performance was measured by using receiver operating characteristic techniques. Performance was correlated with results of a human vision model designed to predict observer performance (just noticeable difference [JND]metrix model). Two approaches were used to generate model metrics; a paired discrimination and channelized model observer approach. RESULTS: The performance of human observers indicated that on-axis viewing with the LCD was better than with the CRT, but off-axis viewing was significantly better with the CRT than LCD (F = 8.8175; P < .0001). The paired discrimination model correctly predicted on-axis, but not off-axis, results. The channelized model observer correctly predicted both on- and off-axis results. CONCLUSION: Off-axis viewing of radiographic images on an LCD monitor degrades human observer performance significantly compared with a CRT display. Care should be taken in the clinic to avoid off-axis viewing during diagnostic interpretation.

Assessment of display performance for medical imaging systems: executive summary of AAPM TG18 report.

Digital imaging provides an effective means to electronically acquire, archive, distribute, and view medical images. Medical imaging display stations are an integral part of these operations. Therefore, it is vitally important to assure that electronic display devices do not compromise image quality and ultimately patient care. The AAPM Task Group 18 (TG18) recently published guidelines and acceptance criteria for acceptance testing and quality control of medical display devices. This paper is an executive summary of the TG18 report. TG18 guidelines include visual, quantitative, and advanced testing methodologies for primary and secondary class display devices. The characteristics, tested in conjunction with specially designed test patterns (i.e., TG18 patterns), include reflection, geometric distortion, luminance, the spatial and angular dependencies of luminance, resolution, noise, glare, chromaticity, and display artifacts. Geometric distortions are evaluated by linear measurements of the TG18-QC test pattern, which should render distortion coefficients less than 2%/5% for primary/secondary displays, respectively. Reflection measurements include specular and diffuse reflection coefficients from which the maximum allowable ambient lighting is determined such that contrast degradation due to display reflection remains below a 20% limit and the level of ambient luminance (Lamb) does not unduly compromise luminance ratio (LR) and contrast at low luminance levels. Luminance evaluation relies on visual assessment of low contrast features in the TG18-CT and TG18-MP test patterns, or quantitative measurements at 18 distinct luminance levels of the TG18-LN test patterns. The major acceptable criteria for primary/ secondary displays are maximum luminance of greater than 170/100 cd/m2, LR of greater than 250/100, and contrast conformance to that of the grayscale standard display function (GSDF) of better than 10%/20%, respectively. The angular response is tested to ascertain the viewing cone within which contrast conformance to the GSDF is better than 30%/60% and LR is greater than 175/70 for primary/secondary displays, or alternatively, within which the on-axis contrast thresholds of the TG18-CT test pattern remain discernible. The evaluation of luminance spatial uniformity at two distinct luminance levels across the display faceplate using TG18-UNL test patterns should yield nonuniformity coefficients smaller than 30%. The resolution evaluation includes the visual scoring of the CX test target in the TG18-QC or TG18-CX test patterns, which should yield scores greater than 4/6 for primary/secondary displays. Noise evaluation includes visual evaluation of the contrast threshold in the TG18-AFC test pattern, which should yield a minimum of 3/2 targets visible for primary/secondary displays. The guidelines also include methodologies for more quantitative resolution and noise measurements based on MTF and NPS analyses. The display glare test, based on the visibility of the low-contrast targets of the TG18-GV test pattern or the measurement of the glare ratio (GR), is expected to yield scores greater than 3/1 and GRs greater than 400/150 for primary/secondary displays. Chromaticity, measured across a display faceplate or between two display devices, is expected to render a u',v' color separation of less than 0.01 for primary displays. The report offers further descriptions of prior standardization efforts, current display technologies, testing prerequisites, streamlined procedures and timelines, and TG18 test patterns.

Evaluation of and compensation for spatial noise of LCDs in medical applications.

Recent developments in liquid crystal display (LCD) technology suggest that this technology will replace the cathode ray tube (CRT) as the most popular softcopy display technology in the medical arena. However, LCDs are far from ideal for medical imaging. One of the principal problems they possess is spatial noise contamination, which requires accurate characterization and appropriate compensation before LCD images can be effectively utilized for reliable diagnosis. This paper presents some work we have conducted recently on characterization of spatial noise of high resolution LCDs. The primary purpose of this work is to explore the properties of spatial noise and propose a method to reduce it. A high quality CCD camera was used for physical evaluation. Spatial noise properties were analyzed and estimated from the camera images via signal modeling and processing. A noise compensation algorithm based on error diffusion was developed to process images before they were displayed. Results shown in this paper suggest that LCD spatial noise can be effectively reduced via appropriate processing.

Use of a human visual system model to predict observer performance with CRT vs LCD display of images.

This Project evaluated a human visual system model (JNDmetrix) based on just noticeable difference (JND) and frequency-channel vision-modeling principles to assess whether a Cathode ray tube (CRT) or a liquid crystal display (LCD) monochrome display monitor would yield better observer performance in radiographic interpretation. Key physical characteristics, such as veiling glare and modulation transfer function (MTF) of the CRT and LCD were measured. Regions of interest from mammographic images with masses of different contrast levels were shown once on each display to six radiologists using a counterbalanced presentation order. The images were analyzed using the JNDmetrix model. Performance as measured by receiver operating characteristic (ROC) analysis was significantly better overall on the LCD display (P = 0.0120). The JNDmetrix model predicted the result (P = 0.0046) and correlation between human and computer observers was high (r (2) (quadratic) = 0.997). The results suggest that observer performance with LCD displays is superior to CRT viewing, at least for on-axis viewing.

Using a human visual system model to optimize soft-copy mammography display: influence of MTF compensation.

RATIONALE AND OBJECTIVES: The investigators developed an efficient method for optimizing cathode ray tube (CRT) monitor performance for digital mammography, based on the correlation between the performance of human observers and the performance of a mathematical computer model of the human visual system. The investigators examined observer performance on soft-copy display of mammographic images that were either unprocessed or processed to compensate for modulation transfer function (MTF) deficiencies in the CRT display. The results were used to validate the human visual system model. MATERIALS AND METHODS: Six radiologists viewed a series of 250 mammographic images with microcalcification clusters with different contrast levels on a CRT monitor. The images were viewed twice: once without image processing and once with processing designed to compensate for MTF deficiencies in the CRT monitor. The images were analyzed with the JNDmetrix Visual Discrimination Model, which is based on the principles of just-noticeable difference measurement and frequency-channel vision modeling. Receiver operating characteristic (ROC) curves were generated for the human observers and compared statistically with the model observers' performance. RESULTS: Both human and model performance was better overall with the MTF-compensated images, especially for microcalcifications in the midlevel contrast range. There was a very high correlation between human and model observers. CONCLUSION: The use of image-processing methods to compensate for limitations in the MTF of CRT monitors can improve the detection performance of radiologists searching for microcalcifications in mammographic images, and a model based on characteristics of the human visual system can be used to predict human observer results accurately.

Searching for nodules: what features attract attention and influence detection?

RATIONALE AND OBJECTIVES: The goal of the study was to determine whether there are certain physical features of pulmonary nodules that attract visual attention and contribute to increased recognition and detection by observers. MATERIALS AND METHODS: A series of posteroanterior chest images with solitary pulmonary nodules were searched by six radiologists as their eye-position was recorded. The signal-to-noise ratio, size, conspicuity, location, and calcification status were measured for each nodule. Dwell parameters were correlated with nodule features and related to detection rates. RESULTS: Only nodule size (F = 5.08, P = .0254) and conspicuity (F = 4.625, P = .0329) influenced total dwell time on nodules, with larger, more conspicuous nodules receiving less visual attention than smaller, less conspicuous nodules. All nodule features examined influenced overall detection performance (P < .05) even though most did not influence visual search and attention. CONCLUSION: Individual nodule features do not attract attention as measured by "first hit" fixation data, but certain features do tend to hold attention once the nodule has been fixated. The combination of all features influences whether or not it is detected.

Using a human visual system model to optimize soft-copy mammography display: influence of display phosphor.

RATIONALE AND OBJECTIVES: The authors developed an efficient method for optimizing cathode ray tube performance for soft-copy digital mammography displays, based on correlation between the performance of human observers and the performance of a mathematical computer model of the human visual system. The authors measured radiologist performance on soft-copy display monitors with different phosphors and used these results to validate the human visual performance model. MATERIALS AND METHODS: Six radiologists viewed a series of 250 mammographic images with microcalcifications of different contrast levels. They viewed images on two soft-copy display monitors with phosphor luminescence-one with P45 and the other with P104. The same images were analyzed with the JNDmetrix Visual Discrimination Model, which is based on psychophysical just-noticeable difference measurement principles and on frequency-channel vision-modeling principles. Receiver operating characteristic curves were generated for the human and model observers' performances, and results were compared statistically. RESULTS: Both human and model performance (area under the receiver operating characteristic curve) was better overall with the P45 than with the P104 monitor, especially for microcalcifications in the midlevel contrast range. There was high correlation between the human and model observers. CONCLUSION: The results indicate that the type of phosphor in a display monitor can influence observer performance significantly and that a model based on characteristics of the human visual system can be used to predict human observer performance accurately.

Pulmonary nodule detection and visual search: P45 and P104 monochrome versus color monitor displays.

RATIONALE AND OBJECTIVES: The faceplate of a cathode-ray tube (CRT) display monitor is covered on the vacuum side with a phosphor screen. The different phosphors that can be used for this screen have distinctly different physical properties that can affect the noise properties of the display. Differences in noise affect the signal-to-noise ratio and, hence, may affect diagnostic performance. This study evaluated observer performance and visual search parameters in the detection of pulmonary nodules, comparing two monochrome CRT monitors with different phosphors (P45 and P104) and a color CRT monitor. MATERIALS AND METHODS: The receiver operating characteristic paradigm was used to evaluate observer performance with a series of radiographic chest images containing solitary pulmonary nodules. Eye position was recorded as the observers searched the images on each type of monitor. RESULTS: Observer performance, as indicated by the area under the receiver operating characteristic curve and compared by means of an analysis of variance test, was best for the P45 monitor, next best for the P104 monitor, and worst for the color monitor. All differences were statistically significant. Eye-position parameters were also affected by monitor type. The time required to fixate the lesion and overall search times were longest with the color monitor. CONCLUSION: The type of phosphor used in the CRT monitor faceplate can affect diagnostic performance and visual search parameters. Care should be taken in the selection of monitors for use in clinical radiology.