Evaluation Challenges for the Application of Extended Reality Devices in Medicine.

Augmented and virtual reality devices are being actively investigated and implemented for a wide range of medical uses. However, significant gaps in the evaluation of these medical devices and applications hinder their regulatory evaluation. Addressing these gaps is critical to demonstrating the devices' safety and effectiveness. We outline the key technical and clinical evaluation challenges discussed during the US Food and Drug Administration's public workshop, "Medical Extended Reality: Toward Best Evaluation Practices for Virtual and Augmented Reality in Medicine" and future directions for evaluation method development. Evaluation challenges were categorized into several key technical and clinical areas. Finally, we highlight current efforts in the standards communities and illustrate connections between the evaluation challenges and the intended uses of the medical extended reality (MXR) devices. Participants concluded that additional research is needed to assess the safety and effectiveness of MXR devices across the use cases.

Beam orientation optimization for coherent X-ray scattering from distributed deep targets.

BACKGROUND: Amyloid deposits in the temporal and frontal lobes in patients with Alzheimer's disease make them potential targets to aid in early diagnosis. Recently, spectral small-angle X-ray scattering techniques have been proposed for interrogating deep targets such as amyloid plaques. RESULTS: We describe an optimization approach for the orientation of beams for deep target characterization. The model predicts the main features of scattering profiles from targets with varying shape, size and location. We found that increasing target size introduced additional smearing due to location uncertainty, and incidence angle affected the scattering profile by altering the path length or effective target size. For temporal and frontal lobe targets, beam effectiveness varied up to 2 orders of magnitude. CONCLUSIONS: Beam orientation optimization might allow for patient-specific optimal paths for improved signal characterization.

Color Rendering in Medical Extended-Reality Applications.

Cross-platform development of medical applications in extended-reality (XR) head-mounted displays (HMDs) often relies on game engines with rendering capabilities currently not standardized in the context of medical visualizations. Many aspects of the visualization pipeline including the characterization of color have yet to be consistently defined across rendering models and platforms. We examined the transfer of color properties from digital objects, through the rendering and image processing steps, to the RGB values sent to the display device. Five rendering pipeline configurations within the Unity engine were evaluated using 24 digital color patches. In the second experiment, the same configurations were evaluated with a tissue slide sample image. Measurements of the change in color associated with each configuration were characterized using the CIE 1976 color difference ([Formula: see text]). We found that the distribution of [Formula: see text] for the first experiment ranges from zero, as in the case using an Unlit Shader, to 25.97, as in the case using default configurations. The default Unity configuration consistently returned the highest [Formula: see text] across all 24 colors and also the largest range of color differences. In the second experiment, [Formula: see text]E ranged from 7.49 to 34.18. The Unlit configuration resulted in the highest [Formula: see text] in three of four selected pixels in the tissue sample image. Changes in color image properties associated with texture import settings were then evaluated in a third experiment using the TG18-QC test pattern. Differences in pixel values were found in all nine of the investigated texture import settings. The findings provide an initial characterization of color transfer and a basis for future work on standardization, consistency, and optimization of color in medical XR applications.

Transverse chromatic aberration in virtual reality head-mounted displays.

We demonstrate a method for measuring the transverse chromatic aberration (TCA) in a virtual reality head-mounted display. The method relies on acquiring images of a digital bar pattern and measuring the displacement of different color bars. This procedure was used to characterize the TCAs in the Oculus Go, Oculus Rift, Samsung Gear, and HTC Vive. The results show noticeable TCAs for the Oculus devices for angles larger than 5° from the center of the field of view. TCA is less noticeable in the Vive in part due to off-axis monochromatic aberrations. Finally, user measurements were conducted, which were in excellent agreement with the laboratory results.

Structural evaluation of an amyloid fibril model using small-angle x-ray scattering.

Amyloid fibrils are highly structured protein aggregates associated with a wide range of diseases including Alzheimer's and Parkinson's. We report a structural investigation of an amyloid fibril model prepared from a commonly used plasma protein (bovine serum albumin (BSA)) using small-angle x-ray scattering (SAXS) technique. As a reference, the size estimates from SAXS are compared to dynamic light scattering (DLS) data and the presence of amyloid-like fibrils is confirmed using Congo red absorbance assay. Our SAXS results consistently show the structural transformation of BSA from spheroid to rod-like elongated structures during the fibril formation process. We observe the elongation of fibrils over two months with fibril length growing from 35.9 ± 3.0 nm to 51.5 ± 2.1 nm. Structurally metastable fibrils with distinct SAXS profiles have been identified. As proof of concept, we demonstrate the use of such distinct SAXS profiles to detect fibrils in the mixture solutions of two species by estimating their volume fractions. This easily detectable and well-characterized amyloid fibril model from BSA can be readily used as a control or standard reference to further investigate SAXS applications in the detection of structurally diverse amyloid fibrils associated with protein aggregation diseases.

Method for Adapting the Grayscale Standard Display Function to the Aging Eye.

Perceptual linearity of grayscale images based on a contrast sensitivity model is a widely recognized and used standard for medical imaging visualization. This approach ensures consistency across devices and provides perception of luminance variations in direct relationship to changes in image values. We analyze the effect of aging of the human eye on the precept of linearity and demonstrate that not only the number of just-noticeable differences diminishes for older subjects but also linearity across the range of luminance values is significantly affected. While loss of JNDs is inevitable for a fixed luminance range, our findings suggest possible corrective approaches for maintaining linearity.

Consistency and standardization of color in medical imaging: a consensus report.

This article summarizes the consensus reached at the Summit on Color in Medical Imaging held at the Food and Drug Administration (FDA) on May 8-9, 2013, co-sponsored by the FDA and ICC (International Color Consortium). The purpose of the meeting was to gather information on how color is currently handled by medical imaging systems to identify areas where there is a need for improvement, to define objective requirements, and to facilitate consensus development of best practices. Participants were asked to identify areas of concern and unmet needs. This summary documents the topics that were discussed at the meeting and recommendations that were made by the participants. Key areas identified where improvements in color would provide immediate tangible benefits were those of digital microscopy, telemedicine, medical photography (particularly ophthalmic and dental photography), and display calibration. Work in these and other related areas has been started within several professional groups, including the creation of the ICC Medical Imaging Working Group.

Computational observers and visualization methods for stereoscopic medical imaging.

As stereoscopic display devices become common, their image quality assessment evaluation becomes increasingly important. Most studies conducted on 3D displays are based on psychophysics experiments with humans rating their experience based on detection tasks. The physical measurements do not map to effects on signal detection performance. Additionally, human observer study results are often subjective and difficult to generalize. We designed a computational stereoscopic observer approach inspired by the mechanisms of stereopsis in human vision for task-based image assessment that makes binary decisions based on a set of image pairs. The stereo-observer is constrained to a left and a right image generated using a visualization operator to render voxel datasets. We analyze white noise and lumpy backgrounds using volume rendering techniques. Our simulation framework generalizes many different types of model observers including existing 2D and 3D observers as well as providing flexibility to formulate a stereo model observer approach following the principles of stereoscopic viewing. This methodology has the potential to replace human observer studies when exploring issues with stereo display devices to be used in medical imaging. We show results quantifying the changes in performance when varying stereo angle as measured by an ideal linear stereoscopic observer. Our findings indicate that there is an increase in performance of about 13-18% for white noise and 20-46% for lumpy backgrounds, where the stereo angle is varied from 0 to 30. The applicability of this observer extends to stereoscopic displays used for in the areas of medical and entertainment imaging applications.

The effect of ambient illumination on handheld display image quality.

Handheld devices such as smartphones and tablets are becoming useful in the medical field, as they allow physicians, radiologists, and researchers to analyze images with the benefit of mobile accessibility. However, for handheld devices to be effective, the display must be able to perform well in a wide range of ambient illumination conditions. We conducted visual experiments to quantify user performance for testing the image quality of two current-generation devices in different ambient illumination conditions while measuring ambient light levels with a real-time illuminance meter. We found and quantified that due to the high reflectivity of handheld devices, performance deteriorates as the user moves from dark areas into environments of greater ambient illumination. The quantitative analysis suggests that differences in display reflection coefficients do not affect the low illumination performance of the device but rather the performance at higher levels of illumination.

Characterization of crosstalk in stereoscopic display devices.

Many different types of stereoscopic display devices are used for commercial and research applications. Stereoscopic displays offer the potential to improve performance in detection tasks for medical imaging diagnostic systems. Due to the variety of stereoscopic display technologies, it remains unclear how these compare with each other for detection and estimation tasks. Different stereo devices have different performance trade-offs due to their display characteristics. Among them, crosstalk is known to affect observer perception of 3D content and might affect detection performance. We measured and report the detailed luminance output and crosstalk characteristics for three different types of stereoscopic display devices. We recorded the effect of other issues on recorded luminance profiles such as viewing angle, use of different eye wear, and screen location. Our results show that the crosstalk signature for viewing 3D content can vary considerably when using different types of 3D glasses for active stereo displays. We also show that significant differences are present in crosstalk signatures when varying the viewing angle from 0 degrees to 20 degrees for a stereo mirror 3D display device. Our detailed characterization can help emulate the effect of crosstalk in conducting computational observer image quality assessment evaluations that minimize costly and time-consuming human reader studies.

Synthetic ß-sheets mimicking fibrillar and oligomeric structures for evaluation of spectral X-ray scattering technique for biomarker quantification.

BACKGROUND: Archetypical cross-ß spines sharpen the boundary between functional and pathological proteins including ß-amyloid, tau, α-synuclein and transthyretin are linked to many debilitating human neurodegenerative and non-neurodegenerative amyloidoses. An increased focus on development of pathogenic ß-sheet specific fluid and imaging structural biomarkers and conformation-specific monoclonal antibodies in targeted therapies has been recently observed. Identification and quantification of pathogenic oligomers remain challenging for existing neuroimaging modalities. RESULTS: We propose two artificial ß-sheets which can mimic the nanoscopic structural characteristics of pathogenic oligomers and fibrils for evaluating the performance of a label free, X-ray based biomarker detection and quantification technique. Highly similar structure with elliptical cross-section and parallel cross-ß motif is observed among recombinant α-synuclein fibril, Aß-42 fibril and artificial ß-sheet fibrils. We then use these ß-sheet models to assess the performance of spectral small angle X-ray scattering (sSAXS) technique for detecting ß-sheet structures. sSAXS showed quantitatively accurate detection of antiparallel, cross-ß artificial oligomers from a tissue mimicking environment and significant distinction between different oligomer packing densities such as diffuse and dense packings. CONCLUSION: The proposed synthetic ß-sheet models mimicked the nanoscopic structural characteristics of ß-sheets of fibrillar and oligomeric states of Aß and α-synuclein based on the ATR-FTIR and SAXS data. The tunability of ß-sheet proportions and shapes of structural motifs, and the low-cost of these ß-sheet models can become useful test materials for evaluating ß-sheet or amyloid specific biomarkers in a wide range of neurological diseases. By using the proposed synthetic ß-sheet models, our study indicates that the sSAXS has potential to evaluate different stages of ß-sheet-enriched structures including oligomers of pathogenic proteins.

In situtumor model for longitudinal in silico imaging trials.

Objective.In this article, we introduce a computational model for simulating the growth of breast cancer lesions accounting for the stiffness of surrounding anatomical structures.Approach.In our model, ligaments are classified as the most rigid structures while the softer parts of the breast are occupied by fat and glandular tissues As a result of these variations in tissue elasticity, the rapidly proliferating tumor cells are met with differential resistance. It is found that these cells are likely to circumvent stiffer terrains such as ligaments, instead electing to proliferate preferentially within the more yielding confines of the breast's soft topography. By manipulating the interstitial tumor pressure in direct proportion to the elastic constants of the tissues surrounding the tumor, this model thus creates the potential for realizing a database of unique lesion morphology sculpted by the distinctive topography of each local anatomical infrastructure. We modeled the growth of simulated lesions within volumes extracted from fatty breast models, developed by Graffet alwith a resolution of 50µm generated with the open-source and readily available Virtual Imaging Clinical Trials for Regulatory Evaluation (VICTRE) imaging pipeline. To visualize and validate the realism of the lesion models, we leveraged the imaging component of the VICTRE pipeline, which replicates the siemens mammomat inspiration mammography system in a digital format. This system was instrumental in generating digital mammogram (DM) images for each breast model containing the simulated lesions.Results.By utilizing the DM images, we were able to effectively illustrate the imaging characteristics of the lesions as they integrated with the anatomical backgrounds. Our research also involved a reader study that compared 25 simulated DM regions of interest (ROIs) with inserted lesions from our models with DM ROIs from the DDSM dataset containing real manifestations of breast cancer. In general the simulation time for the lesions was approximately 2.5 hours, but it varied depending on the lesion's local environment.Significance.The lesion growth model will facilitate and enhance longitudinal in silico trials investigating the progression of breast cancer.

Proceedings Virtual Imaging Trials in Medicine 2024.

This submission comprises the proceedings of the 1st Virtual Imaging Trials in Medicine conference, organized by Duke University on April 22-24, 2024. The listed authors serve as the program directors for this conference. The VITM conference is a pioneering summit uniting experts from academia, industry and government in the fields of medical imaging and therapy to explore the transformative potential of in silico virtual trials and digital twins in revolutionizing healthcare. The proceedings are categorized by the respective days of the conference: Monday presentations, Tuesday presentations, Wednesday presentations, followed by the abstracts for the posters presented on Monday and Tuesday.

Spatial resolution and noise in organic light-emitting diode displays for medical imaging applications.

We report on the resolution and noise characteristics of handheld and workstation organic light-emitting diode (OLED) displays in comparison with liquid crystal displays (LCDs). The results demonstrate advantages, in terms of sharpness, of handheld OLED displays with modulation transfer function (MTF) values exceeding 0.60 at the Nyquist frequencies. The OLED workstation included in this study exhibits significant signal contamination among adjacent pixels resulting in degraded resolution performance indicated by horizontal and vertical MTF values of 0.13 and 0.24 at the Nyquist frequency. On the other hand, its noise characteristics are superior to the LCD workstation tested. While the noise power spectral (NPS) values of the OLED workstation are 8.0×10(-6) mm2 at 1 mm(-1), the LCD workstation has NPS values of 2.6×10(-5) mm2. Although phone-size OLED displays have superior resolution and noise per pixel, the perceived resolution characteristics at appropriate viewing distances are inferior to tablet-size and workstation LCDs. In addition, our results show some degree of dependency of the resolution and noise on luminance level and viewing orientation. We also found a slightly degraded resolution and increased low-frequency noise at off-normal orientations in the handheld displays.

Objective Image Quality Optimization in Augmented Reality Using Spatial Frequency Domain Models.

Augmented reality (AR) blends the digital and physical worlds by overlapping a virtual image onto the see-through physical environment. However, contrast reduction and noise superposition in an AR head-mounted display (HMD) can substantially limit image quality and human perceptual performance in both the digital and physical spaces. To assess image quality in AR, we performed human and model observer studies for various imaging tasks with targets placed in the digital and physical worlds. A target detection model was developed for the complete AR system including the optical see-through. Target detection performance using different observer models developed in the spatial frequency domain was compared with the human observer results. The non-prewhitening model with eye filter and internal noise results closely track human perception performance as measured by the area under the receiver operating characteristic curve (AUC), especially for tasks with high image noise. The AR HMD non-uniformity limits the low-contrast target (less than 0.02) observer performance for low image noise. In augmented reality conditions, the detectability of a target in the physical world is reduced due to the contrast reduction by the overlaid AR display image (AUC less than 0.87 for all the contrast levels evaluated). We propose an image quality optimization scheme to optimize the AR display configurations to match observer detection performance for targets in both the digital and physical worlds. The image quality optimization procedure is validated using both simulation and bench measurements of a chest radiography image with digital and physical targets for various imaging configurations.

Proceedings of the NHLBI Workshop on Artificial Intelligence in Cardiovascular Imaging: Translation to Patient Care.

Artificial intelligence (AI) promises to revolutionize many fields, but its clinical implementation in cardiovascular imaging is still rare despite increasing research. We sought to facilitate discussion across several fields and across the lifecycle of research, development, validation, and implementation to identify challenges and opportunities to further translation of AI in cardiovascular imaging. Furthermore, it seemed apparent that a multidisciplinary effort across institutions would be essential to overcome these challenges. This paper summarizes the proceedings of the National Heart, Lung, and Blood Institute-led workshop, creating consensus around needs and opportunities for institutions at several levels to support and advance research in this field and support future translation.

Spatiotemporal Monte Carlo transport methods in x-ray semiconductor detectors: application to pulse-height spectroscopy in a-Se.

PURPOSE: The authors describe a detailed Monte Carlo (MC) method for the coupled transport of ionizing particles and charge carriers in amorphous selenium (a-Se) semiconductor x-ray detectors, and model the effect of statistical variations on the detected signal. METHODS: A detailed transport code was developed for modeling the signal formation process in semiconductor x-ray detectors. The charge transport routines include three-dimensional spatial and temporal models of electron-hole pair transport taking into account recombination and trapping. Many electron-hole pairs are created simultaneously in bursts from energy deposition events. Carrier transport processes include drift due to external field and Coulombic interactions, and diffusion due to Brownian motion. RESULTS: Pulse-height spectra (PHS) have been simulated with different transport conditions for a range of monoenergetic incident x-ray energies and mammography radiation beam qualities. Two methods for calculating Swank factors from simulated PHS are shown, one using the entire PHS distribution, and the other using the photopeak. The latter ignores contributions from Compton scattering and K-fluorescence. Comparisons differ by approximately 2% between experimental measurements and simulations. CONCLUSIONS: The a-Se x-ray detector PHS responses simulated in this work include three-dimensional spatial and temporal transport of electron-hole pairs. These PHS were used to calculate the Swank factor and compare it with experimental measurements. The Swank factor was shown to be a function of x-ray energy and applied electric field. Trapping and recombination models are all shown to affect the Swank factor.

Predicting perceived image quality: a critique of Lin and Kuo (2011).

A recent study by Lin and Kuo reported on the image quality of a small mobile display under different ambient illumination levels. In this commentary, the present author discusses the limitations of their approach with respect to the rigorous quantification of image quality and the caveats associated with preference studies of new display technologies. Quantitatively predicting image quality using preference-based methods can be useful for initial decisions in early phases of product development, but provides limited value for the rigorous quantification of image quality of display devices.

Spatiotemporal image quality of virtual reality head mounted displays.

Virtual reality (VR) head mounted displays (HMDs) require both high spatial resolution and fast temporal response. However, methods to quantify the VR image quality in the spatiotemporal domain when motion exists are not yet standardized. In this study, we characterize the spatiotemporal capabilities of three VR devices: the HTC VIVE, VIVE Pro, and VIVE Pro 2 during smooth pursuit. A spatiotemporal model for VR HMDs is presented using measured spatial and temporal characteristics. Among the three evaluated headsets, the VIVE Pro 2 improves the display temporal performance using a fast 120 Hz refresh rate and pulsed emission with a small duty cycle of 5%. In combination with a high pixel resolution beyond 2 k [Formula: see text] 2 k per eye, the VIVE Pro 2 achieves an improved spatiotemporal performance compared to the VIVE and VIVE Pro in the high spatial frequency range above 8 cycles per degree during smooth pursuit. The result demonstrates that reducing the display emission duty cycle to less than 20% is beneficial to mitigate motion blur in VR HMDs. Frame rate reduction (e.g., to below 60 Hz) of the input signal compared to the display refresh rate of 120 Hz yields replicated shadow images that can affect the image quality under motion. This work supports the regulatory science research efforts in development of testing methods to characterize the spatiotemporal performance of VR devices for medical use.

Computational models of direct and indirect X-ray breast imaging detectors for in silico trials.

BACKGROUND: To facilitate in silico studies that investigate digital mammography (DM) and breast tomosynthesis (DBT), models replicating the variety in imaging performance of the DM and DBT systems, observed across manufacturers are needed. PURPOSE: The main purpose of this work is to develop generic physics models for direct and indirect detector technology used in commercially available systems, with the goal of making them available open source to manufacturers to further tweak and develop the exact in silico replicas of their systems. METHODS: We recently reported on an in silico version of the SIEMENS Mammomat Inspiration DM/DBT system using an open-source GPU-accelerated Monte Carlo x-ray imaging simulation code (MC-GPU). We build on the previous version of the MC-GPU codes to mimic the imaging performances of two other Food and Drug Administration (FDA)-approved DM/DBT systems, such as Hologic Selenia Dimensions (HSD) and the General Electric Senographe Pristina (GSP) systems. In this work, we developed a hybrid technique to model the optical spread and signal crosstalk observed in the GSP and HSD systems. MC simulations are used to track each x-ray photon till its first interaction within the x-ray detector. On the other hand, the signal spread in the x-ray detectors is modeled using previously developed analytical equations. This approach allows us to preserve the modeling accuracy offered by MC methods in the patient body, while speeding up secondary carrier transport (either electron-hole pairs or optical photons) using analytical equations in the detector. The analytical optical spread model for the indirect detector includes the depth-dependent spread and collection of optical photons and relies on a pre-computed set of point response functions that describe the optical spread as a function of depth. To understand the capabilities of the computational x-ray detector models, we compared image quality metrics like modulation transfer function (MTF), normalized noise power spectrum (NNPS), and detective quantum efficiency (DQE), simulated with our models against measured data. Please note that the purpose of these comparisons with measured data would be to gauge if the model developed as part of this work could replicate commercially used direct and indirect technology in general and not to achieve perfect fits with measured data. RESULTS: We found that the simulated image quality metrics such as MTF, NNPS, and DQE were in reasonable agreement with experimental data. To demonstrate the imaging performance of the three DM/DBT systems, we integrated the detector models with the VICTRE pipeline and simulated DM images of a fatty breast model containing a spiculated mass and a calcium oxalate cluster. In general, we found that the images generated using the indirect model appeared more blurred with a different noise texture and contrast as compared to the systems with direct detectors. CONCLUSIONS: We have presented computational models of three commercially available FDA-approved DM/DBT systems, which implement both direct and indirect detector technology. The updated versions of the MC-GPU codes that can be used to replicate three systems are available in open source format through GitHub.