An augmented reality-guided biopsy system using a high-speed motion tracking and real-time registration platform.

Biopsies play a crucial role in diagnosis of various diseases including cancers. In this study, we developed an augmented reality (AR) system to improve biopsy procedures and increase targeting accuracy. Our AR-guided biopsy system uses a high-speed motion tracking technology and an AR headset to display a holographic representation of the organ, lesions, and other structures of interest superimposed on real physical objects. The first application of our AR system is prostate biopsy. By incorporating preoperative scans, such as computed tomography (CT) or magnetic resonance imaging (MRI), into real-time ultrasound-guided procedures, this innovative AR-guided system enables clinicians to see the lesion as well as the organs in real time. With the enhanced visualization of the prostate, lesion, and surrounding organs, surgeons can perform prostate biopsies with an increased accuracy. Our AR-guided biopsy system yielded an average targeting accuracy of 2.94 ± 1.04 mm and can be applied for real-time guidance of prostate biopsy as well as other biopsy procedures.

Conditional Generative Adversarial Network (cGAN) for Synthesis of Digital Histologic Images from Hyperspectral Images.

Hyperspectral imaging (HSI) has been demonstrated in various digital pathology applications. However, the intrinsic high dimensionality of hyperspectral images makes it difficult for pathologists to visualize the information. The aim of this study is to develop a method to transform hyperspectral images of hemoxylin & eosin (H&E)-stained slides to natural-color RGB histologic images for easy visualization. Hyperspectral images were obtained at 40× magnification with an automated microscopic imaging system and downsampled by various factors to generate data equivalent to different magnifications. High-resolution digital histologic RGB images were cropped and registered to the corresponding hyperspectral images as the ground truth. A conditional generative adversarial network (cGAN) was trained to output natural color RGB images of the histological tissue samples. The generated synthetic RGBs have similar color and sharpness to real RGBs. Image classification was implemented using the real and synthetic RGBs, respectively, with a pretrained network. The classification of tumor and normal tissue using the HSI-synthesized RGBs yielded a comparable but slightly higher accuracy and AUC than the real RGBs. The proposed method can reduce the acquisition time of two imaging modalities while giving pathologists access to the high information density of HSI and the quality visualization of RGBs. This study demonstrated that HSI may provide a potentially better alternative to current RGB-based pathologic imaging and thus make HSI a viable tool for histopathological diagnosis.